desktopcouch

Merge lp://qastaging/~ken-vandine/desktopcouch/desktopcouch-lsof-karmic into lp://qastaging/desktopcouch

  1. desktopcouch-lsof-karmic
  2. Merge into trunk
Proposed by Ken VanDine
Status: Merged
Merged at revision: not available
Proposed branch: lp://qastaging/~ken-vandine/desktopcouch/desktopcouch-lsof-karmic
Merge into: lp://qastaging/desktopcouch
Diff against target: None lines
To merge this branch: bzr merge lp://qastaging/~ken-vandine/desktopcouch/desktopcouch-lsof-karmic
Reviewer Review Type Date Requested Status
dobey (community) Approve
Review via email: mp+9258@code.qastaging.launchpad.net

This proposal supersedes a proposal from 2009-07-23.

To post a comment you must log in.
Revision history for this message
Ken VanDine (ken-vandine) wrote : Posted in a previous version of this proposal

Change to handle the output of lsof in karmic, perhaps it is different from jaunty

Revision history for this message
Tim Cole (tcole) wrote : Posted in a previous version of this proposal

Isn't there another branch in somewhere to get rid of the use of lsof?

review: Needs Information
Revision history for this message
Ken VanDine (ken-vandine) wrote : Posted in a previous version of this proposal

> Isn't there another branch in somewhere to get rid of the use of lsof?

Yeah, you had a branch that dropped it from advertisePort.py, which I had missed. That is merged, but lsof is used in the start script too. I am currently working on revising it a bit to use a common method for checking the port (using your code).

Revision history for this message
Ken VanDine (ken-vandine) wrote : Posted in a previous version of this proposal

I updated my branch to include some refactoring adding a find_pid and find_port method

Revision history for this message
Ken VanDine (ken-vandine) : Posted in a previous version of this proposal
review: Needs Resubmitting
Revision history for this message
Elliot Murphy (statik) wrote : Posted in a previous version of this proposal

great branch, thanks for cleaning this up.

review: Approve
Revision history for this message
dobey (dobey) wrote : Posted in a previous version of this proposal

I don't understand some of the changes here. You got rid of Tim's code that doesn't use lsof at all, to bring back lsof, but your changes seem to break support for running on Jaunty. The lsof output seems to be different enough between the two, to warrant avoiding using it. Since Tim's code fixed the issue of lsof not working on Karmic, I'm a bit confused about what the rest of the branch is for?

review: Needs Information
Revision history for this message
Ken VanDine (ken-vandine) wrote : Posted in a previous version of this proposal

> I don't understand some of the changes here. You got rid of Tim's code that
> doesn't use lsof at all, to bring back lsof, but your changes seem to break
> support for running on Jaunty. The lsof output seems to be different enough
> between the two, to warrant avoiding using it. Since Tim's code fixed the
> issue of lsof not working on Karmic, I'm a bit confused about what the rest of
> the branch is for?

My branch removes lsof completely, tim's branch removed it from one place. I just made his code reusable.

Revision history for this message
dobey (dobey) wrote : Posted in a previous version of this proposal

OK. I misread the reivision diffs because the full diff wasn't generated yet. It looks better in the full diff, though there are some issues.

108 - couchversion = line.split()[-1]
109 + couchversion = line.split()[2]

This change will break. "Apache" is not the version of couchdb, which is what this change would result in for me, looking at the output of couchdb -V. If it's not working on your system, I suspect there's a problem and couchdb is actually trying to start a process.

Also, the comment about poking proc instead of running lsof should be moved to the function that actually pokes proc, since it is irrelevant to finding the PID. I'm guessing this was just overlooked in your splitting of the function.

review: Needs Fixing
Revision history for this message
dobey (dobey) wrote :

I'm approving the code, but the comment in find_pid() about poking proc should be moved to find_port() where proc is actually poked.

review: Approve
lp://qastaging/~ken-vandine/desktopcouch/desktopcouch-lsof-karmic updated
19. By Ken VanDine

moved the commit about lsof vs. /proc digging to the find_port method

20. By Ken VanDine

* reorganized a little to work better as an installed service
* added org.desktopcouch.CouchDB.service for dbus activation

21. By Ken VanDine

moved to data, it only contains the template now

22. By Ken VanDine

Moved pair.py to bin/desktopcouch-pair since this is an executable used on the desktop

23. By Ken VanDine

added packaging

24. By Ken VanDine

Cleanup after some packaging

25. By Ken VanDine

fixed the license

26. By Ken VanDine

* Don't use lsof to determine port, use the find_port method in desktopcouch.
* Don't fall back to the system couch

27. By Ken VanDine

use xdg to find the path for the bookmark template

28. By Ken VanDine

specify the python interpretor

29. By Ken VanDine

specify the copyright

30. By Ken VanDine

added a . to Description

31. By Ken VanDine

Split the package into desktopcouch, python-desktopcouch and python-desktopcouch-records

32. By Ken VanDine

desktop file for the pairing utility

33. By Ken VanDine

added pycompat

34. By Ken VanDine

removed packaging, we will add that to another branch

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'desktopcouch/__init__.py'
2--- desktopcouch/__init__.py 2009-07-08 17:48:11 +0000
3+++ desktopcouch/__init__.py 2009-07-22 21:04:40 +0000
4@@ -14,3 +14,95 @@
5 # You should have received a copy of the GNU Lesser General Public License
6 # along with desktopcouch. If not, see <http://www.gnu.org/licenses/>.
7 "Desktop Couch helper files"
8+
9+import os
10+import re
11+from desktopcouch import local_files
12+import errno
13+import time
14+
15+def find_pid():
16+ # Work out whether CouchDB is running by looking at its pid file
17+ fp = open(local_files.FILE_PID)
18+ pid = int(fp.read())
19+ fp.close()
20+ try:
21+ os.kill(pid, 0)
22+ except os.error, detail:
23+ if detail.errno == errno.ESRCH:
24+ # pidfile is stale
25+ # start CouchDB by running the startup script
26+ print "Desktop CouchDB is not running; starting it."
27+ import start_local_couchdb
28+ start_local_couchdb.start_couchdb()
29+ time.sleep(2) # give the process a chance to start
30+
31+ # Look in the CouchDB log to find the port number, someday.
32+ # Currently, we have to grovel around in /proc instead.
33+ # Oh, the huge manatee... (this replaced an lsof shell recipe
34+ # which was shorter but less reliable)
35+
36+ # get the pid
37+ try:
38+ with open(local_files.FILE_PID) as pid_file:
39+ pid = int(pid_file.read().strip())
40+ except IOError, e:
41+ if e.errno == ENOENT:
42+ raise RuntimeError("desktop-couch not started")
43+ else:
44+ raise
45+ return pid
46+
47+def find_port(pid):
48+ proc_dir = "/proc/%s" % (pid,)
49+
50+ # check to make sure that the process still exists
51+ if not os.path.isdir(proc_dir):
52+ raise RuntimeError("Stale PID file or desktop-couch crashed")
53+
54+ # check to make sure it is actually a desktop-couch instance
55+ with open(os.path.join(proc_dir, 'cmdline')) as cmd_file:
56+ cmd = cmd_file.read()
57+ if re.search('desktop-couch', cmd) is None:
58+ raise RuntimeError("Stale PID file")
59+
60+ # enumerate the process' file descriptors
61+ fd_dir = os.path.join(proc_dir, 'fd')
62+ fd_paths = [os.readlink(os.path.join(fd_dir, fd))
63+ for fd in os.listdir(fd_dir)]
64+
65+ # identify socket fds
66+ socket_matches = [re.match('socket:\\[([0-9]+)\\]', p) for p in fd_paths]
67+ # extract their inode numbers
68+ socket_inodes = [m.group(1) for m in socket_matches if m is not None]
69+
70+ # construct a subexpression which matches any one of these inodes
71+ inode_subexp = "|".join(socket_inodes)
72+ # construct regexp to match /proc/net/tcp entries which are listening
73+ # sockets having one of the given inode numbers
74+ listening_regexp = re.compile(r'''
75+ \s*\d+:\s* # sl
76+ 0100007F:([0-9A-F]{4})\s+ # local_address
77+ 00000000:0000\s+ # rem_address
78+ 0A\s+ # st (0A = listening)
79+ [0-9A-F]{8}: # tx_queue
80+ [0-9A-F]{8}\s+ # rx_queue
81+ [0-9A-F]{2}: # tr
82+ [0-9A-F]{8}\s+ # tm->when
83+ [0-9A-F]{8}\s* # retrnsmt
84+ \d+\s+\d+\s+ # uid, timeout
85+ (?:%s)\s+ # inode
86+ ''' % (inode_subexp,), re.VERBOSE)
87+
88+ # extract the TCP port from the first matching line in /proc/$pid/net/tcp
89+ port = None
90+ with open(os.path.join(proc_dir, 'net', 'tcp')) as tcp_file:
91+ for line in tcp_file:
92+ match = listening_regexp.match(line)
93+ if match is not None:
94+ port = int(match.group(1), 16)
95+ break
96+ if port is None:
97+ raise RuntimeError("Unable to find listening port")
98+
99+ return str(port)
100
101=== modified file 'desktopcouch/local_files.py'
102--- desktopcouch/local_files.py 2009-07-21 19:14:38 +0000
103+++ desktopcouch/local_files.py 2009-07-24 18:34:34 +0000
104@@ -66,7 +66,7 @@
105 sys.exit(1)
106
107 def couch_chain_flag():
108- process = subprocess.Popen([COUCH_EXE, '-V'], shell=True,
109+ process = subprocess.Popen([COUCH_EXE, '-V'], shell=False,
110 stdout=subprocess.PIPE)
111 line = process.stdout.read().split('\n')[0]
112 couchversion = line.split()[-1]
113@@ -82,4 +82,3 @@
114 COUCH_EXEC_COMMAND = [COUCH_EXE, couch_chain_flag(), FILE_INI, '-p', FILE_PID,
115 '-o', FILE_STDOUT, '-e', FILE_STDERR]
116
117-
118
119=== added file 'start-desktop-couchdb.sh'
120--- start-desktop-couchdb.sh 1970-01-01 00:00:00 +0000
121+++ start-desktop-couchdb.sh 2009-07-22 21:05:19 +0000
122@@ -0,0 +1,12 @@
123+#!/bin/bash
124+
125+if [ -f 'desktopcouch/local_files.py' ];
126+then
127+ PYTHONPATH=.
128+fi
129+
130+PYTHONPATH=$PYTHONPATH python utilities/start_local_couchdb.py
131+
132+if [ "$DBUS_SESSION_BUS_ADDRESS" ]; then
133+ PYTHONPATH=$PYTHONPATH python utilities/advertisePort.py & # background, 'cos a mainloop
134+fi
135
136=== added file 'stop-desktop-couchdb.sh'
137--- stop-desktop-couchdb.sh 1970-01-01 00:00:00 +0000
138+++ stop-desktop-couchdb.sh 2009-07-22 21:05:19 +0000
139@@ -0,0 +1,16 @@
140+#!/bin/bash
141+
142+if [ -f 'desktopcouch/local_files.py' ];
143+then
144+ PYTHONPATH=.
145+fi
146+
147+# Stop Desktop CouchDB itself
148+PYTHONPATH=$PYTHONPATH python utilities/stop_local_couchdb.py
149+
150+# Stop advertising the slave's port over D-Bus
151+if [ "$DBUS_SESSION_BUS_ADDRESS" ]; then
152+ dbus-send --session --dest=org.desktopcouch.CouchDB \
153+ --type=method_call / org.desktopcouch.CouchDB.quit
154+fi
155+
156
157=== modified file 'utilities/advertisePort.py'
158--- utilities/advertisePort.py 2009-07-21 19:14:38 +0000
159+++ utilities/advertisePort.py 2009-07-22 21:04:40 +0000
160@@ -34,6 +34,7 @@
161
162 """
163
164+import desktopcouch
165 from desktopcouch import local_files
166
167 import re
168@@ -65,85 +66,8 @@
169 self.death()
170
171 if __name__ == "__main__":
172- # Work out whether CouchDB is running by looking at its pid file
173- fp = open(local_files.FILE_PID)
174- pid = int(fp.read())
175- fp.close()
176- try:
177- os.kill(pid, 0)
178- except os.error, detail:
179- if detail.errno == errno.ESRCH:
180- # pidfile is stale
181- # start CouchDB by running the startup script
182- print "Desktop CouchDB is not running; starting it."
183- import start_local_couchdb
184- start_local_couchdb.start_couchdb()
185- time.sleep(2) # give the process a chance to start
186-
187- # Look in the CouchDB log to find the port number, someday.
188- # Currently, we have to grovel around in /proc instead.
189- # Oh, the huge manatee... (this replaced an lsof shell recipe
190- # which was shorter but less reliable)
191-
192- # get the pid
193- try:
194- with open(local_files.FILE_PID) as pid_file:
195- pid = int(pid_file.read().strip())
196- except IOError, e:
197- if e.errno == ENOENT:
198- raise RuntimeError("desktop-couch not started")
199- else:
200- raise
201- proc_dir = "/proc/%s" % (pid,)
202-
203- # check to make sure that the process still exists
204- if not os.path.isdir(proc_dir):
205- raise RuntimeError("Stale PID file or desktop-couch crashed")
206-
207- # check to make sure it is actually a desktop-couch instance
208- with open(os.path.join(proc_dir, 'cmdline')) as cmd_file:
209- cmd = cmd_file.read()
210- if re.search('desktop-couch', cmd) is None:
211- raise RuntimeError("Stale PID file")
212-
213- # enumerate the process' file descriptors
214- fd_dir = os.path.join(proc_dir, 'fd')
215- fd_paths = [os.readlink(os.path.join(fd_dir, fd))
216- for fd in os.listdir(fd_dir)]
217-
218- # identify socket fds
219- socket_matches = [re.match('socket:\\[([0-9]+)\\]', p) for p in fd_paths]
220- # extract their inode numbers
221- socket_inodes = [m.group(1) for m in socket_matches if m is not None]
222-
223- # construct a subexpression which matches any one of these inodes
224- inode_subexp = "|".join(socket_inodes)
225- # construct regexp to match /proc/net/tcp entries which are listening
226- # sockets having one of the given inode numbers
227- listening_regexp = re.compile(r'''
228- \s*\d+:\s* # sl
229- 0100007F:([0-9A-F]{4})\s+ # local_address
230- 00000000:0000\s+ # rem_address
231- 0A\s+ # st (0A = listening)
232- [0-9A-F]{8}: # tx_queue
233- [0-9A-F]{8}\s+ # rx_queue
234- [0-9A-F]{2}: # tr
235- [0-9A-F]{8}\s+ # tm->when
236- [0-9A-F]{8}\s* # retrnsmt
237- \d+\s+\d+\s+ # uid, timeout
238- (?:%s)\s+ # inode
239- ''' % (inode_subexp,), re.VERBOSE)
240-
241- # extract the TCP port from the first matching line in /proc/$pid/net/tcp
242- port = None
243- with open(os.path.join(proc_dir, 'net', 'tcp')) as tcp_file:
244- for line in tcp_file:
245- match = listening_regexp.match(line)
246- if match is not None:
247- port = int(match.group(1), 16)
248- break
249- if port is None:
250- raise RuntimeError("Unable to find listening port")
251+ pid = desktopcouch.find_pid()
252+ port = desktopcouch.find_port(pid)
253
254 # Advertise the port
255 mainloop = gobject.MainLoop()
256
257=== modified file 'utilities/start_local_couchdb.py'
258--- utilities/start_local_couchdb.py 2009-07-21 19:14:38 +0000
259+++ utilities/start_local_couchdb.py 2009-07-22 21:04:40 +0000
260@@ -33,6 +33,7 @@
261
262 from __future__ import with_statement
263 import os, subprocess, sys
264+import desktopcouch
265 from desktopcouch import local_files
266
267 def dump_ini(data, filename):
268@@ -107,16 +108,10 @@
269 html = fp.read()
270 fp.close()
271
272- # work out port number the horrible lsof way
273 import time
274 time.sleep(1)
275- lsofcmd = 'lsof -i4TCP@localhost | grep $(couchdb -C %s -p %s -s | ' + \
276- 'cut -d" " -f7 | cut -d "," -f1) | grep "TCP " | ' + \
277- 'grep LISTEN | awk \'{print $8}\' | cut -d":" -f2'
278- actual_lsof_cmd = lsofcmd % (local_files.FILE_INI, local_files.FILE_PID)
279- stdout, stderr = subprocess.Popen(actual_lsof_cmd, shell=True,
280- stdout=subprocess.PIPE).communicate()
281- port = stdout.strip()
282+ pid = desktopcouch.find_pid()
283+ port = desktopcouch.find_port(pid)
284
285 fp = open(bookmark_file, "w")
286 fp.write(html.replace("[[COUCHDB_PORT]]", port))

Subscribers

People subscribed via source and target branches