Merge lp://qastaging/~ken-vandine/desktopcouch/desktopcouch-lsof-karmic into lp://qastaging/desktopcouch
- desktopcouch-lsof-karmic
- Merge into trunk
Status: | Superseded |
---|---|
Proposed branch: | lp://qastaging/~ken-vandine/desktopcouch/desktopcouch-lsof-karmic |
Merge into: | lp://qastaging/desktopcouch |
Diff against target: | None lines |
To merge this branch: | bzr merge lp://qastaging/~ken-vandine/desktopcouch/desktopcouch-lsof-karmic |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
dobey (community) | Needs Fixing | ||
Elliot Murphy (community) | Approve | ||
Review via email:
|
This proposal supersedes a proposal from 2009-07-22.
This proposal has been superseded by a proposal from 2009-07-24.
Commit message
Description of the change
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Ken VanDine (ken-vandine) wrote : Posted in a previous version of this proposal | # |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Tim Cole (tcole) wrote : Posted in a previous version of this proposal | # |
Isn't there another branch in somewhere to get rid of the use of lsof?
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Ken VanDine (ken-vandine) wrote : Posted in a previous version of this proposal | # |
> Isn't there another branch in somewhere to get rid of the use of lsof?
Yeah, you had a branch that dropped it from advertisePort.py, which I had missed. That is merged, but lsof is used in the start script too. I am currently working on revising it a bit to use a common method for checking the port (using your code).
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Ken VanDine (ken-vandine) wrote : Posted in a previous version of this proposal | # |
I updated my branch to include some refactoring adding a find_pid and find_port method
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Ken VanDine (ken-vandine) : Posted in a previous version of this proposal | # |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Elliot Murphy (statik) wrote : | # |
great branch, thanks for cleaning this up.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
dobey (dobey) wrote : | # |
I don't understand some of the changes here. You got rid of Tim's code that doesn't use lsof at all, to bring back lsof, but your changes seem to break support for running on Jaunty. The lsof output seems to be different enough between the two, to warrant avoiding using it. Since Tim's code fixed the issue of lsof not working on Karmic, I'm a bit confused about what the rest of the branch is for?
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Ken VanDine (ken-vandine) wrote : | # |
> I don't understand some of the changes here. You got rid of Tim's code that
> doesn't use lsof at all, to bring back lsof, but your changes seem to break
> support for running on Jaunty. The lsof output seems to be different enough
> between the two, to warrant avoiding using it. Since Tim's code fixed the
> issue of lsof not working on Karmic, I'm a bit confused about what the rest of
> the branch is for?
My branch removes lsof completely, tim's branch removed it from one place. I just made his code reusable.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
dobey (dobey) wrote : | # |
OK. I misread the reivision diffs because the full diff wasn't generated yet. It looks better in the full diff, though there are some issues.
108 - couchversion = line.split()[-1]
109 + couchversion = line.split()[2]
This change will break. "Apache" is not the version of couchdb, which is what this change would result in for me, looking at the output of couchdb -V. If it's not working on your system, I suspect there's a problem and couchdb is actually trying to start a process.
Also, the comment about poking proc instead of running lsof should be moved to the function that actually pokes proc, since it is irrelevant to finding the PID. I'm guessing this was just overlooked in your splitting of the function.
- 18. By Ken VanDine
-
Don't set shell=True for subprocess, it causes strange results and isn't needed
- 19. By Ken VanDine
-
moved the commit about lsof vs. /proc digging to the find_port method
- 20. By Ken VanDine
-
* reorganized a little to work better as an installed service
* added org.desktopcouch.CouchDB. service for dbus activation - 21. By Ken VanDine
-
moved to data, it only contains the template now
- 22. By Ken VanDine
-
Moved pair.py to bin/desktopcouc
h-pair since this is an executable used on the desktop - 23. By Ken VanDine
-
added packaging
- 24. By Ken VanDine
-
Cleanup after some packaging
- 25. By Ken VanDine
-
fixed the license
- 26. By Ken VanDine
-
* Don't use lsof to determine port, use the find_port method in desktopcouch.
* Don't fall back to the system couch - 27. By Ken VanDine
-
use xdg to find the path for the bookmark template
- 28. By Ken VanDine
-
specify the python interpretor
- 29. By Ken VanDine
-
specify the copyright
- 30. By Ken VanDine
-
added a . to Description
- 31. By Ken VanDine
-
Split the package into desktopcouch, python-desktopcouch and python-
desktopcouch- records - 32. By Ken VanDine
-
desktop file for the pairing utility
- 33. By Ken VanDine
-
added pycompat
- 34. By Ken VanDine
-
removed packaging, we will add that to another branch
Unmerged revisions
Preview Diff
1 | === modified file 'desktopcouch/__init__.py' |
2 | --- desktopcouch/__init__.py 2009-07-08 17:48:11 +0000 |
3 | +++ desktopcouch/__init__.py 2009-07-22 21:04:40 +0000 |
4 | @@ -14,3 +14,95 @@ |
5 | # You should have received a copy of the GNU Lesser General Public License |
6 | # along with desktopcouch. If not, see <http://www.gnu.org/licenses/>. |
7 | "Desktop Couch helper files" |
8 | + |
9 | +import os |
10 | +import re |
11 | +from desktopcouch import local_files |
12 | +import errno |
13 | +import time |
14 | + |
15 | +def find_pid(): |
16 | + # Work out whether CouchDB is running by looking at its pid file |
17 | + fp = open(local_files.FILE_PID) |
18 | + pid = int(fp.read()) |
19 | + fp.close() |
20 | + try: |
21 | + os.kill(pid, 0) |
22 | + except os.error, detail: |
23 | + if detail.errno == errno.ESRCH: |
24 | + # pidfile is stale |
25 | + # start CouchDB by running the startup script |
26 | + print "Desktop CouchDB is not running; starting it." |
27 | + import start_local_couchdb |
28 | + start_local_couchdb.start_couchdb() |
29 | + time.sleep(2) # give the process a chance to start |
30 | + |
31 | + # Look in the CouchDB log to find the port number, someday. |
32 | + # Currently, we have to grovel around in /proc instead. |
33 | + # Oh, the huge manatee... (this replaced an lsof shell recipe |
34 | + # which was shorter but less reliable) |
35 | + |
36 | + # get the pid |
37 | + try: |
38 | + with open(local_files.FILE_PID) as pid_file: |
39 | + pid = int(pid_file.read().strip()) |
40 | + except IOError, e: |
41 | + if e.errno == ENOENT: |
42 | + raise RuntimeError("desktop-couch not started") |
43 | + else: |
44 | + raise |
45 | + return pid |
46 | + |
47 | +def find_port(pid): |
48 | + proc_dir = "/proc/%s" % (pid,) |
49 | + |
50 | + # check to make sure that the process still exists |
51 | + if not os.path.isdir(proc_dir): |
52 | + raise RuntimeError("Stale PID file or desktop-couch crashed") |
53 | + |
54 | + # check to make sure it is actually a desktop-couch instance |
55 | + with open(os.path.join(proc_dir, 'cmdline')) as cmd_file: |
56 | + cmd = cmd_file.read() |
57 | + if re.search('desktop-couch', cmd) is None: |
58 | + raise RuntimeError("Stale PID file") |
59 | + |
60 | + # enumerate the process' file descriptors |
61 | + fd_dir = os.path.join(proc_dir, 'fd') |
62 | + fd_paths = [os.readlink(os.path.join(fd_dir, fd)) |
63 | + for fd in os.listdir(fd_dir)] |
64 | + |
65 | + # identify socket fds |
66 | + socket_matches = [re.match('socket:\\[([0-9]+)\\]', p) for p in fd_paths] |
67 | + # extract their inode numbers |
68 | + socket_inodes = [m.group(1) for m in socket_matches if m is not None] |
69 | + |
70 | + # construct a subexpression which matches any one of these inodes |
71 | + inode_subexp = "|".join(socket_inodes) |
72 | + # construct regexp to match /proc/net/tcp entries which are listening |
73 | + # sockets having one of the given inode numbers |
74 | + listening_regexp = re.compile(r''' |
75 | + \s*\d+:\s* # sl |
76 | + 0100007F:([0-9A-F]{4})\s+ # local_address |
77 | + 00000000:0000\s+ # rem_address |
78 | + 0A\s+ # st (0A = listening) |
79 | + [0-9A-F]{8}: # tx_queue |
80 | + [0-9A-F]{8}\s+ # rx_queue |
81 | + [0-9A-F]{2}: # tr |
82 | + [0-9A-F]{8}\s+ # tm->when |
83 | + [0-9A-F]{8}\s* # retrnsmt |
84 | + \d+\s+\d+\s+ # uid, timeout |
85 | + (?:%s)\s+ # inode |
86 | + ''' % (inode_subexp,), re.VERBOSE) |
87 | + |
88 | + # extract the TCP port from the first matching line in /proc/$pid/net/tcp |
89 | + port = None |
90 | + with open(os.path.join(proc_dir, 'net', 'tcp')) as tcp_file: |
91 | + for line in tcp_file: |
92 | + match = listening_regexp.match(line) |
93 | + if match is not None: |
94 | + port = int(match.group(1), 16) |
95 | + break |
96 | + if port is None: |
97 | + raise RuntimeError("Unable to find listening port") |
98 | + |
99 | + return str(port) |
100 | |
101 | === modified file 'desktopcouch/local_files.py' |
102 | --- desktopcouch/local_files.py 2009-07-21 19:14:38 +0000 |
103 | +++ desktopcouch/local_files.py 2009-07-22 21:04:40 +0000 |
104 | @@ -69,7 +69,7 @@ |
105 | process = subprocess.Popen([COUCH_EXE, '-V'], shell=True, |
106 | stdout=subprocess.PIPE) |
107 | line = process.stdout.read().split('\n')[0] |
108 | - couchversion = line.split()[-1] |
109 | + couchversion = line.split()[2] |
110 | if couchversion.startswith('0.1'): |
111 | chain = '-a' |
112 | else: |
113 | @@ -82,4 +82,3 @@ |
114 | COUCH_EXEC_COMMAND = [COUCH_EXE, couch_chain_flag(), FILE_INI, '-p', FILE_PID, |
115 | '-o', FILE_STDOUT, '-e', FILE_STDERR] |
116 | |
117 | - |
118 | |
119 | === added file 'start-desktop-couchdb.sh' |
120 | --- start-desktop-couchdb.sh 1970-01-01 00:00:00 +0000 |
121 | +++ start-desktop-couchdb.sh 2009-07-22 21:05:19 +0000 |
122 | @@ -0,0 +1,12 @@ |
123 | +#!/bin/bash |
124 | + |
125 | +if [ -f 'desktopcouch/local_files.py' ]; |
126 | +then |
127 | + PYTHONPATH=. |
128 | +fi |
129 | + |
130 | +PYTHONPATH=$PYTHONPATH python utilities/start_local_couchdb.py |
131 | + |
132 | +if [ "$DBUS_SESSION_BUS_ADDRESS" ]; then |
133 | + PYTHONPATH=$PYTHONPATH python utilities/advertisePort.py & # background, 'cos a mainloop |
134 | +fi |
135 | |
136 | === added file 'stop-desktop-couchdb.sh' |
137 | --- stop-desktop-couchdb.sh 1970-01-01 00:00:00 +0000 |
138 | +++ stop-desktop-couchdb.sh 2009-07-22 21:05:19 +0000 |
139 | @@ -0,0 +1,16 @@ |
140 | +#!/bin/bash |
141 | + |
142 | +if [ -f 'desktopcouch/local_files.py' ]; |
143 | +then |
144 | + PYTHONPATH=. |
145 | +fi |
146 | + |
147 | +# Stop Desktop CouchDB itself |
148 | +PYTHONPATH=$PYTHONPATH python utilities/stop_local_couchdb.py |
149 | + |
150 | +# Stop advertising the slave's port over D-Bus |
151 | +if [ "$DBUS_SESSION_BUS_ADDRESS" ]; then |
152 | + dbus-send --session --dest=org.desktopcouch.CouchDB \ |
153 | + --type=method_call / org.desktopcouch.CouchDB.quit |
154 | +fi |
155 | + |
156 | |
157 | === modified file 'utilities/advertisePort.py' |
158 | --- utilities/advertisePort.py 2009-07-21 19:14:38 +0000 |
159 | +++ utilities/advertisePort.py 2009-07-22 21:04:40 +0000 |
160 | @@ -34,6 +34,7 @@ |
161 | |
162 | """ |
163 | |
164 | +import desktopcouch |
165 | from desktopcouch import local_files |
166 | |
167 | import re |
168 | @@ -65,85 +66,8 @@ |
169 | self.death() |
170 | |
171 | if __name__ == "__main__": |
172 | - # Work out whether CouchDB is running by looking at its pid file |
173 | - fp = open(local_files.FILE_PID) |
174 | - pid = int(fp.read()) |
175 | - fp.close() |
176 | - try: |
177 | - os.kill(pid, 0) |
178 | - except os.error, detail: |
179 | - if detail.errno == errno.ESRCH: |
180 | - # pidfile is stale |
181 | - # start CouchDB by running the startup script |
182 | - print "Desktop CouchDB is not running; starting it." |
183 | - import start_local_couchdb |
184 | - start_local_couchdb.start_couchdb() |
185 | - time.sleep(2) # give the process a chance to start |
186 | - |
187 | - # Look in the CouchDB log to find the port number, someday. |
188 | - # Currently, we have to grovel around in /proc instead. |
189 | - # Oh, the huge manatee... (this replaced an lsof shell recipe |
190 | - # which was shorter but less reliable) |
191 | - |
192 | - # get the pid |
193 | - try: |
194 | - with open(local_files.FILE_PID) as pid_file: |
195 | - pid = int(pid_file.read().strip()) |
196 | - except IOError, e: |
197 | - if e.errno == ENOENT: |
198 | - raise RuntimeError("desktop-couch not started") |
199 | - else: |
200 | - raise |
201 | - proc_dir = "/proc/%s" % (pid,) |
202 | - |
203 | - # check to make sure that the process still exists |
204 | - if not os.path.isdir(proc_dir): |
205 | - raise RuntimeError("Stale PID file or desktop-couch crashed") |
206 | - |
207 | - # check to make sure it is actually a desktop-couch instance |
208 | - with open(os.path.join(proc_dir, 'cmdline')) as cmd_file: |
209 | - cmd = cmd_file.read() |
210 | - if re.search('desktop-couch', cmd) is None: |
211 | - raise RuntimeError("Stale PID file") |
212 | - |
213 | - # enumerate the process' file descriptors |
214 | - fd_dir = os.path.join(proc_dir, 'fd') |
215 | - fd_paths = [os.readlink(os.path.join(fd_dir, fd)) |
216 | - for fd in os.listdir(fd_dir)] |
217 | - |
218 | - # identify socket fds |
219 | - socket_matches = [re.match('socket:\\[([0-9]+)\\]', p) for p in fd_paths] |
220 | - # extract their inode numbers |
221 | - socket_inodes = [m.group(1) for m in socket_matches if m is not None] |
222 | - |
223 | - # construct a subexpression which matches any one of these inodes |
224 | - inode_subexp = "|".join(socket_inodes) |
225 | - # construct regexp to match /proc/net/tcp entries which are listening |
226 | - # sockets having one of the given inode numbers |
227 | - listening_regexp = re.compile(r''' |
228 | - \s*\d+:\s* # sl |
229 | - 0100007F:([0-9A-F]{4})\s+ # local_address |
230 | - 00000000:0000\s+ # rem_address |
231 | - 0A\s+ # st (0A = listening) |
232 | - [0-9A-F]{8}: # tx_queue |
233 | - [0-9A-F]{8}\s+ # rx_queue |
234 | - [0-9A-F]{2}: # tr |
235 | - [0-9A-F]{8}\s+ # tm->when |
236 | - [0-9A-F]{8}\s* # retrnsmt |
237 | - \d+\s+\d+\s+ # uid, timeout |
238 | - (?:%s)\s+ # inode |
239 | - ''' % (inode_subexp,), re.VERBOSE) |
240 | - |
241 | - # extract the TCP port from the first matching line in /proc/$pid/net/tcp |
242 | - port = None |
243 | - with open(os.path.join(proc_dir, 'net', 'tcp')) as tcp_file: |
244 | - for line in tcp_file: |
245 | - match = listening_regexp.match(line) |
246 | - if match is not None: |
247 | - port = int(match.group(1), 16) |
248 | - break |
249 | - if port is None: |
250 | - raise RuntimeError("Unable to find listening port") |
251 | + pid = desktopcouch.find_pid() |
252 | + port = desktopcouch.find_port(pid) |
253 | |
254 | # Advertise the port |
255 | mainloop = gobject.MainLoop() |
256 | |
257 | === modified file 'utilities/start_local_couchdb.py' |
258 | --- utilities/start_local_couchdb.py 2009-07-21 19:14:38 +0000 |
259 | +++ utilities/start_local_couchdb.py 2009-07-22 21:04:40 +0000 |
260 | @@ -33,6 +33,7 @@ |
261 | |
262 | from __future__ import with_statement |
263 | import os, subprocess, sys |
264 | +import desktopcouch |
265 | from desktopcouch import local_files |
266 | |
267 | def dump_ini(data, filename): |
268 | @@ -107,16 +108,10 @@ |
269 | html = fp.read() |
270 | fp.close() |
271 | |
272 | - # work out port number the horrible lsof way |
273 | import time |
274 | time.sleep(1) |
275 | - lsofcmd = 'lsof -i4TCP@localhost | grep $(couchdb -C %s -p %s -s | ' + \ |
276 | - 'cut -d" " -f7 | cut -d "," -f1) | grep "TCP " | ' + \ |
277 | - 'grep LISTEN | awk \'{print $8}\' | cut -d":" -f2' |
278 | - actual_lsof_cmd = lsofcmd % (local_files.FILE_INI, local_files.FILE_PID) |
279 | - stdout, stderr = subprocess.Popen(actual_lsof_cmd, shell=True, |
280 | - stdout=subprocess.PIPE).communicate() |
281 | - port = stdout.strip() |
282 | + pid = desktopcouch.find_pid() |
283 | + port = desktopcouch.find_port(pid) |
284 | |
285 | fp = open(bookmark_file, "w") |
286 | fp.write(html.replace("[[COUCHDB_PORT]]", port)) |
Change to handle the output of lsof in karmic, perhaps it is different from jaunty