desktopcouch

Merge lp://qastaging/~sil/desktopcouch/allow-unpairing into lp://qastaging/desktopcouch

  1. allow-unpairing
  2. Merge into trunk
Proposed by Stuart Langridge
Status: Merged
Approved by: Eric Casteleijn
Approved revision: 57
Merge reported by: Chad Miller
Merged at revision: not available
Proposed branch: lp://qastaging/~sil/desktopcouch/allow-unpairing
Merge into: lp://qastaging/desktopcouch
Diff against target: None lines
To merge this branch: bzr merge lp://qastaging/~sil/desktopcouch/allow-unpairing
Reviewer Review Type Date Requested Status
Eric Casteleijn (community) Approve
Nicola Larosa (community) Approve
Tim Cole (community) Abstain
Review via email: mp+11367@code.qastaging.launchpad.net

Commit message

Pair and unpair servers. Allows local pairing.

To post a comment you must log in.
Revision history for this message
Stuart Langridge (sil) wrote :

Pair and unpair servers. Allows local pairing -- do not package a desktopcouch with this in. Suggested for merge so cardinalfang can then work with passing oauth details back to newly paired DC servers.

Revision history for this message
Tim Cole (tcole) wrote :

The FIXME and "does not work" in the commit messages make me a little charry. What behavior are we supposed to expect from this branch, and why are we merging it to trunk if we don't want to ship it?

Revision history for this message
Tim Cole (tcole) :
review: Abstain
Revision history for this message
Stuart Langridge (sil) wrote :

> The FIXME and "does not work" in the commit messages make me a little charry.
> What behavior are we supposed to expect from this branch, and why are we
> merging it to trunk if we don't want to ship it?

It's being merged to trunk because we need to do some work based on it being done. Keeping this out of trunk and doing the further work based on this branch and then merging the lot makes for a mammoth merge in the future, and doing it in bite-size chunks makes review easier. The "does not work" stuff is because the code now knows how to receive oauth data from a remote desktopcouch, but remote desktopcouches don't send that data (this is the future work to be done). So the code itself works if handed correct data, but the overall approach (actually making local desktopcouches pair) does not (because it is not handed corrrect data).

Revision history for this message
Nicola Larosa (teknico) wrote :

This is apparently part of a work in progress, tests pass, code looks good.

review: Approve
Revision history for this message
Eric Casteleijn (thisfred) wrote :

Looks good, tests pass, but I'm getting a few pyflakes messages:

desktopcouch/__init__.py:19: 'time' imported but unused
desktopcouch/local_files.py:33: redefinition of unused 'configparser' from line 31
desktopcouch/replication_services/ubuntuone.py:18: redefinition of unused 'gnomekeyring' from line 6
desktopcouch/replication_services/__init__.py:3: 'ubuntuone' imported but unused
desktopcouch/replication_services/__init__.py:4: 'example' imported but unused
desktopcouch/pair/couchdb_pairing/network_io.py:24: 'ssl' imported but unused
desktopcouch/pair/couchdb_pairing/network_io.py:32: redefinition of unused 'get_remote_hostname' from line 29

(There are more, but both the pairing and the couchgrid do some awful things intentionally that I don't know how to fix cleanly.)

review: Approve
Revision history for this message
Chad Miller (cmiller) wrote :

text conflict in bin/desktopcouch-pair

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'bin/desktopcouch-pair'
2--- bin/desktopcouch-pair 2009-08-26 18:01:29 +0000
3+++ bin/desktopcouch-pair 2009-09-08 12:45:17 +0000
4@@ -65,10 +65,11 @@
5 from desktopcouch.pair.couchdb_pairing import network_io
6 from desktopcouch.pair.couchdb_pairing import dbus_io
7
8+from desktopcouch.records.server import CouchDatabase
9+from desktopcouch.records.record import Record
10+
11 discovery_tool_version = "1"
12
13-CLOUD_SERVICES = {}
14-
15 def generate_secret(length=7):
16 """Create a secret that is easy to write and read. We hate ambiguity and
17 errors."""
18@@ -445,30 +446,24 @@
19 already-listening tool instance. This sets up a "Bob" in the
20 module's story."""
21
22- # positions: host id, description, pair host, pair port, is_cloud
23- listening_hosts = gtk.TreeStore(str, str, str, int, bool)
24-
25- # If there is an Ubuntu One key in the keyring, add Ubuntu One
26- # as an item to be paired with
27- import gnomekeyring
28- try:
29- matches = gnomekeyring.find_items_sync(
30- gnomekeyring.ITEM_GENERIC_SECRET,
31- {'ubuntuone-realm': "https://ubuntuone.com",
32- 'oauth-consumer-key': "ubuntuone"})
33- except gnomekeyring.NoMatchError:
34- matches = None
35- if matches:
36- # parse "a=b&c=d" to {"a":"b","c":"d"}
37- oauth_data = dict([x.split("=", 1) for x in
38- matches[0].secret.split("&")])
39- oauth_data.update({
40- "consumer_key": "ubuntuone",
41- "consumer_secret": "",
42- })
43- listening_hosts.append(None, ["Ubuntu One", "The Ubuntu One cloud service",
44- "ubuntuone.com", 5984, True])
45- CLOUD_SERVICES["Ubuntu One"] = oauth_data
46+ # positions: host id, descr, host, port, cloud_name
47+ self.listening_hosts = gtk.TreeStore(str, str, str, int, str)
48+
49+ import desktopcouch.replication_services as services
50+
51+ for srv_name in dir(services):
52+ if srv_name.startswith("__"):
53+ continue
54+ srv = getattr(services, srv_name)
55+ try:
56+ if srv.is_active():
57+ all_paired_cloud_servers = [x.key for x in
58+ self.db.execute_view("byservicename", "pairedservers")]
59+ if not srv_name in all_paired_cloud_servers:
60+ self.listening_hosts.append(None, [srv.name, srv.description,
61+ "", 0, srv_name])
62+ except Exception, e:
63+ self.logging.exception("service %r has errors", srv_name)
64
65 self.inviting = None # pylint: disable-msg=W0201
66
67@@ -482,7 +477,7 @@
68 pick_box.pack_start(l, False, False, 0)
69 l.show()
70
71- tv = gtk.TreeView(listening_hosts)
72+ tv = gtk.TreeView(self.listening_hosts)
73 tv.set_headers_visible(False)
74 tv.set_rules_hint(True)
75 tv.show()
76@@ -495,16 +490,22 @@
77 if not iter:
78 return
79 service = model.get_value(iter, 0)
80+ description = model.get_value(iter, 1)
81 hostname = model.get_value(iter, 2)
82 port = model.get_value(iter, 3)
83- is_cloud = model.get_value(iter, 4)
84+ service_name = model.get_value(iter, 4)
85
86- if is_cloud:
87+ if service_name:
88 # Pairing with a cloud service, which doesn't do key exchange
89- pair_with_cloud_service(service, hostname, port)
90+ pair_with_cloud_service(service_name)
91+ # remove from listening list
92+ self.listening_hosts.remove(iter)
93+ # add to already-paired list
94+ srv = getattr(services, service_name)
95+ self.already_paired_hosts.append(None, [service, description,
96+ hostname, port, service_name])
97 return
98
99-
100 self.logging.error("Local pairing is not yet supported. Aborting.")
101 fail_note = gtk.MessageDialog(
102 parent=pick_or_listen.window,
103@@ -514,7 +515,7 @@
104 message_format =_("Sorry, couchdb authentication is not yet enabled, so local pairing is not supported"))
105 fail_note.run()
106 fail_note.destroy()
107- return False
108+ #FIXME return False
109
110
111 self.logging.info("connecting to %s:%s tcp to invite",
112@@ -539,38 +540,155 @@
113 def add_service_to_list(name, description, host, port, version):
114 """When a zeroconf service appears, this adds it to the
115 listing of choices."""
116-
117- listening_hosts.append(None, [name, description, host, port, False, None])
118+ self.listening_hosts.append(None, [name, description, host, port, None])
119
120 def remove_service_from_list(name):
121 """When a zeroconf service disappears, this finds it in the
122 listing and removes it as an option for picking."""
123
124- it = listening_hosts.get_iter_first()
125+ it = self.listening_hosts.get_iter_first()
126 while it is not None:
127- if listening_hosts.get_value(it, 0) == name:
128- listening_hosts.remove(it)
129+ if self.listening_hosts.get_value(it, 0) == name:
130+ self.listening_hosts.remove(it)
131 return True
132- it = listening_hosts.iter_next(it)
133-
134- dbus_io.discover_services(add_service_to_list, remove_service_from_list)
135-
136- cell = gtk.CellRendererText()
137- cell.set_property("weight", pango.WEIGHT_BOLD)
138- cell.set_property("weight-set", True)
139- tv.append_column(hostid_col)
140- hostid_col.pack_start(cell, True)
141- hostid_col.add_attribute(cell, 'text', 0)
142-
143- cell = gtk.CellRendererText()
144- cell.set_property("ellipsize", pango.ELLIPSIZE_END)
145- cell.set_property("ellipsize-set", True)
146- tv.append_column(description_col)
147- description_col.pack_start(cell, True)
148- description_col.add_attribute(cell, 'text', 1)
149-
150- return pick_box
151-
152+ it = self.listening_hosts.iter_next(it)
153+
154+ dbus_io.discover_services(add_service_to_list, remove_service_from_list, show_local=True)
155+
156+ cell = gtk.CellRendererText()
157+ cell.set_property("weight", pango.WEIGHT_BOLD)
158+ cell.set_property("weight-set", True)
159+ tv.append_column(hostid_col)
160+ hostid_col.pack_start(cell, True)
161+ hostid_col.add_attribute(cell, 'text', 0)
162+
163+ cell = gtk.CellRendererText()
164+ cell.set_property("ellipsize", pango.ELLIPSIZE_END)
165+ cell.set_property("ellipsize-set", True)
166+ tv.append_column(description_col)
167+ description_col.pack_start(cell, True)
168+ description_col.add_attribute(cell, 'text', 1)
169+
170+ return pick_box
171+
172+ def create_already_paired_pane(self, container):
173+ """Set up the pane that shows servers which are already paired."""
174+
175+ # positions: host id, descr, host, port, cloud_name
176+ self.already_paired_hosts = gtk.TreeStore(str, str, str, int, str)
177+
178+ import desktopcouch.replication_services as services
179+ for already_paired_record in self.db.execute_view("byserver",
180+ "pairedservers"):
181+ host, port = already_paired_record.value["server"].split(":")
182+ try:
183+ port = int(port)
184+ except:
185+ port = 0
186+ self.already_paired_hosts.append(None, [host, None, host, port, None])
187+
188+ for already_paired_record in self.db.execute_view("byservicename",
189+ "pairedservers"):
190+ srv_name = already_paired_record.value["service_name"]
191+ if srv_name.startswith("__"):
192+ continue
193+ srv = getattr(services, srv_name)
194+ if srv.is_active():
195+ self.already_paired_hosts.append(None, [srv.name, srv.description,
196+ "", 0, srv_name])
197+
198+ hostid_col = gtk.TreeViewColumn(_("service name"))
199+ description_col = gtk.TreeViewColumn(_("service name"))
200+
201+ pick_box = gtk.VBox()
202+ container.pack_start(pick_box, False, False, 10)
203+
204+ l = gtk.Label(_("Desktop Couches that you are already paired with"))
205+ pick_box.pack_start(l, False, False, 0)
206+ l.show()
207+
208+ tv = gtk.TreeView(self.already_paired_hosts)
209+ tv.set_headers_visible(False)
210+ tv.set_rules_hint(True)
211+ tv.show()
212+
213+ def clicked(selection):
214+ """An item in the list of services was clicked, so now we go
215+ about inviting it to pair with us."""
216+
217+ model, iter = selection.get_selected()
218+ if not iter:
219+ return
220+ service = model.get_value(iter, 0)
221+ hostname = model.get_value(iter, 2)
222+ port = model.get_value(iter, 3)
223+ service_name = model.get_value(iter, 4)
224+
225+ if service_name:
226+ # Pairing with a cloud service, which doesn't do key exchange
227+ # delete record
228+ for record in self.db.execute_view("byservicename",
229+ "pairedservers")[service_name]:
230+ self.db.delete_record(record.value["_id"])
231+
232+ # remove from already-paired list
233+ self.already_paired_hosts.remove(iter)
234+ # add to listening list
235+ srv = getattr(services, service_name)
236+ self.listening_hosts.append(None, [service, srv.description,
237+ hostname, port, service_name])
238+ return
239+
240+ self.logging.error("Local pairing is not yet supported. Aborting.")
241+ fail_note = gtk.MessageDialog(
242+ parent=pick_or_listen.window,
243+ flags=gtk.DIALOG_DESTROY_WITH_PARENT,
244+ buttons=gtk.BUTTONS_OK,
245+ type=gtk.MESSAGE_ERROR,
246+ message_format =_("Sorry, couchdb authentication is not yet enabled, so local pairing is not supported"))
247+ fail_note.run()
248+ fail_note.destroy()
249+ #FIXME return False
250+
251+ # delete record
252+ print list(self.db.execute_view("byserver",
253+ "pairedservers")), hostname, port
254+ for record in self.db.execute_view("byserver",
255+ "pairedservers")["%s:%s" % (hostname, port)]:
256+ self.db.delete_record(record.value["_id"])
257+
258+ # remove from already-paired list
259+ self.already_paired_hosts.remove(iter)
260+ # do not add to listening list -- if it's listening then zeroconf
261+ # will pick it up
262+ return
263+
264+ tv_selection = tv.get_selection()
265+ tv_selection.connect("changed", clicked)
266+
267+ scrolled_window = gtk.ScrolledWindow(hadjustment=None, vadjustment=None)
268+ scrolled_window.set_policy(gtk.POLICY_AUTOMATIC, gtk.POLICY_AUTOMATIC)
269+ scrolled_window.set_border_width(10)
270+ scrolled_window.add_with_viewport(tv)
271+ scrolled_window.show()
272+
273+ pick_box.pack_start(scrolled_window, True, False, 0)
274+
275+ cell = gtk.CellRendererText()
276+ cell.set_property("weight", pango.WEIGHT_BOLD)
277+ cell.set_property("weight-set", True)
278+ tv.append_column(hostid_col)
279+ hostid_col.pack_start(cell, True)
280+ hostid_col.add_attribute(cell, 'text', 0)
281+
282+ cell = gtk.CellRendererText()
283+ cell.set_property("ellipsize", pango.ELLIPSIZE_END)
284+ cell.set_property("ellipsize-set", True)
285+ tv.append_column(description_col)
286+ description_col.pack_start(cell, True)
287+ description_col.add_attribute(cell, 'text', 1)
288+
289+ return pick_box
290
291 def create_single_listen_pane(self, container):
292 """This sets up an "Alice" from the module's story.
293@@ -600,7 +718,7 @@
294 message_format =_("Sorry, couchdb authentication is not yet enabled, so local pairing is not supported"))
295 fail_note.run()
296 fail_note.destroy()
297- return False
298+ #FIXME return False
299
300
301 btn.set_sensitive(False)
302@@ -637,6 +755,36 @@
303 #some_row_in_list.connect("clicked", self.listen, target_db_info)
304
305 def __init__(self):
306+
307+ self.db = CouchDatabase("management", create=True)
308+ design_doc = "pairedservers"
309+ if not self.db.view_exists("byserver", design_doc):
310+ map_js = """function(doc) {
311+ if (doc.record_type == 'http://www.freedesktop.org/wiki/Specifications/desktopcouch/paired_server')
312+ if (doc.application_annotations &&
313+ doc.application_annotations["Ubuntu One"] &&
314+ doc.application_annotations["Ubuntu One"].private_application_annotations &&
315+ doc.application_annotations["Ubuntu One"].private_application_annotations.deleted) {
316+ // don't emit deleted items
317+ } else {
318+ emit(doc.server, doc);
319+ }
320+ }"""
321+ self.db.add_view("byserver", map_js, None, design_doc)
322+ if not self.db.view_exists("byservicename", design_doc):
323+ map_js = """function(doc) {
324+ if (doc.record_type == 'http://www.freedesktop.org/wiki/Specifications/desktopcouch/paired_server')
325+ if (doc.application_annotations &&
326+ doc.application_annotations["Ubuntu One"] &&
327+ doc.application_annotations["Ubuntu One"].private_application_annotations &&
328+ doc.application_annotations["Ubuntu One"].private_application_annotations.deleted) {
329+ // don't emit deleted items
330+ } else {
331+ emit(doc.service_name, doc);
332+ }
333+ }"""
334+ self.db.add_view("byservicename", map_js, None, design_doc)
335+
336 self.logging = logging.getLogger(self.__class__.__name__)
337
338 self.window = gtk.Window()
339@@ -651,6 +799,7 @@
340 self.window.add(top_vbox)
341
342 self.pick_pane = self.create_pick_pane(top_vbox)
343+ self.already_paired_pane = self.create_already_paired_pane(top_vbox)
344 self.listen_pane = self.create_single_listen_pane(top_vbox)
345
346 copyright = gtk.Label(_("Copyright 2009 Canonical"))
347@@ -658,6 +807,7 @@
348 copyright.show()
349
350 self.pick_pane.show()
351+ self.already_paired_pane.show()
352 self.listen_pane.show()
353
354 top_vbox.show()
355@@ -673,6 +823,41 @@
356 # successful.
357 #couchdb_io.replicate_to(...)
358
359+ import uuid
360+
361+ # FIXME THIS IS WRONG. We need to get the oauth tokens for the remote
362+ # server somehow!
363+ oauth_data = {"consumer_key": "XXX", "consumer_secret": "XXX",
364+ "oauth_token": "XXX", "oauth_token_secret": "XXX"}
365+ try:
366+ data = {
367+ "record_type": "http://www.freedesktop.org/wiki/Specifications/desktopcouch/paired_server",
368+ "pairing_identifier": str(uuid.uuid4()),
369+ "server": "%s:%s" % (host, port),
370+ "oauth": {
371+ "consumer_key": str(oauth_data["consumer_key"]),
372+ "consumer_secret": str(oauth_data["consumer_secret"]),
373+ "token": str(oauth_data["oauth_token"]),
374+ "token_secret": str(oauth_data["oauth_token_secret"]),
375+ },
376+ "pull_from_server": True
377+ }
378+
379+ d = CouchDatabase("management", create=True)
380+ r = Record(data)
381+ record_id = d.put_record(r)
382+ except Exception, e:
383+ logging.exception("failure writing record for %s:%s", host, port)
384+ fail_note = gtk.MessageDialog(
385+ parent=pick_or_listen.window,
386+ flags=gtk.DIALOG_DESTROY_WITH_PARENT,
387+ buttons=gtk.BUTTONS_OK,
388+ type=gtk.MESSAGE_ERROR,
389+ message_format =_("Couldn't save pairing details for %s") % host)
390+ fail_note.run()
391+ fail_note.destroy()
392+ return
393+
394 success_note = gtk.Dialog(title=_("Paired with %(host)s") % locals(),
395 parent=pick_or_listen.window,
396 flags=gtk.DIALOG_DESTROY_WITH_PARENT,
397@@ -688,39 +873,40 @@
398 lambda *args: pick_or_listen.window.destroy())
399 success_note.show()
400
401-def pair_with_cloud_service(name, hostname, port):
402+def pair_with_cloud_service(service_name):
403 """Write a paired server record for the selected cloud service."""
404- from desktopcouch.records.server import CouchDatabase
405- from desktopcouch.records.record import Record
406 import uuid
407-
408- # Create a paired server record
409- service_data = CLOUD_SERVICES[name]
410- data = {
411- "record_type": "http://www.freedesktop.org/wiki/Specifications/desktopcouch/paired_server",
412- "pairing_identifier": str(uuid.uuid4()),
413- "server": "%s:%s" % (hostname, port),
414- "oauth": {
415- "consumer_key": service_data["consumer_key"],
416- "consumer_secret": service_data["consumer_secret"],
417- "token": service_data["oauth_token"],
418- "token_secret": service_data["oauth_token_secret"],
419- },
420- "pull_from_server": True
421- }
422+ import desktopcouch.replication_services as services
423+ srv = getattr(services, service_name)
424+
425 try:
426+ oauth_data = srv.oauth_data()
427+ data = {
428+ "record_type": "http://www.freedesktop.org/wiki/Specifications/desktopcouch/paired_server",
429+ "pairing_identifier": str(uuid.uuid4()),
430+ "service_name": service_name,
431+ "oauth": {
432+ "consumer_key": str(oauth_data["consumer_key"]),
433+ "consumer_secret": str(oauth_data["consumer_secret"]),
434+ "token": str(oauth_data["oauth_token"]),
435+ "token_secret": str(oauth_data["oauth_token_secret"]),
436+ },
437+ "pull_from_server": True
438+ }
439+
440 d = CouchDatabase("management", create=True)
441 r = Record(data)
442 record_id = d.put_record(r)
443+
444 except Exception, e:
445+ logging.exception("failure in module for service %s", service_name)
446 fail_note = gtk.MessageDialog(
447 parent=pick_or_listen.window,
448 flags=gtk.DIALOG_DESTROY_WITH_PARENT,
449 buttons=gtk.BUTTONS_OK,
450 type=gtk.MESSAGE_ERROR,
451- message_format =_("Couldn't save pairing details for %s") % name)
452+ message_format =_("Couldn't save pairing details for %s") % service_name)
453 fail_note.run()
454- logging.exception(e)
455 fail_note.destroy()
456 return
457
458@@ -729,7 +915,7 @@
459 flags=gtk.DIALOG_DESTROY_WITH_PARENT,
460 buttons=gtk.BUTTONS_OK,
461 type=gtk.MESSAGE_INFO,
462- message_format =_("Successfully paired with %s") % name)
463+ message_format =_("Successfully paired with %s") % service_name)
464 success_note.run()
465 success_note.destroy()
466
467
468=== modified file 'bin/desktopcouch-paired-replication-manager'
469--- bin/desktopcouch-paired-replication-manager 2009-08-26 19:04:39 +0000
470+++ bin/desktopcouch-paired-replication-manager 2009-09-04 17:40:19 +0000
471@@ -37,7 +37,27 @@
472
473 already_replicating = False
474
475+
476+def db_prefix_for_statically_addressed_replicators(addr, port):
477+ """Use the hostname and port to look up what the prefix should be on the
478+ databases. This gives an egalitarian way for non-UbuntuOne servers to have
479+ their own remote-db-name scheme."""
480+ addrport = addr + "_" + str(port)
481+ module_name = addrport.replace(".", "_")
482+ try:
483+ logging.debug("Looking up prefix %r in mod %s", addrport, module_name)
484+ mod = __import__("desktopcouch.replication_hosts", fromlist=[module_name])
485+ return getattr(mod, module_name).db_name_prefix
486+ except ImportError, e:
487+ logging.info("Not changing remote db name. %s", e)
488+ return ""
489+ except Exception, e:
490+ logging.exception("Not changing remote db name.")
491+ return ""
492+
493 class ReplicatorThread(threading.Thread):
494+ """Offload all the replication stuff to a thread so that the main thread
495+ can continue to talk to the zeroconf daemon in a non-blocked way."""
496 def __init__(self, local_port):
497 log.debug("starting up replication thread")
498 super(ReplicatorThread, self).__init__()
499@@ -48,15 +68,38 @@
500 already_replicating = True # just trying to be polite.
501 try:
502 for uuid, addr, port in dbus_io.get_seen_paired_hosts():
503- log.debug("host %s is seen; want to replicate to it", uuid)
504+ log.debug("want to replipush to discovered host %r @ %s",
505+ uuid, addr)
506 for db_name in couchdb_io.get_database_names_replicatable():
507 couchdb_io.replicate(db_name, db_name,
508 target_host=addr, target_port=port,
509 source_port=self.local_port)
510-
511-
512- # TODO: get static addressed paired hosts and replicate to
513- # those too.
514+
515+ for uuid, addr, port, to_pull, to_push in \
516+ couchdb_io.get_static_paired_hosts():
517+
518+ if to_pull:
519+ remote_db_prefix = db_prefix_for_statically_addressed_replicators(addr, port)
520+ for db_name in couchdb_io.get_database_names_replicatable():
521+ remote_db_name = str(remote_db_prefix)+db_name
522+ log.debug("want to replipush %r to static host %r @ %s",
523+ remote_db_name, uuid, addr)
524+ couchdb_io.replicate(db_name, remote_db_name,
525+ target_host=addr, target_port=port,
526+ source_port=self.local_port, target_ssl=True)
527+ if to_push:
528+ for remote_db_name in couchdb_io.get_database_names_replicatable(addr,
529+ port):
530+ if not remote_db_name.startswith(str(remote_db_prefix)):
531+ continue
532+ db_name = remote_db_name[len(str(remote_db_prefix)):]
533+ if db_name.strip("/") == "management":
534+ continue # be paranoid about what we accept.
535+ log.debug("want to replipull %r from static host %r @ %s",
536+ db_name, uuid, addr)
537+ couchdb_io.replicate(remote_db_name, db_name,
538+ source_host=addr, source_port=port,
539+ target_port=self.local_port, source_ssl=True)
540
541 finally:
542 already_replicating = False
543@@ -70,6 +113,7 @@
544
545 r = ReplicatorThread(local_port)
546 r.start()
547+ return r
548
549 def main(args):
550 log_directory = os.path.join(xdg.BaseDirectory.xdg_cache_home,
551@@ -87,6 +131,8 @@
552 logging.getLogger('').addHandler(rotating_log)
553 logging.getLogger('').setLevel(logging.DEBUG)
554
555+ print "Logging to", os.path.join(log_directory, "desktop-couch-replication.log")
556+
557 try:
558 log.info("Starting.")
559
560@@ -117,6 +163,10 @@
561 b.unpublish()
562
563 finally:
564+ try:
565+ t.stop()
566+ except:
567+ pass
568 log.info("Quitting.")
569
570 if __name__ == "__main__":
571
572=== modified file 'desktopcouch/pair/couchdb_pairing/couchdb_io.py'
573--- desktopcouch/pair/couchdb_pairing/couchdb_io.py 2009-08-26 19:04:39 +0000
574+++ desktopcouch/pair/couchdb_pairing/couchdb_io.py 2009-09-03 21:26:51 +0000
575@@ -21,6 +21,7 @@
576
577 from desktopcouch import find_port as desktopcouch_find_port
578 from desktopcouch.records import server
579+import socket
580
581 RECTYPE_BASE = "http://www.freedesktop.org/wiki/Specifications/desktopcouch/"
582 PAIRED_SERVER_RECORD_TYPE = RECTYPE_BASE + "paired_server"
583@@ -30,13 +31,40 @@
584 port = desktopcouch_find_port() # make sure d-c is running.
585 return server.CouchDatabase(name, create=create)
586
587-def get_database_names_replicatable():
588+def get_static_paired_hosts():
589+ db = _get_db("management")
590+ results = db.get_records(create_view=True)
591+ found = dict()
592+ for row in results[PAIRED_SERVER_RECORD_TYPE]:
593+ try:
594+ if row.value["server"] != "":
595+ uuid = row.value["pairing_identifier"]
596+ to_push = row.value.get("push_to_server", True)
597+ to_pull = row.value.get("pull_from_server", False)
598+ addr, port = row.value["server"].split(":") # What about IPv6?
599+ found[(addr, int(port))] = uuid, to_pull, to_push
600+ except KeyError:
601+ pass
602+ unique_hosts = [(v1, h1, h2, v2, v3) for
603+ (h1, h2), (v1, v2, v3) in found.items()]
604+ logging.debug("static pairings are %s", unique_hosts)
605+ return unique_hosts
606+
607+def get_database_names_replicatable(host=None, port=None):
608 """Find a list of local databases, minus dbs that we do not want to
609 replicate (explicitly or implicitly)."""
610
611- port = int(desktopcouch_find_port())
612- couchdb_server = server.Server("http://localhost:%(port)d/" % locals())
613- all = set([db_name for db_name in couchdb_server])
614+ if host is None:
615+ host = "localhost"
616+ if port is None:
617+ port = int(desktopcouch_find_port())
618+
619+ try:
620+ couchdb_server = server.Server("http://%(host)s:%(port)d/" % locals())
621+ all = set([db_name for db_name in couchdb_server])
622+ except socket.error, e:
623+ logging.error("Can't get list of databases from %s", couchdb_server)
624+ return set()
625
626 excluded = set()
627 excluded.add("management")
628@@ -82,51 +110,52 @@
629 logging.debug("found %d %s records", len(values), key)
630 return values
631
632-def create_remote_database(dst_host, dst_port, dst_name):
633+def create_database(dst_host, dst_port, dst_name):
634 dst_url = u"http://%(dst_host)s:%(dst_port)d/" % locals()
635 return server.CouchDatabase(dst_name, dst_url, create=True)
636
637 def replicate(source_database, target_database, target_host=None,
638- target_port=None, source_host=None, source_port=None):
639+ target_port=None, source_host=None, source_port=None,
640+ source_ssl=False, target_ssl=False):
641 """This replication is instant and blocking, and does not persist. """
642
643- data = {}
644-
645+ source_protocol = "https" if source_ssl else "http"
646+ target_protocol = "https" if target_ssl else "http"
647 if source_host:
648 if source_port is None:
649- source = "http://%(source_host)s/%(source_database)s" % locals()
650+ source = "%(source_protocol)s://%(source_host)s/%(source_database)s" % locals()
651 else:
652- source = "http://%(source_host)s:%(source_port)d/%(source_database)s" % locals()
653+ source = "%(source_protocol)s://%(source_host)s:%(source_port)d/%(source_database)s" % locals()
654 else:
655 source = source_database
656
657 if target_host:
658 if target_port is None:
659- target = "http://%(target_host)s/%(target_database)s" % locals()
660+ target = "%(target_protocol)s://%(target_host)s/%(target_database)s" % locals()
661 else:
662- target = "http://%(target_host)s:%(target_port)d/%(target_database)s" % locals()
663+ target = "%(target_protocol)s://%(target_host)s:%(target_port)d/%(target_database)s" % locals()
664 else:
665 target = target_database
666
667 record = dict(source=source, target=target)
668 try:
669- if target_host:
670- # Remote databases must exist before replicating to them.
671- create_remote_database(target_host, target_port, target_database)
672-
673+ url = None # so logging works in exception handler
674+ port = int(desktopcouch_find_port())
675 # TODO: Get admin username and password from keyring. Populate URL.
676-
677- url = None # so logging works in exception handler
678- port = int(desktopcouch_find_port())
679 url = "http://localhost:%d/" % (port,)
680
681+ if target_host:
682+ # Target databases must exist before replicating to them.
683+ logging.debug("creating %r %s:%d", target_database, target_host, target_port)
684+ create_database(target_host, target_port, target_database)
685+
686+
687 ### All until python-couchdb gets a Server.replicate() function
688 import couchdb
689+ logging.debug("asking %r to send %s to %s", url, source, target)
690 server = couchdb.client.Server(url)
691 resp, data = server.resource.post(path='/_replicate', content=record)
692 logging.debug("replicate result: %r %r", resp, data)
693 ###
694- logging.debug("#############################")
695 except:
696 logging.exception("can't talk to couchdb. %r <== %r", url, record)
697- raise
698
699=== added directory 'desktopcouch/replication_services'
700=== added file 'desktopcouch/replication_services/__init__.py'
701--- desktopcouch/replication_services/__init__.py 1970-01-01 00:00:00 +0000
702+++ desktopcouch/replication_services/__init__.py 2009-09-04 22:25:44 +0000
703@@ -0,0 +1,4 @@
704+"""Modules imported here are available as services."""
705+
706+import ubuntuone
707+import example
708
709=== added file 'desktopcouch/replication_services/example.py'
710--- desktopcouch/replication_services/example.py 1970-01-01 00:00:00 +0000
711+++ desktopcouch/replication_services/example.py 2009-09-04 22:25:44 +0000
712@@ -0,0 +1,32 @@
713+# Note that the __init__.py of this package must import this module for it to
714+# be found. Plugin logic is not pretty, and not implemented yet.
715+
716+# Required
717+name = "Example"
718+# Required; should include the words "cloud service" on the end.
719+description = "Example cloud service"
720+
721+# Required
722+def is_active():
723+ """Can we deliver information?"""
724+ return False
725+
726+# Required
727+def oauth_data():
728+ """OAuth information needed to replicate to a server."""
729+ return dict(consumer_key="", consumer_secret="", oauth_token="",
730+ oauth_token_secret="")
731+ # or to symbolize failure
732+ return None
733+
734+# Required
735+def couchdb_location():
736+ """Give a tuple of hostname and port number."""
737+
738+ return "couchdb.example.com", 5984
739+
740+# Access to this as a string fires off functions.
741+# Required
742+db_name_prefix = "foo"
743+# You can be sure that access to this will always, always be through its
744+# __str__ method.
745
746=== added file 'desktopcouch/replication_services/ubuntuone.py'
747--- desktopcouch/replication_services/ubuntuone.py 1970-01-01 00:00:00 +0000
748+++ desktopcouch/replication_services/ubuntuone.py 2009-09-04 22:25:44 +0000
749@@ -0,0 +1,128 @@
750+import hashlib
751+from oauth import oauth
752+import logging
753+import httplib2
754+import simplejson
755+import gnomekeyring
756+
757+name = "Ubuntu One"
758+description = "The Ubuntu One cloud service"
759+
760+def is_active():
761+ """Can we deliver information?"""
762+ return oauth_data() is not None
763+
764+def oauth_data():
765+ """Information needed to replicate to a server."""
766+ try:
767+ import gnomekeyring
768+ matches = gnomekeyring.find_items_sync(
769+ gnomekeyring.ITEM_GENERIC_SECRET,
770+ {'ubuntuone-realm': "https://ubuntuone.com",
771+ 'oauth-consumer-key': "ubuntuone"})
772+ if matches:
773+ # parse "a=b&c=d" to {"a":"b","c":"d"}
774+ oauth_data = dict([x.split("=", 1) for x in
775+ matches[0].secret.split("&")])
776+ oauth_data.update({
777+ "consumer_key": "ubuntuone",
778+ "consumer_secret": "",
779+ })
780+ return oauth_data
781+ except ImportError, e:
782+ logging.info("Can't replicate to Ubuntu One cloud without credentials."
783+ " %s", e)
784+ except gnomekeyring.NoMatchError:
785+ logging.info("This machine hasn't authorized itself to Ubuntu One; "
786+ "replication to the cloud isn't possible until it has. See "
787+ "'ubuntuone-client-applet'.")
788+ except gnomekeyring.NoKeyringDaemonError:
789+ logging.error("No keyring daemon found in this session, so we have "
790+ "no access to Ubuntu One data.")
791+ return None
792+
793+def couchdb_location():
794+ """This can vary more often than the OAuth information. Support SRV
795+ records and whatnot."""
796+
797+ # ...eventually. For now, hard-coded. Maybe YAGNI.
798+ return "couchdb.one.ubuntu.com", 5984
799+
800+def whoami_user_id():
801+ def get_oauth_token(consumer):
802+ """Get the token from the keyring"""
803+ import gobject
804+ gobject.set_application_name("desktopcouch replication to Ubuntu One")
805+ items = gnomekeyring.find_items_sync(
806+ gnomekeyring.ITEM_GENERIC_SECRET,
807+ {'ubuntuone-realm': "https://one.ubuntu.com",
808+ 'oauth-consumer-key': consumer.key})
809+ if len(items):
810+ return oauth.OAuthToken.from_string(items[0].secret)
811+
812+ def get_oauth_request_header(consumer, access_token, http_url):
813+ """Get an oauth request header given the token and the url"""
814+ signature_method = oauth.OAuthSignatureMethod_HMAC_SHA1()
815+ oauth_request = oauth.OAuthRequest.from_consumer_and_token(
816+ http_url=http_url,
817+ http_method="GET",
818+ oauth_consumer=consumer,
819+ token=access_token)
820+ oauth_request.sign_request(signature_method, consumer, access_token)
821+ return oauth_request.to_header()
822+
823+ url = "https://one.ubuntu.com/api/account/"
824+ consumer = oauth.OAuthConsumer("ubuntuone", "hammertime")
825+ try:
826+ access_token = get_oauth_token(consumer)
827+ except gnomekeyring.NoKeyringDaemonError:
828+ logging.info("No keyring daemon is running for this session.")
829+ return None
830+ if not access_token:
831+ logging.info("Could not get access token from keyring")
832+ return None
833+ oauth_header = get_oauth_request_header(consumer, access_token, url)
834+ client = httplib2.Http()
835+ resp, content = client.request(url, "GET", headers=oauth_header)
836+ if resp['status'] == "200":
837+ try:
838+ userinfo = simplejson.loads(content)
839+ return userinfo["id"]
840+ except:
841+ logging.error("Couldn't find id in response %r", content)
842+ return None
843+ else:
844+ logging.error("Couldn't talk to %r. Got HTTP %s", url, resp['status'])
845+ return None
846+
847+
848+class PrefixGetter():
849+ def __init__(self):
850+ self.str = None
851+
852+ def __str__(self):
853+ if self.str is not None:
854+ return self.str
855+
856+ user_id = whoami_user_id()
857+ if user_id is None:
858+ raise ValueError
859+
860+ hasher = hashlib.md5()
861+ hasher.update(str(user_id))
862+ hashed_id_as_hex = hasher.hexdigest()
863+
864+ prefix = "u/%s/%s/%d/" % (
865+ hashed_id_as_hex[0:3],
866+ hashed_id_as_hex[3:6],
867+ user_id)
868+ self.str = prefix
869+ return prefix
870+
871+
872+# Access to this as a string fires off functions.
873+db_name_prefix = PrefixGetter()
874+
875+if __name__ == "__main__":
876+ logging.basicConfig(level=logging.DEBUG, format="%(message)s")
877+ print str(db_name_prefix)

Subscribers

People subscribed via source and target branches