desktopcouch

Merge lp://qastaging/~thisfred/desktopcouch/field-registry-doctest into lp://qastaging/desktopcouch

  1. field-registry-doctest
  2. Merge into trunk
Proposed by Eric Casteleijn
Status: Merged
Approved by: Mark G. Saye
Approved revision: 79
Merged at revision: not available
Proposed branch: lp://qastaging/~thisfred/desktopcouch/field-registry-doctest
Merge into: lp://qastaging/desktopcouch
Diff against target: 322 lines
4 files modified
desktopcouch/records/doc/field_registry.txt (+213/-0)
desktopcouch/records/doc/records.txt (+13/-7)
desktopcouch/records/tests/test_field_registry.py (+5/-1)
desktopcouch/records/tests/test_record.py (+5/-0)
To merge this branch: bzr merge lp://qastaging/~thisfred/desktopcouch/field-registry-doctest
Reviewer Review Type Date Requested Status
Nicola Larosa (community) Approve
Mark G. Saye (community) Approve
John O'Brien (community) Approve
Review via email: mp+12812@code.qastaging.launchpad.net

Commit message

Add a doctest for the field_registry and make this one the existing doctests actually be picked up by the unit tests.

To post a comment you must log in.
Revision history for this message
Eric Casteleijn (thisfred) wrote :

This adds doctests for the field registry and transformers which was a rather critical missing piece in the documentation for developers.

Revision history for this message
John O'Brien (jdobrien) wrote :

code looks clean, tests all pass...I must learn more of this 'doc test'.

review: Approve
lp://qastaging/~thisfred/desktopcouch/field-registry-doctest updated
78. By Eric Casteleijn

textual fixes

79. By Eric Casteleijn

textual fixes

Revision history for this message
Mark G. Saye (markgsaye) wrote :

Looks good, tests pass, small changes discussed on IRC.

review: Approve
Revision history for this message
Nicola Larosa (teknico) wrote :

Well, redundant by now, but since I'm asked directly...

All very interesting, we'll need to talk about this (for the contacts web ui), but well done, thanks.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'desktopcouch/records/doc/field_registry.txt'
2--- desktopcouch/records/doc/field_registry.txt 1970-01-01 00:00:00 +0000
3+++ desktopcouch/records/doc/field_registry.txt 2009-10-05 13:41:09 +0000
4@@ -0,0 +1,213 @@
5+The Field Registry and Transformers
6+
7+Creating a field registry and/or a custom Transformer object is an
8+easy yet flexible way to map data structures between desktopcouch and
9+existing applications.
10+
11+>>> from desktopcouch.records.field_registry import (
12+... SimpleFieldMapping, MergeableListFieldMapping, Transformer)
13+>>> from desktopcouch.records.record import Record
14+
15+Say we have a very simple audiofile record type that defines 'artist'
16+and 'title' string fields. Now also say we have an application that
17+wants to interact with records of this type called 'My Awesome Music
18+Player' or MAMP. The developers of MAMP use a data structure that has
19+the same fields, but uses slightly different names for them:
20+'songtitle' and 'songartist'. We can now define a mapping between the
21+fields:
22+
23+>>> my_registry = {
24+... 'songartist': SimpleFieldMapping('artist'),
25+... 'songtitle': SimpleFieldMapping('title')
26+... }
27+
28+and instantiate a Transformer object:
29+
30+>>> my_transformer = Transformer('My Awesome Music Player', my_registry)
31+
32+If MAMP has the following song object (a plain dictionary):
33+
34+>>> my_song = {
35+... 'songartist': 'Thomas Tantrum',
36+... 'songtitle': 'Shake It Shake It'
37+... }
38+
39+We can have the transformer transform it into a desktopcouch record
40+object:
41+
42+>>> AUDIO_FILE_RECORD_TYPE = 'http://example.org/record_types/audio_file'
43+>>> new_record = Record(record_type=AUDIO_FILE_RECORD_TYPE)
44+>>> my_transformer.from_app(my_song, new_record)
45+
46+Now we can look at the underlying data:
47+
48+>>> new_record._data #doctest: +NORMALIZE_WHITESPACE
49+{'record_type': 'http://example.org/record_types/audio_file',
50+ 'title': 'Shake It Shake It',
51+ 'artist': 'Thomas Tantrum'}
52+
53+You might think that this doesn't really help all that much and that
54+the code you would have had to write to do this yourself would not
55+have been all that much bigger than using the Transformer and you'd be
56+right, but this is not all the transformers do. Let's say the song in
57+MAMP also has a field 'number_of_times_played_in_mamp':
58+
59+>>> my_song = {
60+... 'songartist': 'Thomas Tantrum',
61+... 'songtitle': 'Shake It Shake It',
62+... 'number_of_times_played_in_mamp': 23
63+... }
64+
65+Obviously that is not a field defined by our record type, since it is
66+exceedingly unlikely that any other application would be interested in
67+this data. Let's see what happens if we run the transformation with
68+this field present, but undefined in the field registry:
69+
70+>>> new_record = Record(record_type=AUDIO_FILE_RECORD_TYPE)
71+>>> my_transformer.from_app(my_song, new_record)
72+
73+>>> new_record._data #doctest: +NORMALIZE_WHITESPACE
74+{'record_type': 'http://example.org/record_types/audio_file',
75+ 'title': 'Shake It Shake It',
76+ 'application_annotations': {'My Awesome Music Player': {'application_fields': {'number_of_times_played_in_mamp': 23}}},
77+ 'artist': 'Thomas Tantrum'}
78+
79+The transformer, when it encountered a field it had no knowledge of,
80+assumed it was specific to this application, and instead of ignoring
81+it, stuffed it in the proper place in application_annotations. That's
82+already quite useful.
83+
84+Let's try something a little trickier and more contrived. Say MAMP
85+annotates each song in some other interesting ways: let's say it
86+allows three very specific tags on each song:
87+
88+>>> my_song = {
89+... 'songartist': 'Thomas Tantrum',
90+... 'songtitle': 'Shake It Shake It',
91+... 'number_of_times_played_in_mamp': 23,
92+... 'tag_vocals': 'female vocals',
93+... 'tag_title': 'shaking',
94+... 'tag_subject': 'talking'
95+... }
96+
97+Our record type is a little more enlightened, and allows any number of
98+tags, in a field 'tags', where each tag has a field 'tag' and and a
99+field 'description'. It would be nice if we could keep a mapping
100+between the tags that MAMP cares about, and the ones in our
101+record. We'll have to do just a little more work, but we can. We'll
102+make a new field_registry, and instantiate a new transformer with it:
103+
104+>>> my_registry = {
105+... 'songartist': SimpleFieldMapping('artist'),
106+... 'songtitle': SimpleFieldMapping('title'),
107+... 'tag_vocals': MergeableListFieldMapping(
108+... 'My Awesome Music Player', 'vocals_tag', 'tags', 'tag',
109+... default_values={'description': 'vocals'}),
110+... 'tag_title': MergeableListFieldMapping(
111+... 'My Awesome Music Player', 'title_tag', 'tags', 'tag',
112+... default_values={'description': 'title'}),
113+... 'tag_subject': MergeableListFieldMapping(
114+... 'My Awesome Music Player', 'subject_tag', 'tags', 'tag',
115+... default_values={'description': 'subject'}),
116+... }
117+
118+>>> my_transformer = Transformer('My Awesome Music Player', my_registry)
119+>>> new_record = Record(record_type=AUDIO_FILE_RECORD_TYPE)
120+>>> my_transformer.from_app(my_song, new_record)
121+
122+Since _data will now contain lots of uuids to keep references intact,
123+it's less readable, and a less clear example, so I'll show you what
124+using the higher level API results in:
125+
126+>>> [tag['tag'] for tag in new_record['tags']]
127+['shaking', 'talking', 'female vocals']
128+>>> [tag['description'] for tag in new_record['tags']]
129+['title', 'subject', 'vocals']
130+
131+Let's say we append a tag:
132+
133+>>> new_record['tags'].append({'tag': 'yeah yeah no'})
134+
135+and we do the same thing:
136+
137+>>> [tag['tag'] for tag in new_record['tags']]
138+['shaking', 'talking', 'female vocals', 'yeah yeah no']
139+>>> [tag.get('description') for tag in new_record['tags']]
140+['title', 'subject', 'vocals', None]
141+
142+and say we change the first tag:
143+
144+>>> new_record['tags'][0]['tag'] = 'shaking it'
145+
146+and now look at transforming in the other direction:
147+
148+>>> new_song = {}
149+>>> my_transformer.to_app(new_record, new_song)
150+>>> new_song #doctest: +NORMALIZE_WHITESPACE
151+{'tag_title': 'shaking it',
152+ 'tag_subject': 'talking',
153+ 'tag_vocals': 'female vocals',
154+ 'songtitle': 'Shake It Shake It',
155+ 'songartist': 'Thomas Tantrum',
156+ 'number_of_times_played_in_mamp': 23}
157+
158+We see that we got the data that was in the original song, except with
159+the tag_title value changed to 'shaking it', exactly as we'd expect'.
160+
161+Many more things are possible by creating new Transformers and/or
162+FieldMapping types. I'll give one last example. Let us say that our
163+record_type defines a rating field that's a value between 0 and
164+100. Let's also say that MAMP stores a string with anywhere between
165+zero and five stars.
166+
167+>>> class StarIntMapping(SimpleFieldMapping):
168+... """Map a five star rating system to a score of 0 to 100 as
169+... losslessly as possible.
170+... """
171+...
172+... def getValue(self, record):
173+... """Get the value for the registered field."""
174+... score = record.get(self._fieldname)
175+... stars = score / 20
176+... remainder = score % 20
177+... if remainder >= 5:
178+... stars += 1
179+... return "*" * stars
180+...
181+... def setValue(self, record, value):
182+... """Set the value for the registered field."""
183+... if value is None:
184+... self.deleteValue(record)
185+... return
186+... star_score = len(value) * 20
187+... score = record.get(self._fieldname)
188+... if score is None or abs(star_score - score) > 5:
189+... record[self._fieldname] = star_score
190+... # else we keep the original value, since it was close
191+... # enough and more precise
192+
193+And we make a registry and a transformer:
194+
195+>>> my_registry = {
196+... 'songartist': SimpleFieldMapping('artist'),
197+... 'songtitle': SimpleFieldMapping('title'),
198+... 'stars': StarIntMapping('score'),
199+... }
200+>>> my_transformer = Transformer('My Awesome Music Player', my_registry)
201+
202+Create a song with a rating:
203+
204+>>> my_song = {
205+... 'songartist': 'Thomas Tantrum',
206+... 'songtitle': 'Shake It Shake It',
207+... 'stars': '*****',
208+... 'number_of_times_played_in_mamp': 23
209+... }
210+
211+>>> new_record = Record(record_type=AUDIO_FILE_RECORD_TYPE)
212+>>> my_transformer.from_app(my_song, new_record)
213+>>> new_record['score']
214+100
215+
216+And, I don't know if you've ever heard the song in question, but that
217+is in fact correct! ;)
218
219=== modified file 'desktopcouch/records/doc/records.txt'
220--- desktopcouch/records/doc/records.txt 2009-07-30 16:25:24 +0000
221+++ desktopcouch/records/doc/records.txt 2009-10-05 13:41:09 +0000
222@@ -3,15 +3,16 @@
223 >>> from desktopcouch.records.server import CouchDatabase
224 >>> from desktopcouch.records.record import Record
225
226-Create a database object. Your database needs to exist. If it doesn't, you
227+Create a database object. Your database needs to exist. If it doesn't, you
228 can create it by passing create=True.
229
230 >>> db = CouchDatabase('testing', create=True)
231
232-Create a Record object. Records have a record type, which should be a URL.
233-The URL should point to a human-readable document which describes your
234-record type. (This is not checked, though.) You can pass in an initial set
235-of data.
236+Create a Record object. Records have a record type, which should be a
237+URL. The URL should point to a human-readable document which
238+describes your record type. (This is not checked, though.) You can
239+pass in an initial set of data.
240+
241 >>> r = Record({'a':'b'}, record_type='http://example.com/testrecord')
242
243 Records work like Python dicts.
244@@ -32,6 +33,7 @@
245 There is no ad-hoc query functionality.
246
247 For views, you should specify a design document for most all calls.
248+
249 >>> design_doc = "application"
250
251 To create a view:
252@@ -41,20 +43,24 @@
253 >>> db.add_view("blueberries", map_js, reduce_js, design_doc)
254
255 List views for a given design document:
256+
257 >>> db.list_views(design_doc)
258 ['blueberries']
259
260 Test that a view exists:
261+
262 >>> db.view_exists("blueberries", design_doc)
263 True
264
265-Execute a view. Results from execute_view() take list-like syntax to pick one
266-or more rows to retreive. Use index or slice notation.
267+Execute a view. Results from execute_view() take list-like syntax to
268+pick one or more rows to retrieve. Use index or slice notation.
269+
270 >>> result = db.execute_view("blueberries", design_doc)
271 >>> for row in result["idfoo"]:
272 ... pass # all rows with id "idfoo". Unlike lists, may be more than one.
273
274 Finally, remove a view. It returns a dict containing the deleted view data.
275+
276 >>> db.delete_view("blueberries", design_doc)
277 {'map': 'function(doc) { emit(doc._id, null) }'}
278
279
280=== modified file 'desktopcouch/records/tests/test_field_registry.py'
281--- desktopcouch/records/tests/test_field_registry.py 2009-08-13 16:06:20 +0000
282+++ desktopcouch/records/tests/test_field_registry.py 2009-10-05 13:41:09 +0000
283@@ -17,7 +17,7 @@
284
285 """Test cases for field mapping"""
286
287-import copy
288+import copy, doctest
289 from testtools import TestCase
290 from desktopcouch.records.field_registry import (
291 SimpleFieldMapping, MergeableListFieldMapping, Transformer)
292@@ -111,3 +111,7 @@
293 self.transformer.to_app(record, data)
294 self.assertEqual(
295 {'simpleField': 23, 'strawberryField': 'the value'}, data)
296+
297+ def test_run_doctests(self):
298+ results = doctest.testfile('../doc/field_registry.txt')
299+ self.assertEqual(0, results.failed)
300
301=== modified file 'desktopcouch/records/tests/test_record.py'
302--- desktopcouch/records/tests/test_record.py 2009-08-13 16:06:20 +0000
303+++ desktopcouch/records/tests/test_record.py 2009-10-05 13:41:09 +0000
304@@ -19,6 +19,7 @@
305 """Tests for the RecordDict object on which the Contacts API is built."""
306
307 from testtools import TestCase
308+import doctest
309
310 # pylint does not like relative imports from containing packages
311 # pylint: disable-msg=F0401
312@@ -179,6 +180,10 @@
313 self.assertEqual('http://fnord.org/smorgasbord',
314 self.record.record_type)
315
316+ def test_run_doctests(self):
317+ results = doctest.testfile('../doc/records.txt')
318+ self.assertEqual(0, results.failed)
319+
320
321 class TestRecordFactory(TestCase):
322 """Test Record/Mergeable List factories."""

Subscribers

People subscribed via source and target branches