Merge lp://qastaging/~david4dev/dmedia/udisks into lp://qastaging/dmedia

Really nice work here, David!

You'll need to get the all unit tests working again, though. Most of the failures seem to be in the schema tests, the 2 files being:

  dmedia/schema.py
  dmedia/tests/test_schema.py

You can run just the schema tests like this:

  ./setup.py test --names=schema

And make sure to run all the tests at the end to, to make sure there aren't other failures. Run all the tests like this:

  ./setup.py test

But again, nice work! You jumped right in!

Ok, I now get no failures when I run `./setup.py test` :)

David,

I wonder if you don't have a file or some changes that aren't committed? I'm still getting failures. Run `bzr status` and see what it says, if you would please.

That or maybe the tests make some assumptions that aren't the same on my system? Here's a paste of the failing tests, as of revno 220 in your udisks branch:

http://paste.ubuntu.com/660001/

What Ubuntu version you running? I'm on 64 bit 11.04 Natty, although it's probably about time for me to switch to Oneiric.

Thanks for all your work on this, exciting stuff!

`bzr status` gave me nothing. I'm on 32 bit 11.04.

From the looks of the output you posted, udisks is failing to locate devices. I'm not sure why this is happening and I can't reproduce this for myself.

I just pushed revision 221 which should make it easier to work out why this is happening. Can you please run the tests from this branch and post any errors you get? Also, post the output of `python misc/debug-udisks.py / /tmp ~/.dmedia/ ~/.local/share/desktop-couch/`.

This has stumped me a bit too, plus I've been working a lot of filestore, have had my head elsewhere.

Here's the failures when running `./setup.py test`

http://paste.ubuntu.com/663071/

And here's output of your debug script:

http://paste.ubuntu.com/663078/

I had to copy into the root of the tree so it could import the new udisks model. But that's a heck of a debug script, thanks!

Ah, one thought... I'm running an mdadm RAID10 array... so a test that looks for a specific device probably wont work.

That is one downside to udisk (and other dbus exposed stuff)... can be pretty tricky to write tests against.

Anyway, thanks for all the awesome work on this, David! Also, thanks for sticking up for Novacut so often during the recent "discussions" on the Internet. You know better than anyone that the "there is no code" argument is crap... because you've written a lot of it. ;)

I'm not sure exactly what's going on here but as you mentioned it could be due to RAID (which I know very little about). The debug script is showing that a device can't be found for the '/tmp' folder, although it is found ok for '/'.

Some information about how I am locating the device from the path:

I first find the `os.minor` and `os.major` for the file path. Then I iterate through udisks devices and match these against the 'DeviceMinor' and 'DeviceMajor' properties. If no matches are found you get the DeviceNotFoundError.

Perhaps the major and minor for RAID partitions behave differently to normal partitions?

As I said on IRC, I think the problem is using tmpfs for your /tmp folder. It seems udisks doesn't list the 'device' for the tmpfs which is understandable. I'm not sure of the best way to work around this issue though.

So I've been thinking on this a bit: how about for now we add calls to self.skipTest() for these udisk related tests that are causing trouble, and go ahead and merge this to trunk?

There is some critical schema stuff in here, and I'd like to have it in trunk so we can play with it day-to-day, knock off any rough edges over the next month.

As far as the tests, long term we probably need an adapter that sits between dmedia code and dbus. Normally we'll have, say, a LiveAdapter that makes real dbus calls, but in the tests we'll have a DummyAdapter that returns canned responses setup by the test.

I think one of the most critical bits is probably how we create the drive ID. I'm liking your _drive_id() function, but a few things we should investigate before declaring the v1 schema final:

 * Would it be safe to make the drive ID from the DriveSerial alone?

 * Regardless what values go into the drive ID, we need to make sure these are all stable values, ie, is their some interpretation by udisk or the kernel that may change between versions, or are these exact values from the physical device?

 * Will we have access to these values on other platforms, and will the values be uniform, produce the same hash? I'd say most important is making sure removable drives are giving a uniform ID, so if you took a removable drive from an Ubuntu system running dmeida to an OSX system running dmedia, everything works as expected.

What do you think? I'll go ahead and merge this to trunk, and I can add the skipTest() for the the tests that don't work on my system (easier for me to do as my system is the problem one).

Again, great work, David! I'm quite proud of you for digging into the schema so aggressively!

Thanks!

I like the idea of the 'Adapter's because that then the tests should be deterministic.

The reason why _drive_id() is as it is is because the documentation ( http://hal.freedesktop.org/docs/udisks/Device.html#Device:DriveSerial ) says that DriveSerial and DriveWWN can be blank. I'm not sure how common this is in reality but this means I don't consider it safe to use only the DriveSerial.

As far as I know, these values are provided by the physical device but I'm not sure how these are handled and whether they would be the same on all platforms.