OpenStack Compute (nova)

Merge lp://qastaging/~justin-fathomdb/nova/virtualbox-support into lp://qastaging/~hudson-openstack/nova/trunk

virtualbox-support
Merge into trunk

Proposed by justinsb on 2010-08-18

Status:	Work in progress
Proposed branch:	lp://qastaging/~justin-fathomdb/nova/virtualbox-support
Merge into:	lp://qastaging/~hudson-openstack/nova/trunk
Prerequisite:	lp://qastaging/~justin-fathomdb/nova/raw-disk-images
Diff against target:	1705 lines (+841/-171) 28 files modified bin/nova-compute (+5/-6) dev-scripts/api.sh (+9/-0) dev-scripts/compute.sh (+19/-0) dev-scripts/create-image.sh (+17/-0) dev-scripts/down.sh (+8/-0) dev-scripts/network.sh (+7/-0) dev-scripts/objectstore.sh (+15/-0) dev-scripts/redis.sh (+3/-0) dev-scripts/restart.sh (+8/-0) dev-scripts/up.sh (+10/-0) dev-scripts/upload-image.sh (+13/-0) dev-scripts/volume.sh (+11/-0) nova/compute/disk.py (+27/-36) nova/compute/monitor.py (+1/-1) nova/compute/service.py (+19/-31) nova/network/service.py (+1/-1) nova/process.py (+31/-34) nova/rpc.py (+113/-13) nova/server.py (+1/-1) nova/service.py (+93/-36) nova/threads.py (+40/-0) nova/twistd.py (+4/-2) nova/utils.py (+16/-2) nova/virt/connection.py (+10/-2) nova/virt/images.py (+4/-5) nova/virt/libvirt_conn.py (+23/-0) nova/virt/virtualbox.py (+332/-0) nova/volume/service.py (+1/-1)
To merge this branch:	bzr merge lp://qastaging/~justin-fathomdb/nova/virtualbox-support
Related bugs:	Link a bug report
Related blueprints:	Add virtualbox bindings (Undefined)

Reviewer	Review Type	Date Requested	Status
Nova Core security contacts		2010-08-18	Pending
Review via email: mp+33067@code.qastaging.launchpad.net

Description of the change

Adds initial experimental support for VirtualBox. Currently doesn't work, because VirtualBox doesn't expose an async API. Will work once the Twisted removal is complete. Won't break anything unless users launch with the (undocumented) connection type 'virtualbox'.

lp://qastaging/~justin-fathomdb/nova/virtualbox-support updated on 2010-08-21

166. By justinsb on 2010-08-20: Merged with trunk & lp:~justin-fathomdb/nova/better-error-messages-on-process-fail
167. By justinsb on 2010-08-20: Merged with updated kiss-backend (twisted for some services)
168. By justinsb on 2010-08-20: Merged with latest kiss-backend
169. By justinsb on 2010-08-20: Merged with latest kiss-backend
170. By justinsb on 2010-08-20: Merged with latest kiss-backend
171. By justinsb on 2010-08-20: Merged with latest lp:~justin-fathomdb/nova/better-error-messages-on-process-fail
172. By justinsb on 2010-08-21: Merged latest kiss-backend
173. By justinsb on 2010-08-21: Merged latest improved error message
174. By justinsb on 2010-08-21: Merged latest kiss-backend
175. By justinsb on 2010-08-21: Adding dev-scripts to make development easier
176. By justinsb on 2010-08-21: Added network to up/down helper scripts
177. By justinsb on 2010-08-21: Merged with latest ~justin-fathomdb/nova/kiss-backend/
178. By justinsb on 2010-08-21: Built disk and launched VM
179. By justinsb on 2010-08-21: Don't partition the disk if using raw images
180. By justinsb on 2010-08-21: Merged with latest lp:~justin-fathomdb/nova/raw-disk-images (though I'm fighting a mini merge-battle with myself because the code actually depends on the twisted removal)
181. By justinsb on 2010-08-21: Actually, it looks like the twisted removal & virtualbox support may be inextricably linked...

Revision history for this message

Soren Hansen (soren) wrote on 2010-08-24:

Twisted has no particular problem talking to a synchronous API. It can simply defer the call to a thread and get a response back as a callback once the call returns. What /is/ difficult is calling Twisted from a strictly synchronous application, but I don't see that as an issue here.

With that in mind, how is it that Twisted removal and VirtualBox support are related?

Revision history for this message

justinsb (justin-fathomdb) wrote on 2010-08-24:

The relational is the 'viral' nature of Twisted's async calls... when implementing an interface method that is currently synchronous (get_info, as I recall), that method needs to be made asynchronous if the implementation is now async (in this case, all VirtualBox methods must spawn the VirtualBoxManage command.) But then all its callers need to be made async, and then their callers etc. When I asked about this on the mailing list, it seems that the only thing to do was essentially to make _every_ method async, in case it ever called in future an interface method which had an async implementation. That's a pretty big lock-in to Twisted, and it seems that the momentum was moving against Twisted.

If this patch continued to work with Twisted, I think it counted it would need a dozen additional changes propagating new deferreds up the call chains.

Whatever the decision on Twisted, I'll work with it, but I do think we need _a_ decision. If it's to keep Twisted then I'll have to rework this patch, but that's OK... My vote is to go with Threads/Eventlet, because then the whole code isn't covered in Twisted-specifics and we have more options in future (Python Threads, Jython, Eventlet, source code translators) vs just Twisted.

Unmerged revisions

181. By justinsb on 2010-08-21: Actually, it looks like the twisted removal & virtualbox support may be inextricably linked...
180. By justinsb on 2010-08-21: Merged with latest lp:~justin-fathomdb/nova/raw-disk-images (though I'm fighting a mini merge-battle with myself because the code actually depends on the twisted removal)
179. By justinsb on 2010-08-21: Don't partition the disk if using raw images
178. By justinsb on 2010-08-21: Built disk and launched VM
177. By justinsb on 2010-08-21: Merged with latest ~justin-fathomdb/nova/kiss-backend/
176. By justinsb on 2010-08-21: Added network to up/down helper scripts
175. By justinsb on 2010-08-21: Adding dev-scripts to make development easier
174. By justinsb on 2010-08-21: Merged latest kiss-backend
173. By justinsb on 2010-08-21: Merged latest improved error message
172. By justinsb on 2010-08-21: Merged latest kiss-backend