Merge lp://qastaging/~raxnetworking/nova/bare_bones_melange into lp://qastaging/~hudson-openstack/nova/trunk

I understand the rationale behind wanting to hang on to the name, but if it's not a top level project, it seems to me that the name melange should go away as it is integrated back into nova.

I think that will lessen the confusion quite a bit. Anyone else have thoughts on this?

I agree with Devin. I think if it's getting integrated into nova, then it should get integrated.

As the original creator of the name, I have absolutely no need to hold onto it. I'm all for simplifying.

> I understand the rationale behind wanting to hang on to the name, but if it's
> not a top level project, it seems to me that the name melange should go away
> as it is integrated back into nova.

We can just call this folder IPAM to convey the functionality it provides.
The only problem is the quantum manager part. It calls current ip management features in nova as nova_ipam and the melange one as melange_ipam. It would be great if we had better names to distinguishing these two ipam implementations once we get rid of the word 'melange'.

I agree with Devin, perhaps it should be a facet of ./nova/network, just like quantum is.

This does, however, beg the following questions:
1. should it continue to use it's own database, or should the tables merge with the nova db? Wouldn't this make migrations easier and remove a lot of the duplication of sqlalchemy infrastructure?

2. this adds a new HTTP service, which implies this is a new API just for the networking aspect of nova ... shouldn't networking just be a new facet of OS API? /network/<foo>/... In which case, shouldn't novaclient be the tool for accessing it?

That said, this is a vast improvement over the initial merge-prop.

In the sense of getting it into trunk and iterating from there, it would be nice to see this addressed shortly afterwards: melange.utils seems to be largely applicable to nova in general, perhaps it should just move into that?

Oh, and I don't see factory-boy actually being used anywhere?

Sandy,
Good points, you're sortof hinting at melange not being a standalone service. I think Troy would be best to answer this as he knows more about what melange is to become, where and at what scope it will be running, what its use cases are, and how it will be interacted with. I'm in general happy with it either way, stand alone or integrated.

Being that its code is to be merged with nova, maybe the service should be as well.

oops forgot to mention, as far as the migrations and sqlalchemy infrastructure go, we're untying everything for quantum anyway so it wouldn't be that great of a difference one way or the other.

> I agree with Devin, perhaps it should be a facet of ./nova/network, just like
> quantum is.
Quantum is a standalone service. There is a network manager that integrates with Quantum (and Melange).
>
> This does, however, beg the following questions:
> 1. should it continue to use it's own database, or should the tables merge
> with the nova db? Wouldn't this make migrations easier and remove a lot of the
> duplication of sqlalchemy infrastructure?
>
> 2. this adds a new HTTP service, which implies this is a new API just for the
> networking aspect of nova ... shouldn't networking just be a new facet of OS
> API? /network/<foo>/... In which case, shouldn't novaclient be the tool for
> accessing it?
>
The original thought is that OpenStack would benefit from an IPAM service that was shared across many components. There could be significant benefit to having a single IPAM service that works across zones or hands out IPs across compute, load balancers, firewall-as-a-service or any other IP-based services. We wanted to start with tight Nova integration because that should be the primary user in the near term. But, I worry that deeper integration would lead to yet-another-IPAM project down the line or a messy separation effort. That's why there is the level of separation in the present implementation

Hmm, tricky ... 'cause it's not really Nova then is it?

Arguably, we could just stand up the [melanage] portions of nova that handle the IPAM stuff, but that seems very non-nova.

> In the sense of getting it into trunk and iterating from there, it would be
> nice to see this addressed shortly afterwards: melange.utils seems to be
> largely applicable to nova in general, perhaps it should just move into that?
I guess most of the methods here are generic enough that we could do that.
We'll make a note of that for the next merge prop we intend to make.

> Oh, and I don't see factory-boy actually being used anywhere?
It is used in melange/tests/factories/models.py.

Sandy: it was this aspect that puzzled me when we first proposed merging.

At the Diablo summit, there was no clear path forward for projects outside of the core pieces of OpenStack. Jesse and Vish both expressed a desire to Melange "incubated" inside of Nova so that we could ensure that it integrated well and with the hope that we would have more control over when it could be leveraged to solve problems with the limited IPAM functionality within Nova.

In retrospect, that was probably not the right choice. The incubated projects path developed rather quickly and we were unable to move faster on getting Melange into Nova because of the delays with multi-nic and several bits of network refactoring that drug out. So, that's where we are.

After discussing this with Jay Pipes, we decided the fastest path forward was not to carve all of this out and start over but to continue with the original while breaking the merge prop into smaller, more digestible pieces and address as many comments as possible from the early reviews in the initial merge prop.

Ideally, this probably spins off at some point down the road so that we can leverage it more cleanly as a standalone IPAM that services multiple OpenStack components. However, right now, there are significant limitations in the current IPAM that limit scaling. I think we can Melange this as a vehicle to solve those problems. So, I am hoping we can get some consensus to move forward. If there is concern that it doesn't make sense, then I would ask that we raise that Vish quickly so that we can resolve them.

Yeah, sorry for making this difficult. Certainly wasn't my intention.

The question becomes: how do we deal with packages that use large portions of the Nova infrastructure that, by themselves, don't really belong to the nova stack per se?

My gut says they should depend on nova-common and not nova.

So, given nova-common not being ready for prime time (yet), I'd say we bring it into nova, with the guarantee that nova-common will get more attention and soonish/eventually the package will be able to stand on it's own two feet.

Does that sound fair to both projects?

> So, given nova-common not being ready for prime time (yet), I'd say we bring
> it into nova, with the guarantee that nova-common will get more attention and
> soonish/eventually the package will be able to stand on it's own two feet.

Very succinctly put. This is actually what we are doing with Melange as of now :-)
Right now melange only depends on the 'infrastructure' components in nova. And we do aim to move these dependencies to the openstack commons once its available.

I have moved the merge prop to gerrit at https://review.openstack.org/#change,646
We can continue the discussions there.