Merge lp://qastaging/~james-w/launchpad-work-items-tracker/multi-project into lp://qastaging/launchpad-work-items-tracker

I reverted the multi-projects commit for now, as it wreaked havoc. It causes the ubuntu database to only contain the linaro milestones, and thus breaking all the milestone charts and sending tons of errors to everyone.

What is the rationale here in the first place? linaro has separate projects in Launchpad, thus we should also have separate WI reports/burndowns, such as

  http://people.canonical.com/~platform/workitems/linaro-pm-wg/all.html
  http://people.canonical.com/~platform/workitems/linaro-toolchain-wg/all.html

(you also use a different burndown chart style). Trying to merge different projects with different milestones into one set of charts will just create confusion. Either you need to ignore the milestones from either side (which is obvioulsy not acceptable), or you need to merge the milestones from all projects. But this will create a combinatorial explosion of charts, and also break the automatic reset of per-milestone burndown charts, so it's also not feasible.

So let's discuss that a bit further first.

> I reverted the multi-projects commit for now, as it wreaked havoc. It causes
> the ubuntu database to only contain the linaro milestones, and thus breaking
> all the milestone charts and sending tons of errors to everyone.

Sorry about that.

> What is the rationale here in the first place? linaro has separate projects in
> Launchpad, thus we should also have separate WI reports/burndowns, such as
>
> http://people.canonical.com/~platform/workitems/linaro-pm-wg/all.html
> http://people.canonical.com/~platform/workitems/linaro-toolchain-wg/all.html

We have changed where we put blueprints somewhat this cycle. For instance, the linaro
foundations team has blueprints against at least "ubuntu", and "linaro", and they
don't want to have to try and reconcile two burndown charts in their head.

> (you also use a different burndown chart style). Trying to merge different
> projects with different milestones into one set of charts will just create
> confusion. Either you need to ignore the milestones from either side (which is
> obvioulsy not acceptable), or you need to merge the milestones from all
> projects.

As the milestones are just name->date mappings I don't think that storing all of
them is a problem. I can arrange to have ubuntu's milestones take precedence, so
we can't do something like have a natty-alpha-2 with a different date.

> But this will create a combinatorial explosion of charts,

Because you generate a chart for every milestone? Would it be enough not to generate
empty charts?

> and also
> break the automatic reset of per-milestone burndown charts, so it's also not
> feasible.

I don't follow this part, can you point me to the auto-reset code so that I can
see what will break please?

> So let's discuss that a bit further first.

Sure, I'm happy to do that, but this is an urgent matter for Linaro, so I'm willing
and keen to make the changes to get this landed ASAP. If it's not acceptable in
Ubuntu's instance, then I imagine we will end up with our own instance with changes
similar to these.

Thanks,

James

Found the auto-reset:

   if milestone:
        # get milestone date range
        cur = db.cursor()
        cur.execute('SELECT due_date FROM milestones WHERE name = ?', (milestone,))
        date_end = cur.fetchone()[0]
        cur.execute('SELECT MAX(due_date), MAX(due_date) < date(CURRENT_TIMESTAMP) FROM milestones WHERE due_date < ?', (date_end,))
        (date_start, in_milestone) = cur.fetchone()

        # if we are already within the milestone, start at the previous
        # milestone's end date; if we are before, start at the beginning of the
        # entire cycle, to allow planning
        if in_milestone:
            ms_sql = "AND w.milestone='%s' AND date >= '%s' AND date <= '%s'" % (
                    escape_sql(milestone), date_start, date_end)
        else:
            ms_sql = "AND w.milestone='%s' AND date <= '%s'" % (
                    escape_sql(milestone), date_end)

I'll have to think about that a bit.

Thanks,

James

Right, that's the auto-reset code. BTW, this is not just an issue for resetting the charts, the more important usage is that you need to determine what the "next" milestone is. Ubuntu alpha-2 needs to go until the day Ubuntu alpha-3 starts, not until an intermediate milestone of a different project source.

> > But this will create a combinatorial explosion of charts,
>Because you generate a chart for every milestone? Would it be enough not to generate empty charts?

The svgs are already skipped if they are empty, and empty HTML pages don't cost that much to generate. Of course this won't be a combinatorial explosion, that was a thinko, sorry. It's just additive, so no big deal.

So the main points to consider here are the merging of the milestone table (which is just a small implementation problem), and making milestones "source" specific, so that the individual charts can still know how long a milestone is, and when to reset it.

"Martin Pitt" <email address hidden> wrote:

>Right, that's the auto-reset code. BTW, this is not just an issue for
>resetting the charts, the more important usage is that you need to
>determine what the "next" milestone is. Ubuntu alpha-2 needs to go
>until the day Ubuntu alpha-3 starts, not until an intermediate
>milestone of a different project source.

Yep, got that now. I'd never realised what the milestone charts were doing, so thanks for the pointer.

>The svgs are already skipped if they are empty, and empty HTML pages
>don't cost that much to generate. Of course this won't be a
>combinatorial explosion, that was a thinko, sorry. It's just additive,
>so no big deal.

Ok, I'm going to see if I can come up with a cunning plan to reduce the number of these without configuration.

>So the main points to consider here are the merging of the milestone
>table (which is just a small implementation problem),

I've pushed up one approach to this, but I may revisit it depending on the solution to the next problem.

> and making
>milestones "source" specific, so that the individual charts can still
>know how long a milestone is,

I've just talked to Jamie and we want to go a little further than that to give unified burndowns for linaro and ubuntu alphas that are on the same day, but I've no idea if that is feasible. Either way I will aim for no change of behavior for ubuntu teams.

> Right, that's the auto-reset code. BTW, this is not just an issue for
> resetting the charts, the more important usage is that you need to determine
> what the "next" milestone is. Ubuntu alpha-2 needs to go until the day Ubuntu
> alpha-3 starts, not until an intermediate milestone of a different project
> source.

I hadn't understood what the milestone charts were doing before. Thanks for the pointer.

> The svgs are already skipped if they are empty, and empty HTML pages don't
> cost that much to generate. Of course this won't be a combinatorial explosion,
> that was a thinko, sorry. It's just additive, so no big deal.

Ok. I'm going to see if I can come up with a cunning way to avoid some of
this without configuration.

> So the main points to consider here are the merging of the milestone table
> (which is just a small implementation problem),

I've pushed up a small change that does this, but may alter the approach depending
on the solution to the next issue.

> and making milestones "source"
> specific, so that the individual charts can still know how long a milestone
> is, and when to reset it.

I've just chatted to Jamie, and this doesn't go quite as far as we would like
in Linaro, where we would like to have consolidated burndown charts for milestones
where the Ubuntu and Linaro milestones are on the same date. I will aim to have
no change for Ubuntu teams when doing this though.

Thanks,

James

Updating status to get a better overview on the "active code reviews" page.