Ubuntu CI Engine

Merge lp://qastaging/~doanac/uci-engine/setup-helpers-boost into lp://qastaging/uci-engine

  1. setup-helpers-boost
  2. Merge into trunk
Proposed by Andy Doan
Status: Merged
Approved by: Andy Doan
Approved revision: 452
Merged at revision: 455
Proposed branch: lp://qastaging/~doanac/uci-engine/setup-helpers-boost
Merge into: lp://qastaging/uci-engine
Diff against target: 24 lines (+4/-1)
1 file modified
ci-utils/ci_utils/setup_helper.py (+4/-1)
To merge this branch: bzr merge lp://qastaging/~doanac/uci-engine/setup-helpers-boost
Reviewer Review Type Date Requested Status
Vincent Ladeuil (community) Approve
PS Jenkins bot (community) continuous-integration Approve
Review via email: mp+217493@code.qastaging.launchpad.net

Commit message

setup_helpers: big performance boost for setup.py develop

When using our CI_DEPS_BRANCH, our setup_helpers.py logic was doing a
bzr-pull operation for every dependency we install. This is a big repo
and the pull is slow. This changes just has us to the pull one time
per invocation of setup.py.

In my quick unscientific testing. I'm seeing lander go from >90 seconds
to <25 seconds. I saw about a 50% improvement installing ci-utils, but
it compiles stuff for our bzr dep, so thats probably our worst-case
metric.

Description of the change

make our "setup.py develop" about 2x faster

To post a comment you must log in.
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :

PASSED: Continuous integration, rev:452
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/552/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/552/rebuild

review: Approve (continuous-integration)
Revision history for this message
Vincent Ladeuil (vila) wrote :

I'm not quite sure what you're seeing here but I always setup CI_DEPS_BRANCH to an existing path in my dev envs and don't see a slow down here (my .bzr.log says the pulls complete in ~3secs transfering 3kB).

> make our "setup.py develop" about 2x faster

Which one ?

Anyway, for people not setting CI_DEPS_BRANCH I can imagine that it makes things faster.

review: Approve
Revision history for this message
Andy Doan (doanac) wrote :

On 04/29/2014 03:42 AM, Vincent Ladeuil wrote:
> Review: Approve
>
> I'm not quite sure what you're seeing here but I always setup CI_DEPS_BRANCH to an existing path in my dev envs and don't see a slow down here (my .bzr.log says the pulls complete in ~3secs transfering 3kB).
>
>> make our "setup.py develop" about 2x faster
>
> Which one ?

ah - i see what's happening. I set the following:

  export CI_DEPS_BRANCH=lp:~canonical-ci-engineering/uci-engine/deps
  export CI_DEPS_LOCAL=/code/uci-engine/deps/

So i have a local copy, but it is still able to stay in-sync with trunk
because we do a bzr-pull. If you have CI_DEPS_BRANCH pointed locally it
probably won't change much, but you also are required to stay in sync
manually.

NOTE: the way tarmac.sh runs is more like how I do things, so this
should also make running our test suite a little more efficent.

> Anyway, for people not setting CI_DEPS_BRANCH I can imagine that it makes things faster.

For people not using CI_DEPS_BRANCH this is a no-op.

Revision history for this message
Ubuntu CI Bot (uci-bot) wrote :

The attempt to merge lp:~doanac/uci-engine/setup-helpers-boost into lp:uci-engine failed. Below is the output from the failed tests.

/bin/sh: 1: export: ./tarmac.sh: bad variable name

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.

Subscribers

People subscribed via source and target branches