Merge lp://qastaging/~doanac/uci-engine/lander-code-layout into lp://qastaging/uci-engine

Proposed by Andy Doan
Status: Merged
Approved by: Andy Doan
Approved revision: 636
Merged at revision: 658
Proposed branch: lp://qastaging/~doanac/uci-engine/lander-code-layout
Merge into: lp://qastaging/uci-engine
Diff against target: 398 lines (+144/-74)
10 files modified
charms/precise/lander/charm-helpers.yaml (+1/-0)
charms/precise/lander/config.yaml (+5/-10)
charms/precise/lander/hooks/hooks.py (+78/-52)
charms/precise/lander/unit_tests/test_hooks.py (+48/-1)
ci-utils/ci_utils/amqp_utils.py (+4/-2)
juju-deployer/configs/lander_http_vhost (+1/-1)
juju-deployer/lander.yaml.tmpl (+2/-3)
lander/bin/json_status_cgi.py (+2/-2)
lander/lander/run_worker.py (+2/-2)
testing/run_tests.py (+1/-1)
To merge this branch: bzr merge lp://qastaging/~doanac/uci-engine/lander-code-layout
Reviewer Review Type Date Requested Status
PS Jenkins bot (community) continuous-integration Approve
Francis Ginther Approve
Celso Providelo (community) Approve
Review via email: mp+225046@code.qastaging.launchpad.net

Commit message

lander charm: reorganize code layout and improve install logic

This does a couple of things that were hard to do piece by piece. The
change is a bit big but:

The main goal was getting our code to be in its own directory so it
can start to be treated like read-only. We have an idea in the future
of code rollbacks, so this fits in with that model. This now deploys
code to:

 /srv/<service>/code/r<revno>_<sha1>

 and we maintain a symlink to the latest code with:

  /srv/<service>/code/current

While doing this I got rid of all the bzr-branch logic we no longer
use. This reduces complexity.

I'll also converted our tarball logic to use charmhelpers.fetch.

While changing so much, I also decided to improve our code-extraction
logic by moving it to our config-changed logic. This means that
we now have the ability to update code by just changing the payload
value of the service.

Description of the change

I originally worked on this for the restish-charm. However, thomi is landing an MP that would be broken by this:

 https://code.launchpad.net/~doanac/uci-engine/restish-code-layout/+merge/224520

While waiting, I copied the logic from that MP to give us the same ability in the lander.

This gives us the ability to update/change code for a service with simple "juju set" commands.

ev - has created an asana task that would expand upon this allowing specific units to be at specific code levels. In the interest of a somewhat readable MP, I'd like to do that as follow-up work on this.

To post a comment you must log in.
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :

PASSED: Continuous integration, rev:633
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/969/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/969/rebuild

review: Approve (continuous-integration)
Revision history for this message
Andy Doan (doanac) wrote :

gonna - make a couple of minor changes after discussion with cprov and fginther

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :

PASSED: Continuous integration, rev:634
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/972/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/972/rebuild

review: Approve (continuous-integration)
Revision history for this message
Celso Providelo (cprov) wrote :

Andy,

I am worried about the lack of abstraction to access disk-resources in lander, but that's not enough to block this change to land and experiment "upgrading" with the new code layout.

Hopefully accessing lander juju-relations and lander-logs will get clearer later.

review: Approve
Revision history for this message
Ubuntu CI Bot (uci-bot) wrote :
Download full text (124.6 KiB)

The attempt to merge lp:~doanac/uci-engine/lander-code-layout into lp:uci-engine failed. Below is the output from the failed tests.

Running cm...
Updating source dependencies...
Updating source dependencies...
Updating source dependencies...
Updating source dependencies...
Updating source dependencies...
Updating source dependencies...
Updating source dependencies...
uploading webui-content.tgz to swift
Checking juju status
Preparing local branch upload...
Uploading local branch, fingerprint 096f5a75d81880fb5db22111f1b6b8872cff7209
Private PPAs only is DISABLED (CI_PRIVATE_PPAS_ONLY: 0)
Building charm: chroot-builder
Building charm: rabbitmq-worker
Building charm: lander
Building charm: system-image-server
Building charm: python-django
Building charm: restish
Building charm: webui
Running juju-deployer -v -c /tmp/tmpGx9DVq/deployer/branch-source-builder.yaml -c /tmp/tmpGx9DVq/deployer/gatekeeper.yaml -c /tmp/tmpGx9DVq/deployer/image-builder.yaml -c /tmp/tmpGx9DVq/deployer/lander.yaml -c /tmp/tmpGx9DVq/deployer/nf-stats-service.yaml -c /tmp/tmpGx9DVq/deployer/ppa-assigner.yaml -c /tmp/tmpGx9DVq/deployer/production-only.yaml -c /tmp/tmpGx9DVq/deployer/publisher.yaml -c /tmp/tmpGx9DVq/deployer/relations.yaml -c /tmp/tmpGx9DVq/deployer/test-runner.yaml -c /tmp/tmpGx9DVq/deployer/ticket-system.yaml -c /tmp/tmpGx9DVq/deployer/webui.yaml ci-airline
Tests running...
ci-utils.ci_utils.tests.test_amqp.TestAMQP.testConnectFailed ... OK (0.001 secs)
ci-utils.ci_utils.tests.test_amqp.TestAMQP.testProcessQueue ... OK (0.002 secs)
ci-utils.ci_utils.tests.test_amqp.TestAMQP.testRunForever ... OK (0.102 secs)
ci-utils.ci_utils.tests.test_amqp.TestAMQP.testSent ... OK (0.002 secs)
ci-utils.ci_utils.tests.test_amqp.TestProgressTrigger.testProgress ... OK (0.001 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testCancel ... OK (0.108 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testNoQueue ... OK (0.002 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testNoTicket ... OK (0.003 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testOnMessageCalledProcessError ... OK (0.003 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testOnMessageFail ... OK (0.002 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testOnMessageKilled ... OK (0.003 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testOnMessageSimple ... OK (0.002 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestAMQPWorker.testOnMessageUnexpected ... OK (0.003 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestTimer.testCBRuns ... OK (0.021 secs)
ci-utils.ci_utils.tests.test_amqp_worker.TestTimer.testCanCancel ... OK (0.000 secs)
ci-utils.ci_utils.tests.test_data_store.TestDataStoreConfig.test_invalid_auth_config ... OK (0.000 secs)
ci-utils.ci_utils.tests.test_data_store.TestDataStoreConfig.test_valid_auth_config ... OK (0.000 secs)
ci-utils.ci_utils.tests.test_data_store.TestDataStoreFileName.test_get_file_name ... OK (0.000 secs)
ci-utils.ci_utils.tests.test_data_store.TestDataStoreFileName.test_get_file_name_same_directory ... OK (0.000 secs)
ci-utils.ci_utils.tests.test_data_store.TestDataStoreFileName.test_get...

Revision history for this message
Evan (ev) wrote :

======================================================================
FAIL: testing.test_style.TestPep8.test_pep8_conformance
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/dev/shm/tmpH5DVmw/local/lib/python2.7/site-packages/ucitests-0.1.3-py2.7.egg/ucitests/styles.py", line 84, in test_pep8_conformance
    '\n'.join(self.report._msgs))
  File "/usr/lib/python2.7/unittest/case.py", line 515, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python2.7/unittest/case.py", line 744, in assertListEqual
    self.assertSequenceEqual(list1, list2, msg, seq_type=list)
  File "/usr/lib/python2.7/unittest/case.py", line 726, in assertSequenceEqual
    self.fail(msg)
  File "/usr/lib/python2.7/unittest/case.py", line 412, in fail
    raise self.failureException(msg)
AssertionError: bin/../testing/run_tests.py:418:1: W293 blank line contains whitespace

Revision history for this message
Francis Ginther (fginther) wrote :

I had the same comment as Celso, these ../../../../ paths are a hard to keep straight. But I agree this isn't the place to address that.

Also, a good chunk of the code in lander/hooks/hooks.py looks like it would be used in other charms as well. Does this eventually belong in our own charm helpers? I really have no good suggestions for addressing this given how charms work other then copy/paste.

review: Approve
Revision history for this message
Andy Doan (doanac) wrote :

On 07/09/2014 11:48 AM, Francis Ginther wrote:
> Also, a good chunk of the code in lander/hooks/hooks.py looks like it would be used in other charms as well. Does this eventually belong in our own charm helpers? I really have no good suggestions for addressing this given how charms work other then copy/paste.

yeah- this sucks. we could consolidate a lot of charm code. if we had
our own charm helper.

635. By Andy Doan

merge with trunk

636. By Andy Doan

fix pep8 issue to get tarmac working again

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :

PASSED: Continuous integration, rev:636
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/1033/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/1033/rebuild

review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.

Subscribers

People subscribed via source and target branches