Merge lp://qastaging/~thedac/simplestreams/keystone-v3-support into lp://qastaging/simplestreams

LGTM

PASSED: Continuous integration, rev:451
https://jenkins.ubuntu.com/server/job/simplestreams-ci/37/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-amd64/37
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-arm64/37
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-ppc64el/37
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-s390x/37
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=vm-i386/37

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/simplestreams-ci/37/rebuild

Hi,

Very slow in responding, sorry.

I think this looks really good. There are 2 points I'd like addressed though:
a.) I think we need to take the v3 route only if 'OS_AUTH_VERSION' is set to 3 (or possibly OS_IDENTITY_API_VERSION). i'm not sure which is right, i see both of these in my 'serverstack' creds file. We should respect this if set, rather than your 'get_ks_api_version'

 b.) I suspect it is possible that the 'import' of 'v3' would fail . looking at my "example sync" from
http://bazaar.launchpad.net/~smoser/simplestreams/example-sync/view/head:/setup , this previously worked on trusty, so i'd like to have it continue to work there. I think we need to add some ImportError handling to address that.

David,

Can you try this out and see if it works for you?

https://code.launchpad.net/~smoser/simplestreams/keystone-v3-support/+merge/329379

FAILED: Continuous integration, rev:453
https://jenkins.ubuntu.com/server/job/simplestreams-ci/39/
Executed test runs:
    FAILURE: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-amd64/39/console
    FAILURE: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-arm64/39/console
    FAILURE: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-ppc64el/39/console
    FAILURE: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-s390x/39/console
    FAILURE: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=vm-i386/39/console

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/simplestreams-ci/39/rebuild

This should be ready now. I cannot test against serverstack until we are done with release.

The final remaining issue was swiftclient authentication. By sending it the session that seems to have fixed it in my test environments.

You can test with any credentials. Do not need any special privileges.

On September 1, 2017 7:31:00 PM EDT, David Ames <email address hidden> wrote:
>This should be ready now. I cannot test against serverstack until we
>are done with release.
>
>The final remaining issue was swiftclient authentication. By sending it
>the session that seems to have fixed it in my test environments.
>--
>https://code.launchpad.net/~thedac/simplestreams/keystone-v3-support/+merge/325781
>You are reviewing the proposed merge of
>lp:~thedac/simplestreams/keystone-v3-support into lp:simplestreams.

Four CI failures:

https://jenkins.ubuntu.com/server/job/simplestreams-ci/39/nodes=metal-ppc64el/console

simplestreams/openstack.py:138:8: E111 indentation is not a multiple of four
simplestreams/openstack.py:144:8: E111 indentation is not a multiple of four

======================================================================
FAIL: test_create_glance_properties (tests.unittests.test_glancemirror.TestGlanceMirror)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/jenkins/slaves/axew/workspace/simplestreams-ci/nodes/metal-ppc64el/ss-39/tests/unittests/test_glancemirror.py", line 256, in test_create_glance_properties
    properties)
AssertionError: {'con[19 chars]1', 'os_version': '16.04', 'version_name': 'X'[190 chars]ntu'} != {'con[19 chars]1', 'version_name': 'X', 'simplestreams_metada[144 chars]ntu'}
  {'content_id': 'content-1',
   'item_name': 'disk1.img',
- 'os_distro': 'ubuntu',
- 'os_version': '16.04',
   'product_name': 'foobuntu',
   'simplestreams_metadata': '{"extra": "value", "os": "ubuntu", "version": '
                             '"16.04"}',
   'source_content_id': 'source-1',
   'version_name': 'X'}

======================================================================
FAIL: test_insert_item_full (tests.unittests.test_glancemirror.TestGlanceMirror)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/jenkins/slaves/axew/workspace/simplestreams-ci/nodes/metal-ppc64el/ss-39/tests/unittests/test_glancemirror.py", line 595, in test_insert_item_full
    self.assertEqual(expected_create_kwargs, passed_create_kwargs)
AssertionError: {'con[62 chars]c', 'os_version': '14.04', 'version_name': '20[878 chars]3f1'} != {'con[62 chars]c', 'architecture': 'x86_64', 'version_name': [832 chars]3f1'}
Diff is 1866 characters long. Set self.maxDiff to None to see it.
-------------------- >> begin captured stdout << ---------------------
created image-1: auto-sync/ubuntu-trusty-14.04-amd64-server-20160602-disk1.img

--------------------- >> end captured stdout << ----------------------
-------------------- >> begin captured logging << --------------------
sstreams: DEBUG: getting local copy of http://image-store/fooubuntu-X-disk1.img
--------------------- >> end captured logging << ---------------------

Scott,

Note, I had to revert the following to get unit tests to pass.

=== modified file 'simplestreams/mirrors/glance.py'
--- simplestreams/mirrors/glance.py 2017-08-22 17:41:18 +0000
+++ simplestreams/mirrors/glance.py 2017-09-11 16:44:40 +0000
@@ -272,8 +272,8 @@
                 name_old, name_new = carry_over_property
             else:
                 name_old = name_new = carry_over_property
- if name_new in image_metadata:
- properties[name_new] = image_metadata[name_old]
+ properties[name_new] = image_metadata[name_old]

         if 'arch' in image_metadata:
             properties['architecture'] = canonicalize_arch(

PASSED: Continuous integration, rev:455
https://jenkins.ubuntu.com/server/job/simplestreams-ci/40/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-amd64/40
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-arm64/40
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-ppc64el/40
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-s390x/40
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=vm-i386/40

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/simplestreams-ci/40/rebuild

That change is required. Looking at how to fix the unit tests.

PASSED: Continuous integration, rev:456
https://jenkins.ubuntu.com/server/job/simplestreams-ci/41/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-amd64/41
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-arm64/41
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-ppc64el/41
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=metal-s390x/41
    SUCCESS: https://jenkins.ubuntu.com/server/job/simplestreams-ci/nodes=vm-i386/41

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/simplestreams-ci/41/rebuild

This has been tested against v3 and v2 clouds. Ready for merge.

Thank you David!