Merge lp://qastaging/~harlowja/cloud-init/url-ssl-fixings into lp://qastaging/~cloud-init-dev/cloud-init/trunk

Proposed by Joshua Harlow
Status: Merged
Merged at revision: 801
Proposed branch: lp://qastaging/~harlowja/cloud-init/url-ssl-fixings
Merge into: lp://qastaging/~cloud-init-dev/cloud-init/trunk
Diff against target: 890 lines (+316/-163)
15 files modified
Requires (+6/-5)
cloudinit/config/cc_phone_home.py (+3/-2)
cloudinit/config/cc_power_state_change.py (+1/-1)
cloudinit/distros/parsers/resolv_conf.py (+2/-2)
cloudinit/ec2_utils.py (+5/-0)
cloudinit/sources/DataSourceCloudStack.py (+6/-5)
cloudinit/sources/DataSourceEc2.py (+7/-6)
cloudinit/sources/DataSourceMAAS.py (+26/-19)
cloudinit/url_helper.py (+159/-98)
cloudinit/user_data.py (+3/-2)
cloudinit/util.py (+79/-10)
setup.py (+3/-2)
tests/unittests/test__init__.py (+5/-5)
tests/unittests/test_datasource/test_maas.py (+9/-6)
tests/unittests/test_util.py (+2/-0)
To merge this branch: bzr merge lp://qastaging/~harlowja/cloud-init/url-ssl-fixings
Reviewer Review Type Date Requested Status
Scott Moser Needs Fixing
Review via email: mp+149481@code.qastaging.launchpad.net
To post a comment you must log in.
695. By harlowja <email address hidden>

More work on requests integration.

696. By harlowja <email address hidden>

Update to code on trunk.

697. By harlowja <email address hidden>

Why did this file showup.

698. By harlowja <email address hidden>

Get tests working and further adjustments.

699. By harlowja <email address hidden>

Fix the maas callback mechanism now that requests is used.

Revision history for this message
Joshua Harlow (harlowja) wrote :

So I looked into the requests code, they are using in a underlying layer the HTTP[d]Connection objects and the regular ssl object, this should allow it to use the system certificates as a fallback. So thats good news!

Revision history for this message
Scott Moser (smoser) wrote :

I'm sorry this is such a pain.
$ python -c 'from cloudinit import util; util.read_file_or_url("http://brickies.net")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "cloudinit/util.py", line 689, in read_file_or_url
    ssl_details=ssl_details)
  File "cloudinit/url_helper.py", line 163, in readurl
    r = requests.request(**req_args)
  File "/usr/lib/python2.7/dist-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
TypeError: request() got an unexpected keyword argument 'config'

$ dpkg-query --show python-requests
python-requests 1.1.0-1

See https://launchpad.net/ubuntu/+source/requests for all the ubuntu versions:
 precise: 0.8.2-1
 quantal: 0.12.1-1ubuntu6
 raring: 1.1.0-1
 latest-release: 1.1.0

Your committed code does work on precise and quantal.

review: Needs Fixing
Revision history for this message
Scott Moser (smoser) wrote :

Also:
$ python -c 'from cloudinit import util; import sys; print util.read_file_or_url(sys.argv[1])' https://stackoverflow.com/
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "cloudinit/util.py", line 701, in read_file_or_url
    ssl_details=ssl_details)
  File "cloudinit/url_helper.py", line 173, in readurl
    if isinstance(e, (exceptions.HTTPError)) and e.response:
AttributeError: 'HTTPError' object has no attribute 'response'

$ dpkg-query --show python-requests
python-requests 0.8.2-1

700. By Joshua Harlow

Update to handle requests >= 1.0 which doesn't use the config dict.

701. By Joshua Harlow

Fix how the http error doesn't always have the response attached
in earlier versions of requests (pre 0.10.8).

Revision history for this message
Joshua Harlow (harlowja) wrote :

Ok dokie, think those are all fixed by more checks on the requests version and the http error exception object.

702. By Scott Moser

merge from trunk at revno 799

703. By Scott Moser

merge from trunk rev 800

704. By Scott Moser

appease pylint and pep8

* cloudinit/distros/parsers/resolv_conf.py
  added some pylint overrides with 'plXXXXX' syntax.
  example: # pl51222 pylint: disable=E0102

  The pl51222 there means: http://www.logilab.org/ticket/51222

  This specific issue is present in 12.04 pylint, but not 13.04.

* pylint doesn't like the requests special handling we have.
  which makes sense as it is only checking versus one specific version.
* general pep8 and pylint cleanups.

705. By Scott Moser

do not bother retrying on ssl errors

if the error is an ssl error, its extremely unlikely that it would be fixed by
waiting a few seconds and trying again.

706. By Scott Moser

set 'allow_redirects' to True by default

the previous implementation of url_helper.readurl() would default
to allow_redirects being true.

So, for backwards compat, we should keep that behavior.

707. By Scott Moser

make get_instance_userdata and get_instance_metadata more like botos

this shouldn't change anything, only the signatures of the methods.

708. By Scott Moser

fix typo

709. By Joshua Harlow

Move back to using boto for now.

710. By Joshua Harlow

Add doc about issue 1401 in boto.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.