Merge lp://qastaging/~cemil-azizoglu/mir/xenial-touch-ci-failure into lp://qastaging/mir

FAILED: Continuous integration, rev:3197
http://jenkins.qa.ubuntu.com/job/mir-ci/5803/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-android-vivid-i386-build/5247
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-clang-vivid-amd64-build/4153
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-vivid-touch/5196
    FAILURE: http://jenkins.qa.ubuntu.com/job/mir-xenial-amd64-ci/131/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-xenial-i386-ci/131
        deb: http://jenkins.qa.ubuntu.com/job/mir-xenial-i386-ci/131/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-builder-vivid-armhf/5196
        deb: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-builder-vivid-armhf/5196/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-runner-touch/7722
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/25914

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/mir-ci/5803/rebuild

^^^ The mir-mediumtests-xenial-touch job has gone missing (also mir-mediumtests-builder-armhf). So we can't tell if this branch is working.

There is a new failure "TestClientInput.usb_direct_input_devices_work" but with some luck, Alberto's branch about to land will solve all the failures of that sort.

PASSED: Continuous integration, rev:3197
http://jenkins.qa.ubuntu.com/job/mir-ci/5806/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-android-vivid-i386-build/5254
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-clang-vivid-amd64-build/4160
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-vivid-touch/5203
    FAILURE: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-xenial-touch/129/console
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-xenial-amd64-ci/134
        deb: http://jenkins.qa.ubuntu.com/job/mir-xenial-amd64-ci/134/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-xenial-i386-ci/134
        deb: http://jenkins.qa.ubuntu.com/job/mir-xenial-i386-ci/134/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-builder-vivid-armhf/5203
        deb: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-builder-vivid-armhf/5203/artifact/work/output/*zip*/output.zip
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-runner-touch/7728
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/25921
    SUCCESS: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-builder-xenial-armhf/127
        deb: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-builder-xenial-armhf/127/artifact/work/output/*zip*/output.zip
    FAILURE: http://jenkins.qa.ubuntu.com/job/mir-mediumtests-runner-xenial-touch/3/console
    SUCCESS: http://s-jenkins.ubuntu-ci:8080/job/touch-flash-device/25925

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/mir-ci/5806/rebuild

Oh there it is. Still failing though.

Packaging problems? Can't see anything we did wrong. Maybe the archive is in transition.

It was because mir-mediumtests-runner-xenial-touch is using rc-proposed. fginther is on it.

There may be one more potential failure but we'll see once this one is cleared.

In other proposals it's bug 1499229 that's causing failure on the xenial-touch tasks lately.

Seems reasonable even if not complete.

I'm just wondering what this failure really is?; remove_if_stale. That's in the Mir code so did we do something wrong?

> Seems reasonable even if not complete.
>
> I'm just wondering what this failure really is?; remove_if_stale. That's in
> the Mir code so did we do something wrong?

It's nothing we did wrong. It fails in CI because CI uses '-O2' optimization level. When I run locally with '-O0' it doesn't trigger.

The following is from valgrind website http://valgrind.org/docs/manual/manual-core.html.

"On rare occasions, compiler optimisations (at -O2 and above, and sometimes -O1) have been observed to generate code which fools Memcheck into wrongly reporting uninitialised value errors".

> > Seems reasonable even if not complete.
> >
> > I'm just wondering what this failure really is?; remove_if_stale. That's in
> > the Mir code so did we do something wrong?
>
> It's nothing we did wrong. It fails in CI because CI uses '-O2' optimization
> level. When I run locally with '-O0' it doesn't trigger.
>
> The following is from valgrind website http://valgrind.org/docs/manual/manual-
> core.html.
>
> "On rare occasions, compiler optimisations (at -O2 and above, and sometimes
> -O1) have been observed to generate code which fools Memcheck into wrongly
> reporting uninitialised value errors".

The other suppressions (futex) are also caused by the '-O2' optimization level.

My concern is still that we might be papering over a real problem in the call stack leading up to remove_if_stale. Although it's possible -O2 is the root cause, I'm not so sure yet. Not important enough to block on, so one more reviewer and let it land...

Meh, land it anyway. We need all the CI fixes we can get, as quickly as we can get.