Mir

Merge lp://qastaging/~afrantzis/mir/mir-client-ensure-global-symbol-resolution into lp://qastaging/mir

Proposed by Alexandros Frantzis
Status: Work in progress
Proposed branch: lp://qastaging/~afrantzis/mir/mir-client-ensure-global-symbol-resolution
Merge into: lp://qastaging/mir
Diff against target: 219 lines (+151/-1)
7 files modified
debian/rules (+2/-1)
include/client/mir_toolkit/mir_client_ensure_global_symbol_resolution.h (+40/-0)
src/client/CMakeLists.txt (+2/-0)
src/client/mir_client_ensure_global_symbol_resolution.cpp (+28/-0)
tests/CMakeLists.txt (+5/-0)
tests/special-linkage-tests/CMakeLists.txt (+24/-0)
tests/special-linkage-tests/test_client_symbol_resolution.cpp (+50/-0)
To merge this branch: bzr merge lp://qastaging/~afrantzis/mir/mir-client-ensure-global-symbol-resolution
Reviewer Review Type Date Requested Status
Daniel van Vugt Needs Fixing
PS Jenkins bot (community) continuous-integration Needs Fixing
Alan Griffiths Approve
Review via email: mp+196110@code.qastaging.launchpad.net

Commit message

client: Provide a way to ensure libmirclient's symbols are available for symbol resolution in other shared objects

This is useful if libmirclient, or another library linking to libmirclient, is loaded
at runtime with dlopen() using RTLD_LOCAL (e.g., through a plugin mechanism)
and libmirclient's symbols need to be resolved at runtime in other shared objects
(like Mesa's libEGL).

Description of the change

client: Provide a way to ensure libmirclient's symbols are available for symbol resolution in other shared objects

This is useful if libmirclient, or another library linking to libmirclient, is loaded
at runtime with dlopen() using RTLD_LOCAL (e.g., through a plugin mechanism)
and libmirclient's symbols need to be resolved at runtime in other shared objects
(like Mesa's libEGL).

The specific problem that we are trying to solve is that with Qt client applications on the desktop, EGL fails to recognize native MirConnections as valid EGLNativeDisplayType objects, falling back to X11 and crashing. The reason is that the symbols needed for the MirConnection validation and which are exported by libmirclient, are not available to libEGL for dynamic runtime resolution. The problem occurs because Qt loads the platform shared object at runtime with local symbol visibility (RTLD_LOCAL), which is reasonable in general, but not suitable for our use case.

(See reference diagram: http://people.ubuntu.com/~afrantzis/mir-egl-symbol-resolution.png for the following)
Concerning our options for using this:

Ideally we would want to deal with the issue in libEGL, but unfortunately we cannot, since it would defeat the purpose of the runtime resolution, which is to transparently support applications linking against either libmirserver or libmirclient or both (or neither) and doing the right thing.

Next choice would be the Qt platform plugin (libqubuntumirclient), but the plugin doesn't have any knowledge of libmirclient, it's all abstracted through platform-api. We could make it work by linking with libmirclient at either runtime or loadtime and call the mir_client_ensure_global_symbol_resolution() from here.

Moving down a step we reach the platform-api from where we do have direct access to libmirclient. It's easy to call mir_client_ensure_global_symbol_resolution() from here. The trade-off is that we are then going to force this behavior on all users of the platform-api's mirclient backend (which might be OK).

We could even make libmirclient itself force all of its symbols to become available for global resolution, but this seems too intrusive for now (but it is an option if we find that this problem is widespread).

To post a comment you must log in.
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Alexandros Frantzis (afrantzis) wrote :

The CI failures are caused by CI scripts not disabling the new test category (special-tests). It only makes sense to change CI if the MP is approved, so please review to break the vicious circle.

Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

Wouldn't it be as easy (and have mess in fewer places) for platform-api to call dlopen() directly?

review: Needs Information
Revision history for this message
Alexandros Frantzis (afrantzis) wrote :

> Wouldn't it be as easy (and have mess in fewer places) for platform-api to call dlopen() directly?

That's another option, but:

1. platform-api (or wherever this ends up in) will need to contain the logic to load the correct libmirserver.so.X library, which is not as straightforward as doing it in mirclient (since we know our soname)
2. it exposes the mechanism by which this is performed which platform-api doesn't really care about
3. it will not be as easy to reuse the code in other similar cases or move the point of application

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

> > Wouldn't it be as easy (and have mess in fewer places) for platform-api to
> call dlopen() directly?
>
> That's another option, but:
>
> 1. platform-api (or wherever this ends up in) will need to contain the logic
> to load the correct libmirserver.so.X library, which is not as straightforward
> as doing it in mirclient (since we know our soname)
> 2. it exposes the mechanism by which this is performed which platform-api
> doesn't really care about
> 3. it will not be as easy to reuse the code in other similar cases or move the
> point of application

OK then

review: Approve
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

I'm concerned, because using RTLD_LAZY|RTLD_GLOBAL is usually a bad sign that you've done something wrong. You should never need either. But you seem to have good reasons I don't yet fully understand.

review: Abstain
Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

Actually, having slept at the weekend I'm concerned that "MIR_ENABLE_SPECIAL_TESTS" is a horribly uninformative name.

review: Needs Fixing
Revision history for this message
Alexandros Frantzis (afrantzis) wrote :

> Actually, having slept at the weekend I'm concerned that
> "MIR_ENABLE_SPECIAL_TESTS" is a horribly uninformative name.

The name was vague on purpose so that this category could include all kinds of tests that have special needs, mostly in terms of special build flags or linking requirements. For example I was thinking that the c89 test we have could be also placed here.

Alternatively, we could just put this in acceptance tests (it is an acceptance test after all), but we need to have a different executable due to the special linking, which is why I have opted to separate it.

Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

> > Actually, having slept at the weekend I'm concerned that
> > "MIR_ENABLE_SPECIAL_TESTS" is a horribly uninformative name.
>
> The name was vague on purpose so that this category could include all kinds of
> tests that have special needs, mostly in terms of special build flags or
> linking requirements. For example I was thinking that the c89 test we have
> could be also placed here.
>
> Alternatively, we could just put this in acceptance tests (it is an acceptance
> test after all), but we need to have a different executable due to the special
> linking, which is why I have opted to separate it.

Hmm, or our so-called "unit-tests" that need an LD_PRELOAD to work.

I see the motivation, but I'm not sure these form a coherent "SPECIAL" category. Amy trying to come up with a name I like...

Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

> I see the motivation, but I'm not sure these form a coherent "SPECIAL"
> category. Amy trying to come up with a name I like...

On IRC we came up with MIR_ENABLE_SPECIAL_LINKAGE_TESTS

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

Ugly, but appears to address the problem. (And haven't thought of a more elegant solution within code we "own".)

review: Approve
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

1. Needs Fixing: Text conflict in debian/rules

2. Needs information:
"The specific problem that we are trying to solve is that with Qt client applications on the desktop, EGL fails to recognize native MirConnections as valid EGLNativeDisplayType objects, falling back to X11 and crashing. The reason is that the symbols needed for the MirConnection validation and which are exported by libmirclient, are not available to libEGL for dynamic runtime resolution."
How is failing to find a symbol not a fatal error? If we need validation functions and presently can't find them then what function is being called?

Experience with many global/local symbol resolution problems in plugin systems tells me there's possibly a nicer solution but I still don't fully understand the problem being solved.

review: Needs Fixing
Revision history for this message
Chris Halse Rogers (raof) wrote :

Hm. Looks like this could be more awesomely solved in the Mir EGL platform with judicious use of RTLD_NOLOAD;
first try resolving client & server symbols, then if neither are resolved checking whether either mirclient or mirserver are loaded with RTLD_NOLOAD, and reopening them with RTLD_GLOBAL if so.

It's only available on glibc, but whatever.

Longer term it might be nicer to dynamically load the EGL platforms and use actual link-time linking.

Revision history for this message
Chris Halse Rogers (raof) wrote :

I also don't object to the existing approach, except that we'll want to drop it when we solve this correctly.

Revision history for this message
Kevin DuBois (kdub) wrote :

Is this still active?

Unmerged revisions

1250. By Alexandros Frantzis

tests: Rename special-tests to special-linkage-tests

1249. By Alexandros Frantzis

special-tests: Add explicit dependency to mirclient library

1248. By Alexandros Frantzis

build: Disable special tests when building the package for armhf

1247. By Alexandros Frantzis

client: Provide function to ensure libmirclient's symbols are available for symbol resolution in other shared objects

This is useful if libmirclient, or another library linking to libmirclient, is loaded
at runtime with dlopen() using RTLD_LOCAL (e.g., through a plugin mechanism)
and libmirclient's symbols need to be resolved at runtime in other shared objects
(like Mesa's libEGL).

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.

Subscribers

People subscribed via source and target branches