Merge lp://qastaging/~hopem/charms/trusty/percona-cluster/min-cluster-size into lp://qastaging/~openstack-charmers-archive/charms/trusty/percona-cluster/next

charm_lint_check #6364 percona-cluster-next for hopem mp265108
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/6364/

charm_unit_test #5996 percona-cluster-next for hopem mp265108
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/5996/

charm_amulet_test #5175 percona-cluster-next for hopem mp265108
    AMULET FAIL: amulet-test missing

AMULET Results (max last 2 lines):
INFO:root:Search string not found in makefile target commands.
ERROR:root:No make target was executed.

Full amulet test output: http://paste.ubuntu.com/11893665/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5175/

charm_unit_test #6137 percona-cluster-next for hopem mp265108
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/6137/

charm_lint_check #6505 percona-cluster-next for hopem mp265108
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/6505/

charm_amulet_test #5227 percona-cluster-next for hopem mp265108
    AMULET FAIL: amulet-test missing

AMULET Results (max last 2 lines):
INFO:root:Search string not found in makefile target commands.
ERROR:root:No make target was executed.

Full amulet test output: http://paste.ubuntu.com/11908703/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5227/

charm_unit_test #6141 percona-cluster-next for hopem mp265108
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/6141/

charm_lint_check #6509 percona-cluster-next for hopem mp265108
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/6509/

charm_amulet_test #5231 percona-cluster-next for hopem mp265108
    AMULET FAIL: amulet-test missing

AMULET Results (max last 2 lines):
INFO:root:Search string not found in makefile target commands.
ERROR:root:No make target was executed.

Full amulet test output: http://paste.ubuntu.com/11909255/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5231/

charm_unit_test #6142 percona-cluster-next for hopem mp265108
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/6142/

charm_lint_check #6510 percona-cluster-next for hopem mp265108
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/6510/

charm_amulet_test #5232 percona-cluster-next for hopem mp265108
    AMULET FAIL: amulet-test missing

AMULET Results (max last 2 lines):
INFO:root:Search string not found in makefile target commands.
ERROR:root:No make target was executed.

Full amulet test output: http://paste.ubuntu.com/11909396/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5232/

Ed,

This looks good and appears to solve bug 1475585.

I am new here but it seems we want to keep the *_hooks.py as clean as possible and have these helper functions in the *_utils.py. For example is_bootstrapped and get_wsrep_value and possibly all of the new functions could be moved over to the utils
functions into perconal_utils.py

This should really get its own amulet test. Setup with min-cluster-size, add min-cluster-size -1, verify percona has not started
, add another unit and verify it all comes up.

Lastly, is "boostrap-pxc mysql" idempotent? If any config change changes the config file this gets run. On a stable cluster would "boostrap-pxc mysql" being run break anything?

Thanks for the review David.

I totally agree that the helper functions should go into percona_utils.py and I will move them across.

I'll see what I can do amulet test-wise.

With regards to bootstrap-pxc, this should be safe since it will only be called once at bootstrap time (ideally once all nodes are configured but that is not a hard requirement). If it were called prior to more units being added to the cluster, on subsequent runs of config_changed() we should only ever be calling 'restart' although bootstrap-pxc is idempotent and, in fact, can be done before you have all units in the cluster. The charm will also only restart percona if the config file changes.

Oh and yes, bootstrap-pxc is idempotent.

> Oh and yes, bootstrap-pxc is idempotent.

Excellent.

Just to be clear bootstrap-pxc *will* be run more than once. After the cluster has been bootstrapped any config-changed run that changes the config file will "re-bootstrap" on the leader. This may not be your intent.

In config-changed bootstrapped is defaulted to False and therefore the leader will run render_config_restart_on_changed with bootstrap=True:

            elif clustered and is_leader():
                log("Leader unit - bootstrap required=%s" % (not bootstrapped),
                    DEBUG)
                render_config_restart_on_changed(clustered, hosts,
                                                 bootstrap=not bootstrapped)

And in render_config_restart_on_changed if the config file has been changed it will "re-bootstrap" rather than restart:

    if file_hash(MY_CNF) != pre_hash:
        if bootstrap:
            service('bootstrap-pxc', 'mysql')
            notify_bootstrapped()
            update_shared_db_rels()
        else:
            service_restart('mysql')