lp:~mwhudson/curtin

Author: Michael Hudson-Doyle
Author Date: 2023-08-31 00:53:37 UTC

flakes

Author: Michael Hudson-Doyle
Author Date: 2023-08-30 22:54:20 UTC

flakes

Author: Michael Hudson-Doyle
Author Date: 2023-08-30 09:15:10 UTC

ensure grub-pc is installed on non-UEFI amd64 installs

I was sure curtin did this already but LP: #2033496 says not.

Author: Michael Hudson-Doyle
Author Date: 2023-08-10 00:29:14 UTC

storage_config: include partition uuids for gpt partitions

Author: Michael Hudson-Doyle
Author Date: 2023-08-01 01:19:12 UTC

bla

Author: Michael Hudson-Doyle
Author Date: 2023-08-01 01:44:10 UTC

remove leftover debugging

Author: Michael Hudson-Doyle
Author Date: 2023-07-31 23:57:54 UTC

sigh, need to stick to attr for now

Author: Michael Hudson-Doyle
Author Date: 2023-07-12 23:40:58 UTC

have ChrootableTarget ignore allow_daemons when target == "/"

For one thing, disabling deamons in the root filesystem is almost
certainly never wanted and for another this allows ChrootableTarget to
be used with "/" as non-root (for example, allowing you to run
get_efibootmgr("/") as non-root when running subiquity integration
tests).

Author: Michael Hudson-Doyle
Author Date: 2023-07-05 03:30:34 UTC

fix existing tests (whitespace...) and add one for new functionality

Author: Michael Hudson-Doyle
Author Date: 2023-06-16 04:36:35 UTC

be more careful to preserve semantics

Author: Michael Hudson-Doyle
Author Date: 2023-06-12 22:38:38 UTC

use a config object in uefi boot entry code

a very small start!

Author: Michael Hudson-Doyle
Author Date: 2023-06-12 21:08:58 UTC

restore argument used in vmtest

Author: Michael Hudson-Doyle
Author Date: 2023-06-12 21:00:32 UTC

tox.ini: update pylint

and fix two new reports

Author: Michael Hudson-Doyle
Author Date: 2023-06-11 11:46:51 UTC

update schema for partition flag to include msftres

Author: Michael Hudson-Doyle
Author Date: 2023-06-08 01:26:37 UTC

support "msftres" as a (GPT-only) partition flag

We want to mark the recovery / reset partition as a "Microsoft Reserved
Partition" as GUI disk utilities hide such partitions, so add support
for this as a value for a partition flag. I chose "msftres" because
that's what parted calls this (although it maps msftres to type code
0x27 for a DOS partition table, which I think is wrong so I'm not doing
that).

Author: Michael Hudson-Doyle
Author Date: 2023-06-08 23:31:16 UTC

address review comments

Author: Michael Hudson-Doyle
Author Date: 2023-06-07 23:53:23 UTC

remove BadUsage exception entirely, no longer used anywhere

Author: Michael Hudson-Doyle
Author Date: 2023-06-07 02:45:19 UTC

address various review comments & fix tests

Author: Michael Hudson-Doyle
Author Date: 2022-03-29 20:38:37 UTC

update an example

Author: Michael Hudson-Doyle
Author Date: 2022-11-28 20:47:29 UTC

add integration test

Author: Michael Hudson-Doyle
Author Date: 2022-11-28 20:41:37 UTC

fix block meta 4k tests with recentish losetup

In recent releases of util-linux, --sector-size and --partscan don't
play well together, so drop the --partscan and call partprobe after
losetup returns. This is fixed in util-linux master but not the latest
release.

Author: Michael Hudson-Doyle
Author Date: 2022-11-27 22:33:07 UTC

fix "curtin swap" exit code

Today I discovered that there is a "curtin swap" command that just does
the "configure swap" part of curthooks and then I fairly quickly after
that discovered that probably noone has used it because it
unconditionally exits with code 2 even if everything succeeded.

Author: Michael Hudson-Doyle
Author Date: 2022-11-27 22:33:07 UTC

fix curtin swap exit code

Author: Michael Hudson-Doyle
Author Date: 2022-11-25 01:47:21 UTC

call wipefs -a -f in addition to zeroing in quick_zero

LP: #1997920

Author: Michael Hudson-Doyle
Author Date: 2022-09-05 04:09:24 UTC

block_meta: add 'device' action to directly refer to a block device

Author: Michael Hudson-Doyle
Author Date: 2022-09-07 23:57:14 UTC

document path and warn if it is wrong

Author: Michael Hudson-Doyle
Author Date: 2022-09-07 23:38:11 UTC

add docs

Author: Michael Hudson-Doyle
Author Date: 2022-07-27 23:28:00 UTC

bah

Author: Michael Hudson-Doyle
Author Date: 2022-03-14 07:44:26 UTC

Merge branch 'v2-disk-identification' into v2-vmtests-2

Author: Michael Hudson-Doyle
Author Date: 2022-03-09 22:00:09 UTC

examples: even more tweaks for v2

bcache-partitions.yaml: remove the ptable setting on a drive that will
be part of a bcache. A quirk of v1 partitioning is that setting ptable:
gpt on a disk but then creating no partitions will leave the disk with
no partition table.

lvm_iscsi.yaml: enlarge some partitions, one for / and one because of
the logical partition overhead.

mirrorboot.yaml: make / slightly larger.

Author: Michael Hudson-Doyle
Author Date: 2022-03-10 02:10:05 UTC

block_meta_v2: call make_dname when required

Author: Michael Hudson-Doyle
Author Date: 2022-03-08 21:48:42 UTC

exclusive=False here

Author: Michael Hudson-Doyle
Author Date: 2022-02-24 01:34:29 UTC

examples: stop assuming curtin will allow for overhead of logical partitions

Curtin currently has the slightly surprising (to me at least) behaviour
of enlarging an extended partition to account for the overhead that
comes from the fact that logical partitions have to have gaps between
them. I do not intend to preserve this in v2 partitioning (and will
handle this in subiquity) so change the two example files that depend on
this.

Also correct an offset in examples/tests/multipath-reuse.yaml.

Author: Michael Hudson-Doyle
Author Date: 2022-03-01 22:17:21 UTC

examples: enlarge / for some more vmtests

These ones only fail on ppc64el for whatever reason.

Author: Michael Hudson-Doyle
Author Date: 2022-02-28 20:46:00 UTC

skip BionicTestPartitionExistingRAID.test_correct_ptype

The collect script fails on Bionic because lsblk -o PTTYPE does not work
there, so just skip the test there. This was the only failing vmtest
overnight!!

Author: Michael Hudson-Doyle
Author Date: 2022-02-24 01:48:46 UTC

examples: boost size of / in multipath-reuse.yaml

missed this previously because it failed for other reasons.

Author: Michael Hudson-Doyle
Author Date: 2022-02-24 01:24:42 UTC

fix v2 partitioning with 4k sectors

sfdisk *does* take input in the sector size of the device

Author: Michael Hudson-Doyle
Author Date: 2022-02-24 00:05:14 UTC

oops

Author: Michael Hudson-Doyle
Author Date: 2022-02-23 22:44:16 UTC

block_meta_v2: change how we invoke sfdisk again, restore partprobe call

It's a bit disappointingly empirical but some multipath vmtests fail
with the current sfdisk arguments with "device busy" type errors. I
think this points to a bug somewhere else but the current partition code
works in this situation so massage things so that the new code does too.

Now we pass --no-tell-kernel to sfdisk, we do need to tell the kernel
about the changes so put back a call to partprobe to do this. This is
actually needed independent of the above change, because it seems sfdisk
with default arguments does not cause the kernel to remove the device
node for a deleted partition!

Author: Michael Hudson-Doyle
Author Date: 2022-02-23 22:19:45 UTC

block_meta_v2: pass start and size to sfdisk in KiB

Turns out the default unit is the device's sector size, not 512 byte
sectors as I had hitherto assumed.

Author: Michael Hudson-Doyle
Author Date: 2022-02-21 21:57:02 UTC

vmtests: boost the size of / in a few tests

apparently 3G is no longer enough.

Author: Michael Hudson-Doyle
Author Date: 2022-02-23 07:24:49 UTC

Merge branch 'no-centos6' into v2-vmtests

Author: Michael Hudson-Doyle
Author Date: 2022-02-22 20:33:56 UTC

Remove CentOS 6 tests.

It's EOL, the required images are no longer in the MAAS image streams
and at least one vmtest fails if you manage to run it somehow.

Author: Michael Hudson-Doyle
Author Date: 2022-02-22 20:17:06 UTC

block_meta: call realpath on partition node returned by sfdisk

sfdisk --json on a device mapper block device will "helpfully" return
paths like /device/mapper/mpatha-part1 for the "node" of each partition.
This doesn't work so well when you then assume that the basename of the
returned path is the kernel name for the device.

This should fix reusing a partition on a multipath disk.

Author: Michael Hudson-Doyle
Author Date: 2022-02-22 07:51:34 UTC

Merge branch 'always-devpath' into vmtest-testing

Author: Michael Hudson-Doyle
Author Date: 2022-02-22 07:51:17 UTC

fix test?

Author: Michael Hudson-Doyle
Author Date: 2022-02-21 23:53:56 UTC

remove offset from injected prep partition

as v2 partitioning respects offsets, this now causes failures

Author: Michael Hudson-Doyle
Author Date: 2022-02-21 20:51:40 UTC

refactor things to allow a test case to override storage config version, add some v2 tests

Author: Michael Hudson-Doyle
Author Date: 2022-02-18 02:04:32 UTC

block_meta_v2: fix implicit offset calculation for dos partitions

Author: Michael Hudson-Doyle
Author Date: 2022-02-18 02:03:51 UTC

block_meta_v2: do not use aliases for partition types

These are not supported by sfdisk in focal.

Author: Michael Hudson-Doyle
Author Date: 2022-02-17 21:27:58 UTC

remove hirsute from vmtests, add jammy

A bit late, as seems to be usual for this!

Author: Michael Hudson-Doyle
Author Date: 2022-02-08 01:36:16 UTC

Update pylint version in tox.ini

The currently specified version does not work with the typing module
from Python 3.9, thus causing spurious isses on impish.

Author: Michael Hudson-Doyle
Author Date: 2022-02-08 01:26:54 UTC

lint

Author: Michael Hudson-Doyle
Author Date: 2022-02-07 22:46:16 UTC

stop running CI on Python 2

Author: Michael Hudson-Doyle
Author Date: 2021-12-01 00:34:09 UTC

fix some more tests

Author: Michael Hudson-Doyle
Author Date: 2021-12-01 00:31:42 UTC

py27 compat, yay

Author: Michael Hudson-Doyle
Author Date: 2021-11-29 20:02:25 UTC

lint

Author: Michael Hudson-Doyle
Author Date: 2021-11-29 03:45:34 UTC

block_meta: refactor partition verification slightly

I want to call these functions from the new partitioning code and this
tweaking makes this easier.

Author: Michael Hudson-Doyle
Author Date: 2021-10-27 20:26:32 UTC

mock 1.0.1 compat, yay

Author: Michael Hudson-Doyle
Author Date: 2021-10-20 21:00:41 UTC

apt_config: tweaks to allow some functionality in partial tree as non-root

Subiquity is going to start doing apt configuration outside of curtin
soon and I want to emulate this in dry-run mode by copying only the apt
config to a temporary directory and configuring that as indicated in the
UI. It's (probably?) too much to ask to be able to run the full
apt-config command but the stuff around generating the sources.list file
doesn't really need to do anything special -- it only depends on non-apt
stuff in target to determine the architecture, which can be passed as a
parameter.

Author: Michael Hudson-Doyle
Author Date: 2021-10-18 03:23:14 UTC

test and fix wiping behaviour

Author: Michael Hudson-Doyle
Author Date: 2021-10-10 23:26:07 UTC

respond to review comments

Author: Michael Hudson-Doyle
Author Date: 2021-01-12 01:51:08 UTC

beginnings, good idea, lot of work

Author: Michael Hudson-Doyle
Author Date: 2021-05-10 02:26:59 UTC

curthook: always install shim-signed if available (when UEFI booted)

Rather than just on amd64. It's been available on arm64 since at least
20.04 (and is in bionic-updates).

Author: Lukas Märdian
Author Date: 2021-08-02 10:54:03 UTC

block:lvm: search encrypted volumes for LVM

Fixes LP: #1895192

Author: Michael Hudson-Doyle
Author Date: 2021-08-11 23:25:41 UTC

mdadm: allow installation to a syncing array

LP: #1939563

Author: Michael Hudson-Doyle
Author Date: 2021-08-06 02:42:03 UTC

more wordsmithing

Author: Michael Hudson-Doyle
Author Date: 2021-08-03 23:10:22 UTC

curthooks: do not unconditionally copy e/n/interfaces to target

When I tried to install a more-minimal server image using curtin, it
failed in copy_interfaces because the image did not have an /etc/network
directory at all (full ubuntu-server images have this directory because
ethtool installs files in there). Fix this by shuffling around how the
$state_dir/interfaces file is handled a bit: basically it used to always
be created by the install commmand and then updated by the net-meta
command. Change this to have the file only be created by net-meta, then
curthooks can copy it if and only if it exists.

Author: Michael Hudson-Doyle
Author Date: 2021-08-02 03:11:06 UTC

flakes

Author: Michael Hudson-Doyle
Author Date: 2021-08-02 01:16:57 UTC

fix flakes

Author: Michael Hudson-Doyle
Author Date: 2021-07-30 03:17:51 UTC

disk_handler: fix partitioning a new RAID

The commmit "disk_handler: check wipe field when deciding whether to
reformat raids" fixed the case of putting a partition on an existing
raid but broke the case of putting a partition on a new RAID.

I adapted a vmtest to exercise this path too.

Author: Michael Hudson-Doyle
Author Date: 2021-07-30 02:00:07 UTC

update comment

Author: Michael Hudson-Doyle
Author Date: 2021-07-15 21:35:53 UTC

add bug references to comment

Author: Michael Hudson-Doyle
Author Date: 2021-07-01 03:44:41 UTC

address review comments by improving test & documentation

Author: Michael Hudson-Doyle
Author Date: 2021-06-21 02:38:11 UTC

disk_handler: allow a reformat flag to override preserve

Currently preserve=true for raid means "preserve the RAID array" and
ALSO "preserve the partition table". So this adds a check so that if you
add reformat=true to a raid action with preserve=true, the RAID array
still gets a new partition table.

LP: #1932976

Author: Michael Hudson-Doyle
Author Date: 2021-06-14 01:24:36 UTC

address review comments

 * do not rely on the fact that containers do not have their own
   metadata
 * add a unit test

Author: Michael Hudson-Doyle
Author Date: 2021-05-10 00:30:52 UTC

log failures a bit more clearly as per review

Author: Michael Hudson-Doyle
Author Date: 2021-04-13 22:10:26 UTC

block_meta: fix wiping of existing dasd partition

The partition verification code does not need the part_path variable in
this case but the partition wiping code does.

Author: Michael Hudson-Doyle
Author Date: 2021-04-12 22:23:50 UTC

block_meta: pass --yes to lvcreate alongside --wipesignatures=y

Otherwise it will not actually wipe the signatures it finds.

LP: #1923487

Author: Michael Hudson-Doyle
Author Date: 2021-04-11 23:33:52 UTC

mdadm: RAID levels 5 and 10 can both be created with 2 devices

RAID5 with 2 devices is a bit strange but it's not really curtin's place
to check this IMO.

RAID10 with 2 is not that strange though, perhaps linux "RAID10" should
have been called something else so it doesn't get confused with
"RAID1+0" so much, which is really something else.

Author: Michael Hudson-Doyle
Author Date: 2021-03-24 00:52:10 UTC

sort list

Author: Michael Hudson-Doyle
Author Date: 2021-03-23 03:36:28 UTC

do not create swapfiles on ext2

Author: Michael Hudson-Doyle
Author Date: 2021-03-23 03:19:53 UTC

add a tiny partition to a randomly selected vmtest

Author: Michael Hudson-Doyle
Author Date: 2021-03-02 10:33:52 UTC

fix things some more

Author: Michael Hudson-Doyle
Author Date: 2021-01-27 00:50:08 UTC

Release 21.2

Bump the version in curtin/__init__.py to 21.2.

LP: #1913357

Author: Michael Hudson-Doyle
Author Date: 2021-01-26 21:26:15 UTC

Revert "apt_config: stop using the deprecated apt-key command"

This reverts commit e099e32c5757b7aa0bc4fc2aeddb91d195a6df2b.

Unfortunately the gpg keys are not being added to the right place on
disk.

LP: #1912801

Author: Michael Hudson-Doyle
Author Date: 2021-01-26 02:22:47 UTC

partition_handler: fix NameError when reusing a vtoc partition

Author: Michael Hudson-Doyle
Author Date: 2021-01-26 02:06:55 UTC

apt_config: put added gpg in correct place in filesystem

Author: Michael Hudson-Doyle
Author Date: 2021-01-20 02:11:02 UTC

fix two multipath bugs:

1) find_mpath_id asked multipathd to list all maps and read out the name
   (or wwid if no name) of the device from that and sometimes timed out,
   but this information is already present in udev as DM_NAME. And
   DM_NAME is the wwid if user_friendly_names is disabled, so there's no
   need to be conditional about that either.

2) fix calculating start and offset of partition on multipathed 4k disk

   calc_dm_partition_info returns start and offset in 512-byte sectors
   but calc_partition_info only adjusted the sectors in the
   non-multipath case.

   I also did some code tidying, including moving the responsibility for
   raising an exception if dmsetup produces no output to
   calc_dm_partition_info.

LP: #1878041

Author: Michael Hudson-Doyle
Author Date: 2021-01-21 10:41:55 UTC

fix calculating start and offset of partition on multipathed 4k disk

calc_dm_partition_info returns start and offset in 512-byte sectors but
calc_partition_info only adjusted the sectors in the non-multipath case.

I also did some code tidying, including moving the responsibility for
raising an exception if dmsetup produces no output to
calc_dm_partition_info.

LP: #1878041

Author: Michael Hudson-Doyle
Author Date: 2021-01-19 23:36:48 UTC

oops

Author: Michael Hudson-Doyle
Author Date: 2021-01-15 03:08:44 UTC

Release 21.1

Bump the version in curtin/__init__.py to 21.1.

LP: #1911841

Author: Michael Hudson-Doyle
Author Date: 2021-01-14 00:31:12 UTC

storage_config: set ptable to vtoc for 'virt' dasds as well as 'ECKD'

Author: Michael Hudson-Doyle
Author Date: 2021-01-12 21:41:23 UTC

simplify mocking in TestSlaveKnames

Author: Michael Hudson-Doyle
Author Date: 2021-01-12 03:24:29 UTC

fix flakes

Author: Michael Hudson-Doyle
Author Date: 2021-01-07 22:18:26 UTC

block_meta: pass disk_path to partition_verify and to sfdisk_info

Currently there is a failure trying to reuse an existing partition of a
multipath device because get_blockdev_for_partition doesn't work on a
multipath partition. Probably get_blockdev_for_partition should be fixed
but as partition_handler has the disk path anyway, I think it makes for
saner code to just pass that around instead.

Author: Michael Hudson-Doyle
Author Date: 2020-11-19 20:54:05 UTC

allow adding a vtoc partition without a device id

this is needed for the "dasd passed to kvm via virtio" use case

Author: Michael Hudson-Doyle
Author Date: 2020-12-08 01:02:00 UTC

py27 compat

Author: Michael Hudson-Doyle
Author Date: 2020-12-08 01:00:41 UTC

py27 compat

Author: Michael Hudson-Doyle
Author Date: 2020-12-04 01:30:05 UTC

only default ECKD dasds to vtoc ptables