lp:~dbungert/curtin

Author: Dan Bungert
Author Date: 2024-07-24 20:45:12 UTC

distro: fix handling of pre-installed OEM kernel

In the case of a pre-installed OEM kernel, we need to handle removals
differently. Since Subiquity is managing OEM kernel installs, we need
that outside input to to know what the "before" state of the kernels is,
and the existing handling of ensure_one_kernel works well with that if
we just pass along that "before" state.

Have Subiquity feed the initial state (calculated before attempting to
install OEM kernels) will handle the OEM case - we start and finish with
one kernel so leave it alone - along with the two kernel case - we have
some other kernel pre-installed then Subiquity installs the OEM kernel,
and the remove list is that pre-installed kernel.

Author: Dan Bungert
Author Date: 2024-07-19 21:11:37 UTC

curthooks: support explicit kernel removal

Author: Dan Bungert
Author Date: 2024-07-22 19:38:26 UTC

maybe

Author: Dan Bungert
Author Date: 2024-07-17 20:26:15 UTC

curthooks: attempt to skip kernel installs

For the OEM case, it helps if we don't redundantly try to install
kernels already installed, as we may not have that repo in our list
right this second.

Author: Dan Bungert
Author Date: 2024-07-12 19:10:36 UTC

integration: quick_zero test

LP: #2061073

Author: Dan Bungert
Author Date: 2024-07-11 19:31:10 UTC

vmtests: fix basic_scsi.yaml running out of space

Author: Dan Bungert
Author Date: 2024-07-11 19:31:10 UTC

vmtests: fix basic_scsi.yaml running out of space

Author: Dan Bungert
Author Date: 2024-07-09 21:42:44 UTC

tox: unpin flake8 version, sort

Author: Dan Bungert
Author Date: 2024-07-01 21:14:59 UTC

README: note about github

Author: Dan Bungert
Author Date: 2024-06-06 23:57:28 UTC

distro: ensure_one_kernel debug logs

Author: Dan Bungert
Author Date: 2024-05-29 21:03:06 UTC

curthooks: support kernel pkg removal

Author: Dan Bungert
Author Date: 2024-05-22 20:44:41 UTC

curthooks: create and refactor to KernelConfig

Author: Dan Bungert
Author Date: 2024-04-15 18:34:20 UTC

default GRUB_CMDLINE_LINUX_DEFAULT to include quiet/splash

Author: Dan Bungert
Author Date: 2024-04-13 17:25:54 UTC

util: log release of flock

Author: Dan Bungert
Author Date: 2024-03-20 03:40:26 UTC

block/zfs: flock for ZFS

Add flock calls around encrypted ZFS creation areas

LP: #2057661

Author: Dan Bungert
Author Date: 2024-03-20 19:05:31 UTC

lint: fix whitespace

Author: Dan Bungert
Author Date: 2024-03-08 18:55:06 UTC

Revert "Bind ischroot->true in ChrootableTarget and --mount-proc in unshare"

This reverts commit c5bb06b9645dfa7c69b7f54f860c68c9d8f560d7.

Author: Dan Bungert
Author Date: 2024-02-29 23:47:25 UTC

do we need this for zpool export?

Author: Dan Bungert
Author Date: 2024-02-28 23:15:32 UTC

util: human2bytes actually return int

Author: Dan Bungert
Author Date: 2023-05-16 17:59:37 UTC

integration: add run_bm(..., debug=True) helper

The debug argument for run_bm allows for running curtin outside of the
integration test itself, but with all the setup being done for you by
the integration test. This means that custom tweaks can be done to the
curtin code before running, setting breakpoints, etc.

Author: Dan Bungert
Author Date: 2024-02-23 00:24:46 UTC

dm_crypt: tests related to keyfile removal

Author: Dan Bungert
Author Date: 2024-02-21 19:14:55 UTC

zpool: create keystore encrypted setups

Add support for the 'luks_keystore' style encrypted ZFS install. Native
ZFS encryption is supported by a small dm_crypt dataset containing the
real key. This is a functional transcription of the way that Ubiquity
has implemented encrypted guided ZFS.

(cherry picked from commit 5bf15d6373def082104d64a352c53ed079b537dd)

Author: Dan Bungert
Author Date: 2024-02-20 22:14:30 UTC

block_meta: improve get_volume_uuid

blkid can return 3 different types of UUIDs, and they aren't
interchangable. Clarify which one we're using.

Also, leverage the existing blkid() wrapper to do so.

Author: Dan Bungert
Author Date: 2024-02-20 22:58:09 UTC

better

Author: Dan Bungert
Author Date: 2023-05-16 17:59:37 UTC

integration: add run_bm(..., debug=True) helper

The debug argument for run_bm allows for running curtin outside of the
integration test itself, but with all the setup being done for you by
the integration test. This means that custom tweaks can be done to the
curtin code before running, setting breakpoints, etc.

Author: Dan Bungert
Author Date: 2023-06-08 18:09:53 UTC

maybe

Author: Dan Bungert
Author Date: 2024-02-08 16:05:53 UTC

block: fix kname kernel ref

Author: Dan Bungert
Author Date: 2024-02-06 23:52:44 UTC

block_meta: fix fstab path to cryptoswap

Curtin has this system by which it looks for the most robust path to the
device. Sadly, the existing mechanism fails for cryptoswap.

At install time, this device shows up with an ID that looks like
CRYPT-LUKS2 and insted ends up with CRYPT-PLAIN but the underlying
device remains luks (has the header).

This change appears to not affect other cases like LVM on LUKS, based on
regression testing of the existing Subiquity LVM+LUKS guided install
type.

I believe the name appropriate as a selection mechanism because we
already know it needs to be unique.

Author: Dan Bungert
Author Date: 2024-02-06 23:52:44 UTC

block_meta: fix fstab path to cryptoswap

Curtin has this system by which it looks for the most robust path to the
device. Sadly, the existing mechanism fails for cryptoswap.

At install time, this device shows up with an ID that looks like
CRYPT-LUKS2 and insted ends up with CRYPT-PLAIN but the underlying
device remains luks (has the header).

This change appears to not affect other cases like LVM on LUKS, based on
regression testing of the existing Subiquity LVM+LUKS guided install
type.

I believe the name appropriate as a selection mechanism because we
already know it needs to be unique.

Author: Dan Bungert
Author Date: 2023-11-11 13:32:40 UTC

block_meta: dm_crypt extensions for cryptswap

Author: Dan Bungert
Author Date: 2023-11-12 00:09:53 UTC

notes

Author: Dan Bungert
Author Date: 2023-10-05 23:51:04 UTC

install: do not log merged config

Author: Dan Bungert
Author Date: 2023-09-26 21:01:34 UTC

Revert "vmtests: add mantic after jammy"

This reverts commit abebab1e1e30a16d4be891a051060e81254ff0c5.

Author: Dan Bungert
Author Date: 2023-09-20 20:57:46 UTC

thoughts on devsync

Author: Dan Bungert
Author Date: 2023-09-19 19:01:20 UTC

vmtests: make loading error verbose

Author: Dan Bungert
Author Date: 2023-09-13 23:08:44 UTC

extract: log source information

If you try hard enough and follow mount commands backwards you can
surely get this same information, but when analyzing what the extract is
actually doing it's nice to just be able to see what the image stack is.

Author: Dan Bungert
Author Date: 2023-09-13 01:12:32 UTC

storage_config: handle partitions on 4k disk

In LP: #2035034, a user reported that the layout of existing
partitions was showing as 1/8th the correct size. Update storage_config
to no longer hard-code a 512 byte sector and read the value instead.

Author: Dan Bungert
Author Date: 2023-08-30 22:37:35 UTC

swap: add logging

The suggested swap size code just sort of returns 0 sometimes and
figuring out why is a complicated analysis of several variables.

Provide diagnostic info on those values, and the reason for the
decision.

Author: Dan Bungert
Author Date: 2023-08-30 22:37:35 UTC

swap: add logging

The suggested swap size code just sort of returns 0 sometimes and
figuring out why is a complicated analysis of several variables.

Provide diagnostic info on those values, and the reason for the
decision.

Author: Dan Bungert
Author Date: 2023-08-25 18:53:16 UTC

docs: link to sources.list manpage for apt deb822

Author: Dan Bungert
Author Date: 2023-08-24 17:41:02 UTC

zpool: allow selective default property override

zpool fs or pool properties may now use underlying fs defaults with
key: null.

Author: Dan Bungert
Author Date: 2023-08-23 20:45:50 UTC

zfs: only use default props if none are supplied

Author: Dan Bungert
Author Date: 2023-08-03 01:39:12 UTC

several: drop python3-mock

Author: Dan Bungert
Author Date: 2023-08-03 00:10:39 UTC

vmtest-system-setup: update for latest mantic

-pep8, not available
+parameterized, in use by tests

Author: Dan Bungert
Author Date: 2023-07-25 16:33:24 UTC

swap: support avail parameter

Add an avail parameter to allow swap size projections based on expected
available space. Add unit tests for existing and new (avail) usage.
Add docstring to `suggested_swapsize()`.

Author: Dan Bungert
Author Date: 2023-07-25 16:33:24 UTC

swap: support avail parameter

Add an avail parameter to allow swap size projections based on expected
available space. Add unit tests for existing and new (avail) usage.
Add docstring to `suggested_swapsize()`.

Author: Dan Bungert
Author Date: 2023-06-05 21:42:38 UTC

env hacks

Author: Dan Bungert
Author Date: 2023-05-25 20:23:30 UTC

several: fix lints

Author: Alexsander de Souza
Author Date: 2023-05-25 19:48:55 UTC

add python3-attr as dependency

(cherry picked from commit 1019040c497bdbf96613d7eb0dc682f92275d5b0)

Author: Alexsander de Souza
Author Date: 2023-05-25 19:48:55 UTC

add python3-attr as dependency

(cherry picked from commit 1019040c497bdbf96613d7eb0dc682f92275d5b0)

Author: Dan Bungert
Author Date: 2023-05-23 21:01:34 UTC

block: move ENXIO workaround up a level

Author: Dan Bungert
Author Date: 2023-05-19 18:53:00 UTC

block: only use no-tell-kernel if sfdisk supports

Author: Dan Bungert
Author Date: 2023-05-17 00:23:39 UTC

block: retry loop post dd image

On Bionic, I observed that the volume group present in the test image
would often not show up. This retry loop helps.

Author: Dan Bungert
Author Date: 2023-05-17 18:51:50 UTC

integration: fix tests with empty ptable

Older sfdisk seems to not like a completely empty ptable while setting
these attributes. Adding an unused partition is enough to allow
focusing on the task at hand.

Author: Dan Bungert
Author Date: 2023-05-17 18:51:50 UTC

integration: fix tests with empty ptable

Older sfdisk seems to not like a completely empty ptable while setting
these attributes. Adding an unused partition is enough to allow
focusing on the task at hand.

Author: Dan Bungert
Author Date: 2023-05-15 23:52:36 UTC

requirements: add missing content

Author: Dan Bungert
Author Date: 2023-05-19 18:53:00 UTC

block: only use no-tell-kernel if sfdisk supports

Author: Dan Bungert
Author Date: 2023-05-17 17:53:37 UTC

unittest: fix tests that scan log results

Author: Dan Bungert
Author Date: 2023-05-19 21:17:33 UTC

several: support older attrs version

Author: Dan Bungert
Author Date: 2023-04-26 11:23:31 UTC

more deps

Author: Dan Bungert
Author Date: 2023-04-18 14:31:48 UTC

block: ntfsresize with force options

To match Ubiquity behavior, we should run ntfsresize with the force
options and sending 'y\n' on input.

(cherry picked from commit 6caa799551b19dc85cb1e901e013b12be326aff1)

Author: Dan Bungert
Author Date: 2023-04-18 14:31:48 UTC

block: ntfsresize with force options

To match Ubiquity behavior, we should run ntfsresize with the force
options and sending 'y\n' on input.

Author: Dan Bungert
Author Date: 2023-02-07 17:56:37 UTC

.

Author: Dan Bungert
Author Date: 2023-02-07 01:40:09 UTC

integration: compare against sysfs results

Ignoring size on extended partitions, sysfs seems to match what sfdisk
is providing.

Author: Dan Bungert
Author Date: 2023-02-06 20:06:04 UTC

swap: clarify debug message

This log message is shown, among other scenarios, if a swap partition is
used, which is very confusing.
Clarify that we're talking about swap file.

Author: Dan Bungert
Author Date: 2023-01-28 00:36:24 UTC

dm_crypt preserve open

Author: Dan Bungert
Author Date: 2023-01-24 18:24:26 UTC

vmtests: drop python3-attr dep

Author: Dan Bungert
Author Date: 2023-01-24 17:07:35 UTC

block: remove fstrings

Author: Dan Bungert
Author Date: 2023-01-18 18:10:25 UTC

test

Author: Dan Bungert
Author Date: 2022-11-04 20:05:55 UTC

apt: clarify lineage of key file name

Author: Dan Bungert
Author Date: 2022-10-27 23:51:40 UTC

tests/apt: use tmpdir as target

This refactor helps make way for discontinuing usage of apt-key.
When trying to do it all at once, there were problems with the file path
that needed to be used for the key file.

The major change is that these tests are run with a tmpdir as the
target, instead of '/'. This is also independently a nice improvement,
since misbehaved tests or code would really try to write to the host
system.

The tests were written assuming apt key files would have full paths, but
the examples (and presumably most real-world usage) were based on
filenames, so move the tests to use file names.

Add back a full path test, since that is supported behavior today.

Also removes the unused _myjoin function.

Author: Dan Bungert
Author Date: 2022-10-14 19:02:28 UTC

kernel: expand flash_kernel env to cover all flows

While Subiquity should always be providing a metapackage name, expand
the environment adjustment for flash_kernel to handle the fallback
cases.

Author: Dan Bungert
Author Date: 2022-10-03 16:22:30 UTC

block/v2: fix wipe check for extended

As the comment in _wipe_for_action notes, we never want to wipe the
extended, so do that check first.

Author: Dan Bungert
Author Date: 2022-09-30 18:04:53 UTC

doc: clarify size units

Author: Dan Bungert
Author Date: 2022-09-28 22:43:13 UTC

many: fix several instances of whitespace lint

`tox -e tip-pycodestyle` reported several instances of:
  E275 missing whitespace after keyword

Author: Dan Bungert
Author Date: 2022-09-23 15:43:34 UTC

vmtests: +python3-attr

Author: Dan Bungert
Author Date: 2022-09-06 17:46:57 UTC

block: install cryptsetup if dm_crypt

Co-authored-by: J-P Nurmi <jp.nurmi@canonical.com>

Author: Dan Bungert
Author Date: 2022-08-30 16:55:17 UTC

curthooks: narrow the chzdev_export error skip

Author: Dan Bungert
Author Date: 2022-08-22 22:50:27 UTC

nicer terminology for non-API items

Author: Dan Bungert
Author Date: 2022-08-22 22:50:27 UTC

nicer terminology for non-API items

Author: Dan Bungert
Author Date: 2022-07-29 21:09:48 UTC

block: genericize kname generation

Author: Dan Bungert
Author Date: 2022-07-27 16:43:51 UTC

block_meta: fix for pmem kname lookup

Author: Dan Bungert
Author Date: 2022-07-12 17:34:15 UTC

block/v2: preserve last-lba,table-length

Author: Dan Bungert
Author Date: 2022-06-27 22:26:32 UTC

changelog: 22.1-0ubuntu1

Author: Dan Bungert
Author Date: 2022-06-22 22:40:06 UTC

version to 22.1

Author: Dan Bungert
Author Date: 2022-05-20 19:44:52 UTC

deb: fix dependencies

Required for bionic build, also more correct anyhow.

Author: Dan Bungert
Author Date: 2022-05-09 17:23:07 UTC

tox: drop xenial-py3 from default env list

Running xenial-era python packages against modern python is not likely
to be successful! If this is an interesting test then we should support
that all the way - with a matching python version and other
dependencies, or not at all.

Author: Olivier Gayot
Author Date: 2022-04-27 16:55:18 UTC

Make sure curthooks do not discard supplied proxy settings

Just like we witnessed for the APT preferences, the proxy settings are
not honored when Subiquity installs packages from the "packages"
autoinstall section. This also applies for unattended-upgrades.

This happened because the installation of packages and execution of
unattended-upgrades occur after running curthooks.
Curtooks call handle_apt with an almost empty configuration.

Therefore, we would discard the proxy settings by removing the
etc/apt/apt.conf.d/90curtin-aptproxy file.

Fixed by not removing etc/apt/preferences.d/90curtin.pref when the
configuration does not contain APT preferences.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Dan Bungert
Author Date: 2022-05-03 13:57:32 UTC

test commit - CI should fail

Author: Dan Bungert
Author Date: 2022-04-29 23:00:48 UTC

block/v2: preserve disk label id

Windows will report a boot failure if the disk label ID changes and
nothing else.

Author: Dan Bungert
Author Date: 2022-04-27 20:48:58 UTC

block/v2: allow setting raw partition_type value

To preserve partition types that do not map to known flag values, we
must be able to capture that original type information and carry it
around in the storage config until used.

If we do not do this, preserved partitions get their type normalized to
another value (usually the equivalent of flags 'linux' or 'boot').

Author: Dan Bungert
Author Date: 2022-04-25 16:52:04 UTC

block/v2: resize-friendly ordering of wipe

Author: Dan Bungert
Author Date: 2022-04-26 17:36:35 UTC

change md path?

Author: Dan Bungert
Author Date: 2022-04-21 20:07:43 UTC

block/v2: combine needs and prepare resize

Author: Dan Bungert
Author Date: 2022-04-20 00:25:00 UTC

block/v2: resize of ntfs

Author: Dan Bungert
Author Date: 2022-04-26 15:08:33 UTC

vmtests: remove out of date skip

Author: Dan Bungert
Author Date: 2022-04-14 21:19:42 UTC

block: provide get_resize_fstypes

So callers can know which fs types we can and can't resize.

Author: Dan Bungert
Author Date: 2022-04-14 21:03:27 UTC

block-meta: remove extension of extended part size

In v1 action mode, the size of the extended partition is currently
extended by 1MiB to counter for the 1MiB gap in between logical
partiton. Remove that. Callers are expected to provide appropriate
sizing.

Author: Dan Bungert
Author Date: 2022-04-12 17:41:11 UTC

block-meta: resize and preserve correctness

Author: William Wilson
Author Date: 2022-03-14 22:49:52 UTC

Add riscv64 to supported UEFI architectures

As we prepare a RISC-V installer image, we need to add 'riscv64' to the
supported UEFI architectures. This commit does so by setting values
for grub_name and grub_target.