Checkbox

Merge ~kristinchuang/checkbox/+git/checkbox-configure:fix-testplan-missing-parts into ~checkbox-dev/checkbox/+git/checkbox-configure:master

  1. Git
  2. lp:~kristinchuang/checkbox/+git/checkbox-configure
  3. fix-testplan-missing-parts
  4. Merge into master
Proposed by Kristin Chuang
Status: Merged
Approved by: Kristin Chuang
Approved revision: e73ee02f884d25af152be27a5fc532acea314d6c
Merged at revision: 7fd6717c70646bf9702fe9e67b8116e0df71f4e5
Proposed branch: ~kristinchuang/checkbox/+git/checkbox-configure:fix-testplan-missing-parts
Merge into: ~checkbox-dev/checkbox/+git/checkbox-configure:master
Diff against target: 305 lines (+144/-101)
2 files modified
checkbox-provider-template/units/test-plan-classic.pxu.in (+70/-50)
checkbox-provider-template/units/test-plan.pxu.in (+74/-51)
Reviewer Review Type Date Requested Status
Sylvain Pineau (community) Approve
Betty Lin (community) Approve
Vic Liu Approve
Review via email: mp+400173@code.qastaging.launchpad.net

Commit message

Change: update missing/invalid jobs and nested parts

Description of the change

Update the test plan templates to enhance the overall user experience:

1. Split com.canonical.certification::after-suspend.* into smaller parts (e.g. com.canonical.certification::after-suspend-audio.* , com.canonical.certification::after-suspend-ethernet.*) that align with the parts listed under client-cert-iot-ubuntucore-20[1] to avoid dependency issues.

2. Bring back the nested part "device-connections-tp"[2] that was accidentally removed in the previous test plan cleanup.

3. Breakdown usb nested parts (com.canonical.certification::usb.* and com.canonical.certification::after-suspend-usb.*) into usb, usb3 & usb-c parts to allow generating a more precise test plan.

4. Include the job "com.canonical.certification::image/model-grade-not-dangerous" in core test plans.

5. Minor text formatting alignments

[1] https://git.launchpad.net/plainbox-provider-certification-client/tree/units/client-cert-iot-ubuntucore-20.pxu
[2] https://git.launchpad.net/~checkbox-dev/checkbox/+git/checkbox-configure/tree/checkbox-provider-template/units/device/jobs.pxu#n21

To post a comment you must log in.
Revision history for this message
Vic Liu (zongminl) wrote :

- text formatting alignments were adjusted but still not aligned between manual and automated

- keywords used to exclude wireless/wifi_ap.* job are not aligned between manual and automated plans

- In manual plan, there are some jobs excluded but they are only nested in automated plan:
  - iot-fwts-automated
  - after-suspend-gpio-automated
  - after-suspend-location-automated

- WWAN related jobs are missing in exclude section of manual plan:
after-suspend-manual-wwan.*

- In automated plan, there are some jobs excluded but they are only nested in manual plan:
  - monitor-manual
  - usb-c-manual
  - after-suspend-monitor-manual

- The wording in comment of excluding "after-suspend-ipv6.*" should be more clearer, because there should always be a network interface either ethernet or wifi, this exclusion is only required when suspend is not supported in this project.

- after-suspend-socketcan.* is not nested in the cert automated plan

- after-suspend-wireless/wifi_ap.* is not nested in the cert automated plan

review: Needs Fixing
Revision history for this message
Kristin Chuang (kristinchuang) wrote :

Thanks Vic for the very detailed review!

> - text formatting alignments were adjusted but still not aligned between
> manual and automated

Manual, automated & stress text formats now aligned

> - keywords used to exclude wireless/wifi_ap.* job are not aligned between
> manual and automated plans

keywords throughout the plans aligned to "wireless/wifi_ap.*"

> - In manual plan, there are some jobs excluded but they are only nested in
> automated plan:
> - iot-fwts-automated
> - after-suspend-gpio-automated
> - after-suspend-location-automated

aligned with client-cert - removed fwts/gpio/location jobs from manual "exclude:" section

> - WWAN related jobs are missing in exclude section of manual plan:
> after-suspend-manual-wwan.*

added "after-suspend.*wwan.*" into manual "exclude:" section

> - In automated plan, there are some jobs excluded but they are only nested in
> manual plan:
> - monitor-manual
> - usb-c-manual
> - after-suspend-monitor-manual

aligned with client-cert - removed monitor/usb-c/after-suspend-monitor jobs from automated "exclude:" section

> - The wording in comment of excluding "after-suspend-ipv6.*" should be more
> clearer, because there should always be a network interface either ethernet or
> wifi, this exclusion is only required when suspend is not supported in this
> project.

Wording updated

> - after-suspend-socketcan.* is not nested in the cert automated plan
> - after-suspend-wireless/wifi_ap.* is not nested in the cert automated plan

aligned with client-cert - removed "after-suspend-socketcan.*" and "after-suspend-wireless/wifi_ap.*" from the automated "exclude:" section

Revision history for this message
Vic Liu (zongminl) wrote :

LGTM, +1

review: Approve
Revision history for this message
Betty Lin (bettyl) wrote :

LGTM +1

review: Approve
Revision history for this message
Sylvain Pineau (sylvain-pineau) wrote :

very clean and detailed, thanks. +1

review: Approve

There was an error fetching revisions from git servers. Please try again in a few minutes. If the problem persists, contact Launchpad support.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.

Subscribers

People subscribed via source and target branches