Repository: redhat-openshift-ecosystem/openshift-preflight
Pull request #912 (2023-03-02 20:24:46)

add offline flag and necessary code to tar artifacts directory
This is the work for `--offline` flag to tar up the artifacts directory.

- Fixes: #781 
- Add new `--offline` flag for `check container` command
   - make this flag mutually exclusive with `--submit` flag's presence
- Add a method to `tar` the artifacts directory.

 

openshift-ci[bot]: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#" title="Author self-approved">acornett21</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=redhat-openshift-ecosystem%2Fopenshift-preflight).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/redhat-openshift-ecosystem/openshift-preflight/blob/main/OWNERS)~~ [acornett21]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
coveralls: 
[![Coverage Status](https://coveralls.io/builds/57485774/badge)](https://coveralls.io/builds/57485774)

Coverage: 81.881% (-0.2%) from 82.09% when pulling **c3243de7067f266d53529a983c3d0189d9632306 on acornett21:offline_flag** into **bb882d41245510e8d51b0982471b78444f0b4dc9 on redhat-openshift-ecosystem:main**.

dcibot: - SUCCESS https://www.distributed-ci.io/jobs/b68a6f1a-408f-485a-9d6b-f58739622635/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/327d2a4a-11c3-43e3-97e4-21b4b0a6a6b0/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/d2af56d9-3cf0-45de-b7e2-a5574eb87f45/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
openshift-ci[bot]: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#" title="Author self-approved">acornett21</a>*, *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#pullrequestreview-1328970148" title="LGTM">bcrochet</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=redhat-openshift-ecosystem%2Fopenshift-preflight).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/redhat-openshift-ecosystem/openshift-preflight/blob/main/OWNERS)~~ [acornett21,bcrochet]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/cc4cf5dc-e41c-4c58-8731-4cc71dbe828c/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
openshift-ci[bot]: New changes are detected. LGTM label has been removed.
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/148916ac-2643-42f3-b683-cdd727f66325/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
openshift-ci[bot]: @skattoju4: changing LGTM is restricted to collaborators

<details>

In response to [this](https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#pullrequestreview-1329586855):

>


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
openshift-ci[bot]: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#" title="Author self-approved">acornett21</a>*, *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#pullrequestreview-1328970148" title="LGTM">bcrochet</a>*, *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#pullrequestreview-1329586855" title="Approved">skattoju4</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=redhat-openshift-ecosystem%2Fopenshift-preflight).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/redhat-openshift-ecosystem/openshift-preflight/blob/main/OWNERS)~~ [acornett21,bcrochet]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
openshift-ci[bot]: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#" title="Author self-approved">acornett21</a>*, *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#pullrequestreview-1328970148" title="LGTM">bcrochet</a>*, *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#pullrequestreview-1329587665" title="Approved">skattoju</a>*, *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#pullrequestreview-1329586855" title="Approved">skattoju4</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=redhat-openshift-ecosystem%2Fopenshift-preflight).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/redhat-openshift-ecosystem/openshift-preflight/blob/main/OWNERS)~~ [acornett21,bcrochet,skattoju]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
openshift-ci[bot]: New changes are detected. LGTM label has been removed.
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/df1516d2-8736-46ea-8028-869a883c465a/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
openshift-merge-robot: PR needs rebase.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
openshift-ci[bot]: @acornett21: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
ci/prow/4.8-images | cdfcade50fb312d8debcc8370611b5c631f06e2f | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.8-images/1635742467900837888) | true | `/test 4.8-images`

[Full PR test history](https://prow.ci.openshift.org/pr-history?org=redhat-openshift-ecosystem&repo=openshift-preflight&pr=912). [Your PR dashboard](https://prow.ci.openshift.org/pr?query=is:pr+state:open+author:acornett21).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
openshift-ci[bot]: @acornett21: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
ci/prow/4.8-images | cdfcade50fb312d8debcc8370611b5c631f06e2f | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.8-images/1635742467900837888) | true | `/test 4.8-images`
ci/prow/4.12-images | cdfcade50fb312d8debcc8370611b5c631f06e2f | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.12-images/1635742467733065728) | true | `/test 4.12-images`

[Full PR test history](https://prow.ci.openshift.org/pr-history?org=redhat-openshift-ecosystem&repo=openshift-preflight&pr=912). [Your PR dashboard](https://prow.ci.openshift.org/pr?query=is:pr+state:open+author:acornett21).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
openshift-ci[bot]: @acornett21: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
ci/prow/4.8-images | cdfcade50fb312d8debcc8370611b5c631f06e2f | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.8-images/1635742467900837888) | true | `/test 4.8-images`
ci/prow/4.12-images | cdfcade50fb312d8debcc8370611b5c631f06e2f | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.12-images/1635742467733065728) | true | `/test 4.12-images`
ci/prow/4.12-e2e | cdfcade50fb312d8debcc8370611b5c631f06e2f | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.12-e2e/1635742467544322048) | true | `/test 4.12-e2e`

[Full PR test history](https://prow.ci.openshift.org/pr-history?org=redhat-openshift-ecosystem&repo=openshift-preflight&pr=912). [Your PR dashboard](https://prow.ci.openshift.org/pr?query=is:pr+state:open+author:acornett21).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
openshift-ci[bot]: @acornett21: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
ci/prow/4.8-images | cdfcade50fb312d8debcc8370611b5c631f06e2f | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.8-images/1635742467900837888) | true | `/test 4.8-images`
ci/prow/4.12-images | cdfcade50fb312d8debcc8370611b5c631f06e2f | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.12-images/1635742467733065728) | true | `/test 4.12-images`
ci/prow/4.12-e2e | cdfcade50fb312d8debcc8370611b5c631f06e2f | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.12-e2e/1635742467544322048) | true | `/test 4.12-e2e`
ci/prow/4.8-e2e | cdfcade50fb312d8debcc8370611b5c631f06e2f | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.8-e2e/1635742467821146112) | true | `/test 4.8-e2e`

[Full PR test history](https://prow.ci.openshift.org/pr-history?org=redhat-openshift-ecosystem&repo=openshift-preflight&pr=912). [Your PR dashboard](https://prow.ci.openshift.org/pr?query=is:pr+state:open+author:acornett21).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
dcibot: ERROR unable to rebase PR 912 from redhat-openshift-ecosystem/openshift-preflight branch main
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/9e569d91-0f54-47f7-a784-2bf07afea8a2/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/ba44d237-3810-4e3c-9703-13787d4a8411/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/bb59fea4-6499-43bf-a157-2193f152c3c1/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
acornett21: > A few changes and questions. Seems to function. Not sure I love the artifacts.tar being written to the artifacts directory, but alas.

I'd be open to other ideas, should we force that the user giving the `offline` flag _also_ has to give us the `artifacts` flag and provide a custom directory? I think that might be an option.
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/a6c2b2d2-e623-4b76-a318-2fff4fcbe1d5/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
komish: > > A few changes and questions. Seems to function. Not sure I love the artifacts.tar being written to the artifacts directory, but alas.
> 
> I'd be open to other ideas, should we force that the user giving the `offline` flag _also_ has to give us the `artifacts` flag and provide a custom directory? I think that might be an option.

I don't think this buys us anything. We'd still write to /custompath/artifacts.tar, alongside the artifacts that would otherwise be included.
acornett21: @komish I added logic to remove a tar if it already exists, I think this makes the most sense to not run into the situation you described where someone wants to keep re-running `check container`
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/1b42f1ad-3e9d-468c-94f7-173909c5451d/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/8f5575ea-62cf-4f82-a57d-37a682d63d4a/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/25740f64-fd6f-4649-9715-725fc8113991/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/2f63393c-e65a-4c8a-8078-970d99d355c5/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/d3ea9476-629b-409a-bb0e-c0bcb227f89f/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
acornett21: > I think this needs to include preflight.log too, though.

@komish I've added logic to write the log to the same directory where the artifacts are written to.
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/3005e66b-a7b2-44ff-ba08-36e3019993fb/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
openshift-ci[bot]: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#" title="Author self-approved">acornett21</a>*, *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#pullrequestreview-1359665507" title="LGTM">bcrochet</a>*, *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#pullrequestreview-1329587665" title="Approved">skattoju</a>*, *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#pullrequestreview-1329586855" title="Approved">skattoju4</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=redhat-openshift-ecosystem%2Fopenshift-preflight).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/redhat-openshift-ecosystem/openshift-preflight/blob/main/OWNERS)~~ [acornett21,bcrochet,skattoju]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
komish: This isn't quite working how I expect. I'm seeing a log format that doesn't match our standard formatter on first runs with the offline flag.

```
 ⤿ ./preflight check container --offline quay.io/opdev/simple-demo-operator:latest
INFO[0000] certification library version                 version="0.0.0 <commit: cd9998494513973fb4fd6e4e657f38f9890dc2ee>"
INFO[0005] check completed                               check=HasLicense result=PASSED
INFO[0005] check completed                               check=HasUniqueTag result=PASSED
INFO[0005] check completed                               check=LayerCountAcceptable result=PASSED
INFO[0005] check completed                               check=HasNoProhibitedPackages result=PASSED
INFO[0005] check completed                               check=HasRequiredLabel result=PASSED
INFO[0005] USER 65532:65532 specified that is non-root  
INFO[0005] check completed                               check=RunAsNonRoot result=PASSED
INFO[0006] check completed                               check=HasModifiedFiles result=PASSED
INFO[0007] check completed                               check=BasedOnUbi result=PASSED

... truncated ...
```

I get this log format only on initial runs, and I get our usual log format on reruns (e.g. when I already have an artifacts directory already in the working directory). 

It almost looks like we _aren't_ initializing our logger when we run with the offline flag on the first attempt, but then are if we're overwriting the preflight log.

---

I also don't _love_ that we aren't writing preflight.log where it was before. This will probably break folks that have automation oriented around preflight.log being in the current working directory.

---

Finally, if we set the logfile using `PFLT_LOGFILE` on a rerun, we get both files in the archive.

```
 ⤿ PFLT_LOGFILE=foo.log ./preflight check container --offline quay.io/opdev/simple-demo-operator:latest
... truncated ...


 ⤿ tar xvf artifacts/artifacts.tar 
cert-image.json
foo.log
preflight.log
results.json
rpm-manifest.json
```

I think that is probably passable for the moment. The easy workaround is "clean up your directory".


dcibot: - SUCCESS https://www.distributed-ci.io/jobs/def0e277-61d0-4153-9077-96340ef535fb/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
acornett21: @komish I've addressed most of the issues from your comment [here](https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#issuecomment-1487473339). 

The only thing that really can't be addressed is the setting of `PFLT_LOGFILE` in one run and the unsetting in another run. The issue that you point out would exist even in _without_ the `--offline` flag.

Note: PR #939 has a change that is also in this PR, so I need that to merge first so I can rebase.
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/060c717e-bb8a-4e97-b3d9-e7b5bac54221/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912
openshift-ci[bot]: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#" title="Author self-approved">acornett21</a>*, *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#pullrequestreview-1359665507" title="LGTM">bcrochet</a>*, *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#pullrequestreview-1369435428" title="LGTM">komish</a>*, *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#pullrequestreview-1329587665" title="Approved">skattoju</a>*, *<a href="https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912#pullrequestreview-1329586855" title="Approved">skattoju4</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=redhat-openshift-ecosystem%2Fopenshift-preflight).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/redhat-openshift-ecosystem/openshift-preflight/blob/main/OWNERS)~~ [acornett21,bcrochet,komish,skattoju]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
openshift-ci[bot]: New changes are detected. LGTM label has been removed.
openshift-merge-robot: PR needs rebase.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
openshift-ci[bot]: @acornett21: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
ci/prow/4.8-images | feffa77dbab20affdbece50841799fb8ee8c00d0 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.8-images/1643352719781531648) | true | `/test 4.8-images`

[Full PR test history](https://prow.ci.openshift.org/pr-history?org=redhat-openshift-ecosystem&repo=openshift-preflight&pr=912). [Your PR dashboard](https://prow.ci.openshift.org/pr?query=is:pr+state:open+author:acornett21).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
openshift-ci[bot]: @acornett21: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
ci/prow/4.8-images | feffa77dbab20affdbece50841799fb8ee8c00d0 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.8-images/1643352719781531648) | true | `/test 4.8-images`
ci/prow/4.8-e2e | feffa77dbab20affdbece50841799fb8ee8c00d0 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.8-e2e/1643352719752171520) | true | `/test 4.8-e2e`

[Full PR test history](https://prow.ci.openshift.org/pr-history?org=redhat-openshift-ecosystem&repo=openshift-preflight&pr=912). [Your PR dashboard](https://prow.ci.openshift.org/pr?query=is:pr+state:open+author:acornett21).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
openshift-ci[bot]: @acornett21: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
ci/prow/4.8-images | feffa77dbab20affdbece50841799fb8ee8c00d0 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.8-images/1643352719781531648) | true | `/test 4.8-images`
ci/prow/4.8-e2e | feffa77dbab20affdbece50841799fb8ee8c00d0 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.8-e2e/1643352719752171520) | true | `/test 4.8-e2e`
ci/prow/4.12-e2e | feffa77dbab20affdbece50841799fb8ee8c00d0 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.12-e2e/1643352719689256960) | true | `/test 4.12-e2e`

[Full PR test history](https://prow.ci.openshift.org/pr-history?org=redhat-openshift-ecosystem&repo=openshift-preflight&pr=912). [Your PR dashboard](https://prow.ci.openshift.org/pr?query=is:pr+state:open+author:acornett21).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
openshift-ci[bot]: @acornett21: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
ci/prow/4.8-images | feffa77dbab20affdbece50841799fb8ee8c00d0 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.8-images/1643352719781531648) | true | `/test 4.8-images`
ci/prow/4.8-e2e | feffa77dbab20affdbece50841799fb8ee8c00d0 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.8-e2e/1643352719752171520) | true | `/test 4.8-e2e`
ci/prow/4.12-e2e | feffa77dbab20affdbece50841799fb8ee8c00d0 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.12-e2e/1643352719689256960) | true | `/test 4.12-e2e`
ci/prow/4.12-images | feffa77dbab20affdbece50841799fb8ee8c00d0 | [link](https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-openshift-ecosystem_openshift-preflight/912/pull-ci-redhat-openshift-ecosystem-openshift-preflight-main-4.12-images/1643352719718617088) | true | `/test 4.12-images`

[Full PR test history](https://prow.ci.openshift.org/pr-history?org=redhat-openshift-ecosystem&repo=openshift-preflight&pr=912). [Your PR dashboard](https://prow.ci.openshift.org/pr?query=is:pr+state:open+author:acornett21).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
dcibot: ERROR unable to rebase PR 912 from redhat-openshift-ecosystem/openshift-preflight branch main
dcibot: - SUCCESS https://www.distributed-ci.io/jobs/4719f1f5-e98e-4e0f-8a18-1d3182c4caa9/jobStates
 from https://github.com/redhat-openshift-ecosystem/openshift-preflight/pull/912

Patch:
diff --git a/.gitignore b/.gitignore
index 01de4f9..9133fdc 100644
--- a/.gitignore
+++ b/.gitignore
@@ -35,6 +35,7 @@ results-junit.xml
 *BasicSpecCheck.json
 *OlmSuiteCheck.json
 hashes.txt
+artifacts.tar
 
 # E2E customzied env log file.
 sgol.txt
diff --git a/artifacts/filesystem_writer.go b/artifacts/filesystem_writer.go
index 5f2a794..7918095 100644
--- a/artifacts/filesystem_writer.go
+++ b/artifacts/filesystem_writer.go
@@ -52,6 +52,20 @@ func (w *FilesystemWriter) WriteFile(filename string, contents io.Reader) (strin
 	return fullFilePath, nil
 }
 
+// Exists checks if a file exists with a filename
+func (w *FilesystemWriter) Exists(filename string) (bool, error) {
+	fullFilePath := filepath.Join(w.Path(), filename)
+
+	return afero.Exists(w.fs, fullFilePath)
+}
+
+// Remove removes contents from dir at filename.
+func (w *FilesystemWriter) Remove(filename string) error {
+	fullFilePath := filepath.Join(w.Path(), filename)
+
+	return w.fs.Remove(fullFilePath)
+}
+
 // Path is the full artifacts path.
 func (w *FilesystemWriter) Path() string {
 	return w.dir
diff --git a/cmd/preflight/cmd/check_container.go b/cmd/preflight/cmd/check_container.go
index ddd3bce..e13d4d3 100644
--- a/cmd/preflight/cmd/check_container.go
+++ b/cmd/preflight/cmd/check_container.go
@@ -1,8 +1,13 @@
 package cmd
 
 import (
+	"archive/tar"
+	"bytes"
 	"context"
 	"fmt"
+	"io"
+	"os"
+	"path/filepath"
 	rt "runtime"
 	"strings"
 
@@ -50,6 +55,14 @@ func checkContainerCmd(runpreflight runPreflight) *cobra.Command {
 	flags.Bool("insecure", false, "Use insecure protocol for the registry. Default is False. Cannot be used with submit.")
 	_ = viper.BindPFlag("insecure", flags.Lookup("insecure"))
 
+	flags.Bool("offline", false, "Intended to be used in disconnected environments and will tar artifacts used for submission,\n"+
+		"enabling other Red Hat managed tools/process to submit these artifacts at a later time.\n"+
+		"Cannot be used with submit.")
+	_ = viper.BindPFlag("offline", flags.Lookup("offline"))
+
+	// Make --submit mutually exclusive to --offline
+	checkContainerCmd.MarkFlagsMutuallyExclusive("submit", "offline")
+
 	// Make --submit mutually exclusive to --insecure
 	checkContainerCmd.MarkFlagsMutuallyExclusive("submit", "insecure")
 
@@ -116,7 +129,7 @@ func checkContainerRunE(cmd *cobra.Command, args []string, runpreflight runPrefl
 	// Run the  container check.
 	cmd.SilenceUsage = true
 
-	return runpreflight(
+	err = runpreflight(
 		ctx,
 		checkcontainer.Run,
 		cli.CheckConfig{
@@ -127,6 +140,46 @@ func checkContainerRunE(cmd *cobra.Command, args []string, runpreflight runPrefl
 		&runtime.ResultWriterFile{},
 		resultSubmitter,
 	)
+
+	if err != nil {
+		return err
+	}
+
+	// checking for offline flag, if present tar up the contents of the artifacts directory
+	if cfg.Offline {
+		src := artifactsWriter.Path()
+		var buf bytes.Buffer
+
+		// check to see if a tar file already exist to account for someone re-running
+		exists, err := artifactsWriter.Exists(check.DefaultArtifactsTarFileName)
+		if err != nil {
+			return fmt.Errorf("unable to check if tar already exists: %v", err)
+		}
+
+		// remove the tar file if it exists
+		if exists {
+			err = artifactsWriter.Remove(check.DefaultArtifactsTarFileName)
+			if err != nil {
+				return fmt.Errorf("unable to remove existing tar: %v", err)
+			}
+		}
+
+		// tar the directory
+		err = artifactsTar(ctx, src, &buf)
+		if err != nil {
+			return fmt.Errorf("unable to tar up artifacts directory: %v", err)
+		}
+
+		// writing the tar file to disk
+		_, err = artifactsWriter.WriteFile(check.DefaultArtifactsTarFileName, &buf)
+		if err != nil {
+			return fmt.Errorf("could not artifacts tar to artifacts dir: %w", err)
+		}
+
+		logger.Info("artifact tar written to disk", "filename", check.DefaultArtifactsTarFileName)
+	}
+
+	return nil
 }
 
 func checkContainerPositionalArgs(cmd *cobra.Command, args []string) error {
@@ -214,3 +267,75 @@ func generateContainerCheckOptions(cfg *runtime.Config) []container.Option {
 
 	return o
 }
+
+// artifactsTar takes a source path and a writer; a tar writer loops over the files in the source
+// directory, writes the appropriate header information and copies the file into the tar writer
+//
+//nolint:unparam // ctx is unused. Keep for future use.
+func artifactsTar(ctx context.Context, src string, w io.Writer) error {
+	// ensure the src actually exists before trying to tar it
+	if _, err := os.Stat(src); err != nil {
+		return fmt.Errorf("unable to tar files - %v", err.Error())
+	}
+
+	tw := tar.NewWriter(w)
+	defer tw.Close()
+
+	// getting a list of DirEntry's
+	files, err := os.ReadDir(src)
+	if err != nil {
+		return err
+	}
+
+	// iterating over the list
+	for _, file := range files {
+		// ignoring directories and only processing files
+		if file.IsDir() {
+			continue
+		}
+
+		// getting the FileInfo from the DirEntry, account for errors with this call ie ErrNotExist
+		fileInfo, err := file.Info()
+		if err != nil {
+			return err
+		}
+
+		// continue on non-regular files
+		if !fileInfo.Mode().IsRegular() {
+			continue
+		}
+
+		// create a new dir/file header
+		header, err := tar.FileInfoHeader(fileInfo, fileInfo.Name())
+		if err != nil {
+			return err
+		}
+
+		// write the header
+		if err := tw.WriteHeader(header); err != nil {
+			return err
+		}
+
+		err = func() error {
+			// open files to tar
+			f, err := os.Open(filepath.Join(src, fileInfo.Name()))
+			if err != nil {
+				return err
+			}
+			defer f.Close()
+
+			// copy file data into tar writer
+			if _, err := io.Copy(tw, f); err != nil {
+				return err
+			}
+
+			return nil
+		}()
+
+		if err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
diff --git a/cmd/preflight/cmd/root.go b/cmd/preflight/cmd/root.go
index 99b4919..0f3f12c 100644
--- a/cmd/preflight/cmd/root.go
+++ b/cmd/preflight/cmd/root.go
@@ -5,6 +5,7 @@ import (
 	"context"
 	"io"
 	"os"
+	"path/filepath"
 
 	"github.com/redhat-openshift-ecosystem/openshift-preflight/artifacts"
 	"github.com/redhat-openshift-ecosystem/openshift-preflight/internal/viper"
@@ -97,12 +98,29 @@ func preRunConfig(cmd *cobra.Command, args []string) {
 		mw := io.MultiWriter(os.Stderr, logFile)
 		l.SetOutput(mw)
 	} else {
-		l.Debug("Failed to log to file, using default stderr")
+		l.Infof("Failed to log to file, using default stderr")
 	}
 	if ll, err := logrus.ParseLevel(viper.GetString("loglevel")); err == nil {
 		l.SetLevel(ll)
 	}
 
+	// if we are in the offline flow redirect log file to exist in the directory where all other artifact exist
+	if viper.GetBool("offline") {
+		artifacts := viper.GetString("artifacts")
+
+		// ignoring error since OpenFile will error and we'll still have the multiwriter from above
+		_ = os.Mkdir(artifacts, 0o777)
+
+		artifactsLogFile, err := os.OpenFile(filepath.Join(artifacts, logname), os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o600)
+		if err == nil {
+			mw := io.MultiWriter(os.Stderr, logFile, artifactsLogFile)
+			l.SetOutput(mw)
+		}
+
+		// setting log level to trace, to provide the most detailed logs possible
+		l.SetLevel(logrus.TraceLevel)
+	}
+
 	if !configFileUsed {
 		l.Debug("config file not found, proceeding without it")
 	}
diff --git a/internal/check/defaults.go b/internal/check/defaults.go
index 5cf7448..1fddda1 100644
--- a/internal/check/defaults.go
+++ b/internal/check/defaults.go
@@ -1,10 +1,11 @@
 package check
 
 var (
-	DefaultCertImageFilename   = "cert-image.json"
-	DefaultRPMManifestFilename = "rpm-manifest.json"
-	DefaultTestResultsFilename = "results.json"
-	DefaultPyxisHost           = "catalog.redhat.com/api/containers"
-	DefaultPyxisEnv            = "prod"
-	SystemdDir                 = "/etc/systemd/system"
+	DefaultCertImageFilename    = "cert-image.json"
+	DefaultRPMManifestFilename  = "rpm-manifest.json"
+	DefaultTestResultsFilename  = "results.json"
+	DefaultArtifactsTarFileName = "artifacts.tar"
+	DefaultPyxisHost            = "catalog.redhat.com/api/containers"
+	DefaultPyxisEnv             = "prod"
+	SystemdDir                  = "/etc/systemd/system"
 )
diff --git a/internal/engine/engine.go b/internal/engine/engine.go
index 6f0c523..7c7b09a 100644
--- a/internal/engine/engine.go
+++ b/internal/engine/engine.go
@@ -41,6 +41,7 @@ import (
 	"github.com/google/go-containerregistry/pkg/name"
 	cranev1 "github.com/google/go-containerregistry/pkg/v1"
 	"github.com/google/go-containerregistry/pkg/v1/cache"
+	"github.com/google/go-containerregistry/pkg/v1/mutate"
 	"github.com/google/go-containerregistry/pkg/v1/remote"
 )
 
@@ -75,6 +76,12 @@ type CraneEngine struct {
 	results  certification.Results
 }
 
+func export(img cranev1.Image, w io.Writer) error {
+	fs := mutate.Extract(img)
+	_, err := io.Copy(w, fs)
+	return err
+}
+
 func (c *CraneEngine) ExecuteChecks(ctx context.Context) error {
 	logger := logr.FromContextOrDiscard(ctx)
 	logger.Info("target image", "image", c.Image)
@@ -157,7 +164,7 @@ func (c *CraneEngine) ExecuteChecks(ctx context.Context) error {
 		// extraction. These errors will be returned by the reader end
 		// on subsequent reads. If err == nil, the reader will return
 		// EOF.
-		w.CloseWithError(crane.Export(img, w))
+		w.CloseWithError(export(img, w))
 	}()
 
 	logger.V(log.DBG).Info("extracting container filesystem", "path", containerFSPath)
diff --git a/internal/policy/container/base_on_ubi.go b/internal/policy/container/base_on_ubi.go
index 0adcb44..db43353 100644
--- a/internal/policy/container/base_on_ubi.go
+++ b/internal/policy/container/base_on_ubi.go
@@ -50,7 +50,7 @@ func (p *BasedOnUBICheck) getImageLayers(image cranev1.Image) ([]cranev1.Hash, e
 func (p *BasedOnUBICheck) certifiedImagesFound(ctx context.Context, layerHashes []cranev1.Hash) (bool, error) {
 	certImages, err := p.LayerHashCheckEngine.CertifiedImagesContainingLayers(ctx, layerHashes)
 	if err != nil {
-		return false, fmt.Errorf("pyxis query for uncompressed top layers ids failed: %w", err)
+		return false, fmt.Errorf("pyxis query for uncompressed top layers ids %+q failed: %w", layerHashes, err)
 	}
 	if len(certImages) >= 1 {
 		return true, nil
diff --git a/internal/policy/container/has_modified_files.go b/internal/policy/container/has_modified_files.go
index a2c73b6..a3bc348 100644
--- a/internal/policy/container/has_modified_files.go
+++ b/internal/policy/container/has_modified_files.go
@@ -2,11 +2,15 @@ package container
 
 import (
 	"archive/tar"
+	"bufio"
 	"context"
+	"errors"
 	"fmt"
 	"io"
+	"os"
+	"path/filepath"
+	"regexp"
 	"strings"
-	"sync"
 
 	"github.com/redhat-openshift-ecosystem/openshift-preflight/internal/check"
 	"github.com/redhat-openshift-ecosystem/openshift-preflight/internal/image"
@@ -14,7 +18,9 @@ import (
 	"github.com/redhat-openshift-ecosystem/openshift-preflight/internal/rpm"
 
 	"github.com/go-logr/logr"
+	v1 "github.com/google/go-containerregistry/pkg/v1"
 	rpmdb "github.com/knqyf263/go-rpmdb/pkg"
+	"github.com/spf13/afero"
 )
 
 var _ check.Check = &HasModifiedFilesCheck{}
@@ -24,140 +30,215 @@ var _ check.Check = &HasModifiedFilesCheck{}
 // modified in subsequent layers.
 type HasModifiedFilesCheck struct{}
 
+const whiteoutPrefix = ".wh."
+
+type packageMeta struct {
+	Name    string
+	Version string
+	Release string
+	Arch    string
+	Vendor  string
+}
+
+func (pm packageMeta) Compare(other packageMeta) int {
+	return 0
+}
+
 type packageFilesRef struct {
-	// LayerFiles contains a slice of files modified by layer.
-	LayerFiles [][]string
-	// PackagesFiles is a map of modified files. the anonymous struct
-	// here is just used to allow us to search package files by name
-	// as the key of this map, and doesn't have any value.
-	PackageFiles map[string]struct{}
+	// LayerFiles contains a slice of files created/modified in layer
+	LayerFiles []string
+	// LayerPackages is a map of the packages in a layer. The key is
+	// the NVR of the package. The value is metadata about the package
+	// that we use for processing
+	LayerPackages map[string]packageMeta
+	// LayerPackageFiles maps files to a package name-version-release
+	LayerPackageFiles map[string]string
+	HasRPMDB          bool
 }
 
+// Validate runs the check of whether any Red Hat files were modified
 func (p *HasModifiedFilesCheck) Validate(ctx context.Context, imgRef image.ImageReference) (bool, error) {
-	packageFiles, err := p.getDataToValidate(ctx, imgRef)
+	fs := afero.NewOsFs()
+	layerIDs, packageFiles, packageDist, err := p.gatherDataToValidate(ctx, imgRef, fs)
 	if err != nil {
 		return false, fmt.Errorf("could not generate modified files list: %v", err)
 	}
-	return p.validate(ctx, packageFiles)
+
+	return p.validate(ctx, layerIDs, packageFiles, packageDist)
 }
 
-// getDataToValidate returns the list of files (packageFilesRef.PackageFiles)
-// installed via packages from the container image,and the list of files (packageFilesRef.LayerFiles)
-// modified/added via layers in the image.
-func (p *HasModifiedFilesCheck) getDataToValidate(ctx context.Context, imgRef image.ImageReference) (*packageFilesRef, error) {
+// gatherDataToValidate returns a map from layer digest to a struct containing the list of files
+// (packageFilesRef.LayerPackageFiles) installed via packages (packageFilesRef.LayerPackages)
+// from the container image, and the list of files (packageFilesRef.LayerFiles) modified/added
+// via layers in the image.
+func (p *HasModifiedFilesCheck) gatherDataToValidate(ctx context.Context, imgRef image.ImageReference, fs afero.Fs) ([]string, map[string]packageFilesRef, string, error) {
 	logger := logr.FromContextOrDiscard(ctx)
 
-	// Get a list of packages from the RPM database. This avoids having to rely on
-	// rpm, dnf, yum, etc. being installed in the image.
-	pkgList, err := rpm.GetPackageList(ctx, imgRef.ImageFSPath)
+	layerDir, err := afero.TempDir(fs, "", "rpm-layers-")
 	if err != nil {
-		return nil, fmt.Errorf("could not get rpm list: %w", err)
+		return nil, nil, "", err
 	}
+	defer func() {
+		_ = fs.RemoveAll(layerDir)
+	}()
 
-	packageFiles, err := p.getInstalledFilesFor(pkgList)
-	if err != nil {
-		return nil, fmt.Errorf("could not list installed files: %w", err)
+	if imgRef.ImageInfo == nil {
+		return nil, nil, "", fmt.Errorf("image reference invalid")
 	}
 
 	layers, err := imgRef.ImageInfo.Layers()
 	if err != nil {
-		return nil, fmt.Errorf("could not get image layers: %w", err)
+		return nil, nil, "", err
 	}
 
-	files := make([][]string, 0, len(layers))
+	layerIDs := make([]string, 0, len(layers))
+	layerRefs := make(map[string]packageFilesRef, len(layers))
+
+	// Uncompress each layer and build maps containing the packages,
+	// the package files, and the files modified by each layer
+	// Also generate a list of the layerIDs so we can keep the
+	// order within the maps.
+	for idx, layer := range layers {
+		layerIDHash, err := layer.Digest()
+		if err != nil {
+			return nil, nil, "", fmt.Errorf("unable to retrieve diff id for layer: %w", err)
+		}
+		layerID := layerIDHash.String()
+
+		layerDir := filepath.Join(layerDir, fmt.Sprintf("%02d-%s", idx, layerID))
+		err = fs.Mkdir(layerDir, 0o755)
+		if err != nil {
+			return nil, nil, "", fmt.Errorf("could not create layer directory: %w", err)
+		}
+
+		layerIDs = append(layerIDs, layerID)
 
-	// Uncompress each layer and build a slice containing the files
-	// modified by each layer.
-	for _, layer := range layers {
-		r, err := layer.Uncompressed()
+		files, err := generateChangesFor(layer)
 		if err != nil {
-			return nil, fmt.Errorf("could not extract layers: %w", err)
-		}
-		pathChan := make(chan string)
-
-		var wg sync.WaitGroup
-		wg.Add(1)
-		go func() {
-			// For each path in the pathChan, add it to the layer's
-			// list of files.
-			layerFiles := make([]string, 0)
-			for path := range pathChan {
-				layerFiles = append(layerFiles, path)
+			return nil, nil, "", err
+		}
+
+		found, pkgList := findRPMDB(ctx, layer)
+		if !found {
+			logger.V(log.TRC).Info("could not find rpm database in layer", "layer", layerID)
+			if idx > 0 {
+				// Just make this is the same as last layer, since the RPM db was not modified
+				lastLayer := layerIDs[idx-1]
+				layerRefs[layerID] = packageFilesRef{
+					LayerFiles:        files,
+					LayerPackages:     layerRefs[lastLayer].LayerPackages,
+					LayerPackageFiles: layerRefs[lastLayer].LayerPackageFiles,
+					HasRPMDB:          false,
+				}
+				continue
 			}
-			// also add it to the overall list of files.
-			files = append(files, layerFiles)
-			wg.Done()
-		}()
-		// add paths to the pathChan
-		err = untar(pathChan, r)
+
+			// If it's the first layer, just make the pkgList empty.
+			pkgList = make([]*rpmdb.PackageInfo, 0)
+		}
+
+		pkgNameList := extractPackageNameVersionRelease(pkgList)
+
+		packageFiles, err := installedFileMapWithExclusions(ctx, pkgList)
 		if err != nil {
-			return nil, fmt.Errorf("failed to extract tarball: %w", err)
+			return nil, nil, "", err
 		}
 
-		wg.Wait() // wait for file list to get appended to the files var
+		layerRefs[layerID] = packageFilesRef{
+			LayerFiles:        files,
+			LayerPackages:     pkgNameList,
+			LayerPackageFiles: packageFiles,
+			HasRPMDB:          true,
+		}
 	}
 
-	files, dropped := p.dropFirstLayerIfEmpty(files)
-	if dropped {
-		// tell the user the first layer was dropped
-		diff0, _ := layers[0].DiffID()
-		diff1, _ := layers[1].DiffID()
-		logger.V(log.DBG).Info(
-			fmt.Sprintf("The first layer (%s) contained no files, so the next layer (%s) is being used as the base layer.",
-				diff0.String(),
-				diff1.String(),
-			))
+	osRelease, err := fs.Open(filepath.Join(imgRef.ImageFSPath, "etc", "os-release"))
+	if err != nil {
+		return nil, nil, "", fmt.Errorf("could not open os-release: %v", err)
+	}
+	defer osRelease.Close()
+	scanner := bufio.NewScanner(osRelease)
+	packageDist := "unknown"
+	for scanner.Scan() {
+		line := scanner.Text()
+		r, _ := regexp.Compile(`PLATFORM_ID="platform:([[:alnum:]]+)"`)
+		m := r.FindStringSubmatch(line)
+		if m == nil {
+			continue
+		}
+		packageDist = m[1]
+		break
 	}
 
-	return &packageFilesRef{files, packageFiles}, nil
-}
-
-// dropFirstLayerIfEmpty will evaluate the length of the first layer and will drop its entry in files if it's empty.
-// This avoids establishing a baseline of files installed in the original layer with an empty list. An example
-// of when this should occur is in cases where the image was built FROM scratch, such as ubi-micro. This is required
-// to ensure we don't attempt to validate against an empty layer.
-func (p *HasModifiedFilesCheck) dropFirstLayerIfEmpty(files [][]string) ([][]string, bool) {
-	var dropped bool
-	if len(files[0]) == 0 {
-		files = files[1:] // shift the empty layer out.
-		dropped = true
+	if err := scanner.Err(); err != nil {
+		return nil, nil, "", fmt.Errorf("error while scanning for package dist: %v", err)
 	}
 
-	return files, dropped
+	return layerIDs, layerRefs, packageDist, nil
 }
 
 // validate compares the list of LayerFiles and PackageFiles to see what PackageFiles
-// have been modified within the additional layers.
-//
-//nolint:unparam // ctx is unused. Keep for future use.
-func (p *HasModifiedFilesCheck) validate(ctx context.Context, packageFilesRef *packageFilesRef) (bool, error) {
+// have been modified within the additional layers. packageDist is the value we expect
+// to find in the base package's Release field.
+func (p *HasModifiedFilesCheck) validate(ctx context.Context, layerIDs []string, packageFiles map[string]packageFilesRef, packageDist string) (bool, error) {
 	logger := logr.FromContextOrDiscard(ctx)
 
-	layerFiles := packageFilesRef.LayerFiles
-	packageFiles := packageFilesRef.PackageFiles
-
-	// Determine the list of files that were a part of the base layer.
-	baseLayer := make(map[string]struct{}, len(layerFiles[0]))
-	for _, filename := range layerFiles[0] {
-		if _, ok := packageFiles[filename]; ok {
-			baseLayer[filename] = struct{}{}
-		}
-	}
-	layers := layerFiles[1:]
+	disallowedModifications := false
+	for idx, layerID := range layerIDs {
+		logger := logger.WithValues("layer", layerID)
+		ref := packageFiles[layerID]
+		for _, modifiedFile := range ref.LayerFiles {
+			// If it's a modifiedFile but this layer has an RPM db, that's allowed, but only if the
+			// package itself is updated.
+			if idx == 0 && ref.HasRPMDB {
+				// Just skip these in the first layer
+				continue
+			}
+			if _, found := ref.LayerPackageFiles[modifiedFile]; !found {
+				// Far as we can tell, this isn't from an RPM
+				continue
+			}
+			previousPackageVersion, prevFound := packageFiles[layerIDs[idx-1]].LayerPackageFiles[modifiedFile]
+			if !prevFound && ref.HasRPMDB {
+				// This is a net-new package file. Pass it.
+				continue
+			}
+			currentPackageVersion := ref.LayerPackageFiles[modifiedFile]
+			previousPackage := packageFiles[layerIDs[idx-1]].LayerPackages[previousPackageVersion]
+			currentPackage := ref.LayerPackages[currentPackageVersion]
+
+			if previousPackageVersion == currentPackageVersion {
+				if !strings.Contains(currentPackage.Release, packageDist) && packageDist != "unknown" {
+					// This means it's _probably_ not a RH package. If the file is changed, warn, but don't fail
+					logger.Info("WARN: an rpm-installed file was modified outside of rpm, but appears to be from a third-party. This could be a failure in the future")
+					continue
+				}
+
+				if currentPackage.Vendor != "Red Hat, Inc." && previousPackage.Vendor != "Red Hat, Inc." {
+					// This means it's _probably_ not a RH package. If the file is changed, warn, but don't fail
+					logger.Info("WARN: an rpm-installed file was modified outside of rpm, but appears to be from a third-party. This could be a failure in the future")
+					continue
+				}
+
+				// Nope, nope, nope. File was modified without using RPM
+				logger.V(log.DBG).Info("found disallowed modification in layer", "file", modifiedFile)
+				disallowedModifications = true
+				continue
+			}
+			// Check that release contains the same arch and OS release
+			previousOsRelease := strings.Contains(previousPackage.Release, packageDist)
+			currentOsRelease := strings.Contains(currentPackage.Release, packageDist)
 
-	// Look for modifications in subsequent layers by determining
-	// if a file in a base layer exists in a subsequent layer.
-	modifiedFilesDetected := false
-	for _, layer := range layers {
-		for _, file := range layer {
-			if _, ok := baseLayer[file]; ok {
-				// This means the files exists in the base layer. This is a fail.
-				logger.V(log.DBG).Info("modified file detected", "filename", file)
-				modifiedFilesDetected = true
+			if (previousOsRelease && !currentOsRelease) || (previousPackage.Arch != currentPackage.Arch) {
+				// If either of these differ, that's a fail
+				return false, nil
 			}
+
+			// This appears like an update. This is allowed.
+			// No further action required
 		}
 	}
-	return !modifiedFilesDetected, nil
+	return !disallowedModifications, nil
 }
 
 func (p HasModifiedFilesCheck) Name() string {
@@ -180,69 +261,248 @@ func (p HasModifiedFilesCheck) Metadata() check.Metadata {
 	}
 }
 
-// Untar takes a destination path and a reader; a tar reader loops over the tarfile
-// creating the file structure at 'dst' along the way, and writing any files
-func untar(pathChan chan<- string, r io.Reader) error {
-	tr := tar.NewReader(r)
+func extractPackageNameVersionRelease(pkgList []*rpmdb.PackageInfo) map[string]packageMeta {
+	pkgNameList := make(map[string]packageMeta, len(pkgList))
+	for _, pkg := range pkgList {
+		pkgNameList[fmt.Sprintf("%s-%s-%s", pkg.Name, pkg.Version, pkg.Release)] = packageMeta{
+			Name:    pkg.Name,
+			Version: pkg.Version,
+			Release: pkg.Release,
+			Arch:    pkg.Arch,
+			Vendor:  pkg.Vendor,
+		}
+	}
+	return pkgNameList
+}
 
-	for {
-		header, err := tr.Next()
+// findRPMDB attempts to extract a valid RPMDB from layers in the order
+// they are provided. If found is false, pkglist should
+// be disregarded as any value there will be invalid.
+func findRPMDB(ctx context.Context, layer v1.Layer) (found bool, pkglist []*rpmdb.PackageInfo) {
+	logger := logr.FromContextOrDiscard(ctx)
+	var err error
+	pkglist, err = extractRPMDB(ctx, layer)
+	if err == nil {
+		id, _ := layer.Digest()
+		logger.V(log.TRC).Info("findRPMDB found an RPM db", "layer", id.String())
+		found = true
+		return
+	}
 
-		switch {
-		// if no more files are found return
-		case err == io.EOF:
-			close(pathChan)
-			return nil
+	return found, pkglist
+}
 
-		// return any other error
-		case err != nil:
-			return err
+// directoryIsExcluded excludes a directory and any file contained in that directory.
+func directoryIsExcluded(ctx context.Context, s string) bool {
+	excl := map[string]struct{}{
+		"etc": {},
+		"var": {},
+		"run": {},
+	}
 
-		// if the header is nil, just skip it (not sure how this happens)
-		case header == nil:
-			continue
+	for k := range excl {
+		if strings.HasPrefix(s, filepath.Clean(k+"/")) || k == s {
+			logger := logr.FromContextOrDiscard(ctx)
+			logger.V(log.TRC).Info("directory excluded", "directory", s)
+			return true
 		}
+	}
 
-		// the target location where the dir/file should be created
-		target := header.Name
+	return false
+}
 
-		// the following switch could also be done using fi.Mode(), not sure if there
-		// a benefit of using one vs. the other.
-		// fi := header.FileInfo()
+// pathIsExcluded checks if s is excluded explicitly as written.
+func pathIsExcluded(ctx context.Context, s string) bool {
+	excl := map[string]struct{}{
+		"etc/resolv.conf": {},
+		"etc/hostname":    {},
+		// etc and etc/ are both required as both can present the directory
+		// in a tarball. Same goes for other directories.
+		"etc":  {},
+		"etc/": {},
+		"run":  {},
+		"run/": {},
+	}
 
-		// check the file type
-		switch header.Typeflag {
-		// if its a dir ignore it
-		case tar.TypeDir:
-			continue
+	_, found := excl[s]
+	if found {
+		logger := logr.FromContextOrDiscard(ctx)
+		logger.V(log.TRC).Info("file excluded", "file", s)
+		return true
+	}
+	return found
+}
 
-		// if it's a file write the name to the channel
-		case tar.TypeReg:
-			// Strip off any leading '/' or './'
-			pathChan <- strings.TrimLeft(target, "./")
+// normalize will clean a filepath of extraneous characters like ./, //, etc.
+// and strip a leading slash. E.g. /foo/../baz --> baz
+func normalize(s string) string {
+	// for the root path, return the root path.
+	if s == "/" {
+		return s
+	}
+	return filepath.Clean(strings.TrimPrefix(s, "/"))
+}
 
-		// if it's a link create it
-		case tar.TypeSymlink:
-			pathChan <- strings.TrimLeft(header.Linkname, "./")
+// installedFileMapWithExclusions gets a map of installed filenames that have been cleaned
+// of extra slashes, dotslashes, and leading slashes.
+func installedFileMapWithExclusions(ctx context.Context, pkglist []*rpmdb.PackageInfo) (map[string]string, error) {
+	const okFlags = rpmdb.RPMFILE_CONFIG |
+		rpmdb.RPMFILE_DOC |
+		rpmdb.RPMFILE_LICENSE |
+		rpmdb.RPMFILE_MISSINGOK |
+		rpmdb.RPMFILE_README
+	m := map[string]string{}
+	for _, pkg := range pkglist {
+		files, err := pkg.InstalledFiles()
+		if err != nil {
+			return m, err
+		}
+
+		for _, file := range files {
+			if int32(file.Flags)&okFlags > 0 {
+				// It is one of the ok flags. Skip it.
+				continue
+			}
+			normalized := normalize(file.Path)
+			if pathIsExcluded(ctx, normalized) || directoryIsExcluded(ctx, normalized) {
+				// It is either an explicitly excluded path or directory. Skip it.
+				continue
+			}
+			m[normalized] = fmt.Sprintf("%s-%s-%s", pkg.Name, pkg.Version, pkg.Release)
 		}
 	}
+	return m, nil
 }
 
-// getInstalledFilesFor returns a map of installed files by pkgs. The returned map only contains
-// relevant keys to aid in lookup; values are unimportant.
-func (p *HasModifiedFilesCheck) getInstalledFilesFor(pkgList []*rpmdb.PackageInfo) (map[string]struct{}, error) {
-	installedFiles := make(map[string]struct{}, len(pkgList))
-	for _, pkg := range pkgList {
-		filenames, err := pkg.InstalledFileNames()
+// generateChangesFor will check layer for file changes, and will return a list of those.
+func generateChangesFor(layer v1.Layer) ([]string, error) {
+	layerReader, err := layer.Uncompressed()
+	if err != nil {
+		return nil, fmt.Errorf("reading layer contents: %w", err)
+	}
+	defer layerReader.Close()
+	tarReader := tar.NewReader(layerReader)
+	var filelist []string
+	for {
+		header, err := tarReader.Next()
+		if errors.Is(err, io.EOF) {
+			break
+		}
+		if err != nil {
+			return nil, fmt.Errorf("reading tar: %w", err)
+		}
+
+		// Some tools prepend everything with "./", so if we don't Clean the
+		// name, we may have duplicate entries, which angers tar-split.
+		header.Name = filepath.Clean(header.Name)
+		// force PAX format to remove Name/Linkname length limit of 100 characters
+		// required by USTAR and to not depend on internal tar package guess which
+		// prefers USTAR over PAX
+		header.Format = tar.FormatPAX
+
+		basename := filepath.Base(header.Name)
+		dirname := filepath.Dir(header.Name)
+		tombstone := strings.HasPrefix(basename, whiteoutPrefix)
+		if tombstone {
+			basename = basename[len(whiteoutPrefix):]
+		}
+		switch {
+		case (header.Typeflag == tar.TypeDir && tombstone) || header.Typeflag == tar.TypeReg:
+			filelist = append(filelist, strings.TrimPrefix(filepath.Join(dirname, basename), "/"))
+		case header.Typeflag == tar.TypeSymlink:
+			filelist = append(filelist, strings.TrimPrefix(header.Name, "/"))
+		default:
+			// TODO: what do we do with other flags?
+			continue
+		}
+	}
+
+	return filelist, nil
+}
+
+// ExtractRPMDB copies /var/lib/rpm/* from the archive and derives a list of packages from
+// the rpm database.
+func extractRPMDB(ctx context.Context, layer v1.Layer) ([]*rpmdb.PackageInfo, error) {
+	layerReader, err := layer.Uncompressed()
+	if err != nil {
+		return nil, fmt.Errorf("reading layer contents: %w", err)
+	}
+	defer layerReader.Close()
+
+	fs := afero.NewOsFs()
+	basepath, err := afero.TempDir(fs, "", "rpmdb")
+	if err != nil {
+		return nil, err
+	}
+	defer func() {
+		_ = fs.RemoveAll(basepath)
+	}()
+
+	tarReader := tar.NewReader(layerReader)
+	for {
+		header, err := tarReader.Next()
+		if errors.Is(err, io.EOF) {
+			break
+		}
+		if err != nil {
+			return nil, fmt.Errorf("reading tar: %w", err)
+		}
+
+		// Some tools prepend everything with "./", so if we don't Clean the
+		// name, we may have duplicate entries, which angers tar-split.
+		header.Name = filepath.Clean(header.Name)
+		header.Format = tar.FormatPAX
+		rpmdirname := "var/lib/rpm"
+		basename := filepath.Base(header.Name)
+		dirname := filepath.Dir(header.Name)
+		tombstone := strings.HasPrefix(basename, whiteoutPrefix)
+
+		// Not a file or directory? Continue...
+		if header.Typeflag != tar.TypeDir && header.Typeflag != tar.TypeReg {
+			continue
+		}
+
+		// Tombstone? Ignore...
+		if tombstone {
+			continue
+		}
+
+		// Not in the RPM directory. Ignore...
+		if !strings.HasPrefix(filepath.Join(dirname, basename), rpmdirname) {
+			continue
+		}
+		// a dir or file with the correct var/lib/rpm prefix that has not been marked with a tombstone is valid.
+		if header.Typeflag == tar.TypeDir {
+			err := os.MkdirAll(filepath.Join(basepath, dirname, basename), header.FileInfo().Mode())
+			if err != nil {
+				return nil, err
+			}
+			continue
+		}
+
+		f, err := fs.OpenFile(filepath.Join(basepath, dirname, basename), os.O_RDWR|os.O_CREATE|os.O_TRUNC, header.FileInfo().Mode())
 		if err != nil {
 			return nil, err
 		}
-		for _, file := range filenames {
-			// A struct is used here, but it is unimportant and
-			// should not have value.
-			installedFiles[file] = struct{}{}
+		err = func() error {
+			// closure here allows us to defer f.Close() in this iteration instead of
+			// waiting for the parent function to complete.
+			defer f.Close()
+			_, err := io.Copy(f, tarReader)
+			if err != nil {
+				return err
+			}
+			return nil
+		}()
+		if err != nil {
+			return nil, err
 		}
 	}
 
-	return installedFiles, nil
+	packageList, err := rpm.GetPackageList(ctx, basepath)
+	if err != nil {
+		return nil, err
+	}
+
+	return packageList, nil
 }
diff --git a/internal/runtime/config.go b/internal/runtime/config.go
index 6e28f81..5d08e08 100644
--- a/internal/runtime/config.go
+++ b/internal/runtime/config.go
@@ -26,6 +26,7 @@ type Config struct {
 	Submit                 bool
 	Platform               string
 	Insecure               bool
+	Offline                bool
 	// Operator-Specific Fields
 	Namespace         string
 	ServiceAccount    string
@@ -67,6 +68,7 @@ func (c *Config) storeContainerPolicyConfiguration(vcfg viper.Viper) {
 	c.CertificationProjectID = vcfg.GetString("certification_project_id")
 	c.Platform = vcfg.GetString("platform")
 	c.Insecure = vcfg.GetBool("insecure")
+	c.Offline = vcfg.GetBool("offline")
 }
 
 // storeOperatorPolicyConfiguration reads operator-policy-specific config


Test patch:
diff --git a/cmd/preflight/cmd/check_container_test.go b/cmd/preflight/cmd/check_container_test.go
index 9d21266..ff6bb26 100644
--- a/cmd/preflight/cmd/check_container_test.go
+++ b/cmd/preflight/cmd/check_container_test.go
@@ -7,7 +7,9 @@ import (
 	"os"
 	"path/filepath"
 
+	"github.com/redhat-openshift-ecosystem/openshift-preflight/artifacts"
 	"github.com/redhat-openshift-ecosystem/openshift-preflight/certification"
+	"github.com/redhat-openshift-ecosystem/openshift-preflight/internal/check"
 	"github.com/redhat-openshift-ecosystem/openshift-preflight/internal/cli"
 	"github.com/redhat-openshift-ecosystem/openshift-preflight/internal/formatters"
 	"github.com/redhat-openshift-ecosystem/openshift-preflight/internal/lib"
@@ -24,7 +26,7 @@ var _ = Describe("Check Container Command", func() {
 	Context("when running the check container subcommand", func() {
 		Context("With all of the required parameters", func() {
 			It("should reach the core logic, but throw an error because of the placeholder values for the container image", func() {
-				_, err := executeCommand(checkContainerCmd(mockRunPreflight), "example.com/example/image:mytag")
+				_, err := executeCommand(checkContainerCmd(mockRunPreflightReturnNil), "example.com/example/image:mytag")
 				Expect(err).To(HaveOccurred())
 			})
 		})
@@ -33,21 +35,21 @@ var _ = Describe("Check Container Command", func() {
 	Context("When validating check container arguments and flags", func() {
 		Context("and the user provided more than 1 positional arg", func() {
 			It("should fail to run", func() {
-				_, err := executeCommand(checkContainerCmd(mockRunPreflight), "foo", "bar")
+				_, err := executeCommand(checkContainerCmd(mockRunPreflightReturnNil), "foo", "bar")
 				Expect(err).To(HaveOccurred())
 			})
 		})
 
 		Context("and the user provided less than 1 positional arg", func() {
 			It("should fail to run", func() {
-				_, err := executeCommand(checkContainerCmd(mockRunPreflight))
+				_, err := executeCommand(checkContainerCmd(mockRunPreflightReturnNil))
 				Expect(err).To(HaveOccurred())
 			})
 		})
 
 		DescribeTable("and the user has enabled the submit flag",
 			func(errString string, args []string) {
-				out, err := executeCommand(checkContainerCmd(mockRunPreflight), args...)
+				out, err := executeCommand(checkContainerCmd(mockRunPreflightReturnNil), args...)
 				Expect(err).To(HaveOccurred())
 				Expect(out).To(ContainSubstring(errString))
 			},
@@ -72,7 +74,7 @@ var _ = Describe("Check Container Command", func() {
 				It("should still execute with no error", func() {
 					submit = true
 
-					err := checkContainerPositionalArgs(checkContainerCmd(mockRunPreflight), []string{"foo"})
+					err := checkContainerPositionalArgs(checkContainerCmd(mockRunPreflightReturnNil), []string{"foo"})
 					Expect(err).ToNot(HaveOccurred())
 					Expect(viper.Instance().GetString("pyxis_api_token")).To(Equal("tokenid"))
 					Expect(viper.Instance().GetString("certification_project_id")).To(Equal("certid"))
@@ -94,7 +96,7 @@ certification_project_id: mycertid`
 					initConfig()
 					submit = true
 
-					err := checkContainerPositionalArgs(checkContainerCmd(mockRunPreflight), []string{"foo"})
+					err := checkContainerPositionalArgs(checkContainerCmd(mockRunPreflightReturnNil), []string{"foo"})
 					Expect(err).ToNot(HaveOccurred())
 					Expect(viper.Instance().GetString("pyxis_api_token")).To(Equal("mytoken"))
 					Expect(viper.Instance().GetString("certification_project_id")).To(Equal("mycertid"))
@@ -110,7 +112,7 @@ certification_project_id: mycertid`
 				DeferCleanup(viper.Instance().Set, "certification_project_id", "")
 			})
 			It("should not change the flag value", func() {
-				err := validateCertificationProjectID(checkContainerCmd(mockRunPreflight), []string{"foo"})
+				err := validateCertificationProjectID(checkContainerCmd(mockRunPreflightReturnNil), []string{"foo"})
 				Expect(err).ToNot(HaveOccurred())
 				Expect(viper.Instance().GetString("certification_project_id")).To(Equal("123456789"))
 			})
@@ -121,7 +123,7 @@ certification_project_id: mycertid`
 				DeferCleanup(viper.Instance().Set, "certification_project_id", "")
 			})
 			It("should strip ospid- from the flag value", func() {
-				err := validateCertificationProjectID(checkContainerCmd(mockRunPreflight), []string{"foo"})
+				err := validateCertificationProjectID(checkContainerCmd(mockRunPreflightReturnNil), []string{"foo"})
 				Expect(err).ToNot(HaveOccurred())
 				Expect(viper.Instance().GetString("certification_project_id")).To(Equal("123456789"))
 			})
@@ -132,7 +134,7 @@ certification_project_id: mycertid`
 				DeferCleanup(viper.Instance().Set, "certification_project_id", "")
 			})
 			It("should throw an error", func() {
-				err := validateCertificationProjectID(checkContainerCmd(mockRunPreflight), []string{"foo"})
+				err := validateCertificationProjectID(checkContainerCmd(mockRunPreflightReturnNil), []string{"foo"})
 				Expect(err).To(HaveOccurred())
 			})
 		})
@@ -142,7 +144,7 @@ certification_project_id: mycertid`
 				DeferCleanup(viper.Instance().Set, "certification_project_id", "")
 			})
 			It("should throw an error", func() {
-				err := validateCertificationProjectID(checkContainerCmd(mockRunPreflight), []string{"foo"})
+				err := validateCertificationProjectID(checkContainerCmd(mockRunPreflightReturnNil), []string{"foo"})
 				Expect(err).To(HaveOccurred())
 			})
 		})
@@ -151,14 +153,129 @@ certification_project_id: mycertid`
 	Context("when running the check container subcommand with a logger provided", func() {
 		Context("with all of the required parameters", func() {
 			It("should reach the core logic, and execute the mocked RunPreflight", func() {
-				_, err := executeCommandWithLogger(checkContainerCmd(mockRunPreflight), logr.Discard(), "example.com/example/image:mytag")
+				_, err := executeCommandWithLogger(checkContainerCmd(mockRunPreflightReturnNil), logr.Discard(), "example.com/example/image:mytag")
 				Expect(err).ToNot(HaveOccurred())
 			})
 		})
+		Context("with all of the required parameters with error mocked", func() {
+			It("should reach the core logic, and execute the mocked RunPreflight and return error", func() {
+				_, err := executeCommandWithLogger(checkContainerCmd(mockRunPreflightReturnErr), logr.Discard(), "example.com/example/image:mytag")
+				Expect(err).To(HaveOccurred())
+			})
+		})
+	})
+
+	Context("when running the check container subcommand with a offline config provided", func() {
+		Context("with all of the required parameters", func() {
+			BeforeEach(func() {
+				tmpDir, err := os.MkdirTemp("", "preflight-submit-test-*")
+				Expect(err).ToNot(HaveOccurred())
+				DeferCleanup(os.RemoveAll, tmpDir)
+
+				// creating test files on in the tmpDir so the tar function has files to tar
+				f1, err := os.Create(filepath.Join(tmpDir, "test-file-1.json"))
+				Expect(err).ToNot(HaveOccurred())
+				defer f1.Close()
+
+				f2, err := os.Create(filepath.Join(tmpDir, "test-file-1.json"))
+				Expect(err).ToNot(HaveOccurred())
+				defer f2.Close()
+
+				viper.Instance().Set("artifacts", tmpDir)
+				DeferCleanup(viper.Instance().Set, "artifacts", artifacts.DefaultArtifactsDir)
+
+				viper.Instance().Set("offline", true)
+				DeferCleanup(viper.Instance().Set, "offline", false)
+			})
+			It("should reach core logic, and the additional offline logic", func() {
+				out, err := executeCommandWithLogger(checkContainerCmd(mockRunPreflightReturnNil), logr.Discard(), "example.com/example/image:mytag")
+				Expect(err).ToNot(HaveOccurred())
+				Expect(out).ToNot(BeNil())
+			})
+		})
+		Context("when an existing artifacts.tar already on disk", func() {
+			BeforeEach(func() {
+				tmpDir, err := os.MkdirTemp("", "preflight-submit-test-*")
+				Expect(err).ToNot(HaveOccurred())
+				DeferCleanup(os.RemoveAll, tmpDir)
+
+				// creating test files on in the tmpDir so the tar function has files to tar
+				f1, err := os.Create(filepath.Join(tmpDir, "test-file-1.json"))
+				Expect(err).ToNot(HaveOccurred())
+				defer f1.Close()
+
+				f2, err := os.Create(filepath.Join(tmpDir, "test-file-1.json"))
+				Expect(err).ToNot(HaveOccurred())
+				defer f2.Close()
+
+				// creating a tar file to mimic a user re-running check container for a second time
+				f3, err := os.Create(filepath.Join(tmpDir, check.DefaultArtifactsTarFileName))
+				Expect(err).ToNot(HaveOccurred())
+				defer f3.Close()
+
+				viper.Instance().Set("artifacts", tmpDir)
+				DeferCleanup(viper.Instance().Set, "artifacts", artifacts.DefaultArtifactsDir)
+
+				viper.Instance().Set("offline", true)
+				DeferCleanup(viper.Instance().Set, "offline", false)
+			})
+			It("should reach the additional offline logic, and remove existing tar file", func() {
+				out, err := executeCommandWithLogger(checkContainerCmd(mockRunPreflightReturnNil), logr.Discard(), "example.com/example/image:mytag")
+				Expect(err).ToNot(HaveOccurred())
+				Expect(out).ToNot(BeNil())
+			})
+		})
+	})
+
+	Context("when artifactsTar is called directly", func() {
+		Context("and the src does not exist", func() {
+			It("should get an unable to tar files error", func() {
+				err := artifactsTar(context.Background(), "", nil)
+				Expect(err).To(HaveOccurred())
+			})
+		})
+		Context("and a bad writer is passed", func() {
+			It("should get an error with trying to write the header", func() {
+				err := artifactsTar(context.Background(), ".", errWriter(0))
+				Expect(err).To(HaveOccurred())
+			})
+		})
+		Context("and a src has no permissions", func() {
+			var tmpDir string
+			var err error
+			var buf bytes.Buffer
+			JustBeforeEach(func() {
+				tmpDir, err = os.MkdirTemp("", "preflight-submit-test-*")
+				Expect(err).ToNot(HaveOccurred())
+				err = os.Chmod(tmpDir, 0o00)
+				Expect(err).ToNot(HaveOccurred())
+				DeferCleanup(os.RemoveAll, tmpDir)
+			})
+			It("should get an error with trying to read directory", func() {
+				err := artifactsTar(context.Background(), tmpDir, &buf)
+				Expect(err).To(HaveOccurred())
+			})
+		})
+		Context("and src only contains another directory", func() {
+			var tmpDir string
+			var err error
+			var buf bytes.Buffer
+			JustBeforeEach(func() {
+				tmpDir, err = os.MkdirTemp("", "preflight-submit-test-*")
+				Expect(err).ToNot(HaveOccurred())
+				err = os.Mkdir(filepath.Join(tmpDir, "test"), 0o755)
+				Expect(err).ToNot(HaveOccurred())
+				DeferCleanup(os.RemoveAll, tmpDir)
+			})
+			It("should continue and not tar any files", func() {
+				err := artifactsTar(context.Background(), tmpDir, &buf)
+				Expect(err).To(BeNil())
+			})
+		})
 	})
 })
 
-func mockRunPreflight(context.Context, func(ctx context.Context) (certification.Results, error), cli.CheckConfig, formatters.ResponseFormatter, lib.ResultWriter, lib.ResultSubmitter) error {
+func mockRunPreflightReturnNil(context.Context, func(ctx context.Context) (certification.Results, error), cli.CheckConfig, formatters.ResponseFormatter, lib.ResultWriter, lib.ResultSubmitter) error {
 	return nil
 }
 
diff --git a/cmd/preflight/cmd/check_operator_test.go b/cmd/preflight/cmd/check_operator_test.go
index 58cff17..241771d 100644
--- a/cmd/preflight/cmd/check_operator_test.go
+++ b/cmd/preflight/cmd/check_operator_test.go
@@ -17,7 +17,7 @@ var _ = Describe("Check Operator", func() {
 	Context("when running the check operator subcommand", func() {
 		Context("without the operator bundle image being provided", func() {
 			It("should return an error", func() {
-				_, err := executeCommand(checkOperatorCmd(mockRunPreflight))
+				_, err := executeCommand(checkOperatorCmd(mockRunPreflightReturnNil))
 				Expect(err).To(HaveOccurred())
 			})
 		})
@@ -30,7 +30,7 @@ var _ = Describe("Check Operator", func() {
 				os.Unsetenv("KUBECONFIG")
 			})
 			It("should return an error", func() {
-				out, err := executeCommand(checkOperatorCmd(mockRunPreflight), "quay.io/example/image:mytag")
+				out, err := executeCommand(checkOperatorCmd(mockRunPreflightReturnNil), "quay.io/example/image:mytag")
 				Expect(err).To(HaveOccurred())
 				Expect(out).To(ContainSubstring("KUBECONFIG could not"))
 			})
@@ -50,7 +50,7 @@ var _ = Describe("Check Operator", func() {
 				os.Setenv("KUBECONFIG", "foo")
 			})
 			It("should return an error", func() {
-				out, err := executeCommand(checkOperatorCmd(mockRunPreflight), "quay.io/example/image:mytag")
+				out, err := executeCommand(checkOperatorCmd(mockRunPreflightReturnNil), "quay.io/example/image:mytag")
 				Expect(err).To(HaveOccurred())
 				Expect(out).To(ContainSubstring("PFLT_INDEXIMAGE could not"))
 			})
@@ -78,7 +78,7 @@ var _ = Describe("Check Operator", func() {
 				os.Setenv("KUBECONFIG", f1.Name())
 			})
 			It("should reach the core logic, and execute the mocked RunPreflight", func() {
-				out, err := executeCommandWithLogger(checkOperatorCmd(mockRunPreflight), logr.Discard(), "quay.io/example/image:mytag")
+				out, err := executeCommandWithLogger(checkOperatorCmd(mockRunPreflightReturnNil), logr.Discard(), "quay.io/example/image:mytag")
 				Expect(err).ToNot(HaveOccurred())
 				Expect(out).ToNot(BeNil())
 			})
@@ -102,7 +102,7 @@ var _ = Describe("Check Operator", func() {
 				os.Setenv("KUBECONFIG", "foo")
 			})
 			It("should return a no such file error", func() {
-				out, err := executeCommandWithLogger(checkOperatorCmd(mockRunPreflight), logr.Discard(), "quay.io/example/image:mytag")
+				out, err := executeCommandWithLogger(checkOperatorCmd(mockRunPreflightReturnNil), logr.Discard(), "quay.io/example/image:mytag")
 				Expect(err).To(HaveOccurred())
 				Expect(out).To(ContainSubstring(": open foo: no such file or directory"))
 			})
@@ -121,7 +121,7 @@ var _ = Describe("Check Operator", func() {
 				os.Setenv("KUBECONFIG", ".")
 			})
 			It("should return an is a directory error", func() {
-				out, err := executeCommandWithLogger(checkOperatorCmd(mockRunPreflight), logr.Discard(), "quay.io/example/image:mytag")
+				out, err := executeCommandWithLogger(checkOperatorCmd(mockRunPreflightReturnNil), logr.Discard(), "quay.io/example/image:mytag")
 				Expect(err).To(HaveOccurred())
 				Expect(out).To(ContainSubstring(": is a directory"))
 			})
@@ -185,7 +185,7 @@ var _ = Describe("Check Operator", func() {
 		})
 
 		It("should succeed when all positional arg constraints and environment constraints are correct", func() {
-			err := checkOperatorPositionalArgs(checkOperatorCmd(mockRunPreflight), posArgs)
+			err := checkOperatorPositionalArgs(checkOperatorCmd(mockRunPreflightReturnNil), posArgs)
 			Expect(err).ToNot(HaveOccurred())
 		})
 	})
diff --git a/cmd/preflight/cmd/root_test.go b/cmd/preflight/cmd/root_test.go
index d687610..6146a31 100644
--- a/cmd/preflight/cmd/root_test.go
+++ b/cmd/preflight/cmd/root_test.go
@@ -116,13 +116,38 @@ var _ = Describe("cmd package utility functions", func() {
 				tmpDir, err = os.MkdirTemp("", "prerun-config-*")
 				Expect(err).ToNot(HaveOccurred())
 				DeferCleanup(os.RemoveAll, tmpDir)
+
+				viper.Instance().Set("logfile", filepath.Join(tmpDir, "foo.log"))
+				DeferCleanup(viper.Instance().Set, "logfile", "preflight.log")
 			})
 			It("should create the logfile", func() {
-				viper.Instance().Set("logfile", filepath.Join(tmpDir, "foo.log"))
 				Expect(cmd.ExecuteContext(context.TODO())).To(Succeed())
 				_, err := os.Stat(filepath.Join(tmpDir, "foo.log"))
 				Expect(err).ToNot(HaveOccurred())
 			})
 		})
+		Context("with the offline flag", func() {
+			var tmpDir string
+			BeforeEach(func() {
+				var err error
+				tmpDir, err = os.MkdirTemp("", "prerun-artifacts-*")
+				Expect(err).ToNot(HaveOccurred())
+				DeferCleanup(os.RemoveAll, tmpDir)
+
+				viper.Instance().Set("artifacts", tmpDir)
+				DeferCleanup(viper.Instance().Set, "artifacts", artifacts.DefaultArtifactsDir)
+
+				viper.Instance().Set("logfile", "preflight.log")
+				DeferCleanup(viper.Instance().Set, "logfile", "preflight.log")
+
+				viper.Instance().Set("offline", true)
+				DeferCleanup(viper.Instance().Set, "offline", false)
+			})
+			It("should create the logfile in the artifacts directory", func() {
+				Expect(cmd.ExecuteContext(context.TODO())).To(Succeed())
+				_, err := os.Stat(filepath.Join(tmpDir, "preflight.log"))
+				Expect(err).ToNot(HaveOccurred())
+			})
+		})
 	})
 })
diff --git a/cmd/preflight/cmd/suite_test.go b/cmd/preflight/cmd/suite_test.go
index 46e332a..b9f4d3b 100644
--- a/cmd/preflight/cmd/suite_test.go
+++ b/cmd/preflight/cmd/suite_test.go
@@ -1,6 +1,7 @@
 package cmd
 
 import (
+	"errors"
 	"os"
 	"path/filepath"
 	"testing"
@@ -23,3 +24,10 @@ var createAndCleanupDirForArtifactsAndLogs = func() {
 	DeferCleanup(os.Unsetenv, "PFLT_ARTIFACTS")
 	DeferCleanup(os.Unsetenv, "PFLT_LOGFILE")
 }
+
+// In order to test some negative paths, this io.Writer will just throw an error
+type errWriter int
+
+func (errWriter) Write(p []byte) (n int, err error) {
+	return 0, errors.New("test error")
+}
diff --git a/internal/policy/container/has_modifed_files_test.go b/internal/policy/container/has_modifed_files_test.go
deleted file mode 100644
index 33bf848..0000000
--- a/internal/policy/container/has_modifed_files_test.go
+++ /dev/null
@@ -1,138 +0,0 @@
-package container
-
-import (
-	"context"
-	"path"
-
-	rpmdb "github.com/knqyf263/go-rpmdb/pkg"
-	. "github.com/onsi/ginkgo/v2"
-	. "github.com/onsi/gomega"
-
-	"github.com/redhat-openshift-ecosystem/openshift-preflight/internal/image"
-)
-
-var _ = Describe("HasModifiedFiles", func() {
-	var (
-		hasModifiedFiles HasModifiedFilesCheck
-		pkgList          packageFilesRef
-	)
-
-	BeforeEach(func() {
-		pkgList = packageFilesRef{
-			LayerFiles: [][]string{
-				{
-					"this",
-					"is",
-					"not",
-					"prohibited",
-				},
-				{
-					"there",
-					"are",
-					"no",
-					"prohibited",
-					"duplicates",
-				},
-			},
-			PackageFiles: map[string]struct{}{
-				"this": {},
-				"is":   {},
-				"not":  {},
-			},
-		}
-	})
-
-	Describe("Checking if it has any modified RPM files", func() {
-		Context("When there are no modified RPM files found", func() {
-			It("should pass validate", func() {
-				ok, err := hasModifiedFiles.validate(context.Background(), &pkgList)
-				Expect(err).ToNot(HaveOccurred())
-				Expect(ok).To(BeTrue())
-			})
-		})
-		Context("When there is a modified RPM file found", func() {
-			var pkgs packageFilesRef
-			BeforeEach(func() {
-				pkgs = pkgList
-				pkgs.LayerFiles[1] = append(pkgs.LayerFiles[1], "this")
-			})
-			It("should not pass Validate", func() {
-				ok, err := hasModifiedFiles.validate(context.Background(), &pkgs)
-				Expect(err).ToNot(HaveOccurred())
-				Expect(ok).To(BeFalse())
-			})
-		})
-	})
-
-	Context("When building the installed file list for installed packages", func() {
-		const (
-			basename = "foobasename"
-			dirname  = "foodirname"
-			dirindex = 0
-		)
-		var goodPkgList []*rpmdb.PackageInfo
-		var badPkgList []*rpmdb.PackageInfo
-
-		BeforeEach(func() {
-			goodPkgList = []*rpmdb.PackageInfo{
-				{
-					BaseNames:  []string{basename},
-					DirIndexes: []int32{dirindex},
-					DirNames:   []string{dirname},
-				},
-			}
-
-			badPkgList = []*rpmdb.PackageInfo{
-				{
-					BaseNames:  []string{basename},
-					DirIndexes: []int32{dirindex},
-					DirNames:   []string{dirname, "extra"},
-				},
-			}
-		})
-		It("should contain all files installed by the package according to its metadata", func() {
-			files, err := hasModifiedFiles.getInstalledFilesFor(goodPkgList)
-			Expect(err).ToNot(HaveOccurred())
-
-			_, ok := files[path.Join(dirname, basename)]
-			Expect(ok).To(BeTrue())
-		})
-
-		It("should fail if the rpm is invalid", func() {
-			_, err := hasModifiedFiles.getInstalledFilesFor(badPkgList)
-			Expect(err).To(HaveOccurred())
-		})
-	})
-
-	Context("When evaluating files in each layer", func() {
-		var files [][]string
-		BeforeEach(func() {
-			files = [][]string{
-				{"foo"},
-				{"bar"},
-			}
-		})
-		It("should shift over to the next layer if the first layer is empty", func() {
-			files[0] = []string{}
-			shiftedFiles, shifted := hasModifiedFiles.dropFirstLayerIfEmpty(files)
-			Expect(shiftedFiles).To(BeEquivalentTo(files[1:]))
-			Expect(shifted).To(BeTrue())
-		})
-
-		It("should start at the first layer if it's not empty", func() {
-			unshiftedFiles, shifted := hasModifiedFiles.dropFirstLayerIfEmpty(files)
-			Expect(unshiftedFiles).To(BeEquivalentTo(files))
-			Expect(shifted).To(BeFalse())
-		})
-	})
-
-	Context("When calling the top level Validate", func() {
-		It("should fail with an invalid ImageReference", func() {
-			passed, err := hasModifiedFiles.Validate(context.TODO(), image.ImageReference{})
-			Expect(err).To(HaveOccurred())
-			Expect(passed).To(BeFalse())
-		})
-	})
-
-	AssertMetaData(&hasModifiedFiles)
-})
diff --git a/internal/policy/container/has_modified_files_test.go b/internal/policy/container/has_modified_files_test.go
new file mode 100644
index 0000000..c8209a5
--- /dev/null
+++ b/internal/policy/container/has_modified_files_test.go
@@ -0,0 +1,357 @@
+package container
+
+import (
+	"bytes"
+	"context"
+	"path"
+
+	"github.com/redhat-openshift-ecosystem/openshift-preflight/internal/image"
+
+	"github.com/bombsimon/logrusr/v4"
+	"github.com/go-logr/logr"
+	rpmdb "github.com/knqyf263/go-rpmdb/pkg"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+	"github.com/sirupsen/logrus"
+)
+
+var _ = Describe("HasModifiedFiles", func() {
+	var (
+		hasModifiedFiles HasModifiedFilesCheck
+		layers           []string
+		pkgRef           map[string]packageFilesRef
+		dist             string
+	)
+
+	BeforeEach(func() {
+		pkgRef = make(map[string]packageFilesRef)
+		pkgRef["firstlayer"] = packageFilesRef{
+			LayerFiles: []string{
+				"this",
+				"is",
+				"not",
+				"prohibited",
+			},
+			LayerPackages: map[string]packageMeta{
+				"foo-1.0-1.d9": {
+					Name:    "foo",
+					Version: "1.0",
+					Release: "1.d9",
+					Arch:    "fooarch",
+					Vendor:  "Red Hat, Inc.",
+				},
+				"bar-1.0-1.d9": {
+					Name:    "bar",
+					Version: "1.0",
+					Release: "1.d9",
+					Arch:    "fooarch",
+					Vendor:  "Red Hat, Inc.",
+				},
+				"baz-2.0-1.d9": {
+					Name:    "baz",
+					Version: "1.0",
+					Release: "1.d9",
+					Arch:    "fooarch",
+					Vendor:  "Red Hat, Inc.",
+				},
+			},
+			LayerPackageFiles: map[string]string{
+				"this": "foo-1.0-1.d9",
+				"is":   "bar-1.0-1.d9",
+				"not":  "baz-2.0-1.d9",
+			},
+			HasRPMDB: true,
+		}
+
+		pkgRef["secondlayer"] = packageFilesRef{
+			LayerFiles: []string{
+				"there",
+				"are",
+				"no",
+				"prohibited",
+				"duplicates",
+			},
+			LayerPackages: map[string]packageMeta{
+				"foo-1.0-1.d9": {
+					Name:    "foo",
+					Version: "1.0",
+					Release: "1.d9",
+					Arch:    "fooarch",
+					Vendor:  "Red Hat, Inc.",
+				},
+				"bar-1.0-1.d9": {
+					Name:    "bar",
+					Version: "1.0",
+					Release: "1.d9",
+					Arch:    "fooarch",
+					Vendor:  "Red Hat, Inc.",
+				},
+				"baz-2.0-1.d9": {
+					Name:    "baz",
+					Version: "2.0",
+					Release: "1.d9",
+					Arch:    "fooarch",
+					Vendor:  "Red Hat, Inc.",
+				},
+				"boz-3.0-1.d9": {
+					Name:    "boz",
+					Version: "3.0",
+					Release: "1.d9",
+					Arch:    "fooarch",
+					Vendor:  "Red Hat, Inc.",
+				},
+			},
+			LayerPackageFiles: map[string]string{
+				"this":  "foo-1.0-1.d9",
+				"is":    "bar-1.0-1.d9",
+				"not":   "baz-2.0-1.d9",
+				"no":    "boz-3.0-1.d9",
+				"there": "boz-3.0-1.d9",
+			},
+			HasRPMDB: true,
+		}
+		pkgRef["lastlayer"] = packageFilesRef{
+			LayerFiles: []string{
+				"prohibited",
+			},
+			LayerPackages: map[string]packageMeta{
+				"foo-1.0-1.d9": {
+					Name:    "foo",
+					Version: "1.0",
+					Release: "1.d9",
+					Arch:    "fooarch",
+					Vendor:  "Red Hat, Inc.",
+				},
+				"bar-1.0-1.d9": {
+					Name:    "bar",
+					Version: "1.0",
+					Release: "1.d9",
+					Arch:    "fooarch",
+					Vendor:  "Red Hat, Inc.",
+				},
+				"baz-2.0-1.d9": {
+					Name:    "baz",
+					Version: "2.0",
+					Release: "1.d9",
+					Arch:    "fooarch",
+					Vendor:  "Red Hat, Inc.",
+				},
+				"boz-3.0-1.d9": {
+					Name:    "boz",
+					Version: "3.0",
+					Release: "1.d9",
+					Arch:    "fooarch",
+					Vendor:  "Red Hat, Inc.",
+				},
+			},
+			LayerPackageFiles: map[string]string{
+				"this": "foo-1.0-1.d9",
+				"is":   "bar-1.0-1.d9",
+				"not":  "baz-2.0-1.d9",
+			},
+		}
+		layers = []string{
+			"firstlayer",
+			"secondlayer",
+			"lastlayer",
+		}
+		dist = "d9"
+	})
+
+	Context("Checking if it has any modified RPM files", func() {
+		When("there are no modified RPM files found", func() {
+			It("should pass validate", func() {
+				ok, err := hasModifiedFiles.validate(context.Background(), layers, pkgRef, dist)
+				Expect(err).ToNot(HaveOccurred())
+				Expect(ok).To(BeTrue())
+			})
+		})
+		When("there is a modified RPM file found", func() {
+			var pkgs map[string]packageFilesRef
+			BeforeEach(func() {
+				pkgs = pkgRef
+				pkgSecondLayer := pkgRef["secondlayer"]
+				pkgSecondLayer.LayerFiles = append(pkgs["secondlayer"].LayerFiles, "this")
+				pkgs["secondlayer"] = pkgSecondLayer
+			})
+			It("should not pass Validate", func() {
+				ok, err := hasModifiedFiles.validate(context.Background(), layers, pkgs, dist)
+				Expect(err).ToNot(HaveOccurred())
+				Expect(ok).To(BeFalse())
+			})
+		})
+		When("a package is updated", func() {
+			var pkgs map[string]packageFilesRef
+			BeforeEach(func() {
+				pkgs = pkgRef
+				pkgSecondLayer := pkgs["secondlayer"]
+				pkgSecondLayerPackageFiles := pkgSecondLayer.LayerPackageFiles
+				pkgSecondLayerPackageFiles["this"] = "foo-2.0-d9"
+				pkgSecondLayer.LayerPackageFiles = pkgSecondLayerPackageFiles
+				pkgs["secondlayer"] = pkgSecondLayer
+			})
+			It("should pass validate", func() {
+				ok, err := hasModifiedFiles.validate(context.Background(), layers, pkgs, dist)
+				Expect(err).ToNot(HaveOccurred())
+				Expect(ok).To(BeTrue())
+			})
+		})
+		When("a package is removed", func() {
+			var pkgs map[string]packageFilesRef
+			BeforeEach(func() {
+				pkgs = pkgRef
+				pkgSecondLayer := pkgs["secondlayer"]
+				pkgSecondLayerPackageFiles := pkgSecondLayer.LayerPackageFiles
+				delete(pkgSecondLayerPackageFiles, "this")
+				pkgSecondLayer.LayerPackageFiles = pkgSecondLayerPackageFiles
+				pkgs["secondlayer"] = pkgSecondLayer
+			})
+			It("should pass validate", func() {
+				ok, err := hasModifiedFiles.validate(context.Background(), layers, pkgs, dist)
+				Expect(err).ToNot(HaveOccurred())
+				Expect(ok).To(BeTrue())
+			})
+		})
+		When("the package release dist changes", func() {
+			var pkgs map[string]packageFilesRef
+			BeforeEach(func() {
+				pkgs = pkgRef
+
+				pkgSecondLayerPackageFiles := pkgs["secondlayer"].LayerPackageFiles
+				delete(pkgSecondLayerPackageFiles, "this")
+				pkgSecondLayerPackageFiles["this"] = "foo-1.0-1.d10"
+
+				pkgSecondLayerPackages := pkgs["secondlayer"].LayerPackages
+				delete(pkgSecondLayerPackages, "foo-1.0-1.d9")
+				pkgSecondLayerPackages["foo-1.0-1.d10"] = packageMeta{
+					Name:    "foo",
+					Version: "1.0",
+					Release: "1.d10",
+					Arch:    "fooarch",
+				}
+
+				pkgs["secondlayer"] = packageFilesRef{
+					LayerPackages:     pkgSecondLayerPackages,
+					LayerPackageFiles: pkgSecondLayerPackageFiles,
+					LayerFiles:        append(pkgs["secondlayer"].LayerFiles, "this"),
+					HasRPMDB:          true,
+				}
+			})
+			It("should fail because of different release dist", func() {
+				ok, err := hasModifiedFiles.validate(context.Background(), layers, pkgs, dist)
+				Expect(err).ToNot(HaveOccurred())
+				Expect(ok).To(BeFalse())
+			})
+		})
+		When("release dist does not match installed OS", func() {
+			When("package is a net-new", func() {
+				When("a file is modified", func() {
+					var pkgs map[string]packageFilesRef
+					var ctx context.Context
+					var logOutput bytes.Buffer
+					BeforeEach(func() {
+						pkgs = pkgRef
+
+						pkgSecondLayerPackages := pkgs["secondlayer"].LayerPackages
+						pkgs["secondlayer"].LayerPackages["other-1.0-1.oth"] = packageMeta{
+							Name:    "other",
+							Version: "1.0",
+							Release: "1.oth",
+							Arch:    "x86_64",
+						}
+						pkgSecondLayerPackageFiles := pkgs["secondlayer"].LayerPackageFiles
+						pkgSecondLayerPackageFiles["otherfile"] = "other-1.0-1.oth"
+						pkgs["secondlayer"] = packageFilesRef{
+							LayerPackages:     pkgSecondLayerPackages,
+							LayerPackageFiles: pkgSecondLayerPackageFiles,
+							LayerFiles:        append(pkgs["secondlayer"].LayerFiles, "otherfile"),
+							HasRPMDB:          true,
+						}
+
+						pkgs["lastlayer"] = packageFilesRef{
+							LayerPackages:     pkgs["secondlayer"].LayerPackages,
+							LayerPackageFiles: pkgs["secondlayer"].LayerPackageFiles,
+							LayerFiles:        append(pkgs["lastlayer"].LayerFiles, "otherfile"),
+							HasRPMDB:          false,
+						}
+
+						l := logrus.New()
+						l.SetLevel(logrus.DebugLevel)
+						l.SetOutput(&logOutput)
+						logger := logrusr.New(l)
+						ctx = logr.NewContext(context.Background(), logger)
+					})
+					It("should warn but not fail", func() {
+						ok, err := hasModifiedFiles.validate(ctx, layers, pkgs, dist)
+						Expect(err).ToNot(HaveOccurred())
+						Expect(ok).To(BeTrue())
+						Expect(logOutput.String()).To(ContainSubstring("WARN"))
+					})
+				})
+			})
+		})
+	})
+
+	When("the first layer is empty", func() {
+		var zeroPkgRef map[string]packageFilesRef
+		var zeroLayers []string
+		BeforeEach(func() {
+			zeroPkgRef = pkgRef
+			zeroPkgRef["zerolayer"] = packageFilesRef{
+				LayerFiles:        []string{},
+				LayerPackages:     make(map[string]packageMeta),
+				LayerPackageFiles: make(map[string]string),
+				HasRPMDB:          false,
+			}
+			zeroLayers = append([]string{"zerolayer"}, layers...)
+		})
+		It("should ignore it", func() {
+			ok, err := hasModifiedFiles.validate(context.Background(), zeroLayers, zeroPkgRef, dist)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(ok).To(BeTrue())
+		})
+	})
+
+	When("building the installed file list for installed packages", func() {
+		const (
+			basename = "foobasename"
+			dirname  = "foodirname"
+			dirindex = 0
+		)
+		var goodPkgList []*rpmdb.PackageInfo
+
+		BeforeEach(func() {
+			goodPkgList = []*rpmdb.PackageInfo{
+				{
+					BaseNames:  []string{basename},
+					DirIndexes: []int32{dirindex},
+					DirNames:   []string{dirname},
+				},
+			}
+		})
+		It("should contain all files installed by the package according to its metadata", func() {
+			files, err := installedFileMapWithExclusions(context.TODO(), goodPkgList)
+			Expect(err).ToNot(HaveOccurred())
+
+			_, ok := files[path.Join(dirname, basename)]
+			Expect(ok).To(BeTrue())
+		})
+
+		It("should fail if the rpm is invalid", func() {
+			badPkgList := goodPkgList
+			badPkgList[0].DirNames = []string{dirname, "extradir"}
+			_, err := installedFileMapWithExclusions(context.TODO(), badPkgList)
+			Expect(err).To(HaveOccurred())
+		})
+	})
+
+	When("calling the top level Validate", func() {
+		It("should fail with an invalid ImageReference", func() {
+			passed, err := hasModifiedFiles.Validate(context.TODO(), image.ImageReference{})
+			Expect(err).To(HaveOccurred())
+			Expect(passed).To(BeFalse())
+		})
+	})
+
+	AssertMetaData(&hasModifiedFiles)
+})
diff --git a/internal/runtime/config_test.go b/internal/runtime/config_test.go
index a814a6a..fd432a8 100644
--- a/internal/runtime/config_test.go
+++ b/internal/runtime/config_test.go
@@ -64,12 +64,12 @@ var _ = Describe("Viper to Runtime Config", func() {
 		})
 	})
 
-	It("should only have 20 struct keys for tests to be valid", func() {
+	It("should only have 23 struct keys for tests to be valid", func() {
 		// If this test fails, it means a developer has added or removed
 		// keys from runtime.Config, and so these tests may no longer be
 		// accurate in confirming that the derived configuration from viper
 		// matches.
 		keys := reflect.TypeOf(Config{}).NumField()
-		Expect(keys).To(Equal(22))
+		Expect(keys).To(Equal(23))
 	})
 })
diff --git a/test/containerfiles/container-fails-modifiedfiles-package.Dockerfile b/test/containerfiles/container-fails-modifiedfiles-package.Dockerfile
new file mode 100644
index 0000000..d6c49c0
--- /dev/null
+++ b/test/containerfiles/container-fails-modifiedfiles-package.Dockerfile
@@ -0,0 +1,17 @@
+FROM registry.access.redhat.com/ubi8/ubi:latest
+
+RUN useradd preflightuser
+
+COPY --chown=preflightuser:preflightuser example-license.txt /licenses/
+
+LABEL name="preflight test image" \ 
+      vendor="preflight test vendor" \
+      version="1" \
+      release="1" \
+      summary="testing the preflight tool" \
+      description="test the preflight tool"
+
+RUN dnf remove -y tar
+
+USER preflightuser
+