Repository: tern-tools/tern (language: python)

Issue:
Ensure that Tern produces SPDX docs conformant to NTIA minimum elements
Make sure that Tern's SPDX SBOMs have all of the minimum elements for an SBOM:
    - Supplier Name
    - Component Name
    - Version of the Component
    - Other Unique Identifiers
    - Dependency Relationship
    - Author of SBOM Data
    - Timestamp

More field mapping information here: https://spdx.github.io/spdx-spec/v2.3/how-to-use/#k22-mapping-ntia-minimum-elements-to-spdx-fields

Tooling to check conformance: https://github.com/spdx/ntia-conformance-checker (perhaps this should also be included in the testing for PRs?)


Ensure that Tern produces SPDX docs conformant to NTIA minimum elements
Make sure that Tern's SPDX SBOMs have all of the minimum elements for an SBOM:
    - Supplier Name
    - Component Name
    - Version of the Component
    - Other Unique Identifiers
    - Dependency Relationship
    - Author of SBOM Data
    - Timestamp

More field mapping information here: https://spdx.github.io/spdx-spec/v2.3/how-to-use/#k22-mapping-ntia-minimum-elements-to-spdx-fields

Tooling to check conformance: https://github.com/spdx/ntia-conformance-checker (perhaps this should also be included in the testing for PRs?)




Discussion / hints:
I'm glad to help with the testing component. And if `ntia-conformance-checker` needs to adapt to fit `tern`'s testing needs, please feel free to drop in issues, feature requests, or make PRs!
When I tested a SPDX  document produced by Tern for the `photon:3.0` container image, I got the following results (same results for tag value format):

```
(ternenv) rose@rose-vm:~/ternenv/ntia-conformance-checker/ntia_conformance_checker$ python3 main.py -v --file ../../tern/photon.json

Is this SBOM NTIA minimum element conformant? False

Individual elements                            | Status
-------------------------------------------------------
All component names provided?                  | True
All component versions provided?               | False
All component identifiers provided?            | True
All component suppliers provided?              | False
SBOM author name provided?                     | False
SBOM creation timestamp provided?              | True
Dependency relationships provided?             | True

Components missing a version: 5e94941e3961b26645fbfdc71a59d439537b98417546bfdab35fa074f121eb15

Components missing an supplier: photon,5e94941e3961b26645fbfdc71a59d439537b98417546bfdab35fa074f121eb15,bash,bzip2-libs,ca-certificates,ca-certificates-pki,curl,curl-libs,e2fsprogs-libs,elfutils-libelf,expat,expat-libs,filesystem,glibc,krb5,libcap,libdb,libgcc,libmetalink,libsolv,libssh2,lua,ncurses-libs,nspr,nss-libs,openssl,photon-release,photon-repos,popt,readline,rpm-libs,sqlite-libs,tdnf,tdnf-cli-libs,toybox,xz-libs,zlib,zstd-libs


```
@rnjudge, does that look right to you? You can get more detailed output by adding a `-v` flag. You can also get JSON output using `--output json` flag. 
@jspeed-meyers not all of it looks correct. Opening an issue/question in the repo now :)
To summarize, the missing NTIA minimum info in Tern reports is as follows:

1) The layer Package does not have a version associated with it.
```
PackageName: 5e94941e3961b26645fbfdc71a59d439537b98417546bfdab35fa074f121eb15
SPDXID: SPDXRef-9b5d4fa77c
PackageFileName: 5e94941e3961b26645fbfdc71a59d439537b98417546bfdab35fa074f121eb15
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageChecksum: SHA256: 9b5d4fa77c08cc6b026c7a37524154531daa103658948bfd49932c4e77abeda6
PackageLicenseConcluded: NOASSERTION
PackageLicenseDeclared: NOASSERTION
PackageCopyrightText: NOASSERTION
PackageComment: <text>
Layer 1:

```

2) Packages do not have `Package Supplier` (7.5) information

3) I believe the SBOM author provided status is incorrect. See https://github.com/spdx/ntia-conformance-checker/issues/52
I'm glad to help with the testing component. And if `ntia-conformance-checker` needs to adapt to fit `tern`'s testing needs, please feel free to drop in issues, feature requests, or make PRs!
When I tested a SPDX  document produced by Tern for the `photon:3.0` container image, I got the following results (same results for tag value format):

```
(ternenv) rose@rose-vm:~/ternenv/ntia-conformance-checker/ntia_conformance_checker$ python3 main.py -v --file ../../tern/photon.json

Is this SBOM NTIA minimum element conformant? False

Individual elements                            | Status
-------------------------------------------------------
All component names provided?                  | True
All component versions provided?               | False
All component identifiers provided?            | True
All component suppliers provided?              | False
SBOM author name provided?                     | False
SBOM creation timestamp provided?              | True
Dependency relationships provided?             | True

Components missing a version: 5e94941e3961b26645fbfdc71a59d439537b98417546bfdab35fa074f121eb15

Components missing an supplier: photon,5e94941e3961b26645fbfdc71a59d439537b98417546bfdab35fa074f121eb15,bash,bzip2-libs,ca-certificates,ca-certificates-pki,curl,curl-libs,e2fsprogs-libs,elfutils-libelf,expat,expat-libs,filesystem,glibc,krb5,libcap,libdb,libgcc,libmetalink,libsolv,libssh2,lua,ncurses-libs,nspr,nss-libs,openssl,photon-release,photon-repos,popt,readline,rpm-libs,sqlite-libs,tdnf,tdnf-cli-libs,toybox,xz-libs,zlib,zstd-libs


```
@rnjudge, does that look right to you? You can get more detailed output by adding a `-v` flag. You can also get JSON output using `--output json` flag. 
@jspeed-meyers not all of it looks correct. Opening an issue/question in the repo now :)
To summarize, the missing NTIA minimum info in Tern reports is as follows:

1) The layer Package does not have a version associated with it.
```
PackageName: 5e94941e3961b26645fbfdc71a59d439537b98417546bfdab35fa074f121eb15
SPDXID: SPDXRef-9b5d4fa77c
PackageFileName: 5e94941e3961b26645fbfdc71a59d439537b98417546bfdab35fa074f121eb15
PackageDownloadLocation: NOASSERTION
FilesAnalyzed: false
PackageChecksum: SHA256: 9b5d4fa77c08cc6b026c7a37524154531daa103658948bfd49932c4e77abeda6
PackageLicenseConcluded: NOASSERTION
PackageLicenseDeclared: NOASSERTION
PackageCopyrightText: NOASSERTION
PackageComment: <text>
Layer 1:

```

2) Packages do not have `Package Supplier` (7.5) information

3) I believe the SBOM author provided status is incorrect. See https://github.com/spdx/ntia-conformance-checker/issues/52

Fix patch:
diff --git a/tern/analyze/default/bundle.py b/tern/analyze/default/bundle.py
index d9a9f551..893ee00b 100644
--- a/tern/analyze/default/bundle.py
+++ b/tern/analyze/default/bundle.py
@@ -52,7 +52,8 @@ def convert_to_pkg_dicts(attr_lists):
                'pkg_licenses': 'pkg_licenses',
                'files': 'files',
                'src_name': 'source_names',
-               'src_version': 'source_versions'}
+               'src_version': 'source_versions',
+               'pkg_supplier': 'pkg_suppliers'}
     pkg_list = []
     len_names = len(attr_lists['names'])
     # make a list of keys that correspond with package property names
diff --git a/tern/analyze/default/command_lib/base.yml b/tern/analyze/default/command_lib/base.yml
index cc334d45..b24b67b4 100644
--- a/tern/analyze/default/command_lib/base.yml
+++ b/tern/analyze/default/command_lib/base.yml
@@ -88,6 +88,14 @@ dpkg:
         container:
           - "dpkg-query -W -f '${Version}\n'"
     delimiter: "\n"
+  pkg_suppliers:
+    invoke:
+      1:
+        container:
+          - "distro=`/bin/cat /etc/os-release | grep NAME | sed -n '2p' | cut -f 2 -d '=' | cut -d '\"' -f2`"
+          - "pkgs=`dpkg-query -W -f '${Package}\n'`"
+          - "for p in $pkgs; do echo $distro; done"
+    delimiter: "\n"
   source_names:
     invoke:
       1:
@@ -148,6 +156,13 @@ apk:
           - "pkgs=`apk info 2>/dev/null`"
           - "for p in $pkgs; do lic=`apk info $p 2>/dev/null | head -1 | awk '{print $1}'`; echo $lic | sed -e \"s/^$p-//\"; done"
     delimiter: "\n"
+  pkg_suppliers:
+    invoke:
+      1:
+        container:
+          - "pkgs=`apk info 2>/dev/null`"
+          - "for p in $pkgs; do echo 'Alpine Linux'; done"
+    delimiter: "\n"
   licenses:
     invoke:
       1:
@@ -192,6 +207,13 @@ pacman:
         container:
           - 'pacman -Q 2>/dev/null | cut -f 2 -d " "'
     delimiter: "\n"
+  pkg_suppliers:
+    invoke:
+      1:
+        container:
+          - 'pkgs=`pacman -Qq 2>/dev/null`'
+          - 'for p in $pkgs; do echo "Arch Linux"; done'
+    delimiter: "\n"
   licenses:
     invoke:
       1:
@@ -238,6 +260,14 @@ rpm:
         container:
           - 'rpm --query --all --queryformat "%{version}\n" 2>/dev/null'
     delimiter: "\n"
+  pkg_suppliers:
+    invoke:
+      1:
+        container:
+          - "distro=`/bin/cat /etc/os-release | grep NAME | sed -n '1p' | cut -f 2 -d '=' | cut -d '\"' -f2`"
+          - "pkgs=`rpm --query --all --queryformat \"%{name}\n\" 2>/dev/null`"
+          - "for p in $pkgs; do echo $distro; done"
+    delimiter: "\n"
   licenses:
     invoke:
       1:
@@ -302,6 +332,13 @@ pip:
           - "pkgs=`pip list --format=freeze 2> /dev/null | cut -f1 -d'='`"
           - "for p in $pkgs; do pip show $p 2> /dev/null | head -2 | tail -1 | cut -f2 -d' '; done"
     delimiter: "\n"
+  pkg_suppliers:
+    invoke:
+      1:
+        container:
+          - "pkgs=`pip list --format=freeze 2> /dev/null | cut -f1 -d'='`"
+          - "for p in $pkgs; do echo 'PyPI'; done"
+    delimiter: "\n"
   licenses:
     invoke:
       1:
@@ -345,6 +382,13 @@ pip3:
           - "pkgs=`pip3 list --format=freeze 2> /dev/null | cut -f1 -d'='`"
           - "for p in $pkgs; do pip3 show $p 2> /dev/null | head -2 | tail -1 | cut -f2 -d' '; done"
     delimiter: "\n"
+  pkg_suppliers:
+    invoke:
+      1:
+        container:
+          - "pkgs=`pip3 list --format=freeze 2> /dev/null | cut -f1 -d'='`"
+          - "for p in $pkgs; do echo 'PyPI'; done"
+    delimiter: "\n"
   licenses:
     invoke:
       1:
diff --git a/tern/analyze/default/command_lib/command_lib.py b/tern/analyze/default/command_lib/command_lib.py
index 18971227..4296c8b1 100644
--- a/tern/analyze/default/command_lib/command_lib.py
+++ b/tern/analyze/default/command_lib/command_lib.py
@@ -37,9 +37,10 @@
     command_lib['snippets'] = yaml.safe_load(f)
 # list of package information keys that the command library can accomodate
 base_keys = {'names', 'versions', 'licenses', 'source_names',
-             'source_versions', 'copyrights', 'proj_urls', 'srcs', 'files'}
+             'source_versions', 'pkg_suppliers', 'copyrights',
+             'proj_urls', 'srcs', 'files'}
 package_keys = {'name', 'version', 'license', 'src_name', 'src_version',
-                'copyright', 'proj_url', 'src', 'files'}
+               'pkg_supplier', 'copyright', 'proj_url', 'src', 'files'}
 
 # global logger
 logger = logging.getLogger(constants.logger_name)
diff --git a/tern/classes/package.py b/tern/classes/package.py
index 9d03d3d1..bd0487f5 100644
--- a/tern/classes/package.py
+++ b/tern/classes/package.py
@@ -24,6 +24,8 @@ class Package:
         pkg_licenses: all licenses found within a package
         src_name: the source package associated with the binary package
         src_version: the source package version
+        pkg_supplier: the package distributor according to SPDX 7.5;
+        required to create NTIA compliant SPDX docs
 
     methods:
         to_dict: returns a dict representation of the instance
@@ -47,6 +49,7 @@ def __init__(self, name):
         self.__pkg_format = ''
         self.__src_name = ''
         self.__src_version = ''
+        self.__pkg_supplier = ''
 
     @property
     def name(self):
@@ -144,6 +147,14 @@ def src_version(self):
     def src_version(self, src_version):
         self.__src_version = src_version
 
+    @property
+    def pkg_supplier(self):
+        return self.__pkg_supplier
+
+    @pkg_supplier.setter
+    def pkg_supplier(self, pkg_supplier):
+        self.__pkg_supplier = pkg_supplier
+
     def get_file_paths(self):
         """Return a list of paths of all the files in a package"""
         return [f.path for f in self.__files]
@@ -217,6 +228,7 @@ def fill(self, package_dict):
             files: <package files>
             src_name: <source package>
             src_ver: <source package version>
+            pkg_supplier: <package distributor/supplier>
         the way to use this method is to instantiate the class with the
         name and then give it a package dictionary to fill in the rest
         return true if package name is the same as the one used to instantiate
diff --git a/tern/formats/spdx/spdx.py b/tern/formats/spdx/spdx.py
index 8c451375..d9caff75 100644
--- a/tern/formats/spdx/spdx.py
+++ b/tern/formats/spdx/spdx.py
@@ -21,7 +21,8 @@ def package(self):
                 'copyright': 'PackageCopyrightText',
                 'download_url': 'PackageDownloadLocation',
                 'src_name': 'SourcePackageName',
-                'src_version': 'SourcePackageVersion'}
+                'src_version': 'SourcePackageVersion',
+                'pkg_supplier': 'PackageSupplier'}
 
     def image_layer(self):
         return {'tar_file': 'PackageFileName'}
diff --git a/tern/formats/spdx/spdxjson/image_helpers.py b/tern/formats/spdx/spdxjson/image_helpers.py
index bdb25c3a..bceae8ae 100644
--- a/tern/formats/spdx/spdxjson/image_helpers.py
+++ b/tern/formats/spdx/spdxjson/image_helpers.py
@@ -84,6 +84,7 @@ def get_image_dict(image_obj, template):
         'name': mapping['PackageName'],
         'SPDXID': spdx_common.get_image_spdxref(image_obj),
         'versionInfo': mapping['PackageVersion'],
+        'supplier': 'NOASSERTION',  # always NOASSERTION
         'downloadLocation': 'NOASSERTION',  # always NOASSERTION
         'filesAnalyzed': False,  # always false
         'licenseConcluded': 'NOASSERTION',  # always NOASSERTION
diff --git a/tern/formats/spdx/spdxjson/layer_helpers.py b/tern/formats/spdx/spdxjson/layer_helpers.py
index a9d0fffc..9128fcb2 100644
--- a/tern/formats/spdx/spdxjson/layer_helpers.py
+++ b/tern/formats/spdx/spdxjson/layer_helpers.py
@@ -150,6 +150,7 @@ def get_layer_dict(layer_obj):
         'name': os.path.basename(layer_obj.tar_file),
         'SPDXID': spdx_common.get_layer_spdxref(layer_obj),
         'versionInfo': layer_obj.layer_index,
+        'supplier': 'NOASSERTION', # always NOASSERTION
         'packageFileName': layer_obj.tar_file,
         'downloadLocation': 'NONE',
         'filesAnalyzed': bool(layer_obj.files_analyzed),
diff --git a/tern/formats/spdx/spdxjson/package_helpers.py b/tern/formats/spdx/spdxjson/package_helpers.py
index 3507b4f4..d9f941f2 100644
--- a/tern/formats/spdx/spdxjson/package_helpers.py
+++ b/tern/formats/spdx/spdxjson/package_helpers.py
@@ -58,12 +58,14 @@ def get_package_dict(package, template):
     JSON dictionary representation of the package. The analyzed files will
     go in a separate dictionary for the JSON document.'''
     mapping = package.to_dict(template)
+    supplier_str = 'Organization: ' + mapping['PackageSupplier']
     pkg_ref, _ = spdx_common.get_package_spdxref(package)
     package_dict = {
         'name': mapping['PackageName'],
         'SPDXID': pkg_ref,
         'versionInfo': mapping['PackageVersion'] if mapping['PackageVersion']
         else 'NOASSERTION',
+        'supplier': supplier_str if mapping['PackageSupplier'] else 'NOASSERTION', 
         'downloadLocation': mapping['PackageDownloadLocation'] if
         mapping['PackageDownloadLocation'] else 'NOASSERTION',
         'filesAnalyzed': False,  # always false for packages
diff --git a/tern/formats/spdx/spdxtagvalue/image_helpers.py b/tern/formats/spdx/spdxtagvalue/image_helpers.py
index 645f0374..ac7f32e1 100644
--- a/tern/formats/spdx/spdxtagvalue/image_helpers.py
+++ b/tern/formats/spdx/spdxtagvalue/image_helpers.py
@@ -101,6 +101,8 @@ def get_image_block(image_obj, template):
     block += 'SPDXID: {}\n'.format(spdx_common.get_image_spdxref(image_obj))
     # Package Version
     block += 'PackageVersion: {}\n'.format(mapping['PackageVersion'])
+    # Package Supplier (always NOASSERTION)
+    block += 'PackageSupplier: NOASSERTION\n'
     # Package Download Location (always NOASSERTION)
     block += 'PackageDownloadLocation: NOASSERTION\n'
     # Files Analyzed (always false)
diff --git a/tern/formats/spdx/spdxtagvalue/layer_helpers.py b/tern/formats/spdx/spdxtagvalue/layer_helpers.py
index d8239ba0..abfc2ae8 100644
--- a/tern/formats/spdx/spdxtagvalue/layer_helpers.py
+++ b/tern/formats/spdx/spdxtagvalue/layer_helpers.py
@@ -117,6 +117,8 @@ def get_layer_block(layer_obj, template):
     block += 'SPDXID: {}\n'.format(spdx_common.get_layer_spdxref(layer_obj))
     # Package Version. For Layer objects, this is just the layer_index
     block += 'PackageVersion: {}\n'.format(layer_obj.layer_index)
+    # Package Supplier (always NOASSERTION)
+    block += 'PackageSupplier: NOASSERTION\n'
     # Package File Name
     block += 'PackageFileName: {}\n'.format(layer_obj.tar_file)
     # Package Download Location (always NONE for layers)
diff --git a/tern/formats/spdx/spdxtagvalue/package_helpers.py b/tern/formats/spdx/spdxtagvalue/package_helpers.py
index 452bd87e..e2fefe43 100644
--- a/tern/formats/spdx/spdxtagvalue/package_helpers.py
+++ b/tern/formats/spdx/spdxtagvalue/package_helpers.py
@@ -86,6 +86,11 @@ def get_package_block(package_obj, template):
     # Package Version
     if mapping['PackageVersion']:
         block += 'PackageVersion: {}\n'.format(mapping['PackageVersion'])
+    # Package Supplier
+    if mapping['PackageSupplier']:
+        block += 'PackageSupplier: Organization: {}\n'.format(mapping['PackageSupplier'])
+    else:
+        block += 'PackageSupplier: NOASSERTION\n'
     # Package Download Location
     if mapping['PackageDownloadLocation']:
         block += 'PackageDownloadLoaction: {}\n'.format(


Test patch:
diff --git a/tests/test_class_package.py b/tests/test_class_package.py
index e92c0d40..ac2f5a2f 100644
--- a/tests/test_class_package.py
+++ b/tests/test_class_package.py
@@ -25,6 +25,7 @@ def setUp(self):
         self.p1.pkg_format = 'deb'
         self.p1.src_name = 'p1src'
         self.p1.src_version = '1.0'
+        self.p1.pkg_supplier = 'VMware'
 
         self.p2 = Package('p2')
 
@@ -65,6 +66,8 @@ def testSetters(self):
                          ('a', '/usr/abc/a'))
         self.assertEqual((self.p2.files[1].name, self.p2.files[1].path),
                          ('b', '/usr/abc/b'))
+        self.p2.pkg_supplier = 'Linux Foundation'
+        self.assertEqual(self.p2.pkg_supplier, 'Linux Foundation')
 
     def testGetters(self):
         self.assertEqual(self.p1.name, 'p1')
@@ -78,6 +81,7 @@ def testGetters(self):
         self.assertEqual(self.p1.pkg_format, 'deb')
         self.assertEqual(self.p1.src_name, 'p1src')
         self.assertEqual(self.p1.src_version, '1.0')
+        self.assertEqual(self.p1.pkg_supplier, 'VMware')
 
     def testAddFile(self):
         p1 = Package('package')
@@ -130,6 +134,7 @@ def testToDict(self):
         self.assertEqual(a_dict['pkg_format'], 'deb')
         self.assertEqual(a_dict['src_name'], 'p1src')
         self.assertEqual(a_dict['src_version'], '1.0')
+        self.assertEqual(a_dict['pkg_supplier'], 'VMware')
 
     def testToDictTemplate(self):
         template1 = TestTemplate1()
@@ -175,7 +180,8 @@ def testFill(self):
                             {'name': 'b.txt', 'path': '/lib/b.txt'}],
                   'pkg_format': 'rpm',
                   'src_name': 'p1src',
-                  'src_version': '1.0'
+                  'src_version': '1.0',
+                  'pkg_supplier': 'VMware'
                   }
         p = Package('p1')
         p.fill(p_dict)