Repository: aboutcode-org/purldb (language: python)

Issue:
Create doc/tutorial for matching with MatchCode
We should deploy a decent public demo system and a create usage documentation for MatchCode in particular:

- [x] Deploy a running MatchCode system publicly accessible for demonstration with some decent number of packages.
- [x] Create and publish on RTD a usage and deployment documentation for code matching with MatchCode 
Create doc/tutorial for matching with MatchCode
We should deploy a decent public demo system and a create usage documentation for MatchCode in particular:

- [x] Deploy a running MatchCode system publicly accessible for demonstration with some decent number of packages.
- [x] Create and publish on RTD a usage and deployment documentation for code matching with MatchCode 
Remove unnecessary api endpoints
We should consider removing some of the API endpoints from purldb:

- approximate_directory_content_index
- approximate_directory_structure_index
- exact_file_index
- exact_package_archive_index
- cditems
- on_demand_queue

These endpoints are no longer needed or should not be exposed
Create doc/tutorial for matching with MatchCode
We should deploy a decent public demo system and a create usage documentation for MatchCode in particular:

- [x] Deploy a running MatchCode system publicly accessible for demonstration with some decent number of packages.
- [x] Create and publish on RTD a usage and deployment documentation for code matching with MatchCode 
Create doc/tutorial for matching with MatchCode
We should deploy a decent public demo system and a create usage documentation for MatchCode in particular:

- [x] Deploy a running MatchCode system publicly accessible for demonstration with some decent number of packages.
- [x] Create and publish on RTD a usage and deployment documentation for code matching with MatchCode 
Create doc/tutorial for matching with MatchCode
We should deploy a decent public demo system and a create usage documentation for MatchCode in particular:

- [x] Deploy a running MatchCode system publicly accessible for demonstration with some decent number of packages.
- [x] Create and publish on RTD a usage and deployment documentation for code matching with MatchCode 
Create doc/tutorial for matching with MatchCode
We should deploy a decent public demo system and a create usage documentation for MatchCode in particular:

- [x] Deploy a running MatchCode system publicly accessible for demonstration with some decent number of packages.
- [x] Create and publish on RTD a usage and deployment documentation for code matching with MatchCode 
purl2all: Create doc for PURL services accessible through the API
We want to make sure that:
- [x] Ensure that the latest PurlDB API deployed 
  -  This also provides the API documentation

Create doc/tutorial for matching with MatchCode
We should deploy a decent public demo system and a create usage documentation for MatchCode in particular:

- [x] Deploy a running MatchCode system publicly accessible for demonstration with some decent number of packages.
- [x] Create and publish on RTD a usage and deployment documentation for code matching with MatchCode 
Remove unnecessary api endpoints
We should consider removing some of the API endpoints from purldb:

- approximate_directory_content_index
- approximate_directory_structure_index
- exact_file_index
- exact_package_archive_index
- cditems
- on_demand_queue

These endpoints are no longer needed or should not be exposed
Create doc/tutorial for matching with MatchCode
We should deploy a decent public demo system and a create usage documentation for MatchCode in particular:

- [x] Deploy a running MatchCode system publicly accessible for demonstration with some decent number of packages.
- [x] Create and publish on RTD a usage and deployment documentation for code matching with MatchCode 
Create doc/tutorial for matching with MatchCode
We should deploy a decent public demo system and a create usage documentation for MatchCode in particular:

- [x] Deploy a running MatchCode system publicly accessible for demonstration with some decent number of packages.
- [x] Create and publish on RTD a usage and deployment documentation for code matching with MatchCode 
Create doc/tutorial for matching with MatchCode
We should deploy a decent public demo system and a create usage documentation for MatchCode in particular:

- [x] Deploy a running MatchCode system publicly accessible for demonstration with some decent number of packages.
- [x] Create and publish on RTD a usage and deployment documentation for code matching with MatchCode 


Discussion / hints:

Fix patch:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 4f09a775..65aec239 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -5,6 +5,10 @@ Next Release
 ----------------
 
 - Improve web template for API web page https://github.com/nexB/purldb/issues/132
+- The API endpoints for ``approximate_directory_content_index``,
+  ``approximate_directory_structure_index``, ``exact_file_index``,
+  ``exact_package_archive_index``, ``cditems``, ``on_demand_queue`` have been
+  removed.
 
 
 v5.0.0
@@ -16,7 +20,7 @@ v5.0.0
 - Support indexing of PURLs listed in https://github.com/nexB/purldb/issues/326,
   https://github.com/nexB/purldb/issues/327, https://github.com/nexB/purldb/issues/328,
   https://github.com/nexB/purldb/issues/329 and https://github.com/nexB/purldb/issues/356.
-- Support ``addon_pipelines`` for symbol and string collection in ``/api/collect`` endpoint. https://github.com/nexB/purldb/pull/393 
+- Support ``addon_pipelines`` for symbol and string collection in ``/api/collect`` endpoint. https://github.com/nexB/purldb/pull/393
 - Store ``source_symbols`` and ``source_strings`` in ``extra_data`` field. https://github.com/nexB/purldb/pull/351
 
 
diff --git a/clearcode/api.py b/clearcode/api.py
deleted file mode 100644
index 42bcd6c1..00000000
--- a/clearcode/api.py
+++ /dev/null
@@ -1,80 +0,0 @@
-# -*- coding: utf-8 -*-
-#
-# Copyright (c) nexB Inc. and others. All rights reserved.
-#
-# ClearCode is a free software tool from nexB Inc. and others.
-# Visit https://github.com/nexB/clearcode-toolkit/ for support and download.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-import base64
-
-from django.urls import include, re_path
-from rest_framework import routers
-from rest_framework import serializers
-from rest_framework import viewsets
-
-from clearcode.models import CDitem
-
-
-class CDitemContentFieldSerializer(serializers.Field):
-    """
-    Custom Field Serializer used to translate between Django ORM binary field and
-    base64-encoded string
-    """
-    def to_representation(self, obj):
-        return base64.b64encode(obj).decode('utf-8')
-
-    def to_internal_value(self, data):
-        return base64.b64decode(data)
-
-
-class CDitemSerializer(serializers.HyperlinkedModelSerializer):
-    """
-    Custom Serializer used to serialize the CDitem model
-    """
-    content = CDitemContentFieldSerializer(required=False)
-    class Meta:
-        model = CDitem
-        fields = (
-            'path',
-            'uuid',
-            'content',
-            'last_modified_date',
-            'last_map_date',
-            'map_error',
-        )
-
-
-class CDitemViewSet(viewsets.ModelViewSet):
-    """
-    API endpoint that allows CDitems to be viewed.
-    """
-    serializer_class = CDitemSerializer
-    lookup_field = 'uuid'
-
-    def get_queryset(self):
-        last_modified_date = self.request.query_params.get('last_modified_date', None)
-        queryset = CDitem.objects.all()
-
-        if last_modified_date:
-            queryset = CDitem.objects.modified_after(last_modified_date)
-
-        return queryset
-
-
-router = routers.DefaultRouter()
-router.register(r'cditems', CDitemViewSet, 'cditems')
-
-urlpatterns = [
-    re_path('^api/', include((router.urls, 'api'))),
-]
diff --git a/docs/source/how-to-guides/index.rst b/docs/source/how-to-guides/index.rst
index 96414eb6..425f1f65 100644
--- a/docs/source/how-to-guides/index.rst
+++ b/docs/source/how-to-guides/index.rst
@@ -7,5 +7,6 @@ through specific use cases:
 .. toctree::
    :maxdepth: 2
 
+   installation
    matchcode
    tutorial_symbol_and_string_collection
diff --git a/docs/source/how-to-guides/installation.rst b/docs/source/how-to-guides/installation.rst
new file mode 100644
index 00000000..018ef2af
--- /dev/null
+++ b/docs/source/how-to-guides/installation.rst
@@ -0,0 +1,104 @@
+Installation
+============
+
+This article will detail the steps needed to set up a PurlDB instance with
+MatchCode and package scanning using Docker.
+
+
+You will first need to create a Docker network for PurlDB. This is necessary for
+MatchCode.io to acess PurlDB's database.::
+
+    docker network create purldb
+
+
+PurlDB
+------
+
+  - Clone the PurlDB repo, create an environment file, and build the Docker image::
+
+        git clone https://github.com/nexB/scancode.io.git && cd scancode.io
+        make envfile
+        docker compose build
+
+    Note: if you are running PurlDB on a remote server, be sure to update
+    ``127.0.0.1`` and ``localhost`` to the address of your server in
+    ``docker-compose.yml`` ::
+
+        - "traefik.http.routers.web.rule=
+          Host(`127.0.0.1`)
+          || Host(`localhost`)"
+
+  - Create an ``.env`` file::
+
+        make envfile
+
+  - Move this ``.env`` file to ``/etc/purldb``:::
+
+        sudo mkdir -p /etc/purldb
+        sudo cp .env /etc/purldb
+
+  - Run your image as a container::
+
+        docker compose up
+
+  - You will also need to create a user account for the scan worker::
+
+        docker compose exec web purldb create-scan-queue-worker-user scan_worker_1
+
+    Save this API key for use in the ``.env`` file of the Package scan worker
+
+  - At this point, PurlDB should be running at port 80 on your Docker host. Go
+    to the PurlDB address on a web browser to access the web UI.
+
+
+Package Scan Worker
+-------------------
+
+  - This should be installed on another machine, if possible.
+
+  - Download the latest release of ScanCode.io at
+    https://github.com/nexB/scancode.io/releases and follow the Docker
+    installation instructions at
+    https://scancodeio.readthedocs.io/en/latest/installation.html
+
+  - In the ``.env`` file for ScanCode.io, set the following environment
+    variables::
+
+        PURLDB_URL = env.str("PURLDB_URL", default="")
+        PURLDB_API_KEY = env.str("PURLDB_API_KEY", default="")
+
+    Use the address of your PurlDB server for ``PURLDB_URL`` and the
+    ``PURLDB_API_KEY`` generated from the PurlDB install.
+
+  - Move this ``.env`` file to ``/etc/scancodeio``:::
+
+        sudo mkdir -p /etc/scancodeio
+        sudo cp .env /etc/scancodeio
+
+  - To run the worker, you must reference the
+    ``docker-compose.purldb-scan-worker.yml`` Compose file::
+
+        docker compose -f docker-compose.purldb-scan-worker.yml up
+
+
+MatchCode.io
+------------
+
+  - This should be installed on the same machine as PurlDB.
+
+  - Clone the PurlDB repo, create an environment file, and build the Docker
+    image using the ``docker-compose.matchcodeio.yml`` Compose file::
+
+        git clone https://github.com/nexB/scancode.io.git && cd scancode.io
+        make envfile
+        docker compose -f docker-compose.matchcodeio.yml build
+
+  - Move this ``.env`` file to ``/etc/matchcodeio``:::
+
+        sudo mkdir -p /etc/matchcodeio
+        sudo cp .env /etc/matchcodeio
+
+  - Run your image as a container::
+
+        docker compose -f docker-compose.matchcodeio.yml up
+
diff --git a/docs/source/matchcode/matchcode-pipeline.rst b/docs/source/matchcode/matchcode-pipeline.rst
index a2109b70..a2f8aa89 100644
--- a/docs/source/matchcode/matchcode-pipeline.rst
+++ b/docs/source/matchcode/matchcode-pipeline.rst
@@ -1,2 +1,145 @@
-Matching pipeline
-=================
+Code Matching
+=============
+
+The aim of this tutorial is to show how to use the MatchCode.io API to perform
+code matching on an archive of files.
+
+.. note::
+    This tutorial assumes that you have a working installation of PurlDB. If you
+    don't, please refer to the `installation
+    <../purldb/overview.html#installation>`_ page.
+
+Throughout this tutorial, we will use ``pkg:npm/deep-equal@1.0.1`` and a
+modified copy of ``index.js`` from it.
+
+.. raw:: html
+
+   <details>
+   <summary><a>BPF.cpp</a></summary>
+   </br>
+
+.. code-block:: javascript
+
+    var pSlice = Array.prototype.slice;
+    var objectKeys = require('./lib/keys.js');
+    var isArguments = require('./lib/is_arguments.js');
+
+    var deepEqual = module.exports = function (actual, expected, opts) {
+    if (!opts) opts = {};
+    // 7.1. All identical values are equivalent, as determined by ===.
+    if (actual === expected) {
+        return true;
+
+    } else if (actual instanceof Date && expected instanceof Date) {
+        return actual.getTime() === expected.getTime();
+
+    // 7.3. Other pairs that do not both pass typeof value == 'object',
+    // equivalence is determined by ==.
+    } else if (!actual || !expected || typeof actual != 'object' && typeof expected != 'object') {
+        return opts.strict ? actual === expected : actual == expected;
+
+    // 7.4. For all other Object pairs, including Array objects, equivalence is
+    // determined by having the same number of owned properties (as verified
+    // with Object.prototype.hasOwnProperty.call), the same set of keys
+    // (although not necessarily the same order), equivalent values for every
+    // corresponding key, and an identical 'prototype' property. Note: this
+    // accounts for both named and indexed properties on Arrays.
+    } else {
+        return objEquiv(actual, expected, opts);
+    }
+    }
+
+    function isBuffer (x) {
+    if (!x || typeof x !== 'object' || typeof x.length !== 'number') return false;
+    if (typeof x.copy !== 'function' || typeof x.slice !== 'function') {
+        return false;
+    }
+    if (x.length > 0 && typeof x[0] !== 'number') return false;
+    return true;
+    }
+
+    function objEquiv(a, b, opts) {
+    var i, key;
+    if (isUndefinedOrNull(a) || isUndefinedOrNull(b))
+        return false;
+    // an identical 'prototype' property.
+    if (a.prototype !== b.prototype) return false;
+    //~~~I've managed to break Object.keys through screwy arguments passing.
+    //   Converting to array solves the problem.
+    if (isArguments(a)) {
+        if (!isArguments(b)) {
+        return false;
+        }
+        a = pSlice.call(a);
+        b = pSlice.call(b);
+        return deepEqual(a, b, opts);
+    }
+    if (isBuffer(a)) {
+        if (!isBuffer(b)) {
+        return false;
+        }
+        if (a.length !== b.length) return false;
+        for (i = 0; i < a.length; i++) {
+        if (a[i] !== b[i]) return false;
+        }
+        return true;
+    }
+    try {
+        var ka = objectKeys(a),
+            kb = objectKeys(b);
+    } catch (e) {//happens when one is a string literal and the other isn't
+        return false;
+    }
+    // having the same number of owned properties (keys incorporates
+    // hasOwnProperty)
+    if (ka.length != kb.length)
+        return false;
+    //the same set of keys (although not necessarily the same order),
+    ka.sort();
+    kb.sort();
+    //~~~cheap key test
+    for (i = ka.length - 1; i >= 0; i--) {
+        if (ka[i] != kb[i])
+        return false;
+    }
+    //equivalent values for every corresponding key, and
+    //~~~possibly expensive deep test
+    for (i = ka.length - 1; i >= 0; i--) {
+        key = ka[i];
+        if (!deepEqual(a[key], b[key], opts)) return false;
+    }
+    return typeof a === typeof b;
+    }
+
+.. raw:: html
+
+   </details>
+   </br>
+
+
+Instructions
+------------
+
+- First, index the package ``pkg:npm/deep-equal@1.0.1``::
+
+    /api/collect/?purl=pkg:npm/deep-equal@1.0.1
+
+- Visit ``/api/matching/`` and POST a new matching request.
+
+.. note::
+    Whether you follow this tutorial and previous instructions using cURL or
+    Python script, the final results should be the same.
+
+.. code-block:: bash
+
+    api_url="http://localhost/api/matching/"
+    content_type="Content-Type: application/json"
+    upload_file="upload_file=@<path to index.js>"
+    curl -X POST "$api_url" -H "$content_type" -F "$upload_file"
+
+- When the match has completed, the results can be seen at ``/api/matching/<uuid>/results``
+
+.. code-block:: json
+
+    {
+    }
diff --git a/docs/source/purldb/index.rst b/docs/source/purldb/index.rst
index 21a31890..0faee2a7 100644
--- a/docs/source/purldb/index.rst
+++ b/docs/source/purldb/index.rst
@@ -6,3 +6,4 @@ PURLdb
 
    overview
    symbol_and_string_collection
+   rest_api
diff --git a/docs/source/purldb/rest_api.rst b/docs/source/purldb/rest_api.rst
new file mode 100644
index 00000000..8d909b06
--- /dev/null
+++ b/docs/source/purldb/rest_api.rst
@@ -0,0 +1,1171 @@
+.. _rest_api:
+
+REST API
+========
+
+To get started with the REST API, visit the **PurlDB API endpoints** at
+http://localhost/api/ or http://localhost:8001/api/ if you run on a
+local development setup.
+
+.. _rest_api_authentication:
+
+Authentication
+--------------
+
+When the authentication setting ``PURLDB_SETTINGS_REQUIRE_AUTHENTICATION``
+is enabled on a PurlDB instance (disabled by default), you will have to include
+an authentication token ``API key`` in the Authorization HTTP header of each request.
+
+The key should be prefixed by the string literal "Token" with whitespace
+separating the two strings. For example::
+
+    Authorization: Token abcdef123456
+
+.. warning::
+    Your API key is like a password and should be treated with the same care.
+
+Example of a cURL-style command line using an API Key for authentication:
+
+.. code-block:: console
+
+    curl -X GET http://localhost/api/packages/ -H "Authorization:Token abcdef123456"
+
+Example of a Python script:
+
+.. code-block:: python
+
+    import requests
+
+    api_url = "http://localhost/api/packages/"
+    headers = {
+        "Authorization": "Token abcdef123456",
+    }
+    params = {
+        "page": "2",
+    }
+    response = requests.get(api_url, headers=headers, params=params)
+    response.json()
+
+packages
+--------
+
+package list
+------------
+
+An API endpoint that provides the ability to list and get packages.
+
+``GET /api/packages/``
+
+.. code-block:: json
+
+    {
+        "count": 1,
+        "next": null,
+        "previous": null,
+        "results": [
+            {
+                "url": "https://public.purldb.io/api/packages/0bbdcf88-ad07-4970-9272-7d5f4c82cc7b/",
+                "uuid": "0bbdcf88-ad07-4970-9272-7d5f4c82cc7b",
+                "filename": "elasticsearch-cli-7.17.9.jar",
+                "package_sets": [
+                    {
+                        "uuid": "6d606be2-57c7-429a-a62e-d07833662c38",
+                        "packages": [
+                            "https://public.purldb.io/api/packages/0bbdcf88-ad07-4970-9272-7d5f4c82cc7b/",
+                            "https://public.purldb.io/api/packages/055a7bab-38c6-4a1d-bb57-777b72ea4c99/"
+                        ]
+                    }
+                ],
+                "package_content": "binary",
+                "purl": "pkg:maven/org.elasticsearch/elasticsearch-cli@7.17.9",
+                "type": "maven",
+                "namespace": "org.elasticsearch",
+                "name": "elasticsearch-cli",
+                "version": "7.17.9",
+                "qualifiers": "",
+                "subpath": "",
+                "primary_language": "Java",
+                "description": "elasticsearch-cli\nElasticsearch subproject :libs:elasticsearch-cli",
+                "release_date": "2023-02-01T00:27:00Z",
+                "parties": [
+                    {
+                        "type": "person",
+                        "role": "developper",
+                        "name": "Elastic",
+                        "email": null,
+                        "url": "https://www.elastic.co"
+                    }
+                ],
+                "keywords": [],
+                "homepage_url": "https://github.com/elastic/elasticsearch.git",
+                "download_url": "https://repo1.maven.org/maven2/org/elasticsearch/elasticsearch-cli/7.17.9/elasticsearch-cli-7.17.9.jar",
+                "bug_tracking_url": null,
+                "code_view_url": "https://github.com/elastic/elasticsearch.git",
+                "vcs_url": "https://github.com/elastic/elasticsearch.git",
+                "repository_homepage_url": null,
+                "repository_download_url": null,
+                "api_data_url": null,
+                "size": null,
+                "md5": null,
+                "sha1": "9cb255ad91d178f39b2bffc9635c46caeffbd344",
+                "sha256": "55e58a1a0b85aa771b85404782740b8cdeb4c37c88f87391e51bbf955c7af808",
+                "sha512": "de2a7ca023b60f5d7a8c6c919495942512cfe9561230a1bf006ac160593573d81cbf356a35240dcc338c7c6aec4b79225ef2266eee5eb9b76a256c74b45e834c",
+                "copyright": null,
+                "holder": null,
+                "declared_license_expression": "elastic-license-v2 AND mongodb-sspl-1.0",
+                "declared_license_expression_spdx": "Elastic-2.0 AND SSPL-1.0",
+                "license_detections": [],
+                "other_license_expression": "apache-2.0 AND (mongodb-sspl-1.0 AND elastic-license-v2)",
+                "other_license_expression_spdx": "Apache-2.0 AND (SSPL-1.0 AND Elastic-2.0)",
+                "other_license_detections": [],
+                "extracted_license_statement": "[{'name': 'Elastic License 2.0', 'url': 'https://raw.githubusercontent.com/elastic/elasticsearch/v7.17.9/licenses/ELASTIC-LICENSE-2.0.txt', 'comments': None, 'distribution': 'repo'}, {'name': 'Server Side Public License, v 1', 'url': 'https://www.mongodb.com/licensing/server-side-public-license', 'comments': None, 'distribution': 'repo'}]",
+                "notice_text": null,
+                "source_packages": [
+                    "pkg:maven/org.elasticsearch/elasticsearch-cli@7.17.9?classifier=sources"
+                ],
+                "extra_data": {},
+                "package_uid": "pkg:maven/org.elasticsearch/elasticsearch-cli@7.17.9?uuid=0bbdcf88-ad07-4970-9272-7d5f4c82cc7b",
+                "datasource_id": null,
+                "file_references": [],
+                "dependencies": [
+                    {
+                        "purl": "pkg:maven/net.sf.jopt-simple/jopt-simple@5.0.2",
+                        "extracted_requirement": "5.0.2",
+                        "scope": "compile",
+                        "is_runtime": false,
+                        "is_optional": true,
+                        "is_resolved": true
+                    },
+                    {
+                        "purl": "pkg:maven/org.elasticsearch/elasticsearch-core@7.17.9",
+                        "extracted_requirement": "7.17.9",
+                        "scope": "compile",
+                        "is_runtime": false,
+                        "is_optional": true,
+                        "is_resolved": true
+                    }
+                ],
+                "resources": "https://public.purldb.io/api/packages/0bbdcf88-ad07-4970-9272-7d5f4c82cc7b/resources/",
+                "history": "https://public.purldb.io/api/packages/0bbdcf88-ad07-4970-9272-7d5f4c82cc7b/history/"
+            }
+        ]
+    }
+
+The packages list can be filtered by the following fields:
+
+    - ``type``
+    - ``namespace``
+    - ``name``
+    - ``version``
+    - ``qualifiers``
+    - ``subpath``
+    - ``download_url``
+    - ``filename``
+    - ``sha1``
+    - ``sha256``
+    - ``md5``
+    - ``size``
+    - ``release_date``
+    - ``package_url``
+
+For example:
+
+.. code-block:: console
+
+    api_url="http://localhost/api/packages/"
+    content_type="Content-Type: application/json"
+    payload="sha1=9cb255ad91d178f39b2bffc9635c46caeffbd344"
+
+    curl -X GET "$api_url?$payload" -H "$content_type"
+
+
+package details
+---------------
+
+The package details view returns all information available about a package.
+
+``GET /api/projects/0bbdcf88-ad07-4970-9272-7d5f4c82cc7b/``
+
+.. code-block:: json
+
+    {
+        "url": "https://public.purldb.io/api/packages/0bbdcf88-ad07-4970-9272-7d5f4c82cc7b/",
+        "uuid": "0bbdcf88-ad07-4970-9272-7d5f4c82cc7b",
+        "filename": "elasticsearch-cli-7.17.9.jar",
+        "package_sets": [
+            {
+                "uuid": "6d606be2-57c7-429a-a62e-d07833662c38",
+                "packages": [
+                    "https://public.purldb.io/api/packages/0bbdcf88-ad07-4970-9272-7d5f4c82cc7b/",
+                    "https://public.purldb.io/api/packages/055a7bab-38c6-4a1d-bb57-777b72ea4c99/"
+                ]
+            }
+        ],
+        "package_content": "binary",
+        "purl": "pkg:maven/org.elasticsearch/elasticsearch-cli@7.17.9",
+        "type": "maven",
+        "namespace": "org.elasticsearch",
+        "name": "elasticsearch-cli",
+        "version": "7.17.9",
+        "qualifiers": "",
+        "subpath": "",
+        "primary_language": "Java",
+        "description": "elasticsearch-cli\nElasticsearch subproject :libs:elasticsearch-cli",
+        "release_date": "2023-02-01T00:27:00Z",
+        "parties": [
+            {
+                "type": "person",
+                "role": "developper",
+                "name": "Elastic",
+                "email": null,
+                "url": "https://www.elastic.co"
+            }
+        ],
+        "keywords": [],
+        "homepage_url": "https://github.com/elastic/elasticsearch.git",
+        "download_url": "https://repo1.maven.org/maven2/org/elasticsearch/elasticsearch-cli/7.17.9/elasticsearch-cli-7.17.9.jar",
+        "bug_tracking_url": null,
+        "code_view_url": "https://github.com/elastic/elasticsearch.git",
+        "vcs_url": "https://github.com/elastic/elasticsearch.git",
+        "repository_homepage_url": null,
+        "repository_download_url": null,
+        "api_data_url": null,
+        "size": null,
+        "md5": null,
+        "sha1": "9cb255ad91d178f39b2bffc9635c46caeffbd344",
+        "sha256": "55e58a1a0b85aa771b85404782740b8cdeb4c37c88f87391e51bbf955c7af808",
+        "sha512": "de2a7ca023b60f5d7a8c6c919495942512cfe9561230a1bf006ac160593573d81cbf356a35240dcc338c7c6aec4b79225ef2266eee5eb9b76a256c74b45e834c",
+        "copyright": null,
+        "holder": null,
+        "declared_license_expression": "elastic-license-v2 AND mongodb-sspl-1.0",
+        "declared_license_expression_spdx": "Elastic-2.0 AND SSPL-1.0",
+        "license_detections": [],
+        "other_license_expression": "apache-2.0 AND (mongodb-sspl-1.0 AND elastic-license-v2)",
+        "other_license_expression_spdx": "Apache-2.0 AND (SSPL-1.0 AND Elastic-2.0)",
+        "other_license_detections": [],
+        "extracted_license_statement": "[{'name': 'Elastic License 2.0', 'url': 'https://raw.githubusercontent.com/elastic/elasticsearch/v7.17.9/licenses/ELASTIC-LICENSE-2.0.txt', 'comments': None, 'distribution': 'repo'}, {'name': 'Server Side Public License, v 1', 'url': 'https://www.mongodb.com/licensing/server-side-public-license', 'comments': None, 'distribution': 'repo'}]",
+        "notice_text": null,
+        "source_packages": [
+            "pkg:maven/org.elasticsearch/elasticsearch-cli@7.17.9?classifier=sources"
+        ],
+        "extra_data": {},
+        "package_uid": "pkg:maven/org.elasticsearch/elasticsearch-cli@7.17.9?uuid=0bbdcf88-ad07-4970-9272-7d5f4c82cc7b",
+        "datasource_id": null,
+        "file_references": [],
+        "dependencies": [
+            {
+                "purl": "pkg:maven/net.sf.jopt-simple/jopt-simple@5.0.2",
+                "extracted_requirement": "5.0.2",
+                "scope": "compile",
+                "is_runtime": false,
+                "is_optional": true,
+                "is_resolved": true
+            },
+            {
+                "purl": "pkg:maven/org.elasticsearch/elasticsearch-core@7.17.9",
+                "extracted_requirement": "7.17.9",
+                "scope": "compile",
+                "is_runtime": false,
+                "is_optional": true,
+                "is_resolved": true
+            }
+        ],
+        "resources": "https://public.purldb.io/api/packages/0bbdcf88-ad07-4970-9272-7d5f4c82cc7b/resources/",
+        "history": "https://public.purldb.io/api/packages/0bbdcf88-ad07-4970-9272-7d5f4c82cc7b/history/"
+    }
+
+packages actions
+----------------
+
+Multiple **actions** are available on packages:
+
+History
+^^^^^^^
+
+Return the history of actions taken on the ``package``, e.g. field updates.
+
+Using cURL to get package history:
+
+.. code-block:: console
+
+    api_url="https://public.purldb.io/api/packages/0bbdcf88-ad07-4970-9272-7d5f4c82cc7b/history/"
+    content_type="Content-Type: application/json"
+
+    curl -X GET "$api_url" -H "$content_type"
+
+.. code-block:: json
+
+    {
+        "history": [
+            {
+                "message": "New Package created from ResourceURI: https://repo1.maven.org/maven2/org/elasticsearch/elasticsearch-cli/7.17.9/elasticsearch-cli-7.17.9.jar via map_uri().",
+                "timestamp": "2023-04-28-20:55:59"
+            }
+        ]
+    }
+
+
+Resources
+^^^^^^^^^
+
+Return the ``resources`` of the ``package`` as a list of mappings.
+
+Using cURL to get package resources:
+
+.. code-block:: console
+
+    api_url="https://public.purldb.io/api/packages/0bbdcf88-ad07-4970-9272-7d5f4c82cc7b/resources/"
+    content_type="Content-Type: application/json"
+
+    curl -X GET "$api_url" -H "$content_type"
+
+.. code-block:: json
+
+    {
+        "count": 7556,
+        "next": "https://public.purldb.io/api/packages/97627c6e-9acb-43e0-b8df-28bd92f2b7e5/resources/?page=2",
+        "previous": null,
+        "results": [
+            {
+                "package": "https://public.purldb.io/api/packages/97627c6e-9acb-43e0-b8df-28bd92f2b7e5/",
+                "purl": "pkg:maven/org.elasticsearch/elasticsearch@7.17.9",
+                "path": "config",
+                "type": "directory",
+                "name": "config",
+                "extension": "",
+                "size": 0,
+                "md5": "",
+                "sha1": "",
+                "sha256": "",
+                "sha512": null,
+                "git_sha1": null,
+                "mime_type": "",
+                "file_type": "",
+                "programming_language": "",
+                "is_binary": false,
+                "is_text": false,
+                "is_archive": false,
+                "is_media": false,
+                "is_key_file": false,
+                "detected_license_expression": "",
+                "detected_license_expression_spdx": "",
+                "license_detections": [],
+                "license_clues": [],
+                "percentage_of_license_text": null,
+                "copyrights": [],
+                "holders": [],
+                "authors": [],
+                "package_data": [],
+                "emails": [],
+                "urls": [],
+                "extra_data": {}
+            }
+        ]
+    }
+
+
+Get enhanced package data
+-------------------------
+
+Return a mapping of enhanced Package data for a given Package
+
+This data is formed by supplanting missing data with other data from packages in
+the same package set.
+
+Using cURL to get enhanced package data:
+
+.. code-block:: console
+
+    api_url="https://public.purldb.io/api/packages/0bbdcf88-ad07-4970-9272-7d5f4c82cc7b/get_enhanced_package_data/"
+    content_type="Content-Type: application/json"
+
+    curl -X GET "$api_url" -H "$content_type"
+
+.. code-block:: json
+
+    {
+        "type": "maven",
+        "namespace": "org.elasticsearch",
+        "name": "elasticsearch-cli",
+        "version": "7.17.9",
+        "qualifiers": "",
+        "subpath": "",
+        "package_sets": [
+            {
+                "uuid": "6d606be2-57c7-429a-a62e-d07833662c38",
+                "packages": [
+                    "pkg:maven/org.elasticsearch/elasticsearch-cli@7.17.9?uuid=0bbdcf88-ad07-4970-9272-7d5f4c82cc7b",
+                    "pkg:maven/org.elasticsearch/elasticsearch-cli@7.17.9?classifier=sources&uuid=055a7bab-38c6-4a1d-bb57-777b72ea4c99"
+                ]
+            }
+        ],
+        "package_content": "binary",
+        "primary_language": "Java",
+        "description": "elasticsearch-cli\nElasticsearch subproject :libs:elasticsearch-cli",
+        "release_date": "2023-02-01T00:27:00Z",
+        "parties": [
+            {
+                "type": "person",
+                "role": "developper",
+                "name": "Elastic",
+                "email": null,
+                "url": "https://www.elastic.co"
+            }
+        ],
+        "keywords": [],
+        "homepage_url": "https://github.com/elastic/elasticsearch.git",
+        "download_url": "https://repo1.maven.org/maven2/org/elasticsearch/elasticsearch-cli/7.17.9/elasticsearch-cli-7.17.9.jar",
+        "size": null,
+        "md5": null,
+        "sha1": "9cb255ad91d178f39b2bffc9635c46caeffbd344",
+        "sha256": "55e58a1a0b85aa771b85404782740b8cdeb4c37c88f87391e51bbf955c7af808",
+        "sha512": "de2a7ca023b60f5d7a8c6c919495942512cfe9561230a1bf006ac160593573d81cbf356a35240dcc338c7c6aec4b79225ef2266eee5eb9b76a256c74b45e834c",
+        "bug_tracking_url": null,
+        "code_view_url": "https://github.com/elastic/elasticsearch.git",
+        "vcs_url": "https://github.com/elastic/elasticsearch.git",
+        "copyright": null,
+        "holder": null,
+        "declared_license_expression": "elastic-license-v2 AND mongodb-sspl-1.0",
+        "declared_license_expression_spdx": "Elastic-2.0 AND SSPL-1.0",
+        "license_detections": [],
+        "other_license_expression": "apache-2.0 AND (mongodb-sspl-1.0 AND elastic-license-v2)",
+        "other_license_expression_spdx": "Apache-2.0 AND (SSPL-1.0 AND Elastic-2.0)",
+        "other_license_detections": [],
+        "extracted_license_statement": "[{'name': 'Elastic License 2.0', 'url': 'https://raw.githubusercontent.com/elastic/elasticsearch/v7.17.9/licenses/ELASTIC-LICENSE-2.0.txt', 'comments': None, 'distribution': 'repo'}, {'name': 'Server Side Public License, v 1', 'url': 'https://www.mongodb.com/licensing/server-side-public-license', 'comments': None, 'distribution': 'repo'}]",
+        "notice_text": null,
+        "source_packages": [
+            "pkg:maven/org.elasticsearch/elasticsearch-cli@7.17.9?classifier=sources"
+        ],
+        "extra_data": {},
+        "dependencies": [
+            {
+                "purl": "pkg:maven/net.sf.jopt-simple/jopt-simple@5.0.2",
+                "extracted_requirement": "5.0.2",
+                "scope": "compile",
+                "is_runtime": false,
+                "is_optional": true,
+                "is_resolved": true
+            },
+            {
+                "purl": "pkg:maven/org.elasticsearch/elasticsearch-core@7.17.9",
+                "extracted_requirement": "7.17.9",
+                "scope": "compile",
+                "is_runtime": false,
+                "is_optional": true,
+                "is_resolved": true
+            }
+        ],
+        "package_uid": "pkg:maven/org.elasticsearch/elasticsearch-cli@7.17.9?uuid=0bbdcf88-ad07-4970-9272-7d5f4c82cc7b",
+        "datasource_id": null,
+        "purl": "pkg:maven/org.elasticsearch/elasticsearch-cli@7.17.9",
+        "repository_homepage_url": null,
+        "repository_download_url": null,
+        "api_data_url": null,
+        "file_references": []
+    }
+
+Reindex package
+^^^^^^^^^^^^^^^
+
+Reindex this package instance. This will trigger a new scan for this package and
+the package data will be updated from the scan data.
+
+Using cURL to reindex a package:
+
+.. code-block:: console
+
+    api_url="https://public.purldb.io/api/packages/0bbdcf88-ad07-4970-9272-7d5f4c82cc7b/reindex_package/"
+    content_type="Content-Type: application/json"
+
+    curl -X GET "$api_url" -H "$content_type"
+
+.. code-block:: json
+
+    {
+        "status": "pkg:maven/org.elasticsearch/elasticsearch@7.17.9 has been queued for reindexing"
+    }
+
+
+resources
+---------
+
+resources list
+--------------
+
+Return a list of resources in the PurlDB.
+
+``GET /api/resources/``
+
+.. code-block:: json
+
+    {
+        "count": 6031130,
+        "next": "https://public.purldb.io/api/resources/?page=2",
+        "previous": null,
+        "results": [
+            {
+                "package": "https://public.purldb.io/api/packages/20b7d376-09c7-45ef-a102-75f7f5eef7e2/",
+                "purl": "pkg:npm/cac@6.7.14",
+                "path": "package/deno/CAC.ts",
+                "type": "file",
+                "name": "",
+                "extension": "",
+                "size": 8133,
+                "md5": "969474f21d02f9a1dad6a2e85f4bbd25",
+                "sha1": "8c7042781582df3d5f39fd2fabf7d2dd365f1669",
+                "sha256": null,
+                "sha512": null,
+                "git_sha1": null,
+                "mime_type": "",
+                "file_type": "",
+                "programming_language": "",
+                "is_binary": false,
+                "is_text": false,
+                "is_archive": false,
+                "is_media": false,
+                "is_key_file": false,
+                "detected_license_expression": "",
+                "detected_license_expression_spdx": "",
+                "license_detections": [
+                    {
+                        "matches": [],
+                        "identifier": "none-f9065fa7-3897-50e1-6fe0-0d7ba36748f6",
+                        "license_expression": "None"
+                    }
+                ],
+                "license_clues": [],
+                "percentage_of_license_text": null,
+                "copyrights": [],
+                "holders": [],
+                "authors": [],
+                "package_data": [],
+                "emails": [],
+                "urls": [],
+                "extra_data": {}
+            }
+        ]
+    }
+
+The resources list can be filtered by the following fields:
+
+    - ``package_uuid``
+    - ``package_url``
+    - ``md5``
+    - ``sha1``
+
+For example:
+
+.. code-block:: console
+
+    api_url="http://localhost/api/resources/"
+    content_type="Content-Type: application/json"
+    payload="sha1=8c7042781582df3d5f39fd2fabf7d2dd365f1669"
+
+    curl -X GET "$api_url?$payload" -H "$content_type"
+
+
+resources actions
+-----------------
+
+One action is available on resources:
+
+Filter by checksum
+^^^^^^^^^^^^^^^^^^
+
+Take a mapping, where the keys are the names of the checksum algorthm and the
+values is a list of checksum values and query those values against the
+packagedb.
+
+Supported checksum fields are:
+
+    - ``md5``
+    - ``sha1``
+
+Multiple checksums field scan be passed in one request.
+
+Using cURL to filter for packages using multiple checksums:
+
+.. code-block:: console
+
+    api_url="https://public.purldb.io/api/resources/filter_by_checksums/"
+    content_type="Content-Type: application/json"
+    data='{
+        "sha1": [
+            "8c7042781582df3d5f39fd2fabf7d2dd365f1669"
+        ],
+        "md5": [
+            "969474f21d02f9a1dad6a2e85f4bbd25"
+        ]
+    }'
+
+    curl -X POST "$api_url" -H "$content_type" -d "$data"
+
+.. code-block:: json
+
+    {
+        "count": 1,
+        "next": null,
+        "previous": null,
+        "results": [
+            {
+                "package": "https://public.purldb.io/api/packages/20b7d376-09c7-45ef-a102-75f7f5eef7e2/",
+                "purl": "pkg:npm/cac@6.7.14",
+                "path": "package/deno/CAC.ts",
+                "type": "file",
+                "name": "",
+                "extension": "",
+                "size": 8133,
+                "md5": "969474f21d02f9a1dad6a2e85f4bbd25",
+                "sha1": "8c7042781582df3d5f39fd2fabf7d2dd365f1669",
+                "sha256": null,
+                "sha512": null,
+                "git_sha1": null,
+                "mime_type": "",
+                "file_type": "",
+                "programming_language": "",
+                "is_binary": false,
+                "is_text": false,
+                "is_archive": false,
+                "is_media": false,
+                "is_key_file": false,
+                "detected_license_expression": "",
+                "detected_license_expression_spdx": "",
+                "license_detections": [
+                    {
+                        "matches": [],
+                        "identifier": "none-f9065fa7-3897-50e1-6fe0-0d7ba36748f6",
+                        "license_expression": "None"
+                    }
+                ],
+                "license_clues": [],
+                "percentage_of_license_text": null,
+                "copyrights": [],
+                "holders": [],
+                "authors": [],
+                "package_data": [],
+                "emails": [],
+                "urls": [],
+                "extra_data": {}
+            }
+        ]
+    }
+
+
+validate purl
+-------------
+
+Take a purl and check whether it's valid PackageURL or not. Optionally set
+check_existence to true to check whether the package exists in real world.
+
+Note: As of now check_existence only supports ``cargo``, ``composer``, ``deb``,
+``gem``, ``golang``, ``hex``, ``maven``, ``npm``, ``nuget`` and ``pypi``
+ecosystems.
+
+``GET /api/validate/?purl=pkg:npm/asdf@1.0.2&check_existence=true``
+
+.. code-block:: json
+
+    {
+        "valid": true,
+        "exists": true,
+        "message": "The provided Package URL is valid, and the package exists in the upstream repo.",
+        "purl": "pkg:npm/asdf@1.0.2"
+    }
+
+
+collect
+-------
+
+Return Package data for the purl passed in the purl query parameter.
+
+If the package does not exist, we will fetch the Package data and return it in
+the same request. Optionally, provide the list of addon_pipelines to run on the
+package. Find all addon pipelines at
+https://scancodeio.readthedocs.io/en/latest/built-in-pipelines.html.
+
+``GET /api/collect/?purl=pkg:npm/asdf@1.0.2``
+
+.. code-block:: json
+
+    [
+        {
+            "url": "https://public.purldb.io/api/packages/4f3a57de-e367-43c6-a7f1-51633d0ecd45/",
+            "uuid": "4f3a57de-e367-43c6-a7f1-51633d0ecd45",
+            "filename": "asdf-1.0.2.tgz",
+            "package_sets": [],
+            "package_content": null,
+            "purl": "pkg:npm/asdf@1.0.2",
+            "type": "npm",
+            "namespace": "",
+            "name": "asdf",
+            "version": "1.0.2",
+            "qualifiers": "",
+            "subpath": "",
+            "primary_language": "JavaScript",
+            "description": "tiny static web server that you can launch instantly in any directory (inspired by https://github.com/ddfreyne/adsf/)",
+            "release_date": null,
+            "parties": [
+                {
+                    "type": "person",
+                    "role": "author",
+                    "name": "alsotang",
+                    "email": "alsotang@gmail.com",
+                    "url": null
+                },
+                {
+                    "type": "person",
+                    "role": "maintainer",
+                    "name": "alsotang",
+                    "email": "alsotang@gmail.com",
+                    "url": null
+                }
+            ],
+            "keywords": [
+                "static",
+                "web",
+                "server"
+            ],
+            "homepage_url": "https://github.com/alsotang/asdf",
+            "download_url": "https://registry.npmjs.org/asdf/-/asdf-1.0.2.tgz",
+            "bug_tracking_url": "https://github.com/alsotang/asdf/issues",
+            "code_view_url": null,
+            "vcs_url": "git+https://github.com/alsotang/asdf.git@53aeca5c74c3d8c1fe88c1f98f8e362389fa1d2a",
+            "repository_homepage_url": null,
+            "repository_download_url": null,
+            "api_data_url": null,
+            "size": null,
+            "md5": null,
+            "sha1": "45b7468df1a6f2ec4826257535f97ea89db943e4",
+            "sha256": null,
+            "sha512": null,
+            "copyright": null,
+            "holder": null,
+            "declared_license_expression": "mit",
+            "declared_license_expression_spdx": "MIT",
+            "license_detections": [],
+            "other_license_expression": null,
+            "other_license_expression_spdx": null,
+            "other_license_detections": [],
+            "extracted_license_statement": "MIT",
+            "notice_text": null,
+            "source_packages": [],
+            "extra_data": {},
+            "package_uid": "pkg:npm/asdf@1.0.2?uuid=4f3a57de-e367-43c6-a7f1-51633d0ecd45",
+            "datasource_id": null,
+            "file_references": [],
+            "dependencies": [
+                {
+                    "purl": "pkg:npm/express",
+                    "extracted_requirement": "^4.9.7",
+                    "scope": "dependencies",
+                    "is_runtime": true,
+                    "is_optional": false,
+                    "is_resolved": false
+                },
+                {
+                    "purl": "pkg:npm/mocha",
+                    "extracted_requirement": "^1.21.5",
+                    "scope": "devDependencies",
+                    "is_runtime": false,
+                    "is_optional": true,
+                    "is_resolved": false
+                },
+                {
+                    "purl": "pkg:npm/should",
+                    "extracted_requirement": "^4.0.4",
+                    "scope": "devDependencies",
+                    "is_runtime": false,
+                    "is_optional": true,
+                    "is_resolved": false
+                },
+                {
+                    "purl": "pkg:npm/supertest",
+                    "extracted_requirement": "^0.14.0",
+                    "scope": "devDependencies",
+                    "is_runtime": false,
+                    "is_optional": true,
+                    "is_resolved": false
+                }
+            ],
+            "resources": "https://public.purldb.io/api/packages/4f3a57de-e367-43c6-a7f1-51633d0ecd45/resources/",
+            "history": "https://public.purldb.io/api/packages/4f3a57de-e367-43c6-a7f1-51633d0ecd45/history/"
+        }
+
+scan_queue
+----------
+
+This endpoint provides a queue of Packages to be scanned by the package scan
+worker. A special key for package scan workers or superusers is needed to access
+this endpoint.
+
+This endpoint is intended for use with a PurlDB package scan worker and is not
+intended for users to use directly.
+
+scan_queue actions
+------------------
+
+get_next_download_url
+^^^^^^^^^^^^^^^^^^^^^
+
+Return a mapping containing a ``download_url`` of a package to be scanned with
+the list of provided ``pipelines`` for the scan request ``scannable_uri_uuid``.
+
+The names of the pipelines that can be run are listed here:
+https://scancodeio.readthedocs.io/en/latest/built-in-pipelines.html
+
+Using cURL to get next download URL:
+
+.. code-block:: console
+
+    api_url="https://public.purldb.io/api/scan_queue/get_next_download_url/"
+    content_type="Content-Type: application/json"
+    authorization="Authorization:Token abcdef123456"
+
+    curl -X GET "$api_url" -H "$content_type" -H "$authorization"
+
+.. code-block:: json
+
+    {
+        "scannable_uri_uuid": "4f3a57de-e367-43c6-a7f1-51633d0ecd45",
+        "download_url": "https://registry.npmjs.org/asdf/-/asdf-1.0.2.tgz",
+        "pipelines": ["scan_codebase", "fingerprint_codebase"]
+    }
+
+Example of a Python script:
+
+.. code-block:: python
+
+    import requests
+
+    api_url = "https://public.purldb.io/api/scan_queue/get_next_download_url/"
+    headers = {
+        "Authorization": "Token abcdef123456",
+    }
+    response = requests.get(api_url, headers=headers, params=params)
+    response.json()
+
+
+update_status
+^^^^^^^^^^^^^
+
+Update the status of scan request ``scannable_uri_uuid`` with ``scan_status``
+
+If ``scan_status`` is 'failed', then a ``scan_log`` string is expected and
+should contain the error messages for that scan.
+
+If ``scan_status`` is 'scanned', then a ``scan_results_file``,
+``scan_summary_file``, and ``project_extra_data`` mapping are expected.
+``scan_results_file``, ``scan_summary_file``, and ``project_extra_data`` are
+then used to update Package data and its Resources.
+
+Using cURL to update status:
+
+.. code-block:: console
+
+    api_url="https://public.purldb.io/api/scan_queue/update_status/"
+    content_type="Content-Type: application/json"
+    authorization="Authorization:Token abcdef123456"
+    data='{
+        "scannable_uri_uuid": "4f3a57de-e367-43c6-a7f1-51633d0ecd45",
+        "scan_status": "failed",
+        "scan_status": "scanned timed out"
+    }'
+
+    curl -X POST "$api_url" -H "$content_type" -H "$authorization" -d "$data"
+
+.. code-block:: json
+
+    {
+        "status": "updated scannable_uri 4f3a57de-e367-43c6-a7f1-51633d0ecd45 scan_status to failed"
+    }
+
+
+
+Package Update Set List
+-----------------------
+
+Take a list of purls (where each item is a mapping containing PURL and
+content_type).
+
+If uuid is given then all purls will be added to package set if it exists else a
+new set would be created and all the purls will be added to that new set.
+
+Note: There is also a slight addition to the logic where a purl already exists
+in the database and so there are no changes done to the purl entry it is passed
+as it is.
+
+Using cURL to update status:
+
+.. code-block:: console
+
+    api_url="https://public.purldb.io/api/scan_queue/update_status/"
+    content_type="Content-Type: application/json"
+    authorization="Authorization:Token abcdef123456"
+    data='{
+        "purls": [
+            {
+                "purl": "pkg:npm/less@1.0.32",
+                "content_type": "CURATION"
+            }
+        ],
+        "uuid" : "b67ceb49-1538-481f-a572-431062f382gg"
+    }'
+
+    curl -X POST "$api_url" -H "$content_type" -H "$authorization" -d "$data"
+
+.. code-block:: json
+
+    [
+        {
+            "purl": "pkg:npm/less@1.0.32",
+            "updated_status":: "Updated"
+        }
+    ]
+
+
+Package Set List
+----------------
+
+Return a list of package sets and the package data of packages within
+
+``GET /api/projects/0bbdcf88-ad07-4970-9272-7d5f4c82cc7b/``
+
+.. code-block:: json
+
+    {
+        "count": 8198,
+        "next": "https://public.purldb.io/api/package_sets/?page=2",
+        "previous": null,
+        "results": [
+            {
+                "uuid": "9d1655c0-16c7-424f-b027-a141cfdbf706",
+                "packages": [
+                    {
+                        "url": "https://public.purldb.io/api/packages/8a433f5e-372c-4fe1-9fc3-1027ecc9678b/",
+                        "uuid": "8a433f5e-372c-4fe1-9fc3-1027ecc9678b",
+                        "filename": "rfc8528-data-util-8.0.6.jar",
+                        "package_sets": [
+                            {
+                                "uuid": "9d1655c0-16c7-424f-b027-a141cfdbf706",
+                                "packages": [
+                                    "https://public.purldb.io/api/packages/8a433f5e-372c-4fe1-9fc3-1027ecc9678b/",
+                                    "https://public.purldb.io/api/packages/99b26f6e-b823-4c72-8408-9996f17d30f4/"
+                                ]
+                            }
+                        ],
+                        "package_content": "binary",
+                        "purl": "pkg:maven/org.opendaylight.yangtools/rfc8528-data-util@8.0.6",
+                        "type": "maven",
+                        "namespace": "org.opendaylight.yangtools",
+                        "name": "rfc8528-data-util",
+                        "version": "8.0.6",
+                        "qualifiers": "",
+                        "subpath": "",
+                        "primary_language": "Java",
+                        "description": "rfc8528-data-util\nRFC8528 data model utilities",
+                        "release_date": "2022-05-25T13:36:00Z",
+                        "parties": [],
+                        "keywords": [],
+                        "homepage_url": null,
+                        "download_url": "https://repo1.maven.org/maven2/org/opendaylight/yangtools/rfc8528-data-util/8.0.6/rfc8528-data-util-8.0.6.jar",
+                        "bug_tracking_url": null,
+                        "code_view_url": null,
+                        "vcs_url": null,
+                        "repository_homepage_url": null,
+                        "repository_download_url": null,
+                        "api_data_url": null,
+                        "size": null,
+                        "md5": null,
+                        "sha1": "31157249a6286d5478b5d01e8a29f5de9c33fb80",
+                        "sha256": "e1d83077ff746ccce4783971d85b5f3efecc2a5504e7ca7801e29d9f131dfdf2",
+                        "sha512": "f633bd2fa6cd1d9a36fb296c373fe750dc48f1310b13bc1d020fe8f6ab7dddaf7ff650b11b910504d25a19b205c7237660e170d19fab9b152eabc6d51bd8525a",
+                        "copyright": "Copyright (c) PANTHEON.tech, s.r.o. and others",
+                        "holder": null,
+                        "declared_license_expression": "epl-1.0",
+                        "declared_license_expression_spdx": "EPL-1.0",
+                        "license_detections": [],
+                        "other_license_expression": "((epl-2.0 OR apache-2.0) AND epl-2.0) AND (epl-1.0 AND epl-2.0)",
+                        "other_license_expression_spdx": "((EPL-2.0 OR Apache-2.0) AND EPL-2.0) AND (EPL-1.0 AND EPL-2.0)",
+                        "other_license_detections": [],
+                        "extracted_license_statement": null,
+                        "notice_text": null,
+                        "source_packages": [
+                            "pkg:maven/org.opendaylight.yangtools/rfc8528-data-util@8.0.6?classifier=sources"
+                        ],
+                        "extra_data": {},
+                        "package_uid": "pkg:maven/org.opendaylight.yangtools/rfc8528-data-util@8.0.6?uuid=8a433f5e-372c-4fe1-9fc3-1027ecc9678b",
+                        "datasource_id": null,
+                        "file_references": [],
+                        "dependencies": [
+                            {
+                                "purl": "pkg:maven/com.google.guava/guava",
+                                "extracted_requirement": null,
+                                "scope": "compile",
+                                "is_runtime": false,
+                                "is_optional": true,
+                                "is_resolved": false
+                            },
+                            {
+                                "purl": "pkg:maven/org.opendaylight.yangtools/concepts",
+                                "extracted_requirement": null,
+                                "scope": "compile",
+                                "is_runtime": false,
+                                "is_optional": true,
+                                "is_resolved": false
+                            },
+                            {
+                                "purl": "pkg:maven/org.opendaylight.yangtools/yang-common",
+                                "extracted_requirement": null,
+                                "scope": "compile",
+                                "is_runtime": false,
+                                "is_optional": true,
+                                "is_resolved": false
+                            },
+                            {
+                                "purl": "pkg:maven/org.opendaylight.yangtools/yang-data-api",
+                                "extracted_requirement": null,
+                                "scope": "compile",
+                                "is_runtime": false,
+                                "is_optional": true,
+                                "is_resolved": false
+                            },
+                            {
+                                "purl": "pkg:maven/org.opendaylight.yangtools/yang-data-spi",
+                                "extracted_requirement": null,
+                                "scope": "compile",
+                                "is_runtime": false,
+                                "is_optional": true,
+                                "is_resolved": false
+                            },
+                            {
+                                "purl": "pkg:maven/org.opendaylight.yangtools/yang-model-api",
+                                "extracted_requirement": null,
+                                "scope": "compile",
+                                "is_runtime": false,
+                                "is_optional": true,
+                                "is_resolved": false
+                            },
+                            {
+                                "purl": "pkg:maven/org.opendaylight.yangtools/yang-model-spi",
+                                "extracted_requirement": null,
+                                "scope": "compile",
+                                "is_runtime": false,
+                                "is_optional": true,
+                                "is_resolved": false
+                            },
+                            {
+                                "purl": "pkg:maven/org.opendaylight.yangtools/yang-parser-api",
+                                "extracted_requirement": null,
+                                "scope": "compile",
+                                "is_runtime": false,
+                                "is_optional": true,
+                                "is_resolved": false
+                            },
+                            {
+                                "purl": "pkg:maven/org.opendaylight.yangtools/rfc8528-data-api",
+                                "extracted_requirement": null,
+                                "scope": "compile",
+                                "is_runtime": false,
+                                "is_optional": true,
+                                "is_resolved": false
+                            },
+                            {
+                                "purl": "pkg:maven/org.opendaylight.yangtools/rfc8528-model-api",
+                                "extracted_requirement": null,
+                                "scope": "compile",
+                                "is_runtime": false,
+                                "is_optional": true,
+                                "is_resolved": false
+                            }
+                        ],
+                        "resources": "https://public.purldb.io/api/packages/8a433f5e-372c-4fe1-9fc3-1027ecc9678b/resources/",
+                        "history": "https://public.purldb.io/api/packages/8a433f5e-372c-4fe1-9fc3-1027ecc9678b/history/"
+                    }
+                ]
+            }
+        ]
+    }
+
+
+to_purl
+-------
+
+Return a ``golang_purl`` PackageURL from ``go_package``, a standard go import
+string or a go.mod string.
+
+``GET /api/to_purl/?go_package=github.com/gorilla/mux%20v1.8.1``
+
+.. code-block:: json
+
+    {
+        "golang_purl": "pkg:golang/github.com/gorilla/mux@v1.8.1"
+    }
+
+
+from_purl
+---------
+
+Return a ``git_repo`` from a standard PackageURL ``package_url``.
+
+``GET /api/from_purl/?package_url=pkg:github/ckeditor/ckeditor4-react``
+
+.. code-block:: json
+
+    {
+        "git_repo": "git+https://github.com/ckeditor/ckeditor4-react.git"
+    }
+
+
+matching
+--------
+
+Given a ScanCode.io JSON output ``upload_file``, match directory and resources
+of the codebase in ``upload_file`` to Packages indexed in the PurlDB.
+
+This endpoint runs the ``matching`` pipeline at https://github.com/nexB/purldb/blob/main/matchcode_pipeline/pipelines/matching.py
+
+Using cURL to upload a scan for matching:
+
+.. code-block:: console
+
+    api_url="https://public.purldb.io/api/matching/"
+    content_type="Content-Type: application/json"
+
+    curl -X POST "$api_url" -H "$content_type" -F "upload_file=@/home/user/scan.json"
+
+.. code-block:: json
+
+    {
+        "url": "http://testserver/api/matching/d7b3a3f3-87de-44d5-852a-e0fb99b10d89/",
+        "uuid": "d7b3a3f3-87de-44d5-852a-e0fb99b10d89",
+        "created_date": "2024-06-03T19:02:28.966557Z",
+        "input_sources": [
+            {
+                "filename": "scan.json",
+                "download_url": "",
+                "is_uploaded": true,
+                "tag": "",
+                "exists": true,
+                "uuid": "2f67a376-6ff7-4762-9ea5-e998d8164156"
+            }
+        ],
+        "runs": [
+            {
+                "url": "http://testserver/api/runs/74c533f7-b31b-451c-8fff-a5a556a410ce/",
+                "pipeline_name": "matching",
+                "status": "not started",
+                "description": "",
+                "project": "http://testserver/api/runs/d7b3a3f3-87de-44d5-852a-e0fb99b10d89/",
+                "uuid": "74c533f7-b31b-451c-8fff-a5a556a410ce",
+                "created_date": "2024-06-03T19:02:28.968804Z",
+                "scancodeio_version": "",
+                "task_id": null,
+                "task_start_date": null,
+                "task_end_date": null,
+                "task_exitcode": null,
+                "task_output": "",
+                "log": "",
+                "execution_time": null
+            }
+        ]
+    }
diff --git a/matchcode/api.py b/matchcode/api.py
deleted file mode 100644
index 9810645a..00000000
--- a/matchcode/api.py
+++ /dev/null
@@ -1,308 +0,0 @@
-#
-# Copyright (c) nexB Inc. and others. All rights reserved.
-# purldb is a trademark of nexB Inc.
-# SPDX-License-Identifier: Apache-2.0
-# See http://www.apache.org/licenses/LICENSE-2.0 for the license text.
-# See https://github.com/nexB/purldb for support or download.
-# See https://aboutcode.org for more information about nexB OSS projects.
-#
-from django.db.models import Q
-from django.forms import widgets
-from django.forms.fields import MultipleChoiceField
-from django_filters.filters import MultipleChoiceFilter
-from django_filters.rest_framework import FilterSet
-from rest_framework.decorators import action
-from rest_framework.response import Response
-from rest_framework.serializers import CharField
-from rest_framework.serializers import FloatField
-from rest_framework.serializers import HyperlinkedRelatedField
-from rest_framework.serializers import ModelSerializer
-from rest_framework.serializers import ReadOnlyField
-from rest_framework.serializers import Serializer
-from rest_framework.viewsets import ReadOnlyModelViewSet
-
-from matchcode_toolkit.fingerprinting import create_halohash_chunks
-from matchcode_toolkit.fingerprinting import hexstring_to_binarray
-from matchcode_toolkit.fingerprinting import split_fingerprint
-from matchcode_toolkit.halohash import byte_hamming_distance
-from matchcode.models import ExactFileIndex
-from matchcode.models import ExactPackageArchiveIndex
-from matchcode.models import ApproximateDirectoryContentIndex
-from matchcode.models import ApproximateDirectoryStructureIndex
-
-
-class BaseFileIndexSerializer(ModelSerializer):
-    sha1 = CharField(source='fingerprint')
-    package = HyperlinkedRelatedField(
-        view_name='api:package-detail',
-        lookup_field='uuid',
-        read_only=True
-    )
-
-
-class ExactFileIndexSerializer(BaseFileIndexSerializer):
-    class Meta:
-        model = ExactFileIndex
-        fields = (
-            'sha1',
-            'package'
-        )
-
-
-class ExactPackageArchiveIndexSerializer(BaseFileIndexSerializer):
-    class Meta:
-        model = ExactPackageArchiveIndex
-        fields = (
-            'sha1',
-            'package'
-        )
-
-
-class BaseDirectoryIndexSerializer(ModelSerializer):
-    fingerprint = ReadOnlyField()
-    package = HyperlinkedRelatedField(
-        view_name='api:package-detail',
-        lookup_field='uuid',
-        read_only=True
-    )
-
-class ApproximateDirectoryContentIndexSerializer(BaseDirectoryIndexSerializer):
-    class Meta:
-        model = ApproximateDirectoryContentIndex
-        fields = (
-            'fingerprint',
-            'package',
-        )
-
-
-class ApproximateDirectoryStructureIndexSerializer(BaseDirectoryIndexSerializer):
-    class Meta:
-        model = ApproximateDirectoryStructureIndex
-        fields = (
-            'fingerprint',
-            'package',
-        )
-
-
-class BaseDirectoryIndexMatchSerializer(Serializer):
-    fingerprint = CharField()
-    matched_fingerprint = CharField()
-    package = HyperlinkedRelatedField(
-        view_name='api:package-detail',
-        lookup_field='uuid',
-        read_only=True
-    )
-    similarity_score = FloatField()
-
-
-class CharMultipleWidget(widgets.TextInput):
-    """
-    Enables the support for `MultiValueDict` `?field=a&field=b`
-    reusing the `SelectMultiple.value_from_datadict()` but render as a `TextInput`.
-    """
-    def value_from_datadict(self, data, files, name):
-        value = widgets.SelectMultiple().value_from_datadict(data, files, name)
-        if not value or value == ['']:
-            return ''
-
-        return value
-
-    def format_value(self, value):
-        """
-        Return a value as it should appear when rendered in a template.
-        """
-        return ', '.join(value)
-
-
-class MultipleCharField(MultipleChoiceField):
-    """
-    Overrides `MultipleChoiceField` to fit in `MultipleCharFilter`.
-    """
-    widget = CharMultipleWidget
-
-    def valid_value(self, value):
-        return True
-
-
-class MultipleCharFilter(MultipleChoiceFilter):
-    """
-    Filters on multiple values for a CharField type using `?field=a&field=b` URL syntax.
-    """
-    field_class = MultipleCharField
-
-
-# TODO: Think of a better name for this filter
-class MultipleCharInFilter(MultipleCharFilter):
-    def filter(self, qs, value):
-        if not value:
-            # Even though not a noop, no point filtering if empty.
-            return qs
-
-        if self.is_noop(qs, value):
-            return qs
-
-        predicate = self.get_filter_predicate(value)
-        old_field_name = next(iter(predicate))
-        new_field_name = f'{old_field_name}__in'
-        predicate[new_field_name] = predicate[old_field_name]
-        predicate.pop(old_field_name)
-
-        q = Q(**predicate)
-        qs = self.get_method(qs)(q)
-
-        return qs.distinct() if self.distinct else qs
-
-
-class MultipleSHA1Filter(MultipleCharFilter):
-    """
-    Overrides `MultipleCharFilter.filter()` to convert the SHA1
-    into a bytearray so it can be queried
-    """
-    def filter(self, qs, value):
-        if not value:
-            return qs
-
-        q = Q()
-        for val in value:
-            v = hexstring_to_binarray(val)
-            q.add(Q(sha1=v), Q.OR)
-
-        return qs.filter(q)
-
-
-class MultipleFingerprintFilter(MultipleCharFilter):
-    """
-    Overrides `MultipleCharFilter.filter()` to process fingerprint from a single
-    string into multiple values used for querying.
-
-    In the BaseDirectoryIndex model, the fingerprint is stored in four chunks of
-    equal size, not as a single field that contains the entire fingerprint. We
-    must process the fingerprint into the correct parts so we can use those
-    parts to query the different fields.
-    """
-    def filter(self, qs, value):
-        if not value:
-            return qs
-
-        q = Q()
-        for val in value:
-            indexed_elements_count, bah128 = split_fingerprint(val)
-            chunk1, chunk2, chunk3, chunk4 = create_halohash_chunks(bah128)
-            q.add(
-                Q(
-                    indexed_elements_count=indexed_elements_count,
-                    chunk1=chunk1,
-                    chunk2=chunk2,
-                    chunk3=chunk3,
-                    chunk4=chunk4
-                ),
-                Q.OR
-            )
-
-        return qs.filter(q)
-
-
-class BaseFileIndexFilterSet(FilterSet):
-    sha1 = MultipleSHA1Filter()
-
-
-class ExactFileIndexFilterSet(BaseFileIndexFilterSet):
-    class Meta:
-        model = ExactFileIndex
-        fields = (
-            'sha1',
-        )
-
-
-class ExactPackageArchiveFilterSet(BaseFileIndexFilterSet):
-    class Meta:
-        model = ExactPackageArchiveIndex
-        fields = (
-            'sha1',
-        )
-
-
-class BaseDirectoryIndexFilterSet(FilterSet):
-    fingerprint = MultipleFingerprintFilter()
-
-
-class ApproximateDirectoryContentFilterSet(BaseDirectoryIndexFilterSet):
-    class Meta:
-        model = ApproximateDirectoryContentIndex
-        fields = (
-            'fingerprint',
-        )
-
-
-class ApproximateDirectoryStructureFilterSet(BaseDirectoryIndexFilterSet):
-    class Meta:
-        model = ApproximateDirectoryStructureIndex
-        fields = (
-            'fingerprint',
-        )
-
-
-class BaseFileIndexViewSet(ReadOnlyModelViewSet):
-    lookup_field = 'sha1'
-
-
-class ExactFileIndexViewSet(BaseFileIndexViewSet):
-    queryset = ExactFileIndex.objects.all()
-    serializer_class = ExactFileIndexSerializer
-    filterset_class = ExactFileIndexFilterSet
-
-
-class ExactPackageArchiveIndexViewSet(BaseFileIndexViewSet):
-    queryset = ExactPackageArchiveIndex.objects.all()
-    serializer_class = ExactPackageArchiveIndexSerializer
-    filterset_class = ExactPackageArchiveFilterSet
-
-
-class BaseDirectoryIndexViewSet(ReadOnlyModelViewSet):
-    lookup_field = 'fingerprint'
-
-    @action(detail=False)
-    def match(self, request):
-        fingerprints = request.query_params.getlist('fingerprint')
-        if not fingerprints:
-            return Response()
-
-        model_class = self.get_serializer().Meta.model
-        results = []
-        unique_fingerprints = set(fingerprints)
-        for fingerprint in unique_fingerprints:
-            matches = model_class.match(fingerprint)
-            for match in matches:
-                _, bah128 = split_fingerprint(fingerprint)
-                # Get fingerprint from the match
-                fp = match.fingerprint()
-                _, match_bah128 = split_fingerprint(fp)
-                hd = byte_hamming_distance(bah128, match_bah128)
-                similarity_score = (128 - hd) / 128
-                results.append(
-                    {
-                        'fingerprint': fingerprint,
-                        'matched_fingerprint': fp,
-                        'package': match.package,
-                        'similarity_score': similarity_score,
-                    }
-                )
-
-        serialized_match_results = BaseDirectoryIndexMatchSerializer(
-            results,
-            context={'request': request},
-            many=True
-        )
-        return Response(serialized_match_results.data)
-
-
-class ApproximateDirectoryContentIndexViewSet(BaseDirectoryIndexViewSet):
-    queryset = ApproximateDirectoryContentIndex.objects.all()
-    serializer_class = ApproximateDirectoryContentIndexSerializer
-    filterset_class = ApproximateDirectoryContentFilterSet
-
-
-class ApproximateDirectoryStructureIndexViewSet(BaseDirectoryIndexViewSet):
-    queryset = ApproximateDirectoryStructureIndex.objects.all()
-    serializer_class = ApproximateDirectoryStructureIndexSerializer
-    filterset_class = ApproximateDirectoryStructureFilterSet
diff --git a/packagedb/api.py b/packagedb/api.py
index 164aec31..747f591d 100644
--- a/packagedb/api.py
+++ b/packagedb/api.py
@@ -14,9 +14,13 @@
 from django.db.models import OuterRef
 from django.db.models import Q
 from django.db.models import Subquery
+from django.forms import widgets
+from django.forms.fields import MultipleChoiceField
 from django_filters.filters import Filter
+from django_filters.filters import MultipleChoiceFilter
 from django_filters.filters import OrderingFilter
 from django_filters.rest_framework import FilterSet
+
 from drf_spectacular.plumbing import build_array_type
 from drf_spectacular.plumbing import build_basic_type
 from drf_spectacular.types import OpenApiTypes
@@ -35,8 +39,6 @@
 from univers.version_range import VersionRange
 from univers.versions import InvalidVersion
 
-from matchcode.api import MultipleCharFilter
-from matchcode.api import MultipleCharInFilter
 # UnusedImport here!
 # But importing the mappers and visitors module triggers routes registration
 from minecode import priority_router
@@ -74,6 +76,66 @@
 logger = logging.getLogger(__name__)
 
 
+class CharMultipleWidget(widgets.TextInput):
+    """
+    Enables the support for `MultiValueDict` `?field=a&field=b`
+    reusing the `SelectMultiple.value_from_datadict()` but render as a `TextInput`.
+    """
+    def value_from_datadict(self, data, files, name):
+        value = widgets.SelectMultiple().value_from_datadict(data, files, name)
+        if not value or value == ['']:
+            return ''
+
+        return value
+
+    def format_value(self, value):
+        """
+        Return a value as it should appear when rendered in a template.
+        """
+        return ', '.join(value)
+
+
+class MultipleCharField(MultipleChoiceField):
+    """
+    Overrides `MultipleChoiceField` to fit in `MultipleCharFilter`.
+    """
+    widget = CharMultipleWidget
+
+    def valid_value(self, value):
+        return True
+
+
+class MultipleCharFilter(MultipleChoiceFilter):
+    """
+    Filters on multiple values for a CharField type using `?field=a&field=b` URL syntax.
+    """
+    field_class = MultipleCharField
+
+
+class MultipleCharInFilter(MultipleCharFilter):
+    """
+    Does a <field>__in = [value] filter instead of field=value filter
+    """
+    def filter(self, qs, value):
+        if not value:
+            # Even though not a noop, no point filtering if empty.
+            return qs
+
+        if self.is_noop(qs, value):
+            return qs
+
+        predicate = self.get_filter_predicate(value)
+        old_field_name = next(iter(predicate))
+        new_field_name = f'{old_field_name}__in'
+        predicate[new_field_name] = predicate[old_field_name]
+        predicate.pop(old_field_name)
+
+        q = Q(**predicate)
+        qs = self.get_method(qs)(q)
+
+        return qs.distinct() if self.distinct else qs
+
+
 class CreateListRetrieveUpdateViewSetMixin(
     mixins.CreateModelMixin,
     mixins.ListModelMixin,
@@ -674,7 +736,7 @@ class CollectViewSet(viewsets.ViewSet):
     Return Package data for the purl passed in the `purl` query parameter.
 
     If the package does not exist, we will fetch the Package data and return
-    it in the same request.  
+    it in the same request.
     Optionally, provide the list of addon_pipelines
     to run on the package. Find all addon pipelines [here.](https://scancodeio.readthedocs.io/en/latest/built-in-pipelines.html)
 
@@ -693,7 +755,7 @@ class CollectViewSet(viewsets.ViewSet):
 
                 # There is no OpenApiTypes.LIST https://github.com/tfranzel/drf-spectacular/issues/341
                 OpenApiParameter(
-                    'addon_pipelines', 
+                    'addon_pipelines',
                     build_array_type(build_basic_type(OpenApiTypes.STR)),
                     'query', description='Addon pipelines',
                     ),
@@ -704,17 +766,17 @@ def list(self, request, format=None):
         serializer = self.serializer_class(data=request.query_params)
         if not serializer.is_valid():
             return Response(
-                {'errors': serializer.errors}, 
+                {'errors': serializer.errors},
                 status=status.HTTP_400_BAD_REQUEST,
                 )
 
         validated_data = serializer.validated_data
         purl = validated_data.get('purl')
-        
+
         kwargs = dict()
         if source_purl := validated_data.get('source_purl', None):
             kwargs["source_purl"] = source_purl
-        
+
         if addon_pipelines := validated_data.get('addon_pipelines', []):
             kwargs["pipelines"] = addon_pipelines
 
@@ -756,8 +818,8 @@ def index_packages(self, request, *args, **kwargs):
         """
         Take a list of `packages` (where each item is a dictionary containing either PURL
         or versionless PURL along with vers range, optionally with source package PURL)
-        and index it.  
-        Also each package can have list of `addon_pipelines` to run on the package.  
+        and index it.
+        Also each package can have list of `addon_pipelines` to run on the package.
         Find all addon pipelines [here.](https://scancodeio.readthedocs.io/en/latest/built-in-pipelines.html)
 
 
@@ -1057,7 +1119,7 @@ def get_resolved_packages(packages, supported_ecosystems):
                 resolved_packages_by_purl[res_purl] = {'purl': res_purl}
         else:
             unsupported_vers.add(vers)
-    
+
     unique_resolved_packages = resolved_packages_by_purl.values()
 
     return list(unique_resolved_packages), list(unsupported_purls), list(unsupported_vers)
diff --git a/purldb_project/urls.py b/purldb_project/urls.py
index 9831452f..90b7d04e 100644
--- a/purldb_project/urls.py
+++ b/purldb_project/urls.py
@@ -14,21 +14,15 @@
 
 from rest_framework import routers
 
-from clearcode.api import CDitemViewSet
 from packagedb.api import PackageViewSet
 from packagedb.api import PackageSetViewSet
 from packagedb.api import PackageUpdateSet
 from packagedb.api import PackageWatchViewSet
 from packagedb.api import ResourceViewSet
-from matchcode.api import ApproximateDirectoryContentIndexViewSet
-from matchcode.api import ApproximateDirectoryStructureIndexViewSet
-from matchcode.api import ExactFileIndexViewSet
-from matchcode.api import ExactPackageArchiveIndexViewSet
-from minecode.api import PriorityResourceURIViewSet
 from minecode.api import ScannableURIViewSet
 from packagedb.api import PurlValidateViewSet
 from packagedb.to_purl import api_to_purl_router
-from packagedb.from_purl import api_from_purl_router 
+from packagedb.from_purl import api_from_purl_router
 from packagedb.api import CollectViewSet
 from drf_spectacular.views import SpectacularAPIView
 from drf_spectacular.views import SpectacularSwaggerView
@@ -39,12 +33,6 @@
 api_router.register('update_packages', PackageUpdateSet, 'update_packages')
 api_router.register('package_sets', PackageSetViewSet)
 api_router.register('resources', ResourceViewSet)
-api_router.register('approximate_directory_content_index', ApproximateDirectoryContentIndexViewSet)
-api_router.register('approximate_directory_structure_index', ApproximateDirectoryStructureIndexViewSet)
-api_router.register('exact_file_index', ExactFileIndexViewSet)
-api_router.register('exact_package_archive_index', ExactPackageArchiveIndexViewSet)
-api_router.register('cditems', CDitemViewSet, 'cditems')
-api_router.register('on_demand_queue', PriorityResourceURIViewSet)
 api_router.register('validate', PurlValidateViewSet, 'validate')
 api_router.register('collect', CollectViewSet, 'collect')
 api_router.register('watch',PackageWatchViewSet)


Test patch:
diff --git a/clearcode/tests/test_api.py b/clearcode/tests/test_api.py
deleted file mode 100644
index 76f3feba..00000000
--- a/clearcode/tests/test_api.py
+++ /dev/null
@@ -1,126 +0,0 @@
-# -*- coding: utf-8 -*-
-#
-# Copyright (c) nexB Inc. and others. All rights reserved.
-#
-# ClearCode is a free software tool from nexB Inc. and others.
-# Visit https://github.com/nexB/clearcode-toolkit/ for support and download.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-import base64
-import datetime
-import gzip
-import json
-
-from django.test import TestCase
-from django.utils import timezone
-from rest_framework import status
-from rest_framework.test import APIClient
-
-from clearcode import api
-from clearcode.models import CDitem
-
-
-class CDitemSerializerTestCase(TestCase):
-
-    def setUp(self):
-        self.cditem_attributes = {
-            'path': 'test/path/file.json',
-            'content': gzip.compress(json.dumps({'test': 'content'}).encode('utf-8'))
-        }
-        self.cditem = CDitem.objects.create(**self.cditem_attributes)
-        self.serializer = api.CDitemSerializer(instance=self.cditem)
-        self.data = self.serializer.data
-
-    def test_contains_expected_fields(self):
-        self.assertCountEqual(self.data.keys(), ['path', 'uuid', 'content', 'last_modified_date', 'last_map_date', 'map_error'])
-
-    def test_path_field_content(self):
-        self.assertEqual(self.data['path'], self.cditem_attributes['path'])
-    
-    def test_content_field_content(self):
-        decoded_test_data = base64.b64decode(self.data['content'])
-        self.assertEqual(decoded_test_data, self.cditem_attributes['content'])
-        self.assertEqual(json.loads(gzip.decompress(decoded_test_data)), {'test': 'content'})
-
-    def test_last_map_date_field_content(self):
-        self.assertIsNone(self.data['last_map_date'])
-
-    def test_map_error_field_content(self):
-        self.assertIsNone(self.data['map_error'])
-
-
-class CDitemAPITestCase(TestCase):
-
-    def setUp(self):
-        self.client = APIClient()
-        self.test_path = 'test/path/file.json'
-
-        self.post_test_path = 'test/post/path/file.json'
-        
-        self.test_data = {'test': 'content'}
-        self.test_content = gzip.compress(json.dumps(self.test_data).encode('utf-8'))
-        
-        self.cditem = CDitem.objects.create(path=self.test_path)
-        self.uuid = self.cditem.uuid
-
-    def test_api_cditems_get(self):
-        response = self.client.get('/api/cditems/{}/'.format(self.uuid))
-        self.assertEqual(response.status_code, status.HTTP_200_OK)
-        self.assertEqual(response.data.get('path'), self.test_path)
-        self.assertEqual(response.data.get('uuid'), str(self.uuid))
-
-    def test_api_cditems_get_list(self):
-        response = self.client.get('/api/cditems/')
-        self.assertEqual(response.status_code, status.HTTP_200_OK)
-        self.assertEqual(1, response.data.get('count'))
-
-    def test_api_cditems_get_list_by_last_modified_date_old_date(self):
-        test_date = datetime.datetime.now() - datetime.timedelta(days=1)
-        test_date_string = '{}-{}-{}'.format(test_date.year, test_date.month, test_date.day)
-
-        response = self.client.get('/api/cditems/?last_modified_date={}'.format(test_date_string))
-        self.assertEqual(response.status_code, status.HTTP_200_OK)
-        self.assertEqual(1, response.data.get('count'))
-
-    def test_api_cditems_get_list_by_last_modified_date_future(self):
-        test_date = datetime.datetime.now() + datetime.timedelta(days=1)
-        test_date_string = '{}-{}-{}'.format(test_date.year, test_date.month, test_date.day)
-
-        response = self.client.get('/api/cditems/?last_modified_date={}'.format(test_date_string))
-        self.assertEqual(response.status_code, status.HTTP_200_OK)
-        self.assertEqual(0, response.data.get('count'))
-    
-    def test_api_cditems_put(self):
-        test_payload = {
-            'path': self.test_path,
-            'content': base64.b64encode(self.test_content).decode('utf-8')
-        }
-
-        response = self.client.put('/api/cditems/{}/'.format(self.uuid), test_payload)
-        self.assertEqual(response.status_code, status.HTTP_200_OK)
-
-        cditem = CDitem.objects.get(path=self.test_path)
-        self.assertEqual(cditem.data, self.test_data)
-    
-    def test_api_cditems_post(self):
-        test_payload = {
-            'path': self.post_test_path,
-            'content': base64.b64encode(self.test_content).decode('utf-8')
-        }
-
-        response = self.client.post('/api/cditems/', test_payload)
-        self.assertEqual(response.status_code, status.HTTP_201_CREATED)
-        self.assertEqual(response.data.get('path'), self.post_test_path)
-
-        cditem = CDitem.objects.get(path=self.post_test_path)
-        self.assertEqual(cditem.data, self.test_data)
diff --git a/matchcode/tests/test_api.py b/matchcode/tests/test_api.py
deleted file mode 100644
index 02d79abf..00000000
--- a/matchcode/tests/test_api.py
+++ /dev/null
@@ -1,168 +0,0 @@
-#
-# Copyright (c) nexB Inc. and others. All rights reserved.
-# purldb is a trademark of nexB Inc.
-# SPDX-License-Identifier: Apache-2.0
-# See http://www.apache.org/licenses/LICENSE-2.0 for the license text.
-# See https://github.com/nexB/purldb for support or download.
-# See https://aboutcode.org for more information about nexB OSS projects.
-#
-
-import os
-
-from django.urls import reverse
-
-from packagedb.models import Package
-
-from matchcode.utils import index_package_directories
-from matchcode.utils import load_resources_from_scan
-from matchcode.utils import MatchcodeTestCase
-from matchcode.tests import FIXTURES_REGEN
-
-
-class ApproximateDirectoryStructureIndexAPITestCase(MatchcodeTestCase):
-    BASE_DIR = os.path.join(os.path.dirname(__file__), 'testfiles')
-
-    def setUp(self):
-        # Execute the superclass' setUp method before creating our own
-        # DB objects
-        super().setUp()
-
-        self.test_package1, _ = Package.objects.get_or_create(
-            filename='plugin-request-2.4.1.tgz',
-            sha1='7295749caddd3c52be472eef6623a7b441ed17d6',
-            size=7269,
-            name='plugin-request',
-            version='2.4.1',
-            download_url='https://registry.npmjs.org/@umijs/plugin-request/-/plugin-request-2.4.1.tgz',
-            type='npm',
-        )
-        load_resources_from_scan(self.get_test_loc('match/nested/plugin-request-2.4.1-ip.json'), self.test_package1)
-        index_package_directories(self.test_package1)
-
-        self.test_package2, _ = Package.objects.get_or_create(
-            filename='underscore-1.10.9.tgz',
-            sha1='ba7a9cfc15873e67821611503a34a7c26bf7264f',
-            size=26569,
-            name='underscore',
-            version='1.10.9',
-            download_url='https://registry.npmjs.org/@types/underscore/-/underscore-1.10.9.tgz',
-            type='npm',
-        )
-        load_resources_from_scan(self.get_test_loc('match/nested/underscore-1.10.9-ip.json'), self.test_package2)
-        index_package_directories(self.test_package2)
-
-    def test_api_approximate_directory_content_index_list_fingerprint_lookup(self):
-        test_fingerprint = '00000007af7d63765c78fa516b5353f5ffa7df45'
-        response = self.client.get(
-            reverse('api:approximatedirectorycontentindex-list'),
-            data={'fingerprint': test_fingerprint}
-        )
-        self.assertEqual(200, response.status_code)
-        results = response.data.get('results', [])
-        self.assertEqual(1, len(results))
-        result = results[0]
-        expected_package = 'http://testserver' + reverse('api:package-detail', args=[self.test_package1.uuid])
-        expected_result = {
-            'fingerprint': '00000007af7d63765c78fa516b5353f5ffa7df45',
-            'package': expected_package
-        }
-        self.assertEqual(expected_result, result)
-
-    def test_api_approximate_directory_structure_index_list_fingerprint_lookup(self):
-        test_fingerprint = '00000004d10982208810240820080a6a3e852486'
-        response = self.client.get(
-            reverse('api:approximatedirectorystructureindex-list'),
-            data={'fingerprint': test_fingerprint}
-        )
-        self.assertEqual(200, response.status_code)
-        results = response.data.get('results', [])
-        self.assertEqual(1, len(results))
-        result = results[0]
-        expected_package = 'http://testserver' + reverse('api:package-detail', args=[self.test_package2.uuid])
-        expected_result = {
-            'fingerprint': '00000004d10982208810240820080a6a3e852486',
-            'package': expected_package
-        }
-        self.assertEqual(expected_result, result)
-
-    def test_api_approximate_directory_content_index_match_no_match(self):
-        test_fingerprint = '000000020e1d2124040134564e1941a6a620db34'
-        response = self.client.get(
-            reverse('api:approximatedirectorycontentindex-match'),
-            data={'fingerprint': test_fingerprint}
-        )
-        results = response.data
-        self.assertEqual(0, len(results))
-
-    def test_api_approximate_directory_structure_index_match_no_match(self):
-        test_fingerprint = '00000004d10982789010240876580a6a3e852485'
-        response = self.client.get(
-            reverse('api:approximatedirectorystructureindex-match'),
-            data={'fingerprint': test_fingerprint}
-        )
-        results = response.data
-        self.assertEqual(0, len(results))
-
-    def test_api_approximate_directory_content_index_match_close_match(self):
-        # This test fingerprint has a hamming distance of 7 from the expected fingerprint
-        test_fingerprint = '00000007af7d63765c78fa516b5353f5ffa7d000'
-        response = self.client.get(
-            reverse('api:approximatedirectorycontentindex-match'),
-            data={'fingerprint': test_fingerprint}
-        )
-        results = response.data
-        self.assertEqual(1, len(results))
-        result = results[0]
-        self.assertEqual(test_fingerprint, result['fingerprint'])
-        expected_matched_fingerprint = '00000007af7d63765c78fa516b5353f5ffa7df45'
-        self.assertEqual(expected_matched_fingerprint, result['matched_fingerprint'])
-        expected_package = 'http://testserver' + reverse('api:package-detail', args=[self.test_package1.uuid])
-        self.assertEqual(expected_package, result['package'])
-        self.assertEqual(0.9453125, result['similarity_score'])
-
-    def test_api_approximate_directory_structure_index_match_close_match(self):
-        # This test fingerprint has a hamming distance of 7 from the expected fingerprint
-        test_fingerprint = '00000004d10982208810240820080a6a3e800000'
-        response = self.client.get(
-            reverse('api:approximatedirectorystructureindex-match'),
-            data={'fingerprint': test_fingerprint}
-        )
-        results = response.data
-        self.assertEqual(1, len(results))
-        result = results[0]
-        self.assertEqual(test_fingerprint, result['fingerprint'])
-        expected_matched_fingerprint = '00000004d10982208810240820080a6a3e852486'
-        self.assertEqual(expected_matched_fingerprint, result['matched_fingerprint'])
-        expected_package = 'http://testserver' + reverse('api:package-detail', args=[self.test_package2.uuid])
-        self.assertEqual(expected_package, result['package'])
-        self.assertEqual(0.9453125, result['similarity_score'])
-
-    def test_api_approximate_directory_content_index_match(self):
-        test_fingerprint = '00000007af7d63765c78fa516b5353f5ffa7df45'
-        response = self.client.get(
-            reverse('api:approximatedirectorycontentindex-match'),
-            data={'fingerprint': test_fingerprint}
-        )
-        results = response.data
-        self.assertEqual(1, len(results))
-        result = results[0]
-        self.assertEqual(test_fingerprint, result['fingerprint'])
-        self.assertEqual(test_fingerprint, result['matched_fingerprint'])
-        expected_package = 'http://testserver' + reverse('api:package-detail', args=[self.test_package1.uuid])
-        self.assertEqual(expected_package, result['package'])
-        self.assertEqual(1.0, result['similarity_score'])
-
-    def test_api_approximate_directory_structure_index_match(self):
-        test_fingerprint = '00000004d10982208810240820080a6a3e852486'
-        response = self.client.get(
-            reverse('api:approximatedirectorystructureindex-match'),
-            data={'fingerprint': test_fingerprint}
-        )
-        results = response.data
-        self.assertEqual(1, len(results))
-        result = results[0]
-        self.assertEqual(test_fingerprint, result['fingerprint'])
-        self.assertEqual(test_fingerprint, result['matched_fingerprint'])
-        expected_package = 'http://testserver' + reverse('api:package-detail', args=[self.test_package2.uuid])
-        self.assertEqual(expected_package, result['package'])
-        self.assertEqual(1.0, result['similarity_score'])
diff --git a/matchcode_pipeline/tests/test_api.py b/matchcode_pipeline/tests/test_api.py
index 69a1f27d..2762d7df 100644
--- a/matchcode_pipeline/tests/test_api.py
+++ b/matchcode_pipeline/tests/test_api.py
@@ -203,7 +203,7 @@ def setUp(self):
         self.auth = f'Token {self.user.auth_token.key}'
         self.csrf_client = APIClient(enforce_csrf_checks=True)
         self.csrf_client.credentials(HTTP_AUTHORIZATION=self.auth)
-    
+
     def test_d2d_pipeline_api_d2d_list(self):
         response = self.csrf_client.get(self.d2d_list_url)
 
@@ -215,12 +215,12 @@ def test_d2d_pipeline_api_d2d_list(self):
         self.assertNotContains(response, 'resource_count')
         self.assertNotContains(response, 'package_count')
         self.assertNotContains(response, 'dependency_count')
-    
+
     @mock.patch('scanpipe.models.Run.execute_task_async')
     def test_d2d_pipeline_api_d2d_create(self, mock_execute_pipeline_task):
         # load upload_file contents
         data = {
-            'input_urls': ['https://github.com/nexB/scancode.io/raw/main/scanpipe/tests/data/d2d-elfs/from-data.zip#from', 
+            'input_urls': ['https://github.com/nexB/scancode.io/raw/main/scanpipe/tests/data/d2d-elfs/from-data.zip#from',
                            'https://github.com/nexB/scancode.io/raw/main/scanpipe/tests/data/d2d-elfs/to-data.zip#to'],
         }
 
@@ -231,7 +231,7 @@ def test_d2d_pipeline_api_d2d_create(self, mock_execute_pipeline_task):
         mock_execute_pipeline_task.assert_called_once()
         response = self.csrf_client.get(response.data['url'])
         self.assertIn('codebase_resources_discrepancies', response.data)
-    
+
     def test_d2d_pipeline_api_run_detail(self):
         run1 = self.project1.add_pipeline('d2d')
         url = reverse('run-detail', args=[run1.uuid])