npm package: @itentialopensource/archive-job-data
Description: [Deprecated] Pre-built to archive jobs and tasks collections
Author: Itential Artifacts
Latest version: 0.0.17-2021.2.8

## _Deprecation Notice_
This Pre-Built has been deprecated as of 04-30-2024 and will be end of life on 04-30-2025. The capabilities of this Pre-Built have been replaced by the [IAP - Workflow Utilities](https://gitlab.com/itentialopensource/pre-built-automations/iap-workflow-utilities)

#  Archive Job Data

## Table of Contents

- [Archive Job Data](#archive-job-data)
  - [Table of Contents](#table-of-contents)
  - [Overview](#overview)
  - [Installation Prerequisites](#installation-prerequisites)
  - [How to Install](#how-to-install)
  - [How to Run](#how-to-run)
  - [Inputs](#inputs)
  - [Additional Information](#additional-information)


## Overview

This pre-built allows IAP users to archive the `jobs` and `tasks` collections to keep IAP performant. It also allows the users to restore the archived jobs and tasks. Users have the ability to choose:
1. The number of days older than which, all qualifying data will be archived.
2. Custom names for the archive collections to store the matching `jobs` and `tasks` documents.
3. To keep or delete the matching documents from the original `jobs` and `tasks` collections after backing them up.
4. Restore the archived `jobs` and `tasks` documents and delete the archived collections.

Users can use the included Operations Manager item to schedule the pre-built to run at regular intervals.

Notes: 
1. For every run, new jobs and tasks collections would be created with the current timestamps appended to the end of the jobs and tasks collection names put in by the user.
2. Only jobs and tasks related to jobs that are either in *complete* or *canceled* state will be considered for archiving. Jobs that are in errored or incomplete state will be left untouched.
3. Getting an `RangeError [ERR_OUT_OF_RANGE]` typically indicates that your bundle size is too large. Try setting it lower within the automation catalog. 
4. Very large databases with a lot of jobs and tasks to archive will typically result in multiple archive collections within the database. 

## Installation Prerequisites

Users must satisfy the following prerequisites:

- Itential Automation Platform
  - `^2021.2`
- Adapter DB Mongo
  - `^0.5.7`
  
## How to Install

To install **Archive Job Data**, verify you are running a supported version of Itential Automation Platform (IAP) as listed above in the Installation Prerequisites section. If you do not currently have App-Artifacts installed as a service on your node, the `.tgz` file or "tarball" can be obtained from the [Nexus repository](https://registry.aws.itential.com/#browse/browse:app-prebuilt_automations). Please refer to the instructions included in the App-Artifacts README to install it.

**Archive Job Data** can be installed from within Admin Essentials. Search for `archive-job-data` and click the **Install** button.

## How to Run

Use the following steps to run the pre-built:
1. Once the pre-built is installed as outlined in the [How to Install](#how-to-install) section above, go to `Operations Manager` and run the `Archive Jobs and Tasks Collections` automation.
2. In the `Copy/Move Jobs data to Collection` and the `Copy/Move Tasks data to Collection` fields, enter the names for collections you want the matching `jobs` and `tasks` collection documents to be copied to.
3. In the `Copy/Move data from before # days` field, enter the number of days (from the current date) oolder than which you want data to be archived. *Note: Please enter a positive number*
4. In the `MongoDB Adapter` dropdown menu, select the adapter that is connected the MongoDB that contains the Tasks and Jobs Collections that you want to create an archive of.
5. In the `Batch Size` field, select how many jobs you want to archive at a time. 
6. Check the `Create Archive Collections` checkbox if you want to create an archive collection that will be stored in the MongoDB. Unchecking this box will mean no archive will be created. 
7. Check the `Delete copied data from collections` checkbox if you want to discard the matching documents from the original `jobs` and `tasks` collections after they have been backed up. 
8. Check the `Verbose` checkbox if you want to see the results of the pre-built at the end of its run and click on `START` and then on `VIEW JOB`.

To restore the archived `jobs` and `tasks`, run the `Restore Archived Jobs and Tasks` workflow.


Note: The adapter ID for mongoDB in all the respective tasks of the workflow must be changed accordingly.

## Inputs

The inputs for the pre-built are outlined in the following table. To run the workflow directly from IAP/API call (and not through Operations Manager), please encapsulate all of these inputs into a root object with the key `formData`.

<table  border='1'  style='border-collapse:collapse'>
<thead>
<tr>
<th>Input</th>
<th>Description</th>
<th>Type</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>createArchiveCollections</code></td>
<td>Set as <code>true</code> or <code>false</code> to either create or skip creating a new collection for <code>jobs</code> and <code>tasks</code> with archived data.</td>
<td><code>boolean</code></td>
</tr>
<tr>
<td><code>deleteCopiedDataFromCollections</code></td>
<td>Set as <code>true</code> or <code>false</code> to either discard or keep the matching documents from the original <code>jobs</code> and <code>tasks</code> collections after they have been backed up.</td>
<td><code>boolean</code></td>
</tr>
<tr>
<td><code>verbose</code></td>
<td>Set as <code>true</code> or <code>false</code> to either see or not see the results of the pre-built at the end of its run.</td>
<td><code>boolean</code></td>
</tr>
<tr>
<td><code>copyMoveJobsDataToCollection</code></td>
<td>Enter the name of the desired collection (new or pre-existing) where you want the matching documents from the <code>jobs</code> collection to be archived.</td>
<td><code>string</code></td>
</tr>
<tr>
<td><code>mongoDbAdapter</code></td>
<td>Select the Mongo DB Adapter instance from which the documents from <code>jobs</code> and <code>tasks</code> collections data will be archived.</td>
<td><code>string</code></td>
</tr>
<tr>
<td><code>copyMoveDataFromBeforeDays</code></td>
<td>Enter the number of days (from the current date) older than which all data will be archived.</td>
<td><code>integer</code></td>
</tr>
<tr>
<td><code>copyMoveTasksDataToCollection</code></td>
<td>Enter the name of the desired collection (new or pre-existing) where you want the matching documents from the <code>tasks</code> collection to be archived.</td>
<td><code>string</code></td>
</tr>
<tr>
<tr>
<td><code>batchSize</code></td>
<td>Enter the number of jobs that should be archived or deleted on a single iteration.</td>
<td><code>integer</code></td>
</tr>
</tbody>
</table>

Inputs for `Restore Archived Jobs and Tasks` workflow are defined below:

<table  border='1'  style='border-collapse:collapse'>
<thead>
<tr>
<th>Input</th>
<th>Description</th>
<th>Type</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>deleteFromArchiveCollections</code></td>
<td>Set as <code>true</code> or <code>false</code> to either delete or not delete the archived <code>jobs</code> and <code>tasks</code> collections.</td>
<td><code>boolean</code></td>
</tr>
<tr>
<td><code>archiveJobsCollectionName</code></td>
<td>Enter the name of the archived jobs collection that you want to be restored.</td>
<td><code>string</code></td>
</tr>
<tr>
<td><code>archiveTasksCollectionName</code></td>
<td>Enter the name of the archived tasks collection that you want to be restored.</td>
<td><code>string</code></td>
</tr>
<tr>
<td><code>numberOfArchivedCollections</code></td>
<td>Enter the number of archived collections that should be restored.</td>
<td><code>integer</code></td>
</tr>
</tbody>
</table>
  
### Sample Input for the overall Pre-Built:

```
{
  "formData": {
    "createArchiveCollections": true,
    "deleteCopiedDataFromCollections": true,
    "verbose": true,
    "copyMoveJobsDataToCollection": "jobs_archive",
    "mongoDbAdapter": "mongoPronghorn",
    "copyMoveDataOlderThanDays": 3,
    "copyMoveTasksDataToCollection": "tasks_archive",
    "batchSize": 2000,

  }
}
```

### Sample Input for the `Restore Archived Jobs and Tasks` workflow:

```
{
	"deleteFromArchiveCollections": true,
	"archiveJobsCollectionName": "Archive_Jobs_2023-08-03T21:17:11.079Z",
	"archiveTasksCollectionName": "Archive_Tasks_2023-08-03T21:17:11.079Z",
	"numberOfArchivedCollections": 9
}
```

## Additional Information

Please use your Itential Customer Success account if you need support when using this Pre-Built.