sw-aqueduct 2.1.1
A Swimlane content delivery system written in Python
License: MIT
Requires Python: >=3.6.2,<4.0.0

# Aqueduct

![PyPI](https://img.shields.io/pypi/v/sw-aqueduct.svg)
![Status](https://img.shields.io/pypi/status/sw-aqueduct.svg)
![Python Version](https://img.shields.io/pypi/pyversions/sw-aqueduct)
![License](https://img.shields.io/pypi/l/sw-aqueduct)

![Read the documentation at https://swimlane.github.io/aqueduct/](https://img.shields.io/badge/docs-success-green)
![Tests](https://github.com/swimlane/aqueduct/actions/workflows/quality.yml/badge.svg)
![Codecov](https://codecov.io/gh/swimlane/aqueduct/branch/main/graph/badge.svg)

[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)][pre-commit]
[![Black](https://img.shields.io/badge/code%20style-black-000000.svg)][black]

[pypi status]: https://pypi.org/project/sw-aqueduct/
[docs]: https://swimlane.github.io/aqueduct
[tests]: https://github.com/swimlane/aqueduct/actions?workflow=Tests
[codecov]: https://app.codecov.io/gh/swimlane/aqueduct
[pre-commit]: https://github.com/pre-commit/pre-commit
[black]: https://github.com/psf/black

> aq·ue·duct | \ˈä-kwə-ˌdəkt\

Definition of _aqueduct_

1. a conduit for water
2. a content delivery system for the [Swimlane](https://swimlane.com/) platform

> ([What's new?](docs/CHANGELOG.md))

> ([Testing Steps](docs/TESTING.md))

_Aqueduct_ is a Python package to migrate content from one (source) [Swimlane](https://swimlane.com/) instance to another (destination) instance in reliable and repeatable process.

## Table of Contents

* [Why?](#why)
* [Features](#features)
    * [Aqueduct Parameters](aqueduct.md)
* [Getting Started](#getting-started)
* [Usage Examples](#usage-examples)
* [Installation](#installation)
* [Getting Help](#getting-help)
* [Contributing](#contributing)

## Why?

> Currently we only support transferring of content from one [Swimlane](https://swimlane.com/) instance to another that are the same semantic versioning number. You can bypass this but it is **NOT** recommended and there are limitations. Please see the [available parameters for more information](docs/aqueduct.md).

Many [Swimlane](https://swimlane.com/) customers build their own custom content (e.g. Use Cases) to fit their needs. We understand that every organization is different and because of this we have built `aqueduct` to facilitate the _migration_ of this content from one instance to another. During the creation of `aqueduct` we have decided that we want to re-enforce best practices when using [Swimlane](https://swimlane.com/). This is mainly in the form of **NOT** updating content in your production systems (e.g. destination systems). Changes should strictly be made within a development (source) instance and then migrated to your production instance once your testing is complete.

## Features

> By **default** aqueduct will __NOT__ perform a live migration of data. You must specify `dry_run=False` in order to migrate content.

You can view all the available parameters for the main Aqueduct class [here](docs/aqueduct.md).

* By default, `aqueduct` will generate a .zip file containing source, altered, and destination JSONs when an error occurs.
* Providing the `dump_content_path` argument will dump *ALL* source, altered, and destination JSONs into a zip file.
* `Aqueduct` now reports back a list of `homework` items that must be completed manually on the destination.
    * You can access the `homework` list by using the `Aqueduct._homework` property.
* By using the `mirror_app_fields_on_destination` parameter, we will force update application fields.
* By using the `update_default_reports` parameter we will update Default reports.
* You can now add and update applets as a separate component.
* Collect related content based on one or more application names. Use the `gather` and `collect` methods.

### Components

`aqueduct` can sync several different content components (listed below). Each of the listed component links below contain details on how content for that component is synced from one instance to another. Please read this for detailed information behind how `aqueduct` works.

> Please be aware that the order of this list of components is forced whether you are syncing all components (default) or specific components from one Swimlane instance to another.

* [keystore](docs/components/keystore.md)
* [packages](docs/components/packages.md)
* [plugins](docs/components/plugins.md)
* [assets](docs/components/assets.md)
* [workspaces](docs/components/workspaces.md)
* [applets](docs/components/applets.md)
* [applications](docs/components/applications.md)
    * [workflows](docs/components/workflows.md)
* [tasks](docs/components/tasks.md)
* [reports](docs/components/reports.md)
* [dashboards](docs/components/dashboards.md)
* [users](docs/components/users.md)
* [groups](docs/components/groups.md)
* [roles](docs/components/roles.md)

## Getting Started

`aqueduct` is a Python-only package hosted on [PyPi](https://pypi.org/project/sw-aqueduct/) and works with Python 3.6 and greater.

```bash
pip install sw-aqueduct
```

> Please see the [Installation](#installation) section for more information.

## Usage Examples

As a [Swimlane](https://swimlane.com/) customer you can utilize `aqueduct` to migrate content from one instance to another. Additionally, you can utilize our `Swimlane Aqueduct` use case and [plugin](https://apphub.swimlane.com/swimbundles/swimlane/sw_swimlane_aqueduct). To do so please reach out to customer support or your PS contact on how to acquire this SSP (Swimlane Solutions Package).

Before we look at the options available for configuring aqueduct, we must first create two `SwimlaneInstance` objects. One is considered the source instance and the other is the destination instance.

```python
from aqueduct import SwimlaneInstance


sw_source = SwimlaneInstance(
    host="https://10.32.100.xxx",
    username="admin",
    password=""
)

sw_dest = SwimlaneInstance(
    host="https://10.32.100.xxx",
    username="admin",
    password=""
)
```

Once we have these instances configured, we can begin to configure `aqueduct`.  You can view all the available parameters for the main Aqueduct class [here](docs/aqueduct.md).

> By default `aqueduct` will **NOT** transfer your content. You must specify the `dry_run=False` parameter in order to actually transfer content.

```python
from aqueduct import Aqueduct

# Below are the default parameters and values when instantiating a Aqueduct object.
aq = Aqueduct(
    source=sw_source,
    destination=sw_dest,
    dry_run=True,
    offline=False,
    update_reports=False,
    update_dashboards=False,
    continue_on_error=False,
    use_unsupported_version=False,
    force_unsupported_version=False,
    dump_content_path=None,
    update_default_reports=False,
    mirror_app_fields_on_destination=False,
)
```

Once we have configured out `Aqueduct` class instance we can now sync content from one instance to another. To do this we must call the `sync` method on the `Aqueduct` instance.

```python
# this will sync all components listed above.
aq.sync()
```

By default we will sync all components but you can select which components you want to sync by passing in a list of [components](#components) as an argument to the `sync` method.

```python
# You can specify one or more of them as well.
aq.sync(components=['applications', 'plugins', 'workspaces', 'roles']
```

You can also include or exclude specific components themselves. Imagine you have an application and a plugin on your source instance that you do not want to transfer to a destination instance. You can specify this in an `exclude` (or `include`) parameter by passing in the following dictionary:

> Please **NOTE** that if you specify a specific component item to exclude you must also provide that same component in the `components` parameter. If you do not, then it will run all components and only filter for that application name / plugin name as specified.

```python
aq.sync(components=['applications', 'plugins'], exclude={'applications': ['My Source Application Name'], 'plugins': ['My Source Plugin Name']})
```

## Installation

You can install **aqueduct** on OS X, Linux, or Windows. You can also install it directly from the source. To install, see the commands under the relevant operating system heading, below.

### Prerequisites

The following libraries are required and installed by `aqueduct`:

```
pyyaml==6.0
fire==0.4.0
attrs==21.4.0
pydantic==1.9.1
swimlane==10.5.0
packaging>-21
requests>=2.27.1
```

### macOS, Linux and Windows:

```bash
pip install sw-aqueduct
```

### macOS using M1 processor

```bash
git clone https://github.com/swimlane/aqueduct.git
cd aqueduct

# Satisfy ModuleNotFoundError: No module named 'setuptools_rust'
brew install rust
pip3 install --upgrade pip
pip3 install setuptools_rust

# Back to our regularly scheduled programming . . .

python setup.py install

# or install via poetry
poetry install --build
```

### Installing from source

```bash
git clone https://github.com/swimlane/aqueduct.git
cd aqueduct
python setup.py install
# or install via poetry
# poetry install
```

## Getting Help

Please create an [issue](https://github.com/swimlane/aqueduct/pulls) if you have questions or run into any issues.

## Security Issues

Please read [SECURITY.md](docs/SECURITY.md) for details on contacting us related to security issues.

## Built With

* [carcass](https://github.com/MSAdministrator/carcass) - Python packaging template

## Contributing

Please read [CONTRIBUTING.md](docs/CONTRIBUTING.md) for details on our code of conduct, and the process for submitting pull requests to us.

## Versioning

We use [SemVer](http://semver.org/) for versioning.

## Authors

* Josh Rickard - *Initial work* - [MSAdministrator](https://github.com/MSAdministrator)

See also the list of [contributors](https://github.com/swimlane/aqueduct/contributors) who participated in this project.

## License

This project is licensed under the MIT License - see the [LICENSE](docs/LICENSE.md) file for details