ad-components 0.1.48
Accelerated Discovery Reusable Components.

# Accelerated Discovery Reusable Components

The central implementation of Accelerated Discover Reusable Components. It serves as a wrapper around client libraries we use locally like Dapr and MLflow.

## 1.Installation
All components will be availble using

```shell
pip install ad-components
```

### CLI
Here's an example usage of the CLI

```
usage: adc [-h] [--verbose] [--version] {<component>} ...

Accelerated Discovery reusable components.

positional arguments:
  <component>    the component that you want to trigger.

optional arguments:
  -h, --help     show this help message and exit.
  --version      show program's version number and exit.
```

## 2. Usage
### 2.0. In your pipeline
To use a component in your pipeline, you need to run it in a Step context

```python
from ad.step import DaprStep
from ad.storage import download, upload

with DaprStep():
    resp = download(download_src, download_dest, binding_name=binding)
    print(f"download resp: {resp}")

    resp = upload(upload_src, upload_dest, binding_name=binding)
    print(f"upload resp: {resp}")
```

Running the components inside a step will make sure the client dependencies are handled correctly.


### 2.1. Storage

#### 2.1.2. Python module
You can invoke the manager using native python. Please note that the package must be present in you python environment.

```python
from ad.storage import download, upload

download_resp = download(
    src, dest,
    # binding_name="s3-state",  # Or any other binding
)

upload_resp = upload(
    src, dest,
    # binding_name="s3-state",  # Or any other binding
)
```

#### 2.1.3. CLI

```shell
usage: adc storage [-h] --src PATH --dest PATH [--binding NAME] [--timeout SEC] {download,upload}

positional arguments:
  {download,upload}     action to be performed on data.

optional arguments:
  -h, --help            show this help message and exit

action arguments:
  --src PATH, -r PATH   path of file to perform action on.
  --dest PATH, -d PATH  object's desired full path in the destination.
  --binding NAME, -b NAME
                        the name of the binding as defined in the components.

dapr arguments:
  --timeout SEC, -t SEC
                        value in seconds we should wait for sidecar to come up.
```

> **Note:** You can replace `adc` with `python ad/main.py ...` if you don't have the package installed in your python environment.

##### Examples
1. To download an object from S3 run
```bash
adc storage download \
    --src test.txt \
    --dest tmp/downloaded.txt
```

2. To upload an object to S3 run
```bash
adc storage upload \
    --src tmp/downloaded.txt \
    --dest local/uploaded.txt
```


## 3. Supported components
### 3.1. Storage
#### 3.1.1. Supported operations
Below is a list of the operations you might intend to perform in your component.

##### Upload
Uploads data from a file to an object in a bucket.

###### Arguments
* `src`: Name of file to download.
* `dest`: Object name in the bucket.
* `binding`: The name of the binding to perform the operation.

##### Download
Downloads data of an object to file.

###### Arguments
* `src`: Object name in the bucket.
* `dest`: Name of file to download.
* `binding`: The name of the binding to perform the operation.


##### Dapr configurations
* `address`: Dapr Runtime gRPC endpoint address.
* `timeout`: Value in seconds we should wait for sidecar to come up


## 4. Publishing
Every change to the python script requires a new version to be pushed PyPi registry.


If you have the right (write) permissions, and a correctly-configured `$HOME/.pypirc` file, run the following command to publish the package

```shell
make
```

### 4.1. Increment the version
To increment the version, go to [adstorage/version.py](adstorage/version.py) and increment the version there. Both the [setup.py](setup.py) and the `CLI` will read the new version correctly.

### 4.2 Configure PyPi registry
To be able to push to the package to our private registry, you need to tell PyPi about it. This one-liner command will take care of it for you

```shell
cat << EOF > $HOME/.pypirc
[distutils]
index-servers =
    pypi

[pypi]
repository: https://upload.pypi.org/legacy/
username: __token__
password: $PYPI_TOKEN
EOF
```

> **Note:** The pip package will fetch the version from `ad/version.py` file, so make sure to increment before pushing.