semvec/pypi-packages huggingface dataset:
download-logs 1.0.0
A tool to download DWX logs from AWS
Requires Python: >=3.5, <4
Keywords: tools,setuptools,development

# Log Tool


Log Tool is a tool to download DWX logs from AWS for debugging purpose.

Specifically, it is to provide an utility to
- collect logs from AWS for a certain DB Catalog
- collect logs for a specific or all computes within the DB Catalog

## Requirements
* python 3.5+
* boto3
* tzlocal
* pytz
* AWS credentials set up

## Usage
Download logs from a certain DB Catalog:
```sh
$ python3 download_logs.py warehouse [-h] --warehouse_id WAREHOUSE_ID
        --env_name ENV_NAME [--env_id ENV_ID]
        --start START_TIME [--end END_TIME]
        [--local_path LOCAL_PATH]
        [--component {metastore}]
```

Download logs from a certain compute:
```sh
$ python3 download_logs.py warehouse [-h] --compute_id COMPUTE_ID
        [--warehouse_id WAREHOUSE_ID]
        --env_name ENV_NAME [--env_id ENV_ID]
        --start START_TIME [--end END_TIME]
        [--local_path LOCAL_PATH]
        [--component {hiveserver2,coordinator,executor,all}]
```
Note: local_path has a default value of "logs"
## Example commands
DB Catalog
```sh
$ python3 download_logs.py warehouse --env_name cslaws50-xvbp-dwx-external
        --env_id env-2zxvbp --start 06/19/2020-10:15:00 --end 06/19/2020-10:30:00
        --local_path local_logs --warehouse_id warehouse-1592579790-b5fq
```
Compute
```sh
$ python3 download_logs.py compute --env_name cslaws50-xvbp-dwx-external --env_id env-2zxvbp
        --start 06/19/2020-10:15:00 --end 06/19/2020-10:35:00
        --compute_id compute-1592580146-tljc --local_path local_logs
```