intertek-data-importer-demo 0.1.0
Add a short description here!
License: MIT

# Intertek Data Importer

Extract and transform EPA data (Monthly) into ingrid csv file compatible to upload into Intertek's INGRID application.

#### Command Usage

```python
>>> getepadata -h
>>> getepadata --help
>>> getepadata --startdate 2022-01 --enddate 2022-03
>>> getepadata -sd 2022-01 -ed 2022-03
>>> getepadata -sd 2022-01 -ed 2022-03 -o data.csv
>>> getepadata -sd 2022-05 -st wy
>>> getepadata -sd 2022-01 -ed 2022-03 --output data.csv
>>> getepadata -sd 2022-01 -ed 2022-03 -o data.csv -st ak
>>> getepadata -sd 2022-01 -ed 2022-03 -o data.csv --state ca,wy
>>> getepadata -sd 2022-01 -ed 2022-03 -o data.csv --state ak --uri https://gaftp.epa.gov/dmdnload/emissions/hourly/monthly/
>>> getepadata -sd 2022-01 -ed 2022-03 -o data.csv -st ca,wy --gcode assets.csv
>>> getepadata -sd 2022-01 -ed 2022-03 -o data.csv -st ca,wy -gc assets.csv
>>> getepadata --startdate=2022-01 --enddate=2022-03 --output=data.csv --state=ca,wy
```

Intertek data importer allows you to get all the files for given start and end date intervals and stores easily. There’s
no need to manually update any codebase, you can easily give inputs (Arguments) to 'getepadata' command to extract and
transform data.

### Command Options

* startdate
    * Info: Starting date for data extraction. Ex. 2022-01
    * format: yyyy-mm
    * Required Argument
    * Options: --startdate / -sd
* enddate
    * Info: End date for data extraction. Ex. 2022-02
    * format: yyyy-mm
    * Optional Argument (If Optional Processes all available year month after start)
    * Options: --enddate / -ed
* output
    * Info: Valid output filename for ingrid data file. Ex. '2022_jf.csv'
    * format: path
    * Optional Argument
    * Defaults: 'ingrid_epa.csv'
    * Options: --output / -o
* state
    * Info: State abbreviation. Ex. ak, ak,wy
    * format: string (length of 2)
    * Optional Argument
    * Defaults: "" (Gets for all states available.)
    * Options: --state / -st
* uri
    * Info: Location of the monthly data folders at EPA.
    * format: URI
    * Optional Argument
    * Defaults: https://gaftp.epa.gov/dmdnload/emissions/hourly/monthly/
    * Options: --uri / -uri
* gcode
    * Info: Specifies Unit ID for plant. (valid csv file)
    * format: path
    * Optional Argument
    * Options: --gcode / -gc

### Output files

* All data will be stored at project level (i.e 'intertek_data_importer') **out/** directory
* Files includes:
    * final output file with name given as per output argument. Ex. '2022_jf.csv'
    * **source/** directory with all fetched zip files.
    * User log file with format as 'UserLog_{current timestamp}.log'.
    * Developer log file with format as '.dev_{current timestamp}.log'.

## Cloning the repository

  ```console
  >>> git clone https://gitlab.com/abhinav15/intertek_data_importer.git 
  >>> cd intertek_data_importer
  ```

## Installing Dependencies and Supported Versions

Intertek Data Importer is available on PyPI:

```shell
python -m pip install intertek-data-importer
```

Intertek Data Importer officially supports Python 3.8+.

## Testing

  To test the data importer getepadata module.
  
  ```python
  >>> python -m unittest
  ```