semvec/pypi-packages huggingface dataset:
marquez-dbt-snowflake 0.16.1
The Marquez Snowflake adapter plugin for dbt (data build tool)
Requires Python: >=3.6
Keywords: marquez

# dbt-snowflake-openlineage

Plugin for [dbt](https://github.com/fishtown-analytics/dbt) (data build tool) for automatic metadata and lineage collection.

This integration is experimental, and there is high possibility that it will change in the future, 
as together with dbt developers we're figuring what's the best way to extract metadata from it.

## Requirements

```
Python 3.6+
sqlparse>=0.3.1
```

## Installation

```
$ pip install marquez-dbt-snowflake
```

## Configuration

You need to configure dbt's [profiles.yaml](https://docs.getdbt.com/dbt-cli/configure-your-profile) file.
To use this plugin you need to use type: openlineage_snowflake.

It handles the same configuration flags as original snowflake plugin, with the addition of OpenLineage related ones:

```bash
    openlineage_url - sets URL of OpenLineage server where events will be pushed
    openlineage_timeout - optional - sets timeout for connecting to OpenLineage server. By default it's set to 5 seconds.
    openlineage_api_key - optional - sets api key for OpenLineage server
```
marquez-dbt-bigquery 0.16.1
The Marquez BigQuery adapter plugin for dbt (data build tool)
Requires Python: >=3.6
Keywords: marquez

# dbt-bigquery-openlineage

Plugin for [dbt](https://github.com/fishtown-analytics/dbt) (data build tool) for automatic metadata and lineage collection.

This integration is experimental, and there is high possibility that it will change in the future, 
as together with dbt developers we're figuring what's the best way to extract metadata from it.

## Requirements

```
Python 3.6+
sqlparse>=0.3.1
```

## Installation

```
$ pip install marquez-dbt-bigquery
```

## Configuration

You need to configure dbt's [profiles.yaml](https://docs.getdbt.com/dbt-cli/configure-your-profile) file.
This plugin introduces additional type: openlineage_bigquery.

It handles the same configuration flags as original bigquery plugin, with the addition of OpenLineage related ones:

```bash
    openlineage_url - sets URL of OpenLineage server where events will be pushed
    openlineage_timeout - optional - sets timeout for connecting to OpenLineage server. By default it's set to 5 seconds.
    openlineage_api_key - optional - sets api key for OpenLineage server
```
tilemani 0.0.1
A collection of tools to retreive OSM data, rasterize them into customizable style, compute geospatial features and build models of urban space
Requires Python: >=3.6

# TileMani 

TileMani (for Tile Mani(pulation) or Tile Mani(ac)) is a collection of tools to
- retreive street network and geometric (eg. buildings, POIs) from OSM database
- rasterize the retrieved data into a `np.array` with customizable style (e.g.
width, color per road type, building type, green area, etc),
- compute spatial features (eg. average number of intersections, road bearings,
between centrality, or density-based metrics like FSI, GSI),
- build/train a model of urban space from graph-, or grid- structured data
reprlearn 0.0.1
A practice example package
Requires Python: >=3.6

# Training scripts
## Dataset-independence
- `train.py`: train one model (eg. beta-vae, IWAE, bivae) on one specific hyperparamter config
    - E.g. Train `BiVAE` on `osmnx_roads` data of the following `cities`, with images of `bgcolors` 
    ```bash
    nohup python train.py --model_name="bivae" \
    --latent_dim=10 --hidden_dims 32 64 128 256 --adv_dim 32 32 32 --adv_weight 1.0 \
    --data_root="/data/hayley-old/osmnx_data/images" \
    --data_name="osmnx_roads" \
    --cities 'la' 'charlotte' 'vegas' 'boston' 'paris' \
         'amsterdam' 'shanghai' 'seoul' 'chicago' 'manhattan' \
         'berlin' 'montreal' 'rome' \
    --bgcolors "k" "r" "g" "b" "y" --n_styles=5 \
    --zooms 14 \
    --gpu_id=2 --max_epochs=300   --terminate_on_nan=True  \
    -lr 3e-4 -bs 32 \
    --log_root="/data/hayley-old/Tenanbaum2000/lightning_logs/2021-05-18/" &
    ```
  
    - E.g.: Train `BIVAE` on Rotated MNIST of optionally specified subset (given as a filepath to `.npy` file containing 
    the indices from the original Training MNIST data) 
    ```bash
    ## Specify which indices to use among the MNIST -- comparable to DIVA's experiments
    ## change 0 to anything inbtw 0,...,9
    nohup python train.py --model_name="bivae" \
    --latent_dim=128 --hidden_dims 32 64 64 64 --adv_dim 32 32 32 \
    --data_name="multi_rotated_mnist" --angles -45 0 45 --n_styles=3 \
    --selected_inds_fp='/data/hayley-old/Tenanbaum2000/data/Rotated-MNIST/supervised_inds_0.npy' \
    --gpu_id=2
    ```
  
    - E.g.: Train Bivae on multi styles of maptiles from specified cities
    ```bash
    # Train BiVAE on Multi Maptiles MNIST
    nohup python train.py --model_name="bivae" \
    --latent_dim=10 --hidden_dims 32 64 128 256 --adv_dim 32 32 32 --adv_weight 15.0 \
    --data_name="multi_maptiles" \
    --cities la paris \
    --styles CartoVoyagerNoLabels StamenTonerBackground --n_styles=3 \
    --zooms 14 \
    --gpu_id=2 --max_epochs=400   --terminate_on_nan=True  \
    -lr 3e-4 -bs 32 \
    --log_root="/data/hayley-old/Tenanbaum2000/lightning_logs/2021-01-23/" &
    ```
  
### Hyperparameter tuning using `Ray Tune`
- `tune_asha.py`: Use `Tune`'s AsyncHyperBandScheduler to search hyperparameter space more efficiently.
    Use `--tune_metric` to specify the value of `tune.run`'s `metric` argument, e.g. `--tune_metric loss` for 
- `tune_asha_with_beta_scheduler.py`:
- `
    



## Dataset-specific
### Rotated MNIST
- `tune_asha_mnists.py`

### osmnx_roads
- `tune_asha_osmnx_roads.py`

##