looker_loader 0.1.50
a tool for flexibly loading schemas from bigquery into looker
Requires Python: >=3.10

## looker-loader
# Write rules for how you want fields and measures to be generated from tables into looker.
Are you tired of adding timeframes to dimension_groups manually, of defining simple measures, of adding descriptions to measures?
looker_loader removes a lot of the tedious work involved with loading new tables into Looker.

# Define field_label, group_label, value_format_name etc in a common source of truth
If you find discrepancies in value formats and labelling across tables to be annoying and tedious, wouldnt it be nice to be able to input a label one place, and have it applied for every table? Looker_loader enables this.

# Setup scheduled refreshes of lookml automatically
Fed up with generating new autogenerated tables every time a new field is added to a table?
Use looker_loader to refresh tables on a schedule!

## Quick Start

1. **Install the package**:
   ```bash
   uv add looker_loader
   ```

2. **Create configuration files**:
   - `loader_config.yml` - Define your BigQuery connections and settings
   - `loader_recipe.yml` - Define rules for field transformations


### loader_config.yml

This file defines your BigQuery connections and loader settings:

```yaml
config:
  loader:
    lexicanum: true
    output_path: ./views
  bigquery:
    - project_id: "your-project-id"
      dataset_id: "your_dataset"
      config:
        regex_include: "^fct_|^dim_|^obt_"
        regex_exclude: "_temp$|_backup$"
```

### loader_recipe.yml

This file defines rules for how different field types should be transformed:

```yaml
recipes:
  - name: primary_key
    filters:
      field_order: [0]
    dimension:
      primary_key: true

  - name: hide_records
    filters:
      db_types: [RECORD]
    dimension:
      hidden: true

  - name: group_ids
    dimension:
      group_label: Identifiers
      value_format_name: id
      measures:
        - type: count_distinct
    filters:
      regex_include: _id$|^pk.*_|_code$

  - name: numbers
    filters:
      types: [number]
      regex_exclude: _id$|^pk_|_code$
    dimension:
      group_label: Numbers
      value_format_name: decimal_1
      measures:
        - type: sum
          label: "Sum {{ parent_label }}"

  - name: time_date_values
    filters:
      regex_include: _date
      types: [date]
    dimension:
      timeframes:
        - raw
        - time
        - date
        - week
        - month
        - quarter
        - year
```


3. **Run the loader**:
   ```bash
   uv run looker_loader
   ```



##  Check the Output

The loader will create LookML files in your specified output directory (default: `./views`). You should see files like:

```
views/
├── fct_sales.view.lkml
├── dim_customers.view.lkml
└── obt_daily_summary.view.lkml
```


### Common Issues

**No tables found**
- Check your `project_id` and `dataset_id` are correct
- Verify your BigQuery permissions
- Check if your `regex_include` pattern matches any tables

**Authentication errors**
- Ensure your Google Cloud credentials are properly set up

## Next Steps

Now that you have the basics working:

- **[Explore Configuration Options](https://github.com/rognerud/looker-loader/tree/main/docs/loader-config.md)** - Learn about all available configuration settings
- **[Create Custom Recipes](https://github.com/rognerud/looker-loader/tree/main/docs/loader-recipe.md)** - Build specialized field transformation rules
- **[Use Lexicanum](https://github.com/rognerud/looker-loader/tree/main/docs/lexicanum.md)** - Enhance field definitions by using a common source of truth