okfn/openspending — Issue #286: [super] Improved and well-factored (ET)L Process
Author: rgrp
Created: 2011-11-04 18:16:07+00:00
State: open

[This comes out of discussion today with @pudo. This is for discussion. @mk270: please review!]

## Terminology

For purposes of this discussion:

Data Descriptor:

* Dataset (metadata)
* model - description of model in openspending
* mapping - between source data and model
* views - description of views (somewhat optional)

Current terminology: Mapping = Model + Mapping

## Overview of the process

Plus examples of how it would appear from command line

### Generate model

ostool model-generate {mycsv.csv - or }
> ... yourmodel.json
>
# edit yourmodel.json
> ...

### Validate model

ostool model-validate yourmodel.json

### Validate data (dry run)

ostool data-check {mycsv.csv} {yourmodel.json}
> OK - {json-dict showing the entry generated}
> FAIL - ...
> OK - ...
Repeat and rinse until all OK then ..

### Load the data

ostool data-load {mycsv.csv} {yourmodel.json} {openspending-url}
=> [Oh dear your model looks like it's version 3 and we are now at version 51 please upgrade see {doc-location}]
=> yourmodel.json saved in the db + success message

### View generation (optional)

ostool view-generate {yourmodel.json} {openspending-url}
...