shazow/composer — Issue #1: Indexer brainstorming
Author: shazow
Created: 2011-10-21 00:36:04+00:00
State: open

While composer is fed static index files, it's clear that the indexer will accept some kind of programmatic input, perhaps in the form of a ``settings.py`` file.

What should an indexer look like? Things an indexer should be able to do:

- Traverse a directory recursively, select a subset of files based on some criteria.
- Inspect the files and pull out metadata out of them (such as the title of a blog post).
- Utilize arbitrary metadata around a file, such as a git repo it's stored in (fetch the author and time created/updated timestamps from the git log).
- Combine the above in arbitrary ways.

The primary goal of the indexer is to generate routes. A route looks like this:

```javascript
{
"context": {
"title": "Goodbye!"
},
"file": "posts/1.md",
"filters": [
"markdown",
"post"
],
"url": "post/1"
}
```

One possible solution is to create a kind of Python DSL (similar to [Pipe](http://dev-tricks.net/pipe-infix-syntax-for-python)):

```python
routes = Walk('templates/') | Select(suffix='.mako') \
| Filter('mako') \
| Url(lambda r: r['file'][:-5].replace('/', '_'))
```

Each of these will pass a routes dictionary to the next and mutate it as it goes along the chain.

Though probably a more explicit solution is better...