mozilla/socorro-crashstats — pull request #364 by peterbe (2013-06-10 21:45:56+00:00)
Title: fixes bug 858245 - Removal of PII from public web API

The way this basically works is that each model now lists their keys. But, because the structure of the response (from the middleware) often very different, the way you list keys needs to be very flexible. The simplest example is:

```python
# for crashstats.crashstats.models.CurrentVersions

API_WHITELIST = (
'end_date',
'featured',
'id',
'product',
'release',
'start_date',
'throttle',
'version',
)
```

A most complex on can be this:

```python
# for crashstats.crashstats.models.CurrentProducts
API_WHITELIST = {
'hits': {
Cleaner.ANY: (
'end_date',
'featured',
'id',
'product',
'release',
'start_date',
'throttle',
'version',
)
}
}
```

How to **scrub** data is best explained with an example:

```python
class ReportList(SocorroMiddleware):

API_CLEAN_SCRUB = (
('user_comments', scrubber.EMAIL),
('user_comments', scrubber.URL),
)
```

"Cleaning" is the only form of scrubbing (apart from removing fields entirely) that is implemented. The cleaning is done by doing replacements with an empty string `''`.

Some models have NO whitelisting at all. That's because my gut tells me they'll never ever contain PII. These are:

* CrashPairsByCrashId
* Status
* CrontabberState

There is only one model that is now **entirely** blacklisted:

* RawCrash

There are a lot of models now that list ALL fields available. This means I might as well have switched off whitelisting (e.g. setting `API_WHITELIST = None`) but I felt this is easier to see and it makes it explicit what's going on. Basically, where this does NOT happen I write down what I deliberately exclude. E.g.:

```python
class CommentsBySignature(SocorroMiddleware):
...

API_WHITELIST = {
'hits': (
'user_comments',
'date_processed',
'uuid',
),
# deliberately not including:
# email
}

API_CLEAN_SCRUB = (
('user_comments', scrubber.EMAIL),
('user_comments', scrubber.URL),
)
```