mozilla/mig — pull request #23 by jvehent (2014-08-29 16:45:02+00:00)
Title: filechecker syntax v2

**THIS IS A PROPOSAL: DO NOT MERGE**

I'm hoping for comments on the JSON syntax, to make sure that it is easy to read and easy to write by hand. The current syntax does a good job at being self explanatory, and I don't want to lose that in the new syntax.

The current parameters syntax of the filechecker module is very compact, but suffers from a number of limitations.
```json
{
"module": "filechecker",
"parameters": {
"/home/*/.ssh/*": {
"regex": {
"private ssh key in homedir": [
"-----BEGIN RSA PRIVATE KEY-----"
]
}
},
"/etc/passwd": {
"regex": {
"test the presence of some users": [
"^ulfr",
"^kang"
]
}
}
}
}
```
* To perform a search on multiple locations, the search block needs to be duplicated for each location.
* Indicators are very hard to point to, there's no indicator ID, which makes adding a condition of the type `ioc1 or ioc2` almost impossible
* The syntax has no room for options. We can add flag to specific the max depth of a search, or prevent the search from crossing file system types.

The proposed syntax below fixes these limitations, but at the cost of a slightly more verbose structure. The goal is to avoid storing information is JSON keys, because that makes the structure hard to parse. Instead, each search maps to a static structure that is easy to extend if needed.
```json
{
"module": "filechecker",
"parameters": {
"searches": [
{
"name": "private ssh key in homedir",
"alias": "pkeyhomedir",
"path": [
"/home/*/.ssh/*"
],
"regex": [
"-----BEGIN RSA PRIVATE KEY-----"
]
},
{
"name": "test the presence of some users",
"alias": "localuser",
"path": [
"/etc/passwd"
],
"regex": [
"^ulfr",
"^kang"
]
},
{
"name": "verify that root uses a strong salted password",
"alias": "strongrootpasswd",
"path": [
"/etc/shadow"
],
"regex": [
"^root:\\$(2(a|y)|5|6)\\$"
]
},
{
"name": "flag if shadow uses md5 for root",
"alias": "rootmd5",
"path": [
"/etc/shadow"
],
"regex": [
"^root:\\$1\\$"
]
},
{
"name": "look into /etc/ only, no recursive walk down the path",
"alias": "etcnorecursive",
"path": [
"/etc/"
],
"regex": [
"^something interesting$"
]
"options": {
"maxdepth": 0
}
},
{
"name": "look into /etc/ and all subsequent directories",
"alias": "etcrecursive",
"path": [
"/etc/"
],
"regex": [
"$another interesting thing$"
],
"options": {
"maxdepth": -1,
"crossfs": false
}
},
{
"name": "detect known dangerous binaries",
"alias": "dangerousbin",
"path": [
"/bin/*",
"/sbin/*",
"/usr/bin/*",
"/usr/sbin/*",
"/opt/*"
],
"sha256": [
"1e2699ff1f9238c58390c1ada53f4f21032ca5e0946bfb54a5a144452e6efc82",
"286c39ec3d8e4f15f353dca350ca7575e0269dba808206f3ce8d1a3ea142b353"
]
}
],
"condition": "pkeyhomedir or !localuser or !strongrootpasswd or rootmd5 or dangerousbin"
}
}
```
This syntax should map to the following Go struct in the filechecker module:
```Go
type Parameters struct {
Searches []search `json:"searches,omitempty"`
Condition string `json:"condition,omitempty"`
}
type search struct {
Name string `json:"name"`
Alias string `json:"alias"`
Path []string `json:"path"`
Regex []string `json:"regex,omitempty"`
Filename []string `json:"filename,omitempty"`
MD5 []string `json:"md5,omitempty"`
SHA1 []string `json:"sha1,omitempty"`
SHA256 []string `json:"sha256,omitempty"`
SHA384 []string `json:"sha384,omitempty"`
SHA512 []string `json:"sha512,omitempty"`
SHA3_224 []string `json:"sha3_224,omitempty"`
SHA3_256 []string `json:"sha3_256,omitempty"`
SHA3_384 []string `json:"sha3_384,omitempty"`
SHA3_512 []string `json:"sha3_512,omitempty"`
Options options `json:"options,omitempty"`
}
type options struct {
MaxDepth float64 `json:"maxdepth,omitempty"`
CrossFS bool `json:"crossfs,omitempty"`
}
```