Change mechanism to parse sign-off tags and add apache style tags

- We now use regular expressions rather than using the string
  function "startswith"

- Additional tags are added which address how apache projects
  annotate the commit message

- Another benefit of this mechanism of matching is that we match
  without case sensitivity, this is especially imporant for apache style
  tags where I noticed variants of Patch, patch and PATCH tags
  which makes the case matching annoying and using tolower(..)
  would reduce performance

- We now also get rid of the massive if conditional and consolidate
  the patterns

- Change parser to exactly match our tag patterns instead of generic
  sign-off pattern

- apache style tags are less strictly formed which demands
  parsing them differently than how we handles kernel tags
  for example there can be an arbitray number of white space
  characters between tag and name and we must check entire
  commit description for sign off tags not just
  the last line because often times tags are not found only
  on the last line, again using regular expressions should
  reduce the performace decrease associated with  parsing
  the entire commit description

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Signed-off-by: Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
Bug Fix: statement incorrectly in loop scope

- seems like the commit message does not need to be inside
  the loop and it means that we are double counting tags
  and wasting cycles on parsing the same message twice

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Signed-off-by: Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
Add Apache Thrift configuration file

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Signed-off-by: Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
Fix: Id case where no email address is found for a developer

- this case happens when a developer does not add there
  email in a commit

- some projects are very strict about always including a developer
  email but in the case of apache style tags the email is most
  often not included, only the developer name is

- developers without an email were previously assigned a generic replacement,
  the problem with this approach is that the email is used for
  merging aliases and thus everyone with the same email gets assigned
  the same person Id

- we now introduce a random string email replacement so that the
  alias merging will resolve unique Ids for people without an
  email address

- an alternative solution would be to change the order in which
  the id service performes alias merging and keep the generic
  replacement email, I think that the generic email is a bad
  idea in general since the nature of an email address is
  unique to the individual or organization that it represents
  and we destroy that notion of uniqueness when we assign a generic
  email, I think we still need to know when an email was artificially
  generated and thats why we need a generic plus a unique component
  for the email as provided by this patch

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Signed-off-by: Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
Set R environment locale to english

- if the locale is not english the parser will silently incorrectly
  parse emails

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Signed-off-by: Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
Fix: Three digit group indexing is too small for large projects

- Now index groups with 4 digits

- previously if the number exceeds three digits an error caused
  execution to fail

- In the case of large projects like the linux kernel we can
  exceed the previous 3 digit limit

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Signed-off-by: Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
Check corpus for precondition violations

- later on in the analysis there are several conditions which
  must be satified otherwise a failure can result

- we introduce a new function where all preconditions can be checked
  for and problematic documents are removed from the corpus

- the particular preconditions added here prevent failures
  from occuring in the threads(..) function in the tm.plugin.mail
  library which fails when multiple reference Ids are found in
  an email

- In the case where multiple reference Ids are found we keep the
  first reference Id and discard the remaining references by manipulating
  the corpus

[WM: Eliminiate two-line string literal created by inserting approx.
200 whitespace characters]

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Signed-off-by: Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
Add communication graph to database

- Add communication graph built up from email exchanges
  to the database

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Signed-off-by: Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
Refactor mechanism to store graph data into the database

- separate the graph indexing manipulation from the database
  writing functionality
- remove redundant cluster user index mapping, we already do
  that to get the edge lists

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Signed-off-by: Wolfgang Mauerer <wolfgang.mauerer@siemens.com>