Change control logic to include case where name is present

- A strange case like "John Doe <>" will result in the
  parser returning a name but no email
- When the random email is not generated then the analysis
  pass will break when it sends the post message to the id
  service with no email

- If a name is given but an email address is not then we
  previously would not enter the else statement where a random
  email address is generated and assigned

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Reviewed-by: Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
Automatically generate analysis time windows

If the configuration file does not specify explicit releases to
construct the analysis timeline, use intervals of 3 months.

[WM: Add patch description]

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Signed-off-by: Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
Save configuration file to temp location

- In the case of manipulating configuration values after loading
  the configuration file we need to save the changes because
  other analysis phases again read the configuration file

- We now save the project configuration file to a temporary
  location since we don't want to overwrite the original

- The project file location variable is then changed to this new
  temp file so that future reads will be on the manipulated
  values, not the originals

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Signed-off-by: Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
Exit cluster analysis if Adjacency matrix is empty

- There is no reason to proceed when there is no edges
  in the adjacency matrix

- Furthermore, when no edges are present the analysis breaks
  when trying to save the empty graphs

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Reviewed-by: Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
Adapt R configuration sanity checks to revision range generation

db3de9f8bd3 introduced automatic means for revision range generation,
but this is not yet known to the R configuration parser, which
exits with an error if no explicit ranges are specified. Honour
the update.

Signed-off-by: Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
Handle empty clusters in the cluster analysis stage

For small projects, it can happen that certain revision ranges
contain only a single contributor, leading to pathological
collaboration graphs. This case is handled partially
in 92fd49b, but some problems in the R code remain. Fix
them by ignoring empty clusters.

Signed-off-by: Wolfgang Mauerer <wolfgang.mauerer@siemens.com>
Create weighted graph from adjacency matrix

- If the weighted parameter is not set to true then the edge
  list that is produced from the adjacency matrix will just be
  a multiplicity of the matrix value

- When writting the graphs to the dot files the edges also are
  written with a multiplicity instead of a separate weight value
  which just makes enormously redundant files

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Scale edge weight before generating dot files

- Graphviz breaks when the edge weights are extremely high

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Do not bother saving singleton communities

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Abort cluster analysis when off diagonal sum is zero

- It turns out the simple matrix sum based abort will
  not handle the case where all edges are self edges,
  when a matrix only has diagonal elements then the
  analysis fails

- We now compute the off diagonal sum to decide whether
  to abort

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Add check to ensure the revisions list is unique

- The revisions list should be unique values, non-unique values
  cause later failures

- The duplicate revisions occur when a project has been very
  inactive and thus has no commits between two different analysis
  windows

- We now check to make sure the windows do not overlap in terms
  of commits

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Add default for global graph query

- There actually should not be a need to indicate a cluster
  method since global graphs have no method applied to them

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
If MySQL timeout occurs, ping server and attempt reconnection

- Currently the mysql connection is made immediately and then
  the scrapping of the VCS starts

- If the scraping takes a long time then a timeout can
  occur for exampe in very large projects like mozilla

- When the timeout occurs then the analysis fails once the
  scraping is done and attempts are made to write data to the db

- If the exception for a lost connection is raised we now
  make a reconnection attempt and re-execute the query

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Change mechanism to retrieve start and end date for analysis

- We need the start and end date for limiting the history for the
  git blame analysis

- In the old approach there is a case where no cmt list exists
  and then the retrieval fails, it also wastefully looped through
  the whole list of commits

- We now retrieve the dates immediately after finding and sorting
  the dates for all commits and thus avoid looping though the
  entire commit list again and simultaneously fixing the emtpy
  commit list issue

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Fix case where ml analysis period is subset of VCS analysis periods

- the mailing list analysis periods may not always include
  the analysis periods for the version control system

- now we check for the case where the period falls outside
  the mailing list analysis range ignore the periods rather
  than fail completely

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Add precondition check for author of e-mail

- More defensive programming directly after loading the corpus
  to avoid failures in later analysis stages

- There are a variety of ways the author can be formatted, however
  the id service will fail if the author is not formatted according
  to the "name <email@something.com>" convention

- We now check for a many different possible cases such as
  name someone at gmail.com
  name someone@gmail.com
  someone@gmail.com name
  <someone@gmail.com> name
  name

- For each case we try to normalize it to the id service's convention

- If no name is present we assign "unknown" and if no email is present
  we create one by appending the name to "could.not.resolve@"

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
The ts analysis requires at least 3 revisions

- With less than 3 revisions the analysis fails

- Check this requirement and log error if not satisfied

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Bug Fix: "if" instead of "elif" causing incorrect exception raising

- In the case of a deadlock error the 1213 exception is thrown and
  caught but the following if block for handling server connection loss
  evaluates to false and the else block is executed and raises the
  exception

- The result is every deadlock error ends up terminating execution
  because the exception is uncaught after being raised

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Add support for analyzing ruby files

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>
Improve mechanism to retrieve a commit date

- Old mechanism used a function that first retrieved all
  commits reachable by the revision to find
  a single date then created a commit object only to retrieve
  the commit date

- Instead we limit the git query to return only one commit date
  and avoid a whole lot of wasted cycles

Signed-off-by: Mitchell Joblin <mitchell.joblin.ext@siemens.com>