24 July 2013 -> Commit 1

Massive Update here:
This is all the commits from Monday the 22nd through wednesday the 24th.

------------
Created NewCompare, a copy of Compare.java. It has some small
modifications (returns List<String> instead of List<Object>) and does
not require files to run, though it can use them. It only gives the
initial marked list, no filtering out after.
--------------
Added all the Dijkstra code, made some modifications so DirectedEdge and
EdgeWeightedDigraph now display strings as well. Dont' know how to use
the Dijkstra algorithm with it yet, still working on that. It is
non-essential though, so it may not be completely necessary.
----------------
Completely reworked the dictionary preparation method for removing data
between files. It now takes up to 5 seconds, whereas it took around 2
minutes before. The slowest part of it is filterCompData(), so I'll need
to rework that too. The new remove...() method uses
LinkedHashSet<String>. It reads every line in both files, adding the
ones from main first. It then rewrites all the ones from main back into
main, and all the ones left are the ones that were not in main. It
writes those to notMain.
--------------------
Fixed the getNthWord method in Reader. It can now also read the first
word, and will find the word you want whether there is a space in front
or not.
-------------
Added Preparer.java. It initializes two TreeMaps for Compare to use. map
is a <String, Integer> that maps the goodTerms frequency to the word.
abbrMap is a <String, String> that maps the abbreviation to its
definition.
----------
Added the goodterms.origin file. This new dictionary contains all java
identifiers used in the databases and the frequency the programmers use
them with. It is the new "comments" dictionary, and is the preferred
dictionary for searches.
---------------
Added some test files, which can be run from PracticeCompare.java. These
files are used to test the analyzeIdentifier() method with 31 different
identifiers, in a mix of both camelCase and under_score.
24 July 2013

Variables now expand if they are the start of another word! ex. Rect =>
Rectangle, or butt => Button.
-----------
Added beginsWith method in Compare, returns the best matching term that
begins with the identifier IF there are any. If not, it still doesn't
expand the terms.
----------
Need to make a way that expands terms through higher probability. Giving
words like "avg" in Newcompare, when termAlgo'd will return the proper
word, but in Compare they won't. Find out why, try and get it to work in
compare.
25 July 2013

Major changes:
1. Added Daniel's java identifier list. Started modifying compare.java
to run with it.
2. Modified stopWordRemoval.java, as for now we won't need to remove
stop words.
3. Added PracticeCompare information so I can test with new data.
--------------
TODO: Finish Pooka_Oracle project for Nasir.
Edit orderedLettersContainedPrecision so I can get better accuracy with
some words.
26 July 2013

1. Fixed OrderedLettersPrecision(). Added the (if orderedPrecision !-
100, don't include) stuff.
-------
2. Changed CompareCopy around a bit, made the algorithm slightly better.
Removed all code from this, it is now probably the main comparison
method.
Added identifier expansion functionality for CompareCopy
-----------
3. Added Daniel's library, using it for expanding anything <=3 in size.
---------------
4. New Methods in PracticeCompare to expand the Pooka_Oracle.csv file.
Still needs some work (see my booklet), but it's getting better. Running
currently at around 1000 lines / minute on the work computer.
26 July 2013 - Part 2

Made everything run through expander, and removed any results that have
Integer, String, Object, or a single letter on the right side of
DanDict.
-----------------
Started working on the better comparison algorithm in PracticeCompare
for once terms are expanded, did not work so I commented it out.
-----------------
Ran responses 2 and responses 3 to compare if either of those are
better, though I'm going to assume that they are both running at around
the same precision because everything runs through Expander anyway.
26 July 2013 - final.

added a jars folder, and the results from responses# are in.
29 July 2013

Moved all dictionaries to a new folder
-------------
Changed parameters of cost() so close variables will always make the
more frequent one be expanded.
----------------
Changed expander to use frequencies, built a frequency table for small
variables
---------------
Added isUnderScored, isCamelCased and isWord methods to
StopWordRemoval.java and Reader.java
30 July 2013

Merging back into master. This is the final commit, as I am removing
this from github.
Merge branch 'Work-Branch'