Original commit
Netra main entry class is now more civilized. TwitterUser is now commented out.
Primary MySQL implementation is in place.
Not have the ability to listen to all tweets.
Updated TweetListener to have a buffer of 1000 objects. Commits to memory once the buffer is full. Also commits when errors occur.
Now have the ability to listen in all twitter communications and store it in database.
Architectural changes
Added hadoop from master heisenberg for archival purposes. MapReduce project WordCounter has been created. The project compiles fine into a jar. This jar has not been tested yet.
Script written to support realtime data analytics.
Analyze data hadoop automation script fixed. Now works flawlessly. However, one does need to startup hadoop cluster before running it.
Added scripts for resource monitoring. analyseData.pl now makes use of this resource management tool and starts it right before running jar file on hadoop. analyseData now also shows execution time for mapreduce.
Resource monitor is now fully functional and more efficient at getting memory stats from bots. analyseData.pl script has been changed to accommodate -statCollectionDirectory flag of Resmon.pl script. Added a folder which contains graphical representation of one of hadoop runs. Also added a info document about the normal hadoop run. Two JDKs and a file containing tweets added for archival purposes.


Former-commit-id: 58c11ca91b34f9cc52e09c7e30dff2daf6c60e4f
Used for changing jdks on current node or across cluster.


Former-commit-id: bc5ba28f6ae9048bc9b71768122951073dbaf074
Formats HDFS namenode and starts up hadoop.


Former-commit-id: 799e0f25542fb8e1ddafddb692ea5b1ac6afb58f
Shuts down entire cluster. Neat handy script. To be executed from master node only.


Former-commit-id: 8d0500740eaec8a691b6062e57f8fa2ed5be4260
Now supports hadoop streaming. Makes a backup of output rather than just dying when it exists. Added flags for starting and cleaning hadoop before executing mapreduce jobs.


Former-commit-id: 8fdc00866e1fb2bd693238b84988a9ea33bf90b7
Editing finished. Lock files removed.


Former-commit-id: 9a6e8f15465c03db07943b3020c126fae59443a9
Fixed major bug. Now it stops hadoop only when initHadoop has been defined.


Former-commit-id: af819dfa91ae50f2bc70f9060eb95767dd071775
WordCount mapreduce application compiled with java 7 update 51 jdk.


Former-commit-id: 56e02b8e1762db6148e31c4b65297f105a5ebf94
added How to execute streaming and native mapreduce using analyseData.pl.


Former-commit-id: 1d7f762dd10950ec0bc089c5827d52aaa4af9f6b