Added S3Outputer and its Output method. Reducers now write their output to S3!
In master_role, it now loops until all tasks have been marked as "complete" by the TaskManager (instead of just sleeping for 5 seconds and then continuing).
Using environment variables to set the S3 bucket name and AWS access credentials.
Merge branch 'tgarv-dev' into dghubble-dev
Cleaned up some code.
Fixed an issue with counting words, which also fixed an issue with memory use. Commented out some prints.
Started work on a JobManager which can be queried to determine when Jobs are complete and store other JobState information that would previously have been thrown in the Job struct.
Creating directories for the Inputer and Outputer interfaces so new types can be placed there.
Moved Inputer and S3Inputter back to accessors.go temporarily.
Turned pings back on to check for liveness of other nodes. Tried to add a test for it, but that's hard to do.
Added in temporary state storage for telling when a job is done. This is temporary! Will be replaced when the job manager is built.
Added basic statsManager to do things like time jobs and tasks
Added task tracking to statsManager, tells how long it takes a task to run.
Updated TaskManager so that instead of assigning the worker for a task randomly, it now checks to make sure the worker is alive first.
Updated test for pings/dead workers to test TaskManager.getWorkerIndex().
Extended the JobManager capabilities. Needs some cleanup. Added unit tests for the JobManager.
Integrated jobManager to replace temporary jobDone map. Added a method to the Task Manager to help reassign Tasks when nodes have failed. The related code for failure handling needs reorganization.
Removed an import line that was breaking the package at compilation when running the tests.
Merge
Fixed error with TaskManager.listTasks where it wasn't returning the correct tasks. Also fixed an error in awaitTasks that was waiting for tasks of the wrong status.