JAVA: 
* Replacing RetryHelper with the one from gcs client
* Fixing EntityCreator example and EndToEndTest - bad usage of DatastoreMutationPool
* Using a RetryHelper in DatastoreMutationPool
* Adding DatastoreOutput and DatastoreOutputWriter


Revision created by MOE tool push_codebase.
MOE_MIGRATION=6695


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@558 441dd3cb-e453-d85f-364a-b28e0c74875c

Java: Replace ApiProxyException with more specific exceptions.



Revision created by MOE tool push_codebase.
MOE_MIGRATION=6696


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@559 441dd3cb-e453-d85f-364a-b28e0c74875c

Python:
1. Always access shard states with iterators so that we don't load all into memory at once.
2. Finish a TODO: Remove old constant values.
3. Replace "retry" with "attempt" in parameters.py


Revision created by MOE tool push_codebase.
MOE_MIGRATION=6697


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@560 441dd3cb-e453-d85f-364a-b28e0c74875c

Java: Update GCS client maven version

Revision created by MOE tool push_codebase.
MOE_MIGRATION=6698


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@561 441dd3cb-e453-d85f-364a-b28e0c74875c

Java: Bumping MR gcs_client dependency to 0.3.2



Revision created by MOE tool push_codebase.
MOE_MIGRATION=6699


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@562 441dd3cb-e453-d85f-364a-b28e0c74875c

Python: Add job id to url.


Revision created by MOE tool push_codebase.
MOE_MIGRATION=6700


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@563 441dd3cb-e453-d85f-364a-b28e0c74875c

Java: Cleanup


Revision created by MOE tool push_codebase.
MOE_MIGRATION=6701


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@564 441dd3cb-e453-d85f-364a-b28e0c74875c

Java: Limiting lookupTask Datastore#get operation to sequential batches of 20 keys at a time.

Revision created by MOE tool push_codebase.
MOE_MIGRATION=6702


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@565 441dd3cb-e453-d85f-364a-b28e0c74875c

Java: More robust handling against low memory on shuffle.



Revision created by MOE tool push_codebase.
MOE_MIGRATION=6703


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@566 441dd3cb-e453-d85f-364a-b28e0c74875c

Java: Start handling shard-level failures

Revision created by MOE tool push_codebase.
MOE_MIGRATION=6704


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@567 441dd3cb-e453-d85f-364a-b28e0c74875c

Python: Fail MapReduceJob.start if provided queue is bad or does not exists.


Revision created by MOE tool push_codebase.
MOE_MIGRATION=6705


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@568 441dd3cb-e453-d85f-364a-b28e0c74875c

Python: Fix one more callsite where base path is determined from url. The bug was urls like "/mapreduce/worker_callback/....../worker_callback/shard_id". The modified tests would fail before the fix.

Revision created by MOE tool push_codebase.
MOE_MIGRATION=6706


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@569 441dd3cb-e453-d85f-364a-b28e0c74875c

Java: Remove the use of @AppEngineInternal as dev appserver complains about it.


Revision created by MOE tool push_codebase.
MOE_MIGRATION=6707


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@570 441dd3cb-e453-d85f-364a-b28e0c74875c

Java: Revert change that depends on StringUtil


Revision created by MOE tool push_codebase.
MOE_MIGRATION=6708


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@571 441dd3cb-e453-d85f-364a-b28e0c74875c

Java: Increasing the GCS timeout.

Revision created by MOE tool push_codebase.
MOE_MIGRATION=6709


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@572 441dd3cb-e453-d85f-364a-b28e0c74875c

Python: MR sets GCS client's urlfetch limit.


Revision created by MOE tool push_codebase.
MOE_MIGRATION=6710


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@573 441dd3cb-e453-d85f-364a-b28e0c74875c

Java: Move MR jobID to the end of the url, so that users can route map, sort, and reduce workers to different modules or versions.



Revision created by MOE tool push_codebase.
MOE_MIGRATION=6711


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@574 441dd3cb-e453-d85f-364a-b28e0c74875c

Python: Max find_all_by_mapreduce_state really non-transactional


Revision created by MOE tool push_codebase.
MOE_MIGRATION=6712


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@575 441dd3cb-e453-d85f-364a-b28e0c74875c

Python: safeguard MR imports of google.appengine.ext.cloudstorage package.



Revision created by MOE tool push_codebase.
MOE_MIGRATION=6713


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@576 441dd3cb-e453-d85f-364a-b28e0c74875c

Java: Adding shard retry and better handling of failed MR.



Revision created by MOE tool push_codebase.
MOE_MIGRATION=6721


git-svn-id: http://appengine-mapreduce.googlecode.com/svn/trunk@577 441dd3cb-e453-d85f-364a-b28e0c74875c