Yelp/mrjob — Issue #303: EMR job not starting with IncompleteSignature error
Author: wtfleming
Created: 2011-12-07 21:26:52+00:00
State: open

EMR job I had been using for the past few months stopped working yesterday (script is run on a ubuntu 10.04 machine in EC2), here is the output

using configs in /etc/mrjob.conf
creating tmp directory /tmp/report.organization.20111207.170002.272854
writing master bootstrap script to /tmp/report.organization.20111207.170002.272854/b.py
Copying non-input files into s3://some-bucket/emr-job-logs/scratch/report.organization.20111207.170002.272854/files/
Waiting 5.0s for S3 eventual consistency
Creating Elastic MapReduce job flow
Traceback (most recent call last):
File "/opt/organization/emr/report.py", line 216, in
MRRunReport.run()
File "/usr/local/lib/python2.6/dist-packages/mrjob-0.2.8-py2.6.egg/mrjob/job.py", line 345, in run
mr_job.execute()
File "/usr/local/lib/python2.6/dist-packages/mrjob-0.2.8-py2.6.egg/mrjob/job.py", line 358, in execute
self.run_job()
File "/usr/local/lib/python2.6/dist-packages/mrjob-0.2.8-py2.6.egg/mrjob/job.py", line 404, in run_job
runner.run()
File "/usr/local/lib/python2.6/dist-packages/mrjob-0.2.8-py2.6.egg/mrjob/runner.py", line 342, in run
self._run()
File "/usr/local/lib/python2.6/dist-packages/mrjob-0.2.8-py2.6.egg/mrjob/emr.py", line 574, in _run
self._launch_emr_job()
File "/usr/local/lib/python2.6/dist-packages/mrjob-0.2.8-py2.6.egg/mrjob/emr.py", line 971, in _launch_emr_job
persistent=False, steps=steps)
File "/usr/local/lib/python2.6/dist-packages/mrjob-0.2.8-py2.6.egg/mrjob/emr.py", line 880, in _create_job_flow
self._job_name, self._opts['s3_log_uri'], **args)
File "/usr/local/lib/python2.6/dist-packages/mrjob-0.2.8-py2.6.egg/mrjob/retry.py", line 73, in call_and_maybe_retry
return f(*args, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/mrjob-0.2.8-py2.6.egg/mrjob/botoemr/connection.py", line 230, in run_jobflow
'RunJobFlow', params, RunJobFlowResponse, verb='POST')
File "/usr/lib/pymodules/python2.6/boto/connection.py", line 631, in get_object
raise self.ResponseError(response.status, response.reason, body)
mrjob.botoemr.connection.EmrResponseError: EmrResponseError: 400 Bad Request

Sender
IncompleteSignature
Request must contain a signature that conforms to AWS standards

requestidsanitized

I was able to get it working today by editing /usr/lib/pymodules/python2.6/boto/connection.py and changing the signatureversion variable from 1 to 2.

It looks like the EMR API is no longer accepting version 1 signatures?

More of a boto issue, but any suggestions on a less hacky way to work around this in mrjob?