Yelp/mrjob — Issue #385: Fair Sharing Scheduler in emr
Author: tokarina
Created: 2012-03-06 04:58:19+00:00
State: open

I'm trying to use the Fair Sharing Scheduler in AWS EMR.

Using the info in this post, I was able to setup the scheduler:
https://forums.aws.amazon.com/thread.jspa?threadID=51135

But... it seems that jobs launched with mrjob somehow do not use it. I'm not sure if my config is wrong, or what. I would appreciate any advice.

-jrf

Here's my fair-scheduler.xml

5
5
25
25
300

6

32
600

and here is how I call a test mrjob:
cat recs.json | python test.py -r emr -c mrjob.conf --emr-job-flow-id j-3QQS10TFU827T --output-dir "s3://mytest/test3" --no-output &>3-100.log