chrishein/s3_uploader — pull request #4 by theSociableme (2013-09-20 16:28:39+00:00)
Title: Prevent hanging when out of sync

I was seeing s3_uploaded hang when filenumber would be larger then the number of files. From the log:

I, [2013-09-20T10:50:24.385000 #50788] INFO -- : Uploaded 18 (14 KB) in 0:0.93
I, [2013-09-20T10:50:24.390000 #50788] INFO -- : [3/8] Uploading 07/31/data.rb to s3://kineticinfo/time_mark_uploader_2/remedy-time-trackers/2013/08/23/07/31/data.rb
I, [2013-09-20T10:50:24.390000 #50788] INFO -- : [5/8] Uploading 08/data.rb to s3://kineticinfo/time_mark_uploader_2/remedy-time-trackers/2013/08/23/08/data.rb
I, [2013-09-20T10:50:24.390000 #50788] INFO -- : [6/8] Uploading 08/33/data.rb to s3://kineticinfo/time_mark_uploader_2/remedy-time-trackers/2013/08/23/08/33/data.rb
I, [2013-09-20T10:50:24.390000 #50788] INFO -- : [2/8] Uploading data.rb to s3://kineticinfo/time_mark_uploader_2/remedy-time-trackers/2013/08/23/data.rb
I, [2013-09-20T10:50:24.390000 #50788] INFO -- : [2/8] Uploading 07/data.rb to s3://kineticinfo/time_mark_uploader_2/remedy-time-trackers/2013/08/23/07/data.rb
I, [2013-09-20T10:50:24.391000 #50788] INFO -- : [9/8] Uploading 10/27/data.rb to s3://kineticinfo/time_mark_uploader_2/remedy-time-trackers/2013/08/23/10/27/data.rb
I, [2013-09-20T10:50:24.390000 #50788] INFO -- : [4/8] Uploading 07/32/data.rb to s3://kineticinfo/time_mark_uploader_2/remedy-time-trackers/2013/08/23/07/32/data.rb
I, [2013-09-20T10:50:24.391000 #50788] INFO -- : [7/8] Uploading 10/data.rb to s3://kineticinfo/time_mark_uploader_2/remedy-time-trackers/2013/08/23/10/data.rb

Notice the 9/8 - This group would hang. After my fixes I stopped seeing this issue. Uploaded 221 batches of files ranging from 3 to 69 files per batch with no hangs.