I'm trying to store my application results in a mongo database.  There are some issues to this, first we generate a lot of data because it is raw image files, up to 50MB per record and 5 records per second at full speed, that's worst case though not typical. This isn't a problem using gridFS. In the mongod.cfg we are using directoryPerDB: true and I create a symlink for the Results database folder to a dedicated SSD where all the results database is kept.  All the settings data is kept in a couple different databases on the OS drive, and the results data has a dedicated drive. This is all working great for our application.

My problem is filling up the results disk. I need to basically have a capped collection to a max size, and then delete only the oldest files.  But I don't see a way to do this with gridFS? Is there a setting  or something I'm missing that would allow capping this?  

I found this answer GridFS disk management but that seems to apply to mongod, and i don't think you can set a quota per database as the Results database is the only one I need to be limited.  

At this point I guess i will write a task that cleans the oldest files out periodically if the total is over a threshold limit, i'm just afraid this is not going to be very efficient.  Are there any recommendations on the best way to handle this?  


Question: Should I have a queue with SMS requests to picked up by a separate process or connect directly to the API?
Answer: Depending on the API for sending text messages, there are two possibilities for meeting those business requirements.


If there are separate API calls to verify a phone number and to send a text message, you can use a queue for the actual sending and verify the validity of the phone number just before you put the text message in the queue.
If there is only an API call to send a text message, which informs you about success/failure after the fact and where the failure codes are descriptive enough, you could send the message initially from the context of the user request. If that fails with a temporary error, add the message to a queue for later retries.