Web api for dynamically adding checks

..and dockerfile that is a bit more efficient at rebuilding python changes

This is still a WIP but I would like to get a bit of feedback
I think we can use your solution with some high default value (it can be `10000`), but with the option to override it via config file. Let's call this option `results_to_keep`.