mozilla/browserid — Issue #787: load_gen: Fix code to better handle failures due to up-front overloading
Author: jbonacci
Created: 2011-12-20 21:36:08+00:00
State: open

We are looking for a fix to handle failures during initial load test "warm up".
These failures are mostly caused by the expensive api flows: sign up and re-authorization

From #identity snippet from 12/20/2011:
We want the code to fail properly under really heavy load.
We want to be able to start load_gen and immediately simulate very high ADU, which will, in turn, really push the api flows mentioned above.
We want to return 503s on operation timeouts and have load_gen report the number of failures due to overloading.
The hope then, by tracking this information, is that as load_gen database warms up, the number of failures will drop off until the DB remains stable.
We should be able to see this stablization.

While under heavy load, we would expect that some percentage of our manual authorization tests will fail.

Ultimate goal here is to address the failures in software, so that we can then focus on warming up load_gen and the database much faster.

The failure handling issue is a "blocker" for this.