Package: dabble
# DabbleDabble is a simple A/B testing framework for Python. Using dabble, youconfigure your tests in code, collect results, and analyze them later tomake informed decisions about design changes, feature implementations, etc.You define an A/B test in dabble with class `ABTest`, which describes thetest name, the names of each of the alternatives, and the set of steps theusers will progress through during the test (in the simplest case, this isjust two steps). You then define one or more `ABParameter`s, which containthe values you wish to vary for each alternative in the test. Each test canhave one or more alternatives, though the most common case is to have 2(hence "A/B testing").## Exampleimport dabbledabble.configure(CookieIdentityProvider('dabble_id'),FSResultStorage('/path/to/results.data'))class Signup(page):path = '/signup'signup_button = ABTest('signup button',alternatives=['red', 'green'],steps=['show', 'signup'])button_color = ABParameter('signup button', ['#ff0000', '#00ff00'])def GET(self):self.signup_button.record('show')return render('index.html', button_color=self.button_color)def POST(self):self.signup_button.record('signup')return redirect('/account')Behind the scenes, dabble has used a cookie for each user on your site toassigne them each an *identity*, so that each user always ever sees the same*alternative*. Users may visit the homepage many times over many browsingsessions, but as long as they have the same cookie present in their browser,they will always see either the red or the green button, depending on whichwas chosen the first time the viewed the page.When a user signs up, the `record()` method of `ABTest` is called, to trackthe user's action. Later on, reports can be generated to determine whetherthe red or the green button induced more users to sign up.## Configuring DabbleIn addition to `ABTest` and `ABParameter`, dabble also needs an`IdentityProvider` and a `ResultsStorage`. Dabble provides severalalternatives for each of these out of the box, and it is alsostraightforward to write your own.`IdentityProvider`s should make their best possible effort to alwaysidentify individuals, rather than browsing sessions (particularly if cookiesare set to expire when the user closes his/her browser). If you are testinga feature that requires users to be logged in, then their username is a goodchoice for identity.`ResultsStorage` stores configuration and results of A/B tests, and providessome facilities for generating reports based on the stored results. Dabbleprovides several backends, including `MongoResultsStorage`, and`FSResultsStorage`.At this time it is not possible to configure different `IdentityProvider`sor `ResultsStorage`s for different tests within the same application.## ReportingDabble will also produce reports on all users who have taken part in an A/Btest, by way of the `report()` method. The report is a dictionary whichdescribes, for each alternative, how many users attempted and converted ateach of the defined steps. For the above example, a report might look like:>>> storage = FSResultStorage('/path/to/results.data')>>> storage.report('signup button'){'test_name': 'signup button','results': [{'alternative': 'red','funnel': [{'stage': ('show', 'signup'),'attempted': 187,'converted': 22,}],},{'alternative': 'green','funnel': [{'stage': ('show', 'signup'),'attempted': 195'converted': 18,}],}],}The `funnel` key in each of the `results` entries will have one elementfewer than the number of steps, since each entry describes the progressionof users from one step to the next.