Package: django-bandits
django-banditsThis package enables optimization of your Django site via multi-armed bandits.This package extendsDjango-Waffleby wrapping the feature flipper functions with bandit algorithms. This allows automatic testing and optimization as users interact with your site.Installationpip install django-banditsorpoetry add django-banditsAdditionally, you can install it directly from GitHub:Pippip install git+https://github.com/hubbs5/django-bandits.gitPoetrypoetry add git+https://github.com/hubbs5/django-bandits.gitDjango SettingsGo to yoursettings.pyfile and addwaffleto your installed apps list:INSTALLED_APPS = [
    ...
    "waffle",
    "django_bandits",
    ...
]Additionally, adddjango_bandits.middleware.UserActivityMiddlewaretoMIDDLEWAREafter CSRF and authentication middleware, e.g.:MIDDLEWARE = [
    ...
    "django.middleware.common.CommonMiddleware",
    "django.middleware.csrf.CsrfViewMiddleware",
    "django.contrib.auth.middleware.AuthenticationMiddleware",
    "django_bandits.middleware.UserActivityMiddleware", # It goes here!
    ...
]Updatesettings.pyto exclude any particular URL from conversion tracking, e.g.:EXCLUDE_FROM_TRACKING = [
    ADMIN_URL,
]Additionally, you can add a REGEX option to filter URLs. Helps to ensure you aren't picking up bot traffic that's trying to break into a Wordpress site or something of the sort.EXCLUDE_FROM_TRACKING_REGEX = r"\.php|wordpress|\/wp-"Finally add the following line to yoursettings.pyfile to ensure the bandit model is overriding Waffle's Flags.WAFFLE_FLAG_MODEL = "django_bandits.BanditFlag"MigrationsCreate migrations by runningpython manage.py makemigrationsfollowed bypython manage.py migrate.Enabling BanditsWith migrations complete, go to the Django site admin page. You should see something like this:SelectFlagsunderDJANGO_BANDITSand click "Add."Make a name for the flag - typically the feature you want to test. Set it to test for whatever group you'd like to test it on. Refer to theWaffle documentationfor details.The bandit needs aFLAG URLto be set - this is the URL where the flag will be shown to users and is needed to track conversions and call the bandit to flip the feature when users reach that URL. Enter this as theSource URL.To track a conversion, then you need to add aTarget URL, which is where you want the user to end up.In the example below, we're tracking the homepage (/) and want to see how many users click on a link to go to the contact page (/contact/).This will consider any user who views your source URL and target URL to be a conversion during the session and will update the count shown in the image above.To add a bandit, you'll need to select one of the bandit options from the list below:Some bandits have customizable parameters (e.g. how frequently a random action is taken such as $\epsilon$). All allow you to set a minimum number of views and confidence interval before a winning version is selected (see more details below).Now that a bandit is enabled, you need to update your templates or views in as done with Waffle to enable the feature flipping flag.In a template, it may be something like this:{% extends "base.html" %}
{% load waffle_tags %}

{% block content %}
{% flag "headline-flag" %}
    <h1>This is my test headline: Waffle Flag is <strong>active</strong></h1>
{% else %}
    <h1>This is my base headline: Waffle Flag is <strong>inactive</strong></h1>
{% endflag %}
    <a href="{% url 'ab_test:target' %}">Click here to see the target page</a>
{% endblock %}Where "headline-flag" is the name of the flag you defined in the admin page.Performance TrackingThe bandits will automatically select a winning option when the given criteria are met. When setting up a bandit, you can select theSIGNIFICANCE LEVEL(default is 0.05) and theMIN VIEWS(default is 100).After theMIN VIEWSthreshold is reached, the model will perform a two-sided t-test to determine if there's significant difference in the two options (i.e. p-value <SIGNIFICANCE LEVEL). If this criteria is met, the bandit will then default to the winning option for all future visits to avoid potential conversion losses from testing.As a user, you can also view the conversion rate and the upper and lower bounds of the confidence intervals at any time within the admin page.