Package: django-spam-classifier
Django Spam ClassifierContact form spam getting you down? We know the feeling. It's demeaning,
draining and relentless.This a very basic Django app that usesdbaclBayesian text classification tool
to filter out contact form spam. It's not perfect, but it works very well on
blocking the really offensive English text spam. The app was written to avoid
depending on external services like reCAPTCHA or Akismet - these services work
well enough, but introduce some privacy concerns.LimitationsCurrently doesn't work so well on non-English text, very short input, garbage
input or HTML only with a single hyperlink. It's possible thatdbaclmay have
options to deal more effectively with this.Additionally,dbaclseems to be not so actively maintained, and is currently
not available on Debian Bullseye. I may switch tobogofilteror other Bayesian
filtering options in the future.Getting startedInstalldjango-spam-classifierInstalldbaclvia your OS package managerAdd aBASE_DIRsettingEnable Djangodjango.contrib.sitesapp and configure your site domain via
Django Admin (used for training links in emails)Add'classifier'to yourINSTALLED_APPSsettingAddpath('', include('classifier.urls')),to your project'surls.pyRunpython manage.py migrateCreate theclassifier_datadirectory to hold the classifier databaseIn contact form callclassifier.is_spam()on all text accepted by your
form:spam, submission = is_spam('\n'.join(submission_fields))
if spam:
    # Throw away the form submission and don't notify anyone.
else:
    # Process the form submission as normal.Doing so will internally usedbaclto classify the submission as spam or
not spam and generate a confidence of 0-100. Spam/not-spam with a high
confidence is processed as you'd expect. If the confidence is below theRECORD_AND_DISCARD_CONFIDENCE, the submission is treated as not spam
because confidence is too low to make a safe decision. The body is recorded
in theSubmissionsmodel and can be manually classified via the Django
Admin. If the confidence is aboveRECORD_AND_DISCARD_CONFIDENCEbut belowSILENTLY_DISCARD_CONFIDENCE, the submission is treated as confidently spam,
but also recorded to theSubmissionsmodel for manual classification.Add a training link to the footer of any notification email you send::email_body = email_body + spam_footer(submission, site)Which will output something like:--
Spam score: spam (15% confidence)
Train as spam: https://example.com/classifier/1704/spam/
Train as not spam: https://example.com/classifier/1704/not-spam/Ensure you have a logging configuration set up so you can see log messagesAdd a cron job to regularly (eg. daily) update the training database with any
new manual classifications you've made:python manage.py trainVisit the Django Admin and classify the low-confidence submissions you receive.Tune the Django settings as desired (optional):CLASSIFIER = {
   'SILENTLY_DISCARD_CONFIDENCE': 90,  # Defaults to 80
  'RECORD_AND_DISCARD_CONFIDENCE': 75,  # Defaults to 60
}DevelopmentCreate a venv and install the development requirements:python3 -m python3.8 -m venv --system-site-packages [VENV-PATH]
source [VENV_PATH]/bin/activate
python -m pip install Django pytzTODO: There is undoubtedly a better way of installing dev-dependencies. Perhaps
poetry or flit? Are they the only tools that handle this? What's generally accepted?Run tests withtoxor:PYTHONPATH=src:.:$PYTHONPATH DJANGO_SETTINGS_MODULE=tests.test_settings pytest testsCreate migrations with:DJANGO_SETTINGS_MODULE=tests.test_settings python -m django makemigrations