django-test-query-counter 0.1.0
A Django Toolkit for controlling Query count when testing
License: MIT
Keywords: django-test-query-counter

=============================
Django Test Query Counter
=============================

.. image:: https://badge.fury.io/py/django-test-query-counter.svg
    :target: https://badge.fury.io/py/django-test-query-counter

.. image:: https://travis-ci.org/sophilabs/django-test-query-counter.svg?branch=master
    :target: https://travis-ci.org/sophilabs/django-test-query-counter

.. image:: https://codecov.io/gh/sophilabs/django-test-query-counter/branch/master/graph/badge.svg
    :target: https://codecov.io/gh/sophilabs/django-test-query-counter


.. image:: logo.png
   :width: 200
   :alt: Logo
   :align: center

A Django Toolkit for controlling Query count when testing. It controls the
number of queries done in the tests stays below a particular threshold between
Test Runs.Specially useful when paired with a CI like Jenkins or Travis to
control each commit doesn't slow down the Database considerably.

Requirements
------------

* Python 3
* Django

Documentation
-------------

The full documentation is at https://django-test-query-counter.readthedocs.io.

Installation
-------------

There are ways to install it into your project

Clone this repository into your project:

.. code-block:: bash

    git clone https://github.com/sophilabs/django-test-query-counter.git

Download the zip file and unpack it:

.. code-block:: bash

    wget https://github.com/sophilabs/django-test-query-counter/archive/master.zip
    unzip master.zip

Install with pip:

.. code-block:: bash

    pip install django-test-query-counter

Add it to your `INSTALLED_APPS`:

.. code-block:: python

    INSTALLED_APPS = (
        ...
        'test_query_count',
        ...
    )

Usage
-----

Run your django tests as you do. After the run the
``reports`` directory with two files ``query_count.json`` and
``query_count_detail.json``.

To check your tests Query Counts run:

``$ python manage.py check_query_count``

Then you would see an output like the following one (if you at least ran the
tests twice):

.. code-block::

    All Tests API Queries are below the allowed threshold.

If the current test run had more queries than the last one, it will tell you:

.. code-block::

    There are test cases with API calls that exceeded threshold:

        In test case app.organizations.tests.api.test_division_api.DivisionViewTest.test_bulk_create_division, POST /api/divisions. Expected at most 11 queries but got 15 queries
        In test case app.shipments.tests.api.test_leg_api.LegViewTest.test_create_bulk_leg, POST /api/legs. Expected at most 59 queries but got 62 queries
        In test case app.plans.tests.functional.test_plan_api.PlannedDatesTest.test_unassign_and_assign_driver_to_leg, POST /api/assignments/assign-driver. Expected at most 261 queries but got 402 queries
    CommandError: There was at least one test with an API call excedding the allowed threshold.

Configuration
-------------

You can customize how the Test Query Count works by defining ``TEST_QUERY_COUNTER``
in your settings file as a dictionary. The following code shows an example

.. code-block:: python

    TEST_QUERY_COUNTER = {
        # Global switch
        'ENABLE': True,

        # Paths where the count files are generated (relative to the current
        # process dir)
        'DETAIL_PATH': 'reports/query_count_detail.json',
        'SUMMARY_PATH': 'reports/query_count.json',

        # Tolerated percentage of count increase on successive
        # test runs.A value of 0 prevents increasing queries altoghether.
        'INCREASE_THRESHOLD': 10
    }


Excluding Tests
---------------

Individual tests or classes can be excluded for the count using the
``@exclude_query_count`` decorator. For example

.. code-block:: python

        # To exclude all methods in the class
        @exclude_query_count()
        class AllTestExcluded(TestCase):
            def test_foo(self):
                self.client.get('path-1')

            def test_foo(self):
                self.client.get('path-2')

        # To exclude one test only
        class OneMethodExcluded():
            def test_foo(self):
                self.client.get('path-1')

            @exclude_query_count()
            def test_foo(self):
                self.client.get('path-2')


More specifically, ``exclude_query_count`` accept parameters to conditionally
exclude a query count by path, method or count. Where ``path`` the or regex of
the excluded path(s). The ``method`` specifies the regex of the method(s) to
exclude, and ``count`` is minimum number of queries tolerated. Requests with
less or same amount as "count" will be excluded.

For example:

.. code-block:: python

    class Test(TestCase):
        @exclude_query_count(path='url-2')
        def test_exclude_path(self):
            self.client.get('/url-1')
            self.client.post('/url-2')

        @exclude_query_count(method='post')
        def test_exclude_method(self):
            self.client.get('/url-1')
            self.client.post('/url-2')

        @exclude_query_count(count=2)
        def test_exclude_count(self):
            #  succesive urls requests are additive
            for i in range(3):
                self.client.get('/url-1')

Implementing into your CI
-------------------------

Currently the query count works locally. However, it shines when it is
integrated with `Jenkins <https://jenkins.io/>`_, or other CIs. You have to do
this manually:

Requirements

* Jenkins with a Job that build you project.

From now on let's suppose your job is available at ``http://127.0.0.1:8080/job/ZAP_EXAMPLE_JOB/``.


1. `Activate Build Archive`: Go to the job `configuration page <https://wiki.jenkins.io/display/JENKINS/Configure+the+Job>`_ and add the `archive artifacts <https://jenkins.io/doc/pipeline/tour/tests-and-artifacts/>`_ build
   Post-build actions.

    .. image:: archive-artifact.png
       :alt: Jenkins Post Build Action showing an archive artifact example
       :align: center

2. Set ``reports/query_count.json`` in the files to archive path

3. Create a new Django custom Command to run the validation against the
   archived Jenkins file. For example:

    .. code-block:: python

        from urllib.request import urlretrieve
        from django.core.management import BaseCommand, CommandError
        from django.conf import settings
        from test_query_counter.apps import RequestQueryCountConfig
        from test_query_counter.query_count import QueryCountEvaluator


        class Command(BaseCommand):
            JENKINS_QUERY_COUNT = 'https://yourci/job/' \
                                  'yourproject/lastSuccessfulBuild/' \
                                  'artifact/app/reports/query_count.json'

            def handle(self, *args, **options):
                current_file_path = RequestQueryCountConfig.get_setting('SUMMARY_FILE')

                with open(current_file_path) as current_file, \
                        open(urlretrieve(self.JENKINS_QUERY_COUNT)[0]) as last_file:
                    violations = QueryCountEvaluator(10, current_file,last_file).run()

                    if violations:
                        raise CommandError('There was at least one test with an API '
                                           'call excedding the allowed threshold.')

4. Add a build step to run this command:

    .. image:: build-action.png
       :alt: Jenkins Build Action showing the script action
       :align: center

   After that, it will run fine, and the build would fail if any Query count is over the limit.

TODO
----

* Include support for stacktraces in ``query_count_detail.json``.
* Generate an HTML report of executed Queries.
* Make query count configurable
* Include Jenkins support out of the box (using `django_jenkins`)

Running Tests
-------------

Does the code actually work?

.. code-block:: bash

    source <YOURVIRTUALENV>/bin/activate
    (myenv) $ pip install tox
    (myenv) $ tox

License
-------

PySchool is MIT Licensed. Copyright (c) 2017 Sophilabs, Inc.


Credits
-------

.. image:: https://s3.amazonaws.com/sophilabs-assets/logo/logo_300x66.gif
    :target: https://sophilabs.co

This tool is maintained and funded by Sophilabs, Inc. The names and logos for
sophilabs are trademarks of sophilabs, inc.

Tools used in rendering this package:

*  Cookiecutter_
*  `cookiecutter-djangopackage`_

.. _Cookiecutter: https://github.com/audreyr/cookiecutter
.. _`cookiecutter-djangopackage`: https://github.com/pydanny/cookiecutter-djangopackage




History
-------

0.1.0 (2017-07-10)
++++++++++++++++++

* First release on PyPI.