powerlaw 2.0.0
Toolbox for testing if a probability distribution fits a power law
License: MIT
Requires Python: >=3.8

powerlaw: A Python Package for Analysis of Heavy-Tailed Distributions
=====================================================================

.. image:: https://github.com/jeffalstott/powerlaw/workflows/Tests/badge.svg
   :target: https://github.com/jeffalstott/powerlaw/actions
   :alt: Tests

``powerlaw`` is a toolbox implementing the statistical methods developed in
`Clauset et al. 2007 <http://arxiv.org/abs/0706.1062>`_ and
`Klaus et al. 2011 <https://doi.org/10.1371/journal.pone.0019779>`_
to fit heavy-tailed distributions like power laws. Academics, please cite as:

Jeff Alstott, Ed Bullmore, Dietmar Plenz. (2014). powerlaw: a Python package
for analysis of heavy-tailed distributions.
`PLoS ONE 9(1): e85777 <https://doi.org/10.1371/journal.pone.0085777>`_

Also available at `arXiv:1305.0215 [physics.data-an] <http://arxiv.org/abs/1305.0215>`_


Basic Usage
------------
The most basic use of this library is to fit some data, extract parameters,
and make comparisons to other distributions:

.. code-block:: python

    import powerlaw
    import numpy as np

    data = np.array([1.7, 3.2, 5.4, 2.1, 1.5, 2.8]) # data can be a list or a numpy array
    fit = powerlaw.Fit(data)

    print(fit.power_law.alpha)
    print(fit.power_law.xmin)

    R, p = fit.distribution_compare('power_law', 'lognormal')

You can also plot various results easily using ``matplotlib``:

.. code-block:: python

    import matplotlib.pyplot as plt
    fig, ax = plt.subplots()

    fit.plot_pdf(ax=ax, label='PDF')
    fit.power_law.plot_pdf(ax=ax, label='Power law fit')

    plt.legend()
    plt.show()


Quick Links
-----------
`Original paper illustrating powerlaw's features, with figures <http://arxiv.org/abs/1305.0215>`__

`Code examples from manuscript, as an IPython Notebook <http://nbviewer.ipython.org/github/jeffalstott/powerlaw/blob/master/manuscript/Manuscript_Code.ipynb>`__
Note: Some results involving lognormals will now be different from the
manuscript, as the lognormal fitting has been improved to allow for
greater numerical precision.

`Documentation <http://pythonhosted.org/powerlaw/>`__


Installation
------------
The package can be installed from PyPi using pip:

.. code-block:: console

    $ pip install powerlaw

Alternatively, you can install directly from the source:

.. code-block:: console

    $ git clone https://github.com/jeffalstott/powerlaw
    $ cd powerlaw
    $ pip install .

This library depends on the usual scientific computing libraries that you
probably already have installed: ``numpy``, ``scipy``, ``matplotlib``, and
``mpmath``.

The package ``tqdm`` is used for creating progress bars.

The requirement of ``mpmath`` will be dropped if/when the scipy functions
``gamma``, ``gammainc`` and ``gammaincc`` are updated to have sufficient numerical
accuracy for negative numbers.


Development
-----------

To run the test suite, we recommend using pytest:

.. code-block:: console

    python -m pytest testing/ -v

The test suite includes comprehensive tests for distribution fitting, comparisons, and statistical validation using reference and synthetic datasets. All tests should pass successfully.

This repository uses GitHub Actions for continuous integration. Tests are automatically run on every push and pull request across multiple Python versions (3.8-3.12) and operating systems (Ubuntu, Windows, macOS). The CI status is shown in the badge above.

The original author of `powerlaw`, Jeff Alstott, is now only writing minor tweaks, but ``powerlaw`` remains open for further development by the community. If there's a feature you'd like to see in ``powerlaw`` you can `submit an issue <https://github.com/jeffalstott/powerlaw/issues>`_, but pull requests are even better. Offers for expansion or inclusion in other projects are welcomed and encouraged.


Mailing List
~~~~~~~~~~~~
Questions/discussions/help go on the Google Group `here <https://groups.google.com/g/powerlaw-general>`__. Also receives update info.


Acknowledgements
----------------
Many thanks to Andreas Klaus, Mika Rubinov and Shan Yu for helpful
discussions. Thanks also to Andreas Klaus,
`Aaron Clauset, Cosma Shalizi <https://aaronclauset.github.io/powerlaws/>`_,
and `Adam Ginsburg <https://github.com/keflavich/plfit>`_ for making
their code available. Their implementations were a critical starting point for
making ``powerlaw``.


Power Laws vs. Lognormals and powerlaw's 'lognormal_positive' option
--------------------------------------------------------------------
When fitting a power law to a data set, one should compare the goodness of fit to that of a `lognormal distribution <https://en.wikipedia.org/wiki/Lognormal_distribution>`__. This is done because lognormal distributions are another heavy-tailed distribution, but they can be generated by a very simple process: multiplying random positive variables together. The lognormal is thus much like the normal distribution, which can be created by adding random variables together; in fact, the log of a lognormal distribution is a normal distribution (hence the name), and the exponential of a normal distribution is the lognormal (which maybe would be better called an expnormal). In contrast, creating a power law generally requires fancy or exotic generative mechanisms (this is probably why you're looking for a power law to begin with; they're sexy). So, even though the power law has only one parameter (``alpha``: the slope) and the lognormal has two (``mu``: the mean of the random variables in the underlying normal and ``sigma``: the standard deviation of the underlying normal distribution), we typically consider the lognormal to be a simpler explanation for observed data, as long as the distribution fits the data just as well. For most data sets, a power law is actually a worse fit than a lognormal distribution, or perhaps equally good, but rarely better. This fact was one of the central empirical results of the paper `Clauset et al. 2007 <http://arxiv.org/abs/0706.1062>`__, which developed the statistical methods that ``powerlaw`` implements.

However, for many data sets, the superior lognormal fit is only possible if one allows the fitted parameter ``mu`` to go negative. Whether or not this is sensible depends on your theory of what's generating the data. If the data is thought to be generated by multiplying random positive variables, ``mu`` is just the log of the distribution's median; a negative ``mu`` just indicates those variables' products are typically below 1. However, if the data is thought to be generated by exponentiating a normal distribution, then ``mu`` is interpreted as the median of the underlying normal data. In that case, the normal data is likely generated by summing random variables (positive and negative), and ``mu`` is those sums' median (and mean). A negative ``mu``, then, indicates that the random variables are typically negative. For some physical systems, this is perfectly possible. For the data you're studying, though, it may be a weird assumption. For starters, all of the data points you're fitting to are positive by definition, since power laws must have positive values (indeed, ``powerlaw`` throws out 0s or negative values). Why would those data be generated by a process that sums and exponentiates *negative* variables?

If you think that your physical system could be modeled by summing and exponentiating random variables, but you think that those random variables should be positive, one possible hacks is ``powerlaw``'s ``lognormal_positive``. This is just a regular lognormal distribution, except ``mu`` must be positive. Note that this does not force the underlying normal distribution to be the sum of only positive variables; it only forces the sums' *average* to be positive, but it's a start. You can compare a power law to this distribution in the normal way shown above::

    R, p = results.distribution_compare('power_law', 'lognormal_positive')

You may find that a lognormal where ``mu`` must be positive gives a much worse fit to your data, and that leaves the power law looking like the best explanation of the data. Before concluding that the data is in fact power law distributed, consider carefully whether a more likely explanation is that the data was generated by multiplying positive random variables, or even by summing and exponentiating random variables; either one would allow for a lognormal with an intelligible negative value of ``mu``.