Package: pysr
PySR searches for symbolic expressions which optimize a particular objective.https://github.com/MilesCranmer/PySR/assets/7593028/c8511a49-b408-488f-8f18-b1749078268fPySR: High-Performance Symbolic Regression in Python and JuliaDocsForumsPapercolab demopipcondaStatspip:conda:If you find PySR useful, please cite the paperarXiv:2305.01582.
If you've finished a project with PySR, please submit a PR to showcase your work on theresearch showcase page!Contents:Why PySR?InstallationQuickstart→ DocumentationContributorsTest statusLinuxWindowsmacOSDockerCondaCoverageWhy PySR?PySR is an open-source tool forSymbolic Regression: a machine learning
task where the goal is to find an interpretable symbolic expression that optimizes some objective.Over a period of several years, PySR has been engineered from the ground up
to be (1) as high-performance as possible,
(2) as configurable as possible, and (3) easy to use.
PySR is developed alongside the Julia librarySymbolicRegression.jl,
which forms the powerful search engine of PySR.
The details of these algorithms are described in thePySR paper.Symbolic regression works best on low-dimensional datasets, but
one can also extend these approaches to higher-dimensional
spaces by using "Symbolic Distillation" of Neural Networks, as explained in2006.11287, where we apply
it to N-body problems. Here, one essentially uses
symbolic regression to convert a neural net
to an analytic equation. Thus, these tools simultaneously present
an explicit and powerful way to interpret deep neural networks.InstallationPipYou can install PySR with pip:pipinstallpysrJulia dependencies will be installed at first import.CondaSimilarly, with conda:condainstall-cconda-forgepysrDockerYou can also use theDockerfileto install PySR in a docker containerClone this repo.Within the repo's directory, build the docker container:dockerbuild-tpysr.You can then start the container with an IPython execution with:dockerrun-it--rmpysripythonFor more details, see thedocker section.TroubleshootingOne issue you might run into can result in a hard crash at import with
a message like "GLIBCXX_...not found". This is due to another one of the Python dependencies
loading an incorrectlibstdc++library. To fix this, you should modify yourLD_LIBRARY_PATHvariable to reference the Julia libraries. For example, if the Julia
version oflibstdc++.sois located in$HOME/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/lib/julia/(which likely differs on your system!), you could add:export LD_LIBRARY_PATH=$HOME/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/lib/julia/:$LD_LIBRARY_PATHto your.bashrcor.zshrcfile.QuickstartYou might wish to try the interactive tutorialhere, which uses the notebook inexamples/pysr_demo.ipynb.In practice, I highly recommend using IPython rather than Jupyter, as the printing is much nicer.
Below is a quick demo here which you can paste into a Python runtime.
First, let's import numpy to generate some test data:importnumpyasnpX=2*np.random.randn(100,5)y=2.5382*np.cos(X[:,3])+X[:,0]**2-0.5We have created a dataset with 100 datapoints, with 5 features each.
The relation we wish to model is $2.5382 \cos(x_3) + x_0^2 - 0.5$.Now, let's create a PySR model and train it.
PySR's main interface is in the style of scikit-learn:frompysrimportPySRRegressormodel=PySRRegressor(niterations=40,# < Increase me for better resultsbinary_operators=["+","*"],unary_operators=["cos","exp","sin","inv(x) = 1/x",# ^ Custom operator (julia syntax)],extra_sympy_mappings={"inv":lambdax:1/x},# ^ Define operator for SymPy as wellelementwise_loss="loss(prediction, target) = (prediction - target)^2",# ^ Custom loss function (julia syntax))This will set up the model for 40 iterations of the search code, which contains hundreds of thousands of mutations and equation evaluations.Let's train this model on our dataset:model.fit(X,y)Internally, this launches a Julia process which will do a multithreaded search for equations to fit the dataset.Equations will be printed during training, and once you are satisfied, you may
quit early by hitting 'q' and then <enter>.After the model has been fit, you can runmodel.predict(X)to see the predictions on a given dataset using the automatically-selected expression,
or, for example,model.predict(X, 3)to see the predictions of the 3rd equation.You may run:print(model)to print the learned equations:PySRRegressor.equations_=[pickscoreequationlosscomplexity00.0000004.432479442.354317111.255691(x0*x0)3.437307320.011629((x0*x0)+-0.28087974)3.358285530.897855((x0*x0)+cos(x3))1.368308640.857018((x0*x0)+(cos(x3)*2.4566472))0.24648385>>>>inf(((cos(x3)+-0.19699033)*2.5382123)+(x0*...0.00000010]This arrow in thepickcolumn indicates which equation is currently selected by yourmodel_selectionstrategy for prediction.
(You may changemodel_selectionafter.fit(X, y)as well.)model.equations_is a pandas DataFrame containing all equations, including callable format
(lambda_format),
SymPy format (sympy_format- which you can also get withmodel.sympy()), and even JAX and PyTorch format
(both of which are differentiable - which you can get withmodel.jax()andmodel.pytorch()).Note thatPySRRegressorstores the state of the last search, and will restart from where you left off the next time you call.fit(), assuming you have setwarm_start=True.
This will cause problems if significant changes are made to the search parameters (like changing the operators). You can runmodel.reset()to reset the state.You will notice that PySR will save two files:hall_of_fame...csvandhall_of_fame...pkl.
The csv file is a list of equations and their losses, and the pkl file is a saved state of the model.
You may load the model from thepklfile with:model=PySRRegressor.from_file("hall_of_fame.2022-08-10_100832.281.pkl")There are several other useful features such as denoising (e.g.,denoise=True),
feature selection (e.g.,select_k_features=3).
For examples of these and other features, see theexamples page.
For a detailed look at more options, see theoptions page.
You can also see the full API atthis page.
There are also tips for tuning PySR onthis page.Detailed ExampleThe following code makes use of as many PySR features as possible.
Note that is just a demonstration of features and you should not use this example as-is.
For details on what each parameter does, check out theAPI page.model=PySRRegressor(procs=4,populations=8,# ^ 2 populations per core, so one is always running.population_size=50,# ^ Slightly larger populations, for greater diversity.ncycles_per_iteration=500,# ^ Generations between migrations.niterations=10000000,# Run foreverearly_stop_condition=("stop_if(loss, complexity) = loss < 1e-6 && complexity < 10"# Stop early if we find a good and simple equation),timeout_in_seconds=60*60*24,# ^ Alternatively, stop after 24 hours have passed.maxsize=50,# ^ Allow greater complexity.maxdepth=10,# ^ But, avoid deep nesting.binary_operators=["*","+","-","/"],unary_operators=["square","cube","exp","cos2(x)=cos(x)^2"],constraints={"/":(-1,9),"square":9,"cube":9,"exp":9,},# ^ Limit the complexity within each argument.# "inv": (-1, 9) states that the numerator has no constraint,# but the denominator has a max complexity of 9.# "exp": 9 simply states that `exp` can only have# an expression of complexity 9 as input.nested_constraints={"square":{"square":1,"cube":1,"exp":0},"cube":{"square":1,"cube":1,"exp":0},"exp":{"square":1,"cube":1,"exp":0},},# ^ Nesting constraints on operators. For example,# "square(exp(x))" is not allowed, since "square": {"exp": 0}.complexity_of_operators={"/":2,"exp":3},# ^ Custom complexity of particular operators.complexity_of_constants=2,# ^ Punish constants more than variablesselect_k_features=4,# ^ Train on only the 4 most important featuresprogress=True,# ^ Can set to false if printing to a file.weight_randomize=0.1,# ^ Randomize the tree much more frequentlycluster_manager=None,# ^ Can be set to, e.g., "slurm", to run a slurm# cluster. Just launch one script from the head node.precision=64,# ^ Higher precision calculations.warm_start=True,# ^ Start from where left off.turbo=True,# ^ Faster evaluation (experimental)julia_project=None,# ^ Can set to the path of a folder containing the# "SymbolicRegression.jl" repo, for custom modifications.update=False,# ^ Don't update Julia packagesextra_sympy_mappings={"cos2":lambdax:sympy.cos(x)**2},# extra_torch_mappings={sympy.cos: torch.cos},# ^ Not needed as cos already defined, but this# is how you define custom torch operators.# extra_jax_mappings={sympy.cos: "jnp.cos"},# ^ For JAX, one passes a string.)DockerYou can also test out PySR in Docker, without
installing it locally, by running the following command in
the root directory of this repo:dockerbuild-tpysr.This builds an image calledpysrfor your system's architecture,
which also contains IPython.You can then run this with:dockerrun-it--rm-v"$PWD:/data"pysripythonwhich will link the current directory to the container's/datadirectory
and then launch ipython.If you have issues building for your system's architecture,
you can emulate another architecture by including--platform linux/amd64,
before thebuildandruncommands.Contributors ✨We are eager to welcome new contributors! Check out our contributorsguidefor tips 🚀.
If you have an idea for a new feature, don't hesitate to share it on theissuesordiscussionspage.Mark Kittisopikul💻💡🚇📦📣👀🔧⚠️T Coxon🐛💻🔌💡🚇🚧👀🔧⚠️📓Dhananjay Ashok💻🌍💡🚧⚠️Johan Blåbäck🐛💻💡🚧📣👀⚠️📓JuliusMartensen🐛💻📖🔌💡🚇🚧📦📣👀🔧📓ngam💻🚇📦👀🔧⚠️Christopher Rowley💻💡🚇📦👀Kaze Wong🐛💻💡🚇🚧📣👀🔬📓Christopher Rackauckas🐛💻🔌💡🚇📣👀🔬🔧⚠️📓Patrick Kidger🐛💻📖🔌💡🚧📣👀🔬🔧⚠️📓Okon Samuel🐛💻📖🚧💡🚇👀⚠️📓William Booth-Clibborn💻🌍📖📓🚧👀🔧⚠️Pablo Lemos🐛💡📣👀🔬📓Jerry Ling🐛💻📖🌍💡📣👀📓Charles Fox🐛💻💡🚧📣👀🔬📓Johann Brehmer💻📖💡📣👀🔬⚠️📓Marius Millea💻💡📣👀📓Coba🐛💻💡👀📓foxtran💻💡🚧🔧📓Shah Mahdi Hasan🐛💻👀📓Pietro Monticone🐛📖💡Mateusz Kubica📖💡Jay Wadekar🐛💡📣🔬Anthony Blaom, PhD🚇💡👀Jgmedina95🐛💡👀Michael Abbott💻💡👀🔧Oscar Smith💻💡Eric Hanson💡📣📓Henrique Becker💻💡👀qwertyjl🐛📖💡📓Rik Huijzer💡🚇Hongyu Wang💡📣🔬Zehao Jin🔬📣Tanner Mengel🔬📣Arthur Grundner🔬📣sjwetzel🔬📣📓Saurav Maheshkar🔧