Repository: pymc-devs/pymc

Issue #5443: Numerical differences between 3.11.4 and 4.0.0b
Created: 2022-02-02T23:37:21Z
Numerical problems with quadratic approximation
As per @ericmjl (pymc-devs/pymc-resources#168 (comment)), pymc 3.11.4 and 4.0.0b are giving different numerical answers to quadratic approximation examples for Chapter 2 in the Rethinking Statistics course. Below is an excerpt from the Chapter 2 example.
pymc 3.11.4 gives standard deviation of 0.16 (correct). 4.0.0b gives 0.64. No other warnings or errors were apparent.
Please provide a minimal, self-contained, and reproducible example.
#!/usr/bin/env python
# coding: utf-8

import numpy as np
import pymc3 as pm

RANDOM_SEED = 8927
np.random.seed(RANDOM_SEED)

# #### Code 2.6
# 
# Computing the posterior using the quadratic approximation (quad).

data = np.repeat((0, 1), (3, 6))
with pm.Model() as normal_approximation:
    p = pm.Uniform("p", 0, 1)  # uniform priors
    w = pm.Binomial("w", n=len(data), p=p, observed=data.sum())  # binomial likelihood
    mean_q = pm.find_MAP()
    std_q = ((1 / pm.find_hessian(mean_q, vars=[p])) ** 0.5)[0]

# display summary of quadratic approximation
print("  Mean, Standard deviation\np {:.2}, {:.2}".format(mean_q["p"], std_q[0]))
Expected Output
  Mean, Standard deviation
p 0.67, 0.16
Actual Output
  Mean, Standard deviation
p 0.67, 0.64

Please provide any additional information below.
Versions and main components

PyMC/PyMC3 Version: 4.0.0b
Aesara/Theano Version:
Python Version: 3.9.7
Operating system: MacOS
How did you install PyMC/PyMC3: conda (using environments.yml in rethinking_2)

Duplicate issue #7082: BUG: pymc.find_hessian() returns incorrect value for version 5.10.3
Created: 2023-12-29T23:49:14Z
Describe the issue:
Hi developers,
I have analytically found the second derivative of following posterior at MAP of 2/3 to be 0.15713. The likelihood is simply Binomial with a Uniform prior on [0, 1]. I can get the exact analytical value from an older version like 4.0.0 but in the new version it returns 0.64!
Pr \left( p \vert W, L \right) &= 840 p^6 \left( 1- p \right)^3
Reproduceable code example:
import numpy as np
import pymc as pm

data = np.repeat((0, 1), (3, 6))
with pm.Model() as normal_approximation:
    p = pm.Uniform("p", 0, 1)  # uniform priors
    w = pm.Binomial("w", n=len(data), p=p, observed=data.sum())  # binomial likelihood
    mean_q = pm.find_MAP()

    p_value = normal_approximation.rvs_to_values[p]
    p_value.tag.transform = None
    p_value.name = p.name

    std_q = ((1 / pm.find_hessian(mean_q, vars=[p])) ** 0.5)[0]

# display summary of quadratic approximation
print("Mean, Standard deviation\np {:.2}, {:.2}".format(mean_q["p"], std_q[0]))
Error message:
No response
PyMC version information:
pymc                      5.10.3               hd8ed1ab_0    conda-forge
pymc-base                 5.10.3             pyhd8ed1ab_0    conda-forge
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pytensor                  2.18.4          py310hc6cd4ac_0    conda-forge
pytensor-base             2.18.4          py310hcc13569_0    conda-forge
python                    3.10.13         hd12c33a_1_cpython    conda-forge
Running on linux through WSL2 (windows 11)
environment created through conda from conda-forge channel
Context for the issue:
I encountered this issue for the statistical rethinking course. You can refer to the code 2.6 in the provided notebook.
https://github.com/pymc-devs/pymc-resources/blob/main/Rethinking_2/Chp_02.ipynb

Closing comment (2023-12-30T21:08:00Z): This is a duplicate of #5443
In the current versions it matters if you define the model with transforms or not. This exact example is used im the docstrings of this utility: https://www.pymc.io/projects/docs/en/stable/api/model/generated/pymc.model.transform.conditioning.remove_value_transforms.html
I imagine you are looking for the value of the hessian with the transform, but you removed it?
Let me know if I am missing something.