Repository: sbi-dev/sbi

Issue:
SNPE with NSF fails when sampling with MCMC
This occurs in a very particular setting: `SNPE` inference with `NSF` density estimator and `sample_with_mcmc=True` (no matter which type of MCMC. 

- it works with `sample_with_mcmc=False`, 
- and it works with `SNLE`! 

I tried to chase it down, but no success so far. You can reproduce it locally by running

```
pytest -s tests/linearGaussian_snpe_test.py::test_c2st_snpe_external_data_on_linearGaussian
```

and setting 
https://github.com/mackelab/sbi/blob/6b5ed7be1d7522546b06c39aec1f206a354cc2ef/tests/linearGaussian_snpe_test.py#L286

to `True`. 

This is the error trace:
```python

>       samples = posterior.sample((num_samples,))

tests/linearGaussian_snpe_test.py:289:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
sbi/inference/posteriors/direct_posterior.py:336: in sample
    samples = self._sample_posterior_mcmc(
sbi/inference/posteriors/base_posterior.py:333: in _sample_posterior_mcmc
    samples = self._slice_np_mcmc(
sbi/inference/posteriors/base_posterior.py:397: in _slice_np_mcmc
    posterior_sampler.gen(int(warmup_steps))
sbi/mcmc/slice_numpy.py:93: in gen
    self._tune_bracket_width(rng)
sbi/mcmc/slice_numpy.py:145: in _tune_bracket_width
    x[i], wi = self._sample_from_conditional(i, x[i], rng)
sbi/mcmc/slice_numpy.py:173: in _sample_from_conditional
    while Li(lx) >= logu and cxi - lx < self.max_width:
sbi/mcmc/slice_numpy.py:162: in <lambda>
    Li = lambda t: self.lp_f(np.concatenate([self.x[:i], [t], self.x[i + 1 :]]))
sbi/inference/posteriors/direct_posterior.py:477: in np_potential
    target_log_prob = self.posterior_nn.log_prob(
.sbi_env/lib/python3.8/site-packages/nflows/distributions/base.py:40: in log_prob
    return self._log_prob(inputs, context)
.sbi_env/lib/python3.8/site-packages/nflows/flows/base.py:39: in _log_prob
    noise, logabsdet = self._transform(inputs, context=embedded_context)
.sbi_env/lib/python3.8/site-packages/torch/nn/modules/module.py:722: in _call_impl
    result = self.forward(*input, **kwargs)
.sbi_env/lib/python3.8/site-packages/nflows/transforms/base.py:56: in forward
    return self._cascade(inputs, funcs, context)
.sbi_env/lib/python3.8/site-packages/nflows/transforms/base.py:50: in _cascade
    outputs, logabsdet = func(outputs, context)
.sbi_env/lib/python3.8/site-packages/torch/nn/modules/module.py:722: in _call_impl
    result = self.forward(*input, **kwargs)
.sbi_env/lib/python3.8/site-packages/nflows/transforms/base.py:56: in forward
    return self._cascade(inputs, funcs, context)
.sbi_env/lib/python3.8/site-packages/nflows/transforms/base.py:50: in _cascade
    outputs, logabsdet = func(outputs, context)
.sbi_env/lib/python3.8/site-packages/torch/nn/modules/module.py:722: in _call_impl
    result = self.forward(*input, **kwargs)
.sbi_env/lib/python3.8/site-packages/nflows/transforms/base.py:56: in forward
    return self._cascade(inputs, funcs, context)
.sbi_env/lib/python3.8/site-packages/nflows/transforms/base.py:50: in _cascade
    outputs, logabsdet = func(outputs, context)
.sbi_env/lib/python3.8/site-packages/torch/nn/modules/module.py:722: in _call_impl
    result = self.forward(*input, **kwargs)
.sbi_env/lib/python3.8/site-packages/nflows/transforms/coupling.py:84: in forward
    transform_split, logabsdet = self._coupling_transform_forward(
.sbi_env/lib/python3.8/site-packages/nflows/transforms/coupling.py:194: in _coupling_transform_forward
    return self._coupling_transform(inputs, transform_params, inverse=False)
.sbi_env/lib/python3.8/site-packages/nflows/transforms/coupling.py:211: in _coupling_transform
    outputs, logabsdet = self._piecewise_cdf(inputs, transform_params, inverse)
.sbi_env/lib/python3.8/site-packages/nflows/transforms/coupling.py:492: in _piecewise_cdf
    return spline_fn(
.sbi_env/lib/python3.8/site-packages/nflows/transforms/splines/rational_quadratic.py:45: in unconstrained_rational_quadratic_spline
    ) = rational_quadratic_spline(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

inputs = tensor([]), unnormalized_widths = tensor([], size=(0, 10)), unnormalized_heights = tensor([], size=(0, 10)), unnormalized_derivatives = tensor([], size=(0, 11))
inverse = False, left = -3.0, right = 3.0, bottom = -3.0, top = 3.0, min_bin_width = 0.001, min_bin_height = 0.001, min_derivative = 0.001

    def rational_quadratic_spline(
        inputs,
        unnormalized_widths,
        unnormalized_heights,
        unnormalized_derivatives,
        inverse=False,
        left=0.0,
        right=1.0,
        bottom=0.0,
        top=1.0,
        min_bin_width=DEFAULT_MIN_BIN_WIDTH,
        min_bin_height=DEFAULT_MIN_BIN_HEIGHT,
        min_derivative=DEFAULT_MIN_DERIVATIVE,
    ):
>       if torch.min(inputs) < left or torch.max(inputs) > right:
E       RuntimeError: operation does not have an identity.

.sbi_env/lib/python3.8/site-packages/nflows/transforms/splines/rational_quadratic.py:77: RuntimeError
```


Discussion:
This is on the new `nflows` version `0.13` and on my feature branch - I will test it on main in a moment: 

Update:
You should get the same error when using `pyknos` and `nflows` 0.12.
I was about to post the same issue. 

In my case, I'm using NSF with SNPE_C. Setting `sample_with_mcmc=False` it works for most of the real observations, but it crashes in some of them (they are difficult and boundary cases, but I can't identify any numerical instability in them), suggesting to switch to `sample_with_mcmc=True`. Finally, after a while, it crashes in the same `rational_quadratic.py` function:
```
Drawing 200 posterior samples:   0%|          | 0/200 [00:00<?, ?it/s]WARNING:root:Only 0% posterior samples are within the
                        prior support. It may take a long time to collect the remaining
                        200 samples. Consider interrupting (Ctrl-C)
                        and switching to `sample_with_mcmc=True`.
Drawing 200 posterior samples:   0%|          | 0/200 [39:10<?, ?it/s]Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/sbi/utils/sbiutils.py", line 176, in sample_posterior_within_prior
    candidates = posterior_nn.sample(sampling_batch_size, context=x).reshape(
  File "/usr/local/lib/python3.8/site-packages/nflows/distributions/base.py", line 65, in sample
    return self._sample(num_samples, context)
  File "/usr/local/lib/python3.8/site-packages/nflows/flows/base.py", line 54, in _sample
    samples, _ = self._transform.inverse(noise, context=embedded_context)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/base.py", line 60, in inverse
    return self._cascade(inputs, funcs, context)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/base.py", line 50, in _cascade
    outputs, logabsdet = func(outputs, context)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/base.py", line 60, in inverse
    return self._cascade(inputs, funcs, context)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/base.py", line 50, in _cascade
    outputs, logabsdet = func(outputs, context)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/base.py", line 60, in inverse
    return self._cascade(inputs, funcs, context)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/base.py", line 50, in _cascade
    outputs, logabsdet = func(outputs, context)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/coupling.py", line 119, in inverse
    transform_split, logabsdet_split = self._coupling_transform_inverse(
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/coupling.py", line 197, in _coupling_transform_inverse
    return self._coupling_transform(inputs, transform_params, inverse=True)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/coupling.py", line 211, in _coupling_transform
    outputs, logabsdet = self._piecewise_cdf(inputs, transform_params, inverse)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/coupling.py", line 492, in _piecewise_cdf
    return spline_fn(
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/splines/rational_quadratic.py", line 45, in unconstrained_rational_quadratic_spline
    ) = rational_quadratic_spline(
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/splines/rational_quadratic.py", line 134, in rational_quadratic_spline
    assert (discriminant >= 0).all()
AssertionError
python-BaseException
```
(i.e., discriminant has, at least, 1 negative value.). 

However, when I switch to `sample_with_mcmc=True`, it breaks for any observation (even for those ones where `sample_with_mcmc=False` worked properly), returning the following error:
```
Tuning bracket width...:   2%|▏         | 1/50 [00:04<03:43,  4.57s/it]
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3417, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-29-30b05a893c71>", line 4, in <module>
    s = posterior_nsf.sample((200,), x=data_brain[i, j] / mean_b0vols[i, j], sample_with_mcmc=True)  # , 0.3, th1+th[i], ph1+ph[j]]))
  File "/usr/local/lib/python3.8/site-packages/sbi/inference/posteriors/direct_posterior.py", line 330, in sample
    samples = self._sample_posterior_mcmc(
  File "/usr/local/lib/python3.8/site-packages/sbi/inference/posteriors/base_posterior.py", line 326, in _sample_posterior_mcmc
    samples = self._slice_np_mcmc(
  File "/usr/local/lib/python3.8/site-packages/sbi/inference/posteriors/base_posterior.py", line 389, in _slice_np_mcmc
    posterior_sampler.gen(int(warmup_steps))
  File "/usr/local/lib/python3.8/site-packages/sbi/mcmc/slice_numpy.py", line 84, in gen
    self._tune_bracket_width(rng)
  File "/usr/local/lib/python3.8/site-packages/sbi/mcmc/slice_numpy.py", line 134, in _tune_bracket_width
    x[i], wi = self._sample_from_conditional(i, x[i], rng)
  File "/usr/local/lib/python3.8/site-packages/sbi/mcmc/slice_numpy.py", line 158, in _sample_from_conditional
    while Li(lx) >= logu and cxi - lx < self.max_width:
  File "/usr/local/lib/python3.8/site-packages/sbi/mcmc/slice_numpy.py", line 147, in <lambda>
    Li = lambda t: self.lp_f(np.concatenate([self.x[:i], [t], self.x[i + 1 :]]))
  File "/usr/local/lib/python3.8/site-packages/sbi/inference/posteriors/direct_posterior.py", line 471, in np_potential
    target_log_prob = self.posterior_nn.log_prob(inputs=theta, context=self.x,)
  File "/usr/local/lib/python3.8/site-packages/nflows/distributions/base.py", line 40, in log_prob
    return self._log_prob(inputs, context)
  File "/usr/local/lib/python3.8/site-packages/nflows/flows/base.py", line 39, in _log_prob
    noise, logabsdet = self._transform(inputs, context=embedded_context)
  File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/base.py", line 56, in forward
    return self._cascade(inputs, funcs, context)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/base.py", line 50, in _cascade
    outputs, logabsdet = func(outputs, context)
  File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/base.py", line 56, in forward
    return self._cascade(inputs, funcs, context)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/base.py", line 50, in _cascade
    outputs, logabsdet = func(outputs, context)
  File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/base.py", line 56, in forward
    return self._cascade(inputs, funcs, context)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/base.py", line 50, in _cascade
    outputs, logabsdet = func(outputs, context)
  File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/coupling.py", line 84, in forward
    transform_split, logabsdet = self._coupling_transform_forward(
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/coupling.py", line 194, in _coupling_transform_forward
    return self._coupling_transform(inputs, transform_params, inverse=False)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/coupling.py", line 211, in _coupling_transform
    outputs, logabsdet = self._piecewise_cdf(inputs, transform_params, inverse)
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/coupling.py", line 492, in _piecewise_cdf
    return spline_fn(
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/splines/rational_quadratic.py", line 45, in unconstrained_rational_quadratic_spline
    ) = rational_quadratic_spline(
  File "/usr/local/lib/python3.8/site-packages/nflows/transforms/splines/rational_quadratic.py", line 77, in rational_quadratic_spline
    if torch.min(inputs) < left or torch.max(inputs) > right:
RuntimeError: operation does not have an identity.
```

Version of packages:
python 3.8
sbi 0.13.2
nflows 0.12
torch 1.6.0
@josepman 

This seems to be an issue with the bound of the `NSF`: during bracket tuning of the MCMC sampler `theta` close to the `NSF` bound (default 3.0) are evaluated. Because they are outside of the bounds, the `inputs` to the spline are empty (see bottom of first stack trace) and we get the error. 

The fix could be to change the `NSF` evaluation to check for empty inputs and to handle this case in some way, e.g., return empty result. 

We will come up with a minimal example to confirm this hypothesis and then suggest a fix in `nflows`.
This is the minimal example. I'll post it on nflows after lunch.

```
from nflows import distributions as distributions_
from nflows import flows, transforms
from nflows.nn import nets
from torch import Tensor, nn, relu, tanh
import torch

def create_alternating_binary_mask(features, even=True):
    """
    Creates a binary mask of a given dimension which alternates its masking.

    :param features: Dimension of mask.
    :param even: If True, even values are assigned 1s, odd 0s. If False, vice versa.
    :return: Alternating binary mask of type torch.Tensor.
    """
    mask = torch.zeros(features).byte()
    start = 0 if even else 1
    mask[start::2] += 1
    return mask

def build_nsf_() -> nn.Module:
    
    x_numel = 3
    y_numel = 3

    transform = transforms.CompositeTransform(
        [
            transforms.CompositeTransform(
                [
                    transforms.PiecewiseRationalQuadraticCouplingTransform(
                        mask=create_alternating_binary_mask(
                            features=x_numel, even=(i % 2 == 0)
                        ),
                        transform_net_create_fn=lambda in_features, out_features: nets.ResidualNet(
                            in_features=in_features,
                            out_features=out_features,
                            hidden_features=50,
                            context_features=y_numel,
                        ),
                        num_bins=10,
                        tails="linear",
                        tail_bound=3.0,
                        apply_unconditional_transform=False,
                    ),
                    transforms.LULinear(x_numel, identity_init=True),
                ]
            )
            for i in range(5)
        ]
    )

    distribution = distributions_.StandardNormal((x_numel,))
    neural_net = flows.Flow(transform, distribution)

    return neural_net

nsf = build_nsf_()
nsf.log_prob(inputs=3.01*torch.ones(1,3), context=torch.zeros(1,3))
```

Here's the full error:
```
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-117-4de2f5f83456> in <module>
     45 
     46 nsf = build_nsf_()
---> 47 nsf.log_prob(inputs=3.01*torch.ones(1,3), context=torch.zeros(1,3))

~/anaconda3/envs/sbi/lib/python3.8/site-packages/nflows/distributions/base.py in log_prob(self, inputs, context)
     38                     "Number of input items must be equal to number of context items."
     39                 )
---> 40         return self._log_prob(inputs, context)
     41 
     42     def _log_prob(self, inputs, context):

~/anaconda3/envs/sbi/lib/python3.8/site-packages/nflows/flows/base.py in _log_prob(self, inputs, context)
     37     def _log_prob(self, inputs, context):
     38         embedded_context = self._embedding_net(context)
---> 39         noise, logabsdet = self._transform(inputs, context=embedded_context)
     40         log_prob = self._distribution.log_prob(noise, context=embedded_context)
     41         return log_prob + logabsdet

~/anaconda3/envs/sbi/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

~/anaconda3/envs/sbi/lib/python3.8/site-packages/nflows/transforms/base.py in forward(self, inputs, context)
     54     def forward(self, inputs, context=None):
     55         funcs = self._transforms
---> 56         return self._cascade(inputs, funcs, context)
     57 
     58     def inverse(self, inputs, context=None):

~/anaconda3/envs/sbi/lib/python3.8/site-packages/nflows/transforms/base.py in _cascade(inputs, funcs, context)
     48         total_logabsdet = torch.zeros(batch_size)
     49         for func in funcs:
---> 50             outputs, logabsdet = func(outputs, context)
     51             total_logabsdet += logabsdet
     52         return outputs, total_logabsdet

~/anaconda3/envs/sbi/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

~/anaconda3/envs/sbi/lib/python3.8/site-packages/nflows/transforms/base.py in forward(self, inputs, context)
     54     def forward(self, inputs, context=None):
     55         funcs = self._transforms
---> 56         return self._cascade(inputs, funcs, context)
     57 
     58     def inverse(self, inputs, context=None):

~/anaconda3/envs/sbi/lib/python3.8/site-packages/nflows/transforms/base.py in _cascade(inputs, funcs, context)
     48         total_logabsdet = torch.zeros(batch_size)
     49         for func in funcs:
---> 50             outputs, logabsdet = func(outputs, context)
     51             total_logabsdet += logabsdet
     52         return outputs, total_logabsdet

~/anaconda3/envs/sbi/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

~/anaconda3/envs/sbi/lib/python3.8/site-packages/nflows/transforms/coupling.py in forward(self, inputs, context)
     82 
     83         transform_params = self.transform_net(identity_split, context)
---> 84         transform_split, logabsdet = self._coupling_transform_forward(
     85             inputs=transform_split, transform_params=transform_params
     86         )

~/anaconda3/envs/sbi/lib/python3.8/site-packages/nflows/transforms/coupling.py in _coupling_transform_forward(self, inputs, transform_params)
    192 class PiecewiseCouplingTransform(CouplingTransform):
    193     def _coupling_transform_forward(self, inputs, transform_params):
--> 194         return self._coupling_transform(inputs, transform_params, inverse=False)
    195 
    196     def _coupling_transform_inverse(self, inputs, transform_params):

~/anaconda3/envs/sbi/lib/python3.8/site-packages/nflows/transforms/coupling.py in _coupling_transform(self, inputs, transform_params, inverse)
    209             transform_params = transform_params.reshape(b, d, -1)
    210 
--> 211         outputs, logabsdet = self._piecewise_cdf(inputs, transform_params, inverse)
    212 
    213         return outputs, torchutils.sum_except_batch(logabsdet)

~/anaconda3/envs/sbi/lib/python3.8/site-packages/nflows/transforms/coupling.py in _piecewise_cdf(self, inputs, transform_params, inverse)
    490             spline_kwargs = {"tails": self.tails, "tail_bound": self.tail_bound}
    491 
--> 492         return spline_fn(
    493             inputs=inputs,
    494             unnormalized_widths=unnormalized_widths,

~/anaconda3/envs/sbi/lib/python3.8/site-packages/nflows/transforms/splines/rational_quadratic.py in unconstrained_rational_quadratic_spline(inputs, unnormalized_widths, unnormalized_heights, unnormalized_derivatives, inverse, tails, tail_bound, min_bin_width, min_bin_height, min_derivative)
     43         outputs[inside_interval_mask],
     44         logabsdet[inside_interval_mask],
---> 45     ) = rational_quadratic_spline(
     46         inputs=inputs[inside_interval_mask],
     47         unnormalized_widths=unnormalized_widths[inside_interval_mask, :],

~/anaconda3/envs/sbi/lib/python3.8/site-packages/nflows/transforms/splines/rational_quadratic.py in rational_quadratic_spline(inputs, unnormalized_widths, unnormalized_heights, unnormalized_derivatives, inverse, left, right, bottom, top, min_bin_width, min_bin_height, min_derivative)
     75     min_derivative=DEFAULT_MIN_DERIVATIVE,
     76 ):
---> 77     if torch.min(inputs) < left or torch.max(inputs) > right:
     78         raise InputOutsideDomain()
     79 

RuntimeError: operation does not have an identity.
```
This will be fixed by https://github.com/bayesiains/nflows/pull/25

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -28,7 +28,7 @@
     "matplotlib",
     "numpy",
     "pillow",
-    "pyknos>=0.12",
+    "pyknos>=0.14",
     "pyro-ppl>=1.3.1",
     "scipy",
     "tensorboard",


Test patch: