Repository: BerkeleyLearnVerify/Scenic

Issue:
External sampler does not work with object positioning
### System Details


1.  Python Version: Python 3.12.4
1.  Scenic Version: Scenic 3.0.0
1.  Operating System / Platform: Apple M2 Air macOS Sonoma 14.5


### Detailed Description

For our use case (railway) we want to make sure we create samples in different regions: plains, tunnel, forest area.

To make this work we created an external sampler. Although we can use the external sampler to define values for arbitrary parameters (defined with the `with` keyword), we cannot assign a region (`on`/`in` keyword) from our distribution.

It would be really nice if we can use external parameters to define regions.

For comparison, this is a working scenario:

```scenic
from scenic_external_sampler.regions import NorthEast, SouthWest

ego = new Object in Uniform(NorthEast, SouthWest)
```

While this scenario fails:

```scenic
from scenic_external_sampler.regions import NorthEast, SouthWest
from scenic_external_sampler.sampler import SimpleSampler, Values
param externalSampler = SimpleSampler

ego = new Object in Values(NorthEast, SouthWest)
```

### Steps To Reproduce

Have a look at a sample repository I created: https://github.com/amolenaar/scenic-external-sampler

You can easily set up an environment with Poetry:

```bash
poetry install
poetry run pytest
```

All tests succeed, except [test_regions_with_externally_sampled_distribution](https://github.com/amolenaar/scenic-external-sampler/blob/main/tests/test_scenario.py#L66). This test tries to set a region from an external parameter. It fails, since the external sampler has not been initialized yet.

It shows a stack trace:

```
.venv/lib/python3.12/site-packages/scenic/syntax/translator.py:105: in scenarioFromString
    return _scenarioFromStream(stream, options, filename, scenario=scenario, **kwargs)
.venv/lib/python3.12/site-packages/scenic/syntax/translator.py:172: in _scenarioFromStream
    compileStream(stream, namespace, compileOptions, filename)
.venv/lib/python3.12/site-packages/scenic/syntax/translator.py:321: in compileStream
    executeCodeIn(code, namespace)
.venv/lib/python3.12/site-packages/scenic/syntax/translator.py:560: in executeCodeIn
    exec(code, namespace)
<string>:1: in <module>
    ???
.venv/lib/python3.12/site-packages/scenic/syntax/veneer.py:1411: in In
    if alwaysProvidesOrientation(region):
.venv/lib/python3.12/site-packages/scenic/syntax/veneer.py:1520: in alwaysProvidesOrientation
    sample = region.sample()
.venv/lib/python3.12/site-packages/scenic/core/distributions.py:149: in sample
    subsamples[child] = child.sample(subsamples)
.venv/lib/python3.12/site-packages/scenic/core/distributions.py:150: in sample
    return self._conditioned.sampleGiven(subsamples)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <scenic_external_sampler.sampler.Values object at 0x13f308350>, value = <DefaultIdentityDict {}>

    def sampleGiven(self, value):
        """Specialization of  `Samplable.sampleGiven` for external parameters.
    
        By default, this method simply looks up the value previously sampled by
        `ExternalSampler.sample`.
        """
>       assert self.sampler is not None
E       AssertionError

.venv/lib/python3.12/site-packages/scenic/core/external_params.py:284: AssertionError
```

### Issue Submission Checklist

- [X] I am reporting an issue, not asking a question
- [X] I checked the open and closed issues, forum, etc. and have not found any solution
- [X] I have provided all necessary code, etc. to reproduce the issue

Discussion:
@Eric-Vin: Hi Arjan! This is definitely a gap in the current Scenic functionality, but I think there might be a workaround that's not too difficult to implement for now. I would recommend creating a subclass [ExternalParameter](https://docs.scenic-lang.org/en/latest/modules/scenic.core.external_params.html#scenic.core.external_params.ExternalParameter) which wraps the region and who's `sampleGiven` samples a point from the region as desired. You could then replace `in` with `at` (and keep `on` the same) to get the desired behavior. Hope this helps with your use case, and please let me know if there are issues with this approach!@amolenaar: Hi @Eric-Vin,

What do you mean by "subclass `ExternalParameter` which wraps the region"?

Something along the lines of:

```python
class Values(ExternalParameter):

    def __init__(self, *values: Any):
        super().__init__()
        self._values = values

    def sampleGiven(self, value):
        if self.sampler is None:
            return self._values[0]
        return super().sampleGiven(value)
```

Just simply return the first option if no samples have been generated by our custom `ExternalSampler`?@dfremont: Hi Arjan, the `ExternalParameter.sampleGiven` method should not get called before the sampler is initialized, so that part is just a bug in Scenic. We'll look into merging at least a quick band-aid fix soon. Eric's suggestion was to have your `sampleGiven` method call your external sampler to compute a point according to any logic you like, but that's orthogonal to the issue you're having here.

Stepping back a bit, if all you want is to make sure you systematically iterate through all of the regions (rather than picking one at random), VerifAI's grid sampler does that. For example:
```scenic
param verifaiSamplerType = "grid"
ego = new Object in VerifaiOptions([Region1, Region2])
```
will pick `Region1` for the first scene and `Region2` for the second. In your case you'd want to then go back to `Region1` and repeat forever (sampling different points from within the regions of course), which you are supposed to be able to do by setting the `repeat` flag of the grid sampler:
```scenic
from dotmap import DotMap
param verifaiSamplerType = "grid"
param verifaiSamplerParams = DotMap(repeat=True)
ego = new Object in VerifaiOptions([Region1, Region2])
```
Unfortunately this doesn't work right now (the repeat flag is ignored and VerifAI raises an exception after both options have been exhausted) due to a bug in the grid sampler which I only just noticed. So we'll work on fixing that too :)@Eric-Vin: The VerifAI solution @dfremont suggested should now function correctly (with latest code on the VerifAI repo's main branch)!@dfremont: @amolenaar Eric pushed a temporary fix for the assertion failure you were getting above: you'll likely get a warning instead, which you can ignore. If the fix doesn't solve your problem, please feel free to reopen this issue.

Solution patch:
diff --git a/src/scenic/syntax/veneer.py b/src/scenic/syntax/veneer.py
index 7b03cc79..4b550fbd 100644
--- a/src/scenic/syntax/veneer.py
+++ b/src/scenic/syntax/veneer.py
@@ -249,6 +249,7 @@ from pathlib import Path
 import sys
 import traceback
 import typing
+import warnings
 
 from scenic.core.distributions import (
     Distribution,
@@ -1521,6 +1522,11 @@ def alwaysProvidesOrientation(region):
             return sample.orientation is not None or sample is nowhere
         except RejectionException:
             return False
+        except Exception as e:
+            warnings.warn(
+                f"While sampling internally to determine if a random region provides an orientation, the following exception was raised: {repr(e)}"
+            )
+            return False
 
 
 def OffsetBy(offset):


Test patch:
diff --git a/tests/syntax/test_specifiers.py b/tests/syntax/test_specifiers.py
index fe5e677f..274e875e 100644
--- a/tests/syntax/test_specifiers.py
+++ b/tests/syntax/test_specifiers.py
@@ -1198,3 +1198,19 @@ def test_color():
             ego = new Object with color (1,1,1,1,1)
             """
         sampleEgoFrom(program)
+
+
+# alwaysProvidesOrientation
+def test_alwaysProvidesOrientation_exception():
+    with pytest.warns(UserWarning):
+        compileScenic(
+            """
+            from scenic.core.distributions import distributionFunction
+
+            @distributionFunction
+            def foo(bar):
+                assert False
+
+            new Object in foo(Range(0,1))
+            """
+        )