Repository: evidentlyai/evidently
Issue: Unexpected parameter value passed into Test
Root cause: Documentation Deficiency

I am trying to write a `TestSuite`, in this example I will reference the `TestShareOfOutListValues` test in particular, but question applies to all tests. I am trying to specify the cat_stattest and cat_stattest_threshold parameters for the test, but get a `TypeError: __init__() got an unexpected keyword argument 'cat_stattest'`. 

[This page](https://docs.evidentlyai.com/reference/all-tests#test-presets) in the documentation refers to the `TestShareOfOutListValues` accepting a `cat_stattest` and `cat_stattest_threshold` parameter


here is my code:
```
test_suite = TestSuite(tests=[
    TestShareOfOutListValues(column_name=column, cat_stattest="jensenshannon", "cat_stattest_threshold"=1)
])
test_suite.run(reference_data=test_ref, current_data=test_current, column_mapping=ColumnMapping(categorical_features=[col])
```

Hi @elutins, 

The `TestShareOfOutListValues` does not accept this parameter. 

I double-checked the docs (the Test is listed under the [Data Quality group](https://docs.evidentlyai.com/reference/all-tests#data-quality)). It appears to be reflected correctly: 
<img width="750" alt="Screenshot 2023-08-11 at 23 56 03" src="https://github.com/evidentlyai/evidently/assets/67064421/3140ccd4-a0bf-4e34-8546-4c1bb9a30a9a">

For this specific Test, you can pass: 
- the standard test condition parameters (such as "less than" (`lt`), "greater than" (`gt`) and so on - documented [here](https://docs.evidentlyai.com/user-guide/tests-and-reports/custom-test-suite#custom-conditions)).
- a test-specific parameter where you pass a list of values - this is relevant if you do not pass the reference dataset and want to manually specify the list of acceptable values.  

I noticed you linked to the "Presets" subheading on the page. In this case, the listed parameters refer to the Test Preset. If you pass the parameters to the Preset, they apply only to the relevant individual Tests inside it.

For example, here is the description of the `NoTargetPerformanceTestPreset`
<img width="765" alt="Screenshot 2023-08-12 at 00 02 05" src="https://github.com/evidentlyai/evidently/assets/67064421/60bdec8c-1332-46fa-8bf4-2d30b97ce632">

It includes the `TestShareOfOutListValues` we discuss, but also includes the `TestShareOfDriftedColumns` that detects distribution drift. The `cat_stattest` parameter is specific to it. If you pass the `cat_stattest` parameters to the Preset, it will affect the drift detection method for categorical features. For this parameter, you must pass one of the applicable values (documented [here](https://docs.evidentlyai.com/user-guide/customization/options-for-statistical-tests#drift-detection-methods-tabular), such as `psi`, `jensenshannon`, etc.) It will not affect other Tests in the Preset - only the ones that refer to data drift detection.

Generally, since you design custom Test Suites from individual Tests, you can safely ignore everything that relates to the Presets and Preset parameters.

**Could you share what is the expected behavior you'd like to see when passing the parameter to the `TestShareOfOutListValues` test?**

The default behavior is to fail if at least one value is out of the list. If you want to set a different condition, you can use the standard parameters like (`lt`, `gt`, `lte`, `gte` and so on).  

hi @elenasamuylova

thank you for clarifying this was super helpful; I believe the confusion arose from the difference between the `TestPresets` and an individual `Test`. 

Follow up question, for test such as `TestShareOfOutListValues`, I am noticing that `TestShareOfOutListValues` is failing because the current dataset contains missing values whereas the reference dataset does not. Is there a way to exclude missing values from being counted in `TestShareOfOutListValues` test? I already have a `TestColumnShareOfMissingValues` to handle checks for missing values.

Hi @elutins, 

I am afraid the test explicitly checks for all new values, and this includes "unknowns" or "nulls". Possible workarounds are:
- Make sure the reference dataset includes missing values
- Pass a custom list of allowed values, including missing values