internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: great-expectations/great_expectations

Issue:
Use cleaner solution for non-truncating division in python 2
Prefer `from __future__ import division` to `1.*x/y`


Discussion:
Done inline during improve_partitions push.

Resolving patch:
diff --git a/scripts/build_glossary_page.py b/scripts/build_glossary_page.py
new file mode 100644
--- /dev/null
+++ b/scripts/build_glossary_page.py
@@ -0,0 +1,36 @@
+import json
+
+
+def build_glossary_tuples(source_json_path):
+    # Open the source_json_path
+    with open(source_json_path) as json_file:
+        # Read the contents as a json
+        data = json.load(json_file)
+        # Convert to list of tuples containing ("term", "definition", "url")
+        data_list = [(x['term'], x['definition']) for x in data.values()]
+        # Order list alphabetically by "term"
+        data_list.sort(key=lambda y: y[0])
+        # return the ordered list.
+        return data_list
+
+
+def build_glossary_page(orderd_list_of_terms_tuples, glossary_file_path):
+    # Open the glossary page for editing
+    with open(glossary_file_path, "w") as glossary_file:
+        # Write the glossary page header
+        glossary_file.write("---\nid: glossary\ntitle: \"Glossary of Terms\"\n---\n")
+        # iterate the glossary list of tuples and write glossary entries.
+        for term, definition in orderd_list_of_terms_tuples:
+            glossary_file.write(f"**{term}:** {definition}\n\n")
+
+
+def all_together_now(source_json_path, glossary_file_path):
+    list_of_terms_tuples = build_glossary_tuples(source_json_path)
+    build_glossary_page(list_of_terms_tuples, glossary_file_path)
+
+
+if __name__ == '__main__':
+    all_together_now(
+        source_json_path="../docs/term_tags/terms.json",
+        glossary_file_path="../docs/glossary.md"
+    )


Test patch:
Repository: great-expectations/great_expectations

Issue:
Use cleaner solution for non-truncating division in python 2
Prefer `from __future__ import division` to `1.*x/y`


Discussion:
Done inline during improve_partitions push.

Resolving patch:
diff --git a/scripts/build_glossary_page.py b/scripts/build_glossary_page.py
--- a/scripts/build_glossary_page.py
+++ b/scripts/build_glossary_page.py
@@ -18,7 +18,7 @@ def build_glossary_page(orderd_list_of_terms_tuples, glossary_file_path):
     # Open the glossary page for editing
     with open(glossary_file_path, "w") as glossary_file:
         # Write the glossary page header
-        glossary_file.write("---\nid: glossary\ntitle: \"Glossary of Terms\"\n---\n")
+        glossary_file.write("---\nid: glossary\ntitle: \"Glossary of Terms\"\n---\n\n")
         # iterate the glossary list of tuples and write glossary entries.
         for term, definition in orderd_list_of_terms_tuples:
             glossary_file.write(f"**{term}:** {definition}\n\n")


Test patch:
Repository: great-expectations/great_expectations

Issue:
Expectation docstrings not present in Jupyter notebooks
**Describe the bug**
Docstrings for `expect_*` methods are not present in Jupyter notebooks.

**To Reproduce**
Steps to reproduce the behavior:
1. Create or edit a suite
2. Enter an expectation method
3. With your cursor in the parens, pull up the docstring shortcut in Jupyter notebooks
4. Note lack of helpful doc string

**Expected behavior**
I expect to see the docstrings and parameters so I know how to use an expectation without looking up the docs.

**Environment (please complete the following information):**
 - Operating System: Windows
 - Great Expectations Version: 0.14.2 

**Additional context**
Add any other context about the problem here.
<img width="963" alt="Screen Shot 2022-01-25 at 4 01 12 PM" src="https://user-images.githubusercontent.com/928247/151079507-24392874-5b37-4e6c-91bc-0402fb1e2568.png">




Discussion:
hi @Aylr, thanks for flagging this -- we'll take a look and get back to you soon!

Resolving patch:
diff --git a/great_expectations/validator/validator.py b/great_expectations/validator/validator.py
--- a/great_expectations/validator/validator.py
+++ b/great_expectations/validator/validator.py
@@ -248,6 +248,7 @@ def validate_expectation(self, name: str):
                         Returns:
                             The Expectation's validation result
         """
+        expectation_impl = get_expectation_impl(name)
 
         def inst_expectation(*args, **kwargs):
             # this is used so that exceptions are caught appropriately when they occur in expectation config
@@ -265,7 +266,6 @@ def inst_expectation(*args, **kwargs):
                 {k: v for k, v in kwargs.items() if k in Validator.RUNTIME_KEYS}
             )
 
-            expectation_impl = get_expectation_impl(name)
             allowed_config_keys = expectation_impl.get_allowed_config_keys()
 
             expectation_kwargs = recursively_convert_to_json_serializable(kwargs)
@@ -352,6 +352,8 @@ def inst_expectation(*args, **kwargs):
             return validation_result
 
         inst_expectation.__name__ = name
+        inst_expectation.__doc__ = expectation_impl.__doc__
+
         return inst_expectation
 
     @property


Test patch:
diff --git a/tests/validator/test_validator.py b/tests/validator/test_validator.py
--- a/tests/validator/test_validator.py
+++ b/tests/validator/test_validator.py
@@ -1078,3 +1078,12 @@ def test_validator_progress_bar_config_disabled(
 
     assert mock_tqdm.called is True
     assert mock_tqdm.call_args[1]["disable"] is True
+
+
+def test_validator_docstrings(multi_batch_taxi_validator):
+    expectation_impl = getattr(
+        multi_batch_taxi_validator, "expect_column_values_to_be_in_set", None
+    )
+    assert expectation_impl.__doc__.startswith(
+        "Expect each column value to be in a given set"
+    )
Repository: great-expectations/great_expectations

Issue:
Use cleaner solution for non-truncating division in python 2
Prefer `from __future__ import division` to `1.*x/y`


Discussion:
Done inline during improve_partitions push.

Resolving patch:
diff --git a/great_expectations/validator/validator.py b/great_expectations/validator/validator.py
--- a/great_expectations/validator/validator.py
+++ b/great_expectations/validator/validator.py
@@ -292,12 +292,16 @@ def inst_expectation(*args, **kwargs):
             configuration = ExpectationConfiguration(
                 expectation_type=name, kwargs=expectation_kwargs, meta=meta
             )
-            configuration.process_evaluation_parameters(
-                self._expectation_suite.evaluation_parameters, True, self._data_context
-            )
 
             exception_info: ExceptionInfo
 
+            if self.interactive_evaluation:
+                configuration.process_evaluation_parameters(
+                    self._expectation_suite.evaluation_parameters,
+                    True,
+                    self._data_context,
+                )
+
             try:
                 expectation = expectation_impl(configuration)
                 """Given an implementation and a configuration for any Expectation, returns its validation result"""


Test patch:
Repository: great-expectations/great_expectations

Issue:
Use cleaner solution for non-truncating division in python 2
Prefer `from __future__ import division` to `1.*x/y`


Discussion:
Done inline during improve_partitions push.

Resolving patch:
diff --git a/great_expectations/util.py b/great_expectations/util.py
--- a/great_expectations/util.py
+++ b/great_expectations/util.py
@@ -1011,9 +1011,15 @@ def filter_properties_dict(
     Returns:
         The (possibly) filtered properties dictionary (or None if no entries remain after filtering is performed)
     """
-    if keep_fields and delete_fields:
+    if keep_fields is None:
+        keep_fields = set()
+
+    if delete_fields is None:
+        delete_fields = set()
+
+    if keep_fields & delete_fields:
         raise ValueError(
-            "Only one of keep_fields and delete_fields filtering directives can be specified."
+            "Common keys between sets of keep_fields and delete_fields filtering directives are illegal."
         )
 
     if clean_falsy:


Test patch:
diff --git a/tests/test_ge_utils.py b/tests/test_ge_utils.py
--- a/tests/test_ge_utils.py
+++ b/tests/test_ge_utils.py
@@ -360,6 +360,7 @@ def test_filter_properties_dict():
             delete_fields={
                 "integer_zero",
                 "scientific_notation_floating_point_number",
+                "string",
             },
             clean_falsy=True,
         )
Repository: great-expectations/great_expectations

Issue:
Use cleaner solution for non-truncating division in python 2
Prefer `from __future__ import division` to `1.*x/y`


Discussion:
Done inline during improve_partitions push.

Resolving patch:
diff --git a/great_expectations/rule_based_profiler/parameter_builder/parameter_builder.py b/great_expectations/rule_based_profiler/parameter_builder/parameter_builder.py
--- a/great_expectations/rule_based_profiler/parameter_builder/parameter_builder.py
+++ b/great_expectations/rule_based_profiler/parameter_builder/parameter_builder.py
@@ -27,6 +27,8 @@
 from great_expectations.validator.validator import Validator
 
 # TODO: <Alex>These are placeholder types, until a formal metric computation state class is made available.</Alex>
+MetricValue = Union[Any, List[Any], np.ndarray]
+MetricValues = Union[MetricValue, np.ndarray]
 MetricComputationDetails = Dict[str, Any]
 MetricComputationResult = make_dataclass(
     "MetricComputationResult", ["metric_values", "details"]
@@ -214,10 +216,9 @@ def get_metrics(
             parameters=parameters,
         )
 
-        # TODO: <Alex>Create a special type encapsulating "value of any type or list of values of any type".</Alex>
-        metric_values: Union[Union[Any, List[Any]], np.ndarray] = []
+        metric_values: MetricValues = []
 
-        metric_value: Union[Any, List[Any]]
+        metric_value: MetricValue
         batch_id: str
         for batch_id in batch_ids:
             metric_domain_kwargs["batch_id"] = batch_id
diff --git a/great_expectations/rule_based_profiler/parameter_builder/simple_date_format_string_parameter_builder.py b/great_expectations/rule_based_profiler/parameter_builder/simple_date_format_string_parameter_builder.py
--- a/great_expectations/rule_based_profiler/parameter_builder/simple_date_format_string_parameter_builder.py
+++ b/great_expectations/rule_based_profiler/parameter_builder/simple_date_format_string_parameter_builder.py
@@ -183,7 +183,7 @@ def _build_parameters(
         match_strftime_metric_value_kwargs: dict
         for fmt_string in self._candidate_strings:
             if self._metric_value_kwargs:
-                match_strftime_metric_value_kwargs: dict = {
+                match_strftime_metric_value_kwargs = {
                     **self._metric_value_kwargs,
                     **{"strftime_format": fmt_string},
                 }
diff --git a/great_expectations/rule_based_profiler/rule_based_profiler.py b/great_expectations/rule_based_profiler/rule_based_profiler.py
--- a/great_expectations/rule_based_profiler/rule_based_profiler.py
+++ b/great_expectations/rule_based_profiler/rule_based_profiler.py
@@ -18,7 +18,6 @@
     GeCloudIdentifier,
 )
 from great_expectations.data_context.util import instantiate_class_from_config
-from great_expectations.rule_based_profiler.config import RuleBasedProfilerConfig
 from great_expectations.rule_based_profiler.config.base import (
     RuleBasedProfilerConfig,
     domainBuilderConfigSchema,


Test patch:
Repository: great-expectations/great_expectations

Issue:
Use cleaner solution for non-truncating division in python 2
Prefer `from __future__ import division` to `1.*x/y`


Discussion:
Done inline during improve_partitions push.

Resolving patch:
diff --git a/great_expectations/expectations/expectation.py b/great_expectations/expectations/expectation.py
--- a/great_expectations/expectations/expectation.py
+++ b/great_expectations/expectations/expectation.py
@@ -1090,7 +1090,7 @@ def _get_docstring_and_short_description(self) -> Tuple[str, str]:
 
         if self.__doc__ is not None:
             docstring = self.__doc__
-            short_description = self.__doc__.split("\n")[0]
+            short_description = next(line for line in self.__doc__.split("\n") if line)
         else:
             docstring = ""
             short_description = ""


Test patch:
Repository: great-expectations/great_expectations

Issue:
Use cleaner solution for non-truncating division in python 2
Prefer `from __future__ import division` to `1.*x/y`


Discussion:
Done inline during improve_partitions push.

Resolving patch:
diff --git a/examples/expectations/column_aggregate_expectation_template.py b/examples/expectations/column_aggregate_expectation_template.py
--- a/examples/expectations/column_aggregate_expectation_template.py
+++ b/examples/expectations/column_aggregate_expectation_template.py
@@ -4,7 +4,7 @@
     https://docs.greatexpectations.io/docs/guides/expectations/creating_custom_expectations/how_to_create_custom_column_aggregate_expectations
 """
 
-from typing import Dict
+from typing import Dict, Optional
 
 from great_expectations.core import ExpectationConfiguration
 from great_expectations.exceptions import InvalidExpectationConfigurationError
diff --git a/examples/expectations/column_map_expectation_template.py b/examples/expectations/column_map_expectation_template.py
--- a/examples/expectations/column_map_expectation_template.py
+++ b/examples/expectations/column_map_expectation_template.py
@@ -5,6 +5,7 @@
 """
 
 import json
+from typing import Optional
 
 from great_expectations.exceptions import InvalidExpectationConfigurationError
 from great_expectations.execution_engine import (
@@ -12,7 +13,10 @@
     SparkDFExecutionEngine,
     SqlAlchemyExecutionEngine,
 )
-from great_expectations.expectations.expectation import ColumnMapExpectation
+from great_expectations.expectations.expectation import (
+    ColumnMapExpectation,
+    ExpectationConfiguration,
+)
 from great_expectations.expectations.metrics import (
     ColumnMapMetricProvider,
     column_condition_partial,


Test patch:
Repository: great-expectations/great_expectations

Issue:
Use cleaner solution for non-truncating division in python 2
Prefer `from __future__ import division` to `1.*x/y`


Discussion:
Done inline during improve_partitions push.

Resolving patch:
diff --git a/great_expectations/rule_based_profiler/config/base.py b/great_expectations/rule_based_profiler/config/base.py
--- a/great_expectations/rule_based_profiler/config/base.py
+++ b/great_expectations/rule_based_profiler/config/base.py
@@ -185,6 +185,11 @@ class Meta:
         allow_none=True,
         missing="great_expectations.rule_based_profiler.parameter_builder",
     )
+    json_serialize = fields.Boolean(
+        required=False,
+        allow_none=True,
+        missing=True,
+    )
     batch_request = fields.Raw(
         required=False,
         allow_none=True,
diff --git a/great_expectations/rule_based_profiler/parameter_builder/__init__.py b/great_expectations/rule_based_profiler/parameter_builder/__init__.py
--- a/great_expectations/rule_based_profiler/parameter_builder/__init__.py
+++ b/great_expectations/rule_based_profiler/parameter_builder/__init__.py
@@ -10,6 +10,9 @@
 from great_expectations.rule_based_profiler.parameter_builder.numeric_metric_range_multi_batch_parameter_builder import (  # isort:skip
     NumericMetricRangeMultiBatchParameterBuilder,
 )
+from great_expectations.rule_based_profiler.parameter_builder.mean_unexpected_map_metric_multi_batch_parameter_builder import (  # isort:skip
+    MeanUnexpectedMapMetricMultiBatchParameterBuilder,
+)
 from great_expectations.rule_based_profiler.parameter_builder.regex_pattern_string_parameter_builder import (  # isort:skip
     RegexPatternStringParameterBuilder,
 )


Test patch:
diff --git a/tests/rule_based_profiler/parameter_builder/test_mean_unexpected_map_metric_multi_batch_parameter_builder.py b/tests/rule_based_profiler/parameter_builder/test_mean_unexpected_map_metric_multi_batch_parameter_builder.py
--- a/tests/rule_based_profiler/parameter_builder/test_mean_unexpected_map_metric_multi_batch_parameter_builder.py
+++ b/tests/rule_based_profiler/parameter_builder/test_mean_unexpected_map_metric_multi_batch_parameter_builder.py
@@ -9,10 +9,8 @@
     get_parameter_value_and_validate_return_type,
 )
 from great_expectations.rule_based_profiler.parameter_builder import (
-    MetricMultiBatchParameterBuilder,
-)
-from great_expectations.rule_based_profiler.parameter_builder.mean_unexpected_map_metric_multi_batch_parameter_builder import (
     MeanUnexpectedMapMetricMultiBatchParameterBuilder,
+    MetricMultiBatchParameterBuilder,
 )
 from great_expectations.rule_based_profiler.types import Domain, ParameterContainer
 from tests.rule_based_profiler.conftest import ATOL, RTOL
Repository: great-expectations/great_expectations

Issue:
Use cleaner solution for non-truncating division in python 2
Prefer `from __future__ import division` to `1.*x/y`


Discussion:
Done inline during improve_partitions push.

Resolving patch:
diff --git a/great_expectations/execution_engine/sqlalchemy_execution_engine.py b/great_expectations/execution_engine/sqlalchemy_execution_engine.py
--- a/great_expectations/execution_engine/sqlalchemy_execution_engine.py
+++ b/great_expectations/execution_engine/sqlalchemy_execution_engine.py
@@ -1,6 +1,8 @@
 import copy
 import datetime
 import logging
+import random
+import string
 import traceback
 import warnings
 from pathlib import Path
@@ -867,9 +869,18 @@ def resolve_metric_bundle(
                     "ids": [],
                     "domain_kwargs": compute_domain_kwargs,
                 }
-            queries[domain_id]["select"].append(
-                engine_fn.label(metric_to_resolve.metric_name)
-            )
+            if self.engine.dialect.name == "clickhouse":
+                queries[domain_id]["select"].append(
+                    engine_fn.label(
+                        metric_to_resolve.metric_name.join(
+                            random.choices(string.ascii_lowercase, k=2)
+                        )
+                    )
+                )
+            else:
+                queries[domain_id]["select"].append(
+                    engine_fn.label(metric_to_resolve.metric_name)
+                )
             queries[domain_id]["ids"].append(metric_to_resolve.id)
         for query in queries.values():
             domain_kwargs = query["domain_kwargs"]


Test patch:
Repository: great-expectations/great_expectations

Issue:
Use cleaner solution for non-truncating division in python 2
Prefer `from __future__ import division` to `1.*x/y`


Discussion:
Done inline during improve_partitions push.

Resolving patch:
diff --git a/great_expectations/expectations/metrics/map_metric_provider.py b/great_expectations/expectations/metrics/map_metric_provider.py
--- a/great_expectations/expectations/metrics/map_metric_provider.py
+++ b/great_expectations/expectations/metrics/map_metric_provider.py
@@ -2045,7 +2045,7 @@ def _sqlalchemy_column_map_condition_values(
         query = query.limit(result_format["partial_unexpected_count"])
     elif (
         result_format["result_format"] == "COMPLETE"
-        and "bigquery" in execution_engine.dialect.name
+        and "bigquery" in execution_engine.engine.dialect.name
     ):
         logger.warning(
             "BigQuery imposes a limit of 10000 parameters on individual queries; "


Test patch:
Repository: great-expectations/great_expectations

Issue:
Use cleaner solution for non-truncating division in python 2
Prefer `from __future__ import division` to `1.*x/y`


Discussion:
Done inline during improve_partitions push.

Resolving patch:
diff --git a/great_expectations/rule_based_profiler/types/__init__.py b/great_expectations/rule_based_profiler/types/__init__.py
--- a/great_expectations/rule_based_profiler/types/__init__.py
+++ b/great_expectations/rule_based_profiler/types/__init__.py
@@ -9,6 +9,8 @@
 from .parameter_container import (  # isort:skip
     DOMAIN_KWARGS_PARAMETER_FULLY_QUALIFIED_NAME,
     FULLY_QUALIFIED_PARAMETER_NAME_SEPARATOR_CHARACTER,
+    FULLY_QUALIFIED_PARAMETER_NAME_VALUE_KEY,
+    FULLY_QUALIFIED_PARAMETER_NAME_METADATA_KEY,
     PARAMETER_KEY,
     VARIABLES_KEY,
     VARIABLES_PREFIX,


Test patch:
Repository: great-expectations/great_expectations

Issue:
Use cleaner solution for non-truncating division in python 2
Prefer `from __future__ import division` to `1.*x/y`


Discussion:
Done inline during improve_partitions push.

Resolving patch:
diff --git a/great_expectations/rule_based_profiler/parameter_builder/regex_pattern_string_parameter_builder.py b/great_expectations/rule_based_profiler/parameter_builder/regex_pattern_string_parameter_builder.py
--- a/great_expectations/rule_based_profiler/parameter_builder/regex_pattern_string_parameter_builder.py
+++ b/great_expectations/rule_based_profiler/parameter_builder/regex_pattern_string_parameter_builder.py
@@ -50,8 +50,8 @@ def __init__(
         name: str,
         metric_domain_kwargs: Optional[Union[str, dict]] = None,
         metric_value_kwargs: Optional[Union[str, dict]] = None,
-        threshold: Union[float, str] = 1.0,
-        candidate_regexes: Optional[Union[Iterable[str], str]] = None,
+        threshold: Union[str, float] = 1.0,
+        candidate_regexes: Optional[Union[str, Iterable[str]]] = None,
         evaluation_parameter_builder_configs: Optional[List[dict]] = None,
         json_serialize: Union[str, bool] = True,
         batch_list: Optional[List[Batch]] = None,


Test patch:
Repository: great-expectations/great_expectations

Issue:
`_sqlalchemy_map_condition_unexpected_count_value`  fails for SQLA version < 1.4
**Describe the bug**
Certain expectations that generate a subquery fail with `sqlalchemy.exc.ProgrammingError: (psycopg2.errors.SyntaxError) subquery in FROM must have an alias` when run against Postgres on SQLAlchemy version earlier than 1.4.

The problem was partially identified in https://github.com/great-expectations/great_expectations/pull/3651, where a helper function was added to explicitly create a subquery with alias in order to avoid the deprecation warnings when running on SQLA >= 1.4. However, the fix is only applied on versions >= 1.4, so doesn't resolve the problem for the earlier versions where it actually results in the error above rather than a deprecation warning

**To Reproduce**
Steps to reproduce the behavior:
1. Install a version of `SQLAlchemy` < 1.4
2. Create an expectation to eg. `expect_column_values_to_be_unique`
3. Run the expectation against a Postgres DB (not sure if a similar error may occur with other DBs)
4. See error

**Expected behavior**
Expectations using subqueries should run successfully on SQLA versions earlier than 1.4

**Environment (please complete the following information):**
 - Operating System: macOS
 - Great Expectations Version 0.14.13
 - SQLAlchemy version < 1.4



Discussion:

Resolving patch:
diff --git a/great_expectations/util.py b/great_expectations/util.py
--- a/great_expectations/util.py
+++ b/great_expectations/util.py
@@ -1401,11 +1401,16 @@ def get_sqlalchemy_selectable(selectable: Union[Table, Select]) -> Union[Table,
     without explicitly turning the inner select() into a subquery first. This helper method ensures that this
     conversion takes place.
 
+    For versions of SQLAlchemy < 1.4 the implicit conversion to a subquery may not always work, so that
+    also needs to be handled here, using the old equivalent method.
+
     https://docs.sqlalchemy.org/en/14/changelog/migration_14.html#change-4617
     """
-    if version.parse(sa.__version__) >= version.parse("1.4"):
-        if isinstance(selectable, Select):
+    if isinstance(selectable, Select):
+        if version.parse(sa.__version__) >= version.parse("1.4"):
             selectable = selectable.subquery()
+        else:
+            selectable = selectable.alias()
     return selectable
 
 


Test patch: