internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: feast-dev/feast

Issue:
Switch from `redis-py-cluster` to `redis-py`
As [documented](https://github.com/Grokzen/redis-py-cluster#redis-py-cluster-eol), `redis-py-cluster` has reached EOL. It is now being merged into [`redis-py`](https://github.com/redis/redis-py#cluster-mode). We previously tried to switch from `redis-py-cluster` to `redis-py` but ran into various issues; see #2328. The upstream [bug](https://github.com/redis/redis-py/issues/2003) has since been fixed, so we can now switch from `redis-py-cluster` to `redis-py`. 


Discussion:
Related, we should try to get redis-cluster integration tests running using testcontainers somehow, and part of the normal configurations tested.

Resolving patch:
diff --git a/sdk/python/feast/infra/online_stores/redis.py b/sdk/python/feast/infra/online_stores/redis.py
--- a/sdk/python/feast/infra/online_stores/redis.py
+++ b/sdk/python/feast/infra/online_stores/redis.py
@@ -42,7 +42,7 @@
 
 try:
     from redis import Redis
-    from rediscluster import RedisCluster
+    from redis.cluster import ClusterNode, RedisCluster
 except ImportError as e:
     from feast.errors import FeastExtrasDependencyImportError
 
@@ -164,7 +164,9 @@ def _get_client(self, online_store_config: RedisOnlineStoreConfig):
                 online_store_config.connection_string
             )
             if online_store_config.redis_type == RedisType.redis_cluster:
-                kwargs["startup_nodes"] = startup_nodes
+                kwargs["startup_nodes"] = [
+                    ClusterNode(**node) for node in startup_nodes
+                ]
                 self._client = RedisCluster(**kwargs)
             else:
                 kwargs["host"] = startup_nodes[0]["host"]
diff --git a/sdk/python/setup.py b/sdk/python/setup.py
--- a/sdk/python/setup.py
+++ b/sdk/python/setup.py
@@ -72,7 +72,7 @@
 ]
 
 GCP_REQUIRED = [
-    "google-cloud-bigquery>=2.28.1",
+    "google-cloud-bigquery>=2,<3",
     "google-cloud-bigquery-storage >= 2.0.0",
     "google-cloud-datastore>=2.1.*",
     "google-cloud-storage>=1.34.*,<1.41",
@@ -80,8 +80,7 @@
 ]
 
 REDIS_REQUIRED = [
-    "redis==3.5.3",
-    "redis-py-cluster>=2.1.3",
+    "redis==4.2.2",
     "hiredis>=2.0.0",
 ]
 
@@ -108,7 +107,7 @@
 
 CI_REQUIRED = (
     [
-        "cryptography==3.3.2",
+        "cryptography==3.4.8",
         "flake8",
         "black==19.10b0",
         "isort>=5",


Test patch:
Repository: feast-dev/feast

Issue:
spark source complain about "table or view not found" error.
## Expected Behavior 
spark offline store

## Current Behavior
if set table of SparkSource with pattern "db.table"，feast will complain about "table or view not found" error.

## Steps to reproduce
1. feast init test_repo
2. modify example.py to use SparkSource, and set table of SparkSource with pattern "db.table"
3. configure featue_store.yaml
4. feast apply

### Specifications

- Version: 0.20.1
- Platform: ubuntu 18/04
- Subsystem:

## Possible Solution



Discussion:

Resolving patch:
diff --git a/sdk/python/feast/infra/offline_stores/contrib/spark_offline_store/spark_source.py b/sdk/python/feast/infra/offline_stores/contrib/spark_offline_store/spark_source.py
--- a/sdk/python/feast/infra/offline_stores/contrib/spark_offline_store/spark_source.py
+++ b/sdk/python/feast/infra/offline_stores/contrib/spark_offline_store/spark_source.py
@@ -177,7 +177,8 @@ def get_table_query_string(self) -> str:
         """Returns a string that can directly be used to reference this table in SQL"""
         if self.table:
             # Backticks make sure that spark sql knows this a table reference.
-            return f"`{self.table}`"
+            table = ".".join([f"`{x}`" for x in self.table.split(".")])
+            return table
         if self.query:
             return f"({self.query})"
 


Test patch:
Repository: feast-dev/feast

Issue:
basicConfig is called at the module level
## Expected Behavior 

```
import feast
logging.basicConfig(level=level, format=FORMAT)
logging.error("msg")
```

should print logging message according to `FORMAT`

## Current Behavior

It uses the format defined in `feast` at the module level.

## Steps to reproduce

Same as in "Expected Behavior"

### Specifications

- Version: 0.18.1
- Platform: Linux
- Subsystem: -

## Possible Solution

I see that `basicConfig` is called here: https://github.com/feast-dev/feast/blob/c9eda79c7b1169ef05a481a96f07960c014e88b9/sdk/python/feast/cli.py#L84 so it is possible that simply removing this call here is enough: https://github.com/feast-dev/feast/blob/0ca62970dd6bc33c00bd5d8b828752814d480588/sdk/python/feast/__init__.py#L30

If there are any other entry points that need to set up logging, they should call the function, but the call in `__init__.py` must be removed.


Discussion:
I think you're right @elshize  - good catch. Would you be willing to submit a PR for this? :) 
Actually I'm going to quickly fix this so that we can include the fix in the release we're planning on cutting tomorrow.

Resolving patch:
diff --git a/sdk/python/feast/__init__.py b/sdk/python/feast/__init__.py
--- a/sdk/python/feast/__init__.py
+++ b/sdk/python/feast/__init__.py
@@ -1,5 +1,3 @@
-import logging
-
 from pkg_resources import DistributionNotFound, get_distribution
 
 from feast.infra.offline_stores.bigquery_source import BigQuerySource
@@ -27,12 +25,6 @@
 from .stream_feature_view import StreamFeatureView
 from .value_type import ValueType
 
-logging.basicConfig(
-    format="%(asctime)s %(levelname)s:%(message)s",
-    datefmt="%m/%d/%Y %I:%M:%S %p",
-    level=logging.INFO,
-)
-
 try:
     __version__ = get_distribution(__name__).version
 except DistributionNotFound:


Test patch:
Repository: feast-dev/feast

Issue:
Parquet Schema Inference only supports File, not directory
When using a FileSource that is in Parquet format, if the source happens to be a directory of partitioned Parquet files, the following lines throw an error:

https://github.com/feast-dev/feast/blob/01d3568168bb9febb9fbda4988283b3886c32a31/sdk/python/feast/infra/offline_stores/file_source.py#L182-L184

`OSError: Expected file path, but /home/ubuntu/project/data/driver_stats_partitioned is a directory`

How to replicate:

1. Start with a demo feast project (`feast init`)
2. Create a partitioned Parquet Dataset.  Use the following to create a dataset with only a single timestamp for inference
```
import pyarrow.parquet as pq
df = pq.read_table("./data/driver_stats.parquet")
df = df.drop(["created"])
pq.write_to_dataset(df, "./data/driver_stats_partitioned")
```
3. Update the file source in `example.py` to look like this:
```
driver_hourly_stats = FileSource(
    path="/home/ubuntu/cado-feast/feature_store/exciting_sunbeam/data/driver_stats_partitioned2",
)
```

4. Run `feast apply`
For now, I've been able to fix by updating the above lines to:
```
schema = ParquetDataset(
    path if filesystem is None else filesystem.open_input_file(path)
).schema.to_arrow_schema()
```


Discussion:

Resolving patch:
diff --git a/sdk/python/feast/infra/offline_stores/file_source.py b/sdk/python/feast/infra/offline_stores/file_source.py
--- a/sdk/python/feast/infra/offline_stores/file_source.py
+++ b/sdk/python/feast/infra/offline_stores/file_source.py
@@ -3,7 +3,7 @@
 
 from pyarrow._fs import FileSystem
 from pyarrow._s3fs import S3FileSystem
-from pyarrow.parquet import ParquetFile
+from pyarrow.parquet import ParquetDataset
 
 from feast import type_map
 from feast.data_format import FileFormat, ParquetFormat
@@ -179,9 +179,9 @@ def get_table_column_names_and_types(
         filesystem, path = FileSource.create_filesystem_and_path(
             self.path, self.file_options.s3_endpoint_override
         )
-        schema = ParquetFile(
+        schema = ParquetDataset(
             path if filesystem is None else filesystem.open_input_file(path)
-        ).schema_arrow
+        ).schema.to_arrow_schema()
         return zip(schema.names, map(str, schema.types))
 
     @staticmethod


Test patch:
Repository: feast-dev/feast

Issue:
Incorrect projects-list.json generated by feast ui when using Postgres as a data source.
## Expected Behavior 
Correct generation of the projects-list.json when running feast ui. 
## Current Behavior
The generated projects-list.json does not contain a name in the dataSources field, causing the parser to fail.
## Steps to reproduce
Setup feast with PostgreSQL as a data source.
### Specifications

- Version:
- Platform:
- Subsystem:

## Possible Solution
Adding name=self.name to to_proto() in postgres_source.py. And in general making the postgres_source.py file more similar to e.g., file_source.py.


Discussion:

Resolving patch:
diff --git a/sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres_source.py b/sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres_source.py
--- a/sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres_source.py
+++ b/sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres_source.py
@@ -18,6 +18,9 @@ def __init__(
         created_timestamp_column: Optional[str] = "",
         field_mapping: Optional[Dict[str, str]] = None,
         date_partition_column: Optional[str] = "",
+        description: Optional[str] = "",
+        tags: Optional[Dict[str, str]] = None,
+        owner: Optional[str] = "",
     ):
         self._postgres_options = PostgreSQLOptions(name=name, query=query)
 
@@ -27,6 +30,9 @@ def __init__(
             created_timestamp_column=created_timestamp_column,
             field_mapping=field_mapping,
             date_partition_column=date_partition_column,
+            description=description,
+            tags=tags,
+            owner=owner,
         )
 
     def __hash__(self):
@@ -57,14 +63,21 @@ def from_proto(data_source: DataSourceProto):
             timestamp_field=data_source.timestamp_field,
             created_timestamp_column=data_source.created_timestamp_column,
             date_partition_column=data_source.date_partition_column,
+            description=data_source.description,
+            tags=dict(data_source.tags),
+            owner=data_source.owner,
         )
 
     def to_proto(self) -> DataSourceProto:
         data_source_proto = DataSourceProto(
+            name=self.name,
             type=DataSourceProto.CUSTOM_SOURCE,
             data_source_class_type="feast.infra.offline_stores.contrib.postgres_offline_store.postgres_source.PostgreSQLSource",
             field_mapping=self.field_mapping,
             custom_options=self._postgres_options.to_proto(),
+            description=self.description,
+            tags=self.tags,
+            owner=self.owner,
         )
 
         data_source_proto.timestamp_field = self.timestamp_field


Test patch:
Repository: feast-dev/feast

Issue:
Missing alias in `PostgreSQLOfflineStore.pull_all_from_table_or_query()`
## Expected Behavior
Call `pull_all_from_table_or_query()` method and retrieve wanted data from offline store.

## Current Behavior
SQL error `psycopg2.errors.SyntaxError: subquery in FROM must have an alias`

## Steps to reproduce
Call `pull_all_from_table_or_query()` on `PostgreSQLOfflineStore`, where `PostgreSQLSource` is initialized with `query="SELECT * FROM table_name"`
### Specifications

- Version: 0.22.1
- Platform: x86_64
- Subsystem: Linux 5.18

## Possible Solution

Adding alias in query located in `PostgreSQLOfflineStore.pull_all_from_table_or_query()`, for example adding `AS dummy_alias`. This was tested on my use-case and it works as expected, and if it looks ok, I could open a PR with this change.

```
query = f"""
            SELECT {field_string}
            FROM {from_expression} AS dummy_alias
            WHERE "{timestamp_field}" BETWEEN '{start_date}'::timestamptz AND '{end_date}'::timestamptz
        """
```


Discussion:

Resolving patch:
diff --git a/sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres.py b/sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres.py
--- a/sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres.py
+++ b/sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres.py
@@ -214,7 +214,7 @@ def pull_all_from_table_or_query(
 
         query = f"""
             SELECT {field_string}
-            FROM {from_expression}
+            FROM {from_expression} AS paftoq_alias
             WHERE "{timestamp_field}" BETWEEN '{start_date}'::timestamptz AND '{end_date}'::timestamptz
         """
 


Test patch:
diff --git a/sdk/python/tests/unit/infra/test_key_encoding_utils.py b/sdk/python/tests/unit/infra/test_key_encoding_utils.py
--- a/sdk/python/tests/unit/infra/test_key_encoding_utils.py
+++ b/sdk/python/tests/unit/infra/test_key_encoding_utils.py
@@ -9,14 +9,14 @@ def test_serialize_entity_key():
     # Should be fine
     serialize_entity_key(
         EntityKeyProto(
-            join_keys=["user"], entity_values=[ValueProto(int64_val=int(2 ** 15))]
+            join_keys=["user"], entity_values=[ValueProto(int64_val=int(2**15))]
         ),
         entity_key_serialization_version=2,
     )
     # True int64, but should also be fine.
     serialize_entity_key(
         EntityKeyProto(
-            join_keys=["user"], entity_values=[ValueProto(int64_val=int(2 ** 31))]
+            join_keys=["user"], entity_values=[ValueProto(int64_val=int(2**31))]
         ),
         entity_key_serialization_version=2,
     )
@@ -25,6 +25,6 @@ def test_serialize_entity_key():
     with pytest.raises(BaseException):
         serialize_entity_key(
             EntityKeyProto(
-                join_keys=["user"], entity_values=[ValueProto(int64_val=int(2 ** 31))]
+                join_keys=["user"], entity_values=[ValueProto(int64_val=int(2**31))]
             ),
         )
Repository: feast-dev/feast

Issue:
Wrong condition in to_snowflake method of snowflake provider
## Expected Behavior 
[to_snowflake](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/infra/offline_stores/snowflake.py#L430) should ignore on_demand_feature_views if on_demand_feature_views is an empty list

## Current Behavior

to_snowflake runs on_demand_feature_views code because on_demand_feature_views is set to [] [here](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/infra/offline_stores/snowflake.py#L387)

## Steps to reproduce

### Specifications

- Version: 0.23.0
- Platform: feast SDK
- Subsystem:

## Possible Solution

`if self.on_demand_feature_views is not None: ` => `if self.on_demand_feature_views is not []: `




Discussion:

Resolving patch:
diff --git a/sdk/python/feast/infra/offline_stores/snowflake.py b/sdk/python/feast/infra/offline_stores/snowflake.py
--- a/sdk/python/feast/infra/offline_stores/snowflake.py
+++ b/sdk/python/feast/infra/offline_stores/snowflake.py
@@ -429,7 +429,7 @@ def _to_arrow_internal(self) -> pa.Table:
 
     def to_snowflake(self, table_name: str, temporary=False) -> None:
         """Save dataset as a new Snowflake table"""
-        if self.on_demand_feature_views is not None:
+        if self.on_demand_feature_views:
             transformed_df = self.to_df()
 
             write_pandas(


Test patch:
Repository: feast-dev/feast

Issue:
Missing key error in snowflake_python_type_to_feast_value_type in type_map for numpy datetime64 with timezone
## Expected Behavior 
Feast should be able to handle different source column data types when updating feature views with inferred features. Specifically all possible `datetime64` python data types with specific timezones should be handled.

## Current Behavior
Snowflake python type `datetime64[ns, america/los_angeles]` does not have a corresponding feast ValueType. There's a ValueType for datetime64[ns] but not a numpy datetime64 with a specific timezone

```
File "/opt/homebrew/anaconda3/envs/feast-python37/lib/python3.7/site-packages/feast/type_map.py", line 536, in snowflake_python_type_to_feast_value_type
    return type_map[snowflake_python_type_as_str.lower()]
KeyError: 'datetime64[ns, america/los_angeles]'
```

## Steps to reproduce

### Specifications

- Version: 0.21.3
- Platform: Mac OSX Monterey 12.4
- Subsystem:

## Possible Solution



Discussion:
Hi @tlam-lyra could you add a pr for a fix to support these types?
@tlam-lyra @kevjumba taking a look.

@tlam-lyra can you tell me what the snowflake datatype is? TIMESTAMP_TZ?

Resolving patch:
diff --git a/sdk/python/feast/infra/utils/snowflake_utils.py b/sdk/python/feast/infra/utils/snowflake_utils.py
--- a/sdk/python/feast/infra/utils/snowflake_utils.py
+++ b/sdk/python/feast/infra/utils/snowflake_utils.py
@@ -88,6 +88,8 @@ def get_snowflake_conn(config, autocommit=True) -> SnowflakeConnection:
             **kwargs,
         )
 
+        conn.cursor().execute("ALTER SESSION SET TIMEZONE = 'UTC'", _is_internal=True)
+
         return conn
     except KeyError as e:
         raise SnowflakeIncompleteConfig(e)


Test patch:
Repository: feast-dev/feast

Issue:
Snowflake materialization fails on feature views with dummy entity
## Expected Behavior 

Same as feature views with entities

## Current Behavior

Materialization query errors, query has duplicate column names caused by a dummy entity having multiple dummy join keys

## Steps to reproduce

### Specifications

- Version:
- Platform:
- Subsystem:

## Possible Solution

even if infer features is ran more than once, only update the dummy entity join key once



Discussion:

Resolving patch:
diff --git a/sdk/python/feast/inference.py b/sdk/python/feast/inference.py
--- a/sdk/python/feast/inference.py
+++ b/sdk/python/feast/inference.py
@@ -156,7 +156,11 @@ def update_feature_views_with_inferred_features_and_entities(
                 )
 
         # Infer a dummy entity column for entityless feature views.
-        if len(fv.entities) == 1 and fv.entities[0] == DUMMY_ENTITY_NAME:
+        if (
+            len(fv.entities) == 1
+            and fv.entities[0] == DUMMY_ENTITY_NAME
+            and not fv.entity_columns
+        ):
             fv.entity_columns.append(Field(name=DUMMY_ENTITY_ID, dtype=String))
 
         # Run inference for entity columns if there are fewer entity fields than expected.


Test patch:
Repository: feast-dev/feast

Issue:
Entity data type (int32) do not match PushSource data type (int64) when pushing to online store
## Expected Behavior 

## Current Behavior

## Steps to reproduce

### Specifications

- Version:
- Platform:
- Subsystem:

## Possible Solution



Discussion:

Resolving patch:
diff --git a/sdk/python/feast/feature_store.py b/sdk/python/feast/feature_store.py
--- a/sdk/python/feast/feature_store.py
+++ b/sdk/python/feast/feature_store.py
@@ -1478,13 +1478,8 @@ def write_to_online_store(
             feature_view = self.get_feature_view(
                 feature_view_name, allow_registry_cache=allow_registry_cache
             )
-        entities = []
-        for entity_name in feature_view.entities:
-            entities.append(
-                self.get_entity(entity_name, allow_registry_cache=allow_registry_cache)
-            )
         provider = self._get_provider()
-        provider.ingest_df(feature_view, entities, df)
+        provider.ingest_df(feature_view, df)
 
     @log_exceptions_and_usage
     def write_to_offline_store(
diff --git a/sdk/python/feast/infra/passthrough_provider.py b/sdk/python/feast/infra/passthrough_provider.py
--- a/sdk/python/feast/infra/passthrough_provider.py
+++ b/sdk/python/feast/infra/passthrough_provider.py
@@ -193,7 +193,6 @@ def online_read(
     def ingest_df(
         self,
         feature_view: FeatureView,
-        entities: List[Entity],
         df: pd.DataFrame,
     ):
         set_usage_attribute("provider", self.__class__.__name__)
@@ -204,7 +203,10 @@ def ingest_df(
                 table, feature_view.batch_source.field_mapping
             )
 
-        join_keys = {entity.join_key: entity.value_type for entity in entities}
+        join_keys = {
+            entity.name: entity.dtype.to_value_type()
+            for entity in feature_view.entity_columns
+        }
         rows_to_write = _convert_arrow_to_proto(table, feature_view, join_keys)
 
         self.online_write_batch(
diff --git a/sdk/python/feast/infra/provider.py b/sdk/python/feast/infra/provider.py
--- a/sdk/python/feast/infra/provider.py
+++ b/sdk/python/feast/infra/provider.py
@@ -123,7 +123,6 @@ def online_write_batch(
     def ingest_df(
         self,
         feature_view: FeatureView,
-        entities: List[Entity],
         df: pd.DataFrame,
     ):
         """
@@ -131,7 +130,6 @@ def ingest_df(
 
         Args:
             feature_view: The feature view to which the dataframe corresponds.
-            entities: The entities that are referenced by the dataframe.
             df: The dataframe to be persisted.
         """
         pass


Test patch:
Repository: feast-dev/feast

Issue:
No "HOME" env variable for windows users with Snowflake components
## Expected Behavior 

## Current Behavior

## Steps to reproduce

https://stackoverflow.com/questions/73604089/error-while-trying-to-run-the-feast-featurestore-function?newreg=828ffb42e07e45d5930ecb6222f31514

### Specifications

- Version:
- Platform:
- Subsystem:

## Possible Solution

look for %UserProfile% and HOME



Discussion:

Resolving patch:
diff --git a/sdk/python/feast/infra/materialization/snowflake_engine.py b/sdk/python/feast/infra/materialization/snowflake_engine.py
--- a/sdk/python/feast/infra/materialization/snowflake_engine.py
+++ b/sdk/python/feast/infra/materialization/snowflake_engine.py
@@ -2,7 +2,6 @@
 import shutil
 from dataclasses import dataclass
 from datetime import datetime
-from pathlib import Path
 from typing import Callable, List, Literal, Optional, Sequence, Union
 
 import click
@@ -45,9 +44,7 @@ class SnowflakeMaterializationEngineConfig(FeastConfigBaseModel):
     type: Literal["snowflake.engine"] = "snowflake.engine"
     """ Type selector"""
 
-    config_path: Optional[str] = (
-        Path(os.environ["HOME"]) / ".snowsql/config"
-    ).__str__()
+    config_path: Optional[str] = os.path.expanduser("~/.snowsql/config")
     """ Snowflake config path -- absolute path required (Cant use ~)"""
 
     account: Optional[str] = None
diff --git a/sdk/python/feast/infra/offline_stores/snowflake.py b/sdk/python/feast/infra/offline_stores/snowflake.py
--- a/sdk/python/feast/infra/offline_stores/snowflake.py
+++ b/sdk/python/feast/infra/offline_stores/snowflake.py
@@ -64,9 +64,7 @@ class SnowflakeOfflineStoreConfig(FeastConfigBaseModel):
     type: Literal["snowflake.offline"] = "snowflake.offline"
     """ Offline store type selector"""
 
-    config_path: Optional[str] = (
-        Path(os.environ["HOME"]) / ".snowsql/config"
-    ).__str__()
+    config_path: Optional[str] = os.path.expanduser("~/.snowsql/config")
     """ Snowflake config path -- absolute path required (Cant use ~)"""
 
     account: Optional[str] = None
diff --git a/sdk/python/feast/infra/online_stores/snowflake.py b/sdk/python/feast/infra/online_stores/snowflake.py
--- a/sdk/python/feast/infra/online_stores/snowflake.py
+++ b/sdk/python/feast/infra/online_stores/snowflake.py
@@ -2,7 +2,6 @@
 import os
 from binascii import hexlify
 from datetime import datetime
-from pathlib import Path
 from typing import Any, Callable, Dict, List, Optional, Sequence, Tuple
 
 import pandas as pd
@@ -31,9 +30,7 @@ class SnowflakeOnlineStoreConfig(FeastConfigBaseModel):
     type: Literal["snowflake.online"] = "snowflake.online"
     """ Online store type selector"""
 
-    config_path: Optional[str] = (
-        Path(os.environ["HOME"]) / ".snowsql/config"
-    ).__str__()
+    config_path: Optional[str] = os.path.expanduser("~/.snowsql/config")
     """ Snowflake config path -- absolute path required (Can't use ~)"""
 
     account: Optional[str] = None


Test patch: