Repository: feast-dev/feast

Issue:
Undefined features should be rejected when being fetched via `get_historical_features` / `get_online_features`
## Context

I want to create versioned feature views. Through various versions, features could be added or removed.

## Expected Behavior 

When doing `feast.get_historical_features`, features that are not defined should be rejected.

## Current Behavior

The features get returned even though they have not been defined. 

## Steps to reproduce

1. Initialize a new feast repository 
2. Define the features:

```python
driver_hourly_stats = FileSource(
    path="/home/benjamintan/workspace/feast-workflow-demo/feature_repo/data/driver_stats.parquet",
    timestamp_field="event_timestamp",
    created_timestamp_column="created",
)

driver = Entity(name="driver_id", value_type=ValueType.INT64, description="driver id",)

driver_hourly_stats_view_v1 = FeatureView(
    name="driver_hourly_stats_v1",
    entities=["driver_id"],
    ttl=timedelta(days=1),
    schema=[
        Field(name="avg_daily_trips", dtype=Int64),
    ],
    online=True,
    batch_source=driver_hourly_stats,
    tags={},
)

driver_hourly_stats_view_v2 = FeatureView(
    name="driver_hourly_stats_v2",
    entities=["driver_id"],
    ttl=timedelta(days=1),
    schema=[
        Field(name="conv_rate", dtype=Float32),
        Field(name="acc_rate", dtype=Float32),
        Field(name="avg_daily_trips", dtype=Int64),
    ],
    online=True,
    batch_source=driver_hourly_stats,
    tags={},
)
```
3. `feast apply`
4. Querying Feast:

```python
fs = FeatureStore(repo_path='.')

entity_df = pd.DataFrame(
    {
        "event_timestamp": [
            pd.Timestamp(dt, unit="ms", tz="UTC").round("ms")
            for dt in pd.date_range(
                start=datetime.now() - timedelta(days=3),
                end=datetime.now(),
                periods=3,
            )
        ],
        "driver_id": [1001, 1002, 1003],
    }
)
```
I _do not_ expect the following to work:

```python
# THIS PART SHOULDN'T WORK
features_wrong = ['driver_hourly_stats_v1:conv_rate',  # doesn't exist in V1
                              'driver_hourly_stats_v1:acc_rate',   # doesn't exist in V1
                              'driver_hourly_stats_v1:avg_daily_trips',
                             ]

hist_features_wrong = fs.get_historical_features(
    entity_df=entity_df,
    features=features_wrong,
)
```

But I do get results:

```
                   event_timestamp  driver_id  ...  acc_rate  avg_daily_trips
0 2022-04-17 09:35:35.658000+00:00       1001  ...  0.536431            742.0
1 2022-04-18 21:35:35.658000+00:00       1002  ...  0.496901            678.0
2 2022-04-20 09:35:35.658000+00:00       1003  ...       NaN     
```

I do not expect this to work because `driver_hourly_stats_v1:conv_rate` and `driver_hourly_stats_v1:acc_rate` were not defined in the `driver_hourly_stats_view_v1` FeatureView. 

And just to double check that `driver_hourly_stats_v1` only has `avg_daily_trips` defined:

```
➜ feast feature-views describe driver_hourly_stats_v1
spec:
  name: driver_hourly_stats_v1
  entities:
  - driver_id
  features:
  - name: avg_daily_trips
    valueType: INT64
  ttl: 86400s
```

### Specifications

- Version: 0.20 (tested this on 0.19 and 0.18)
- Platform: Linux
- Subsystem: Ubuntu

## Possible Solution

The list of features being passed in should be checked against the registry. Currently the feature view name and feature name pairs are not validated _together_. Here's an example that modifies `get_historical_features`:

```python
    @log_exceptions_and_usage
    def get_historical_features(
        self,
        entity_df: Union[pd.DataFrame, str],
        features: Union[List[str], FeatureService],
        full_feature_names: bool = False,
    ) -> RetrievalJob:

        # Build a dictionary of feature view names -> feature names (not sure if this function already exists ...)
        fv_name_features = dict([(fv.name, [f.name.split('-')[0] for f in fv.features]) for fv in self.list_feature_views()])
        
        # Check that input features are found in the `fv_name_features` dictionary
        feature_views_not_found = []
        for feature in features:
            k, v = feature.split(":")
            if v not in fv_name_features[k]:
                feature_views_not_found.append(f'{k}:{v}')

        if feature_views_not_found:
            raise FeatureViewNotFoundException(', '.join(feature_views_not_found))
```

This returns:

```python
feast.errors.FeatureViewNotFoundException: Feature view driver_hourly_stats_v1:conv_rate, driver_hourly_stats_v1:acc_rate does not exist
```

This doesn't handle the case when a `FeatureService` is passed in but it shouldn't be too hard.


This should also apply to `get_online_features`. 



Discussion:

Resolving patch:
diff --git a/sdk/python/feast/feature_store.py b/sdk/python/feast/feature_store.py
--- a/sdk/python/feast/feature_store.py
+++ b/sdk/python/feast/feature_store.py
@@ -875,7 +875,7 @@ def get_historical_features(
                 DeprecationWarning,
             )
 
-        # TODO(achal): _group_feature_refs returns the on demand feature views, but it's no passed into the provider.
+        # TODO(achal): _group_feature_refs returns the on demand feature views, but it's not passed into the provider.
         # This is a weird interface quirk - we should revisit the `get_historical_features` to
         # pass in the on demand feature views as well.
         fvs, odfvs, request_fvs, request_fv_refs = _group_feature_refs(
@@ -2125,8 +2125,12 @@ def _group_feature_refs(
     for ref in features:
         view_name, feat_name = ref.split(":")
         if view_name in view_index:
+            view_index[view_name].projection.get_feature(feat_name)  # For validation
             views_features[view_name].add(feat_name)
         elif view_name in on_demand_view_index:
+            on_demand_view_index[view_name].projection.get_feature(
+                feat_name
+            )  # For validation
             on_demand_view_features[view_name].add(feat_name)
             # Let's also add in any FV Feature dependencies here.
             for input_fv_projection in on_demand_view_index[
@@ -2135,6 +2139,9 @@ def _group_feature_refs(
                 for input_feat in input_fv_projection.features:
                     views_features[input_fv_projection.name].add(input_feat.name)
         elif view_name in request_view_index:
+            request_view_index[view_name].projection.get_feature(
+                feat_name
+            )  # For validation
             request_views_features[view_name].add(feat_name)
             request_view_refs.add(ref)
         else:
diff --git a/sdk/python/feast/feature_view_projection.py b/sdk/python/feast/feature_view_projection.py
--- a/sdk/python/feast/feature_view_projection.py
+++ b/sdk/python/feast/feature_view_projection.py
@@ -64,3 +64,11 @@ def from_definition(base_feature_view: "BaseFeatureView"):
             name_alias=None,
             features=base_feature_view.features,
         )
+
+    def get_feature(self, feature_name: str) -> Field:
+        try:
+            return next(field for field in self.features if field.name == feature_name)
+        except StopIteration:
+            raise KeyError(
+                f"Feature {feature_name} not found in projection {self.name_to_use()}"
+            )


Test patch:
diff --git a/sdk/python/tests/integration/offline_store/test_universal_historical_retrieval.py b/sdk/python/tests/integration/offline_store/test_universal_historical_retrieval.py
--- a/sdk/python/tests/integration/offline_store/test_universal_historical_retrieval.py
+++ b/sdk/python/tests/integration/offline_store/test_universal_historical_retrieval.py
@@ -21,7 +21,7 @@
 from feast.infra.offline_stores.offline_utils import (
     DEFAULT_ENTITY_DF_EVENT_TIMESTAMP_COL,
 )
-from feast.types import Int32
+from feast.types import Float32, Int32
 from feast.value_type import ValueType
 from tests.integration.feature_repos.repo_configuration import (
     construct_universal_feature_views,
@@ -410,6 +410,46 @@ def test_historical_features(environment, universal_data_sources, full_feature_n
     )
 
 
+@pytest.mark.integration
+@pytest.mark.universal
+@pytest.mark.parametrize("full_feature_names", [True, False], ids=lambda v: str(v))
+def test_historical_features_with_shared_batch_source(
+    environment, universal_data_sources, full_feature_names
+):
+    # Addresses https://github.com/feast-dev/feast/issues/2576
+
+    store = environment.feature_store
+
+    entities, datasets, data_sources = universal_data_sources
+    driver_stats_v1 = FeatureView(
+        name="driver_stats_v1",
+        entities=["driver"],
+        schema=[Field(name="avg_daily_trips", dtype=Int32)],
+        source=data_sources.driver,
+    )
+    driver_stats_v2 = FeatureView(
+        name="driver_stats_v2",
+        entities=["driver"],
+        schema=[
+            Field(name="avg_daily_trips", dtype=Int32),
+            Field(name="conv_rate", dtype=Float32),
+        ],
+        source=data_sources.driver,
+    )
+
+    store.apply([driver(), driver_stats_v1, driver_stats_v2])
+
+    with pytest.raises(KeyError):
+        store.get_historical_features(
+            entity_df=datasets.entity_df,
+            features=[
+                # `driver_stats_v1` does not have `conv_rate`
+                "driver_stats_v1:conv_rate",
+            ],
+            full_feature_names=full_feature_names,
+        ).to_df()
+
+
 @pytest.mark.integration
 @pytest.mark.universal_offline_stores
 def test_historical_features_with_missing_request_data(
diff --git a/sdk/python/tests/integration/online_store/test_universal_online.py b/sdk/python/tests/integration/online_store/test_universal_online.py
--- a/sdk/python/tests/integration/online_store/test_universal_online.py
+++ b/sdk/python/tests/integration/online_store/test_universal_online.py
@@ -19,7 +19,7 @@
     RequestDataNotFoundInEntityRowsException,
 )
 from feast.online_response import TIMESTAMP_POSTFIX
-from feast.types import String
+from feast.types import Float32, Int32, String
 from feast.wait import wait_retry_backoff
 from tests.integration.feature_repos.repo_configuration import (
     Environment,
@@ -324,6 +324,60 @@ def get_online_features_dict(
     return dict1
 
 
+@pytest.mark.integration
+@pytest.mark.universal
+def test_online_retrieval_with_shared_batch_source(environment, universal_data_sources):
+    # Addresses https://github.com/feast-dev/feast/issues/2576
+
+    fs = environment.feature_store
+
+    entities, datasets, data_sources = universal_data_sources
+    driver_stats_v1 = FeatureView(
+        name="driver_stats_v1",
+        entities=["driver"],
+        schema=[Field(name="avg_daily_trips", dtype=Int32)],
+        source=data_sources.driver,
+    )
+    driver_stats_v2 = FeatureView(
+        name="driver_stats_v2",
+        entities=["driver"],
+        schema=[
+            Field(name="avg_daily_trips", dtype=Int32),
+            Field(name="conv_rate", dtype=Float32),
+        ],
+        source=data_sources.driver,
+    )
+
+    fs.apply([driver(), driver_stats_v1, driver_stats_v2])
+
+    data = pd.DataFrame(
+        {
+            "driver_id": [1, 2],
+            "avg_daily_trips": [4, 5],
+            "conv_rate": [0.5, 0.3],
+            "event_timestamp": [
+                pd.to_datetime(1646263500, utc=True, unit="s"),
+                pd.to_datetime(1646263600, utc=True, unit="s"),
+            ],
+            "created": [
+                pd.to_datetime(1646263500, unit="s"),
+                pd.to_datetime(1646263600, unit="s"),
+            ],
+        }
+    )
+    fs.write_to_online_store("driver_stats_v1", data.drop("conv_rate", axis=1))
+    fs.write_to_online_store("driver_stats_v2", data)
+
+    with pytest.raises(KeyError):
+        fs.get_online_features(
+            features=[
+                # `driver_stats_v1` does not have `conv_rate`
+                "driver_stats_v1:conv_rate",
+            ],
+            entity_rows=[{"driver_id": 1}, {"driver_id": 2}],
+        )
+
+
 @pytest.mark.integration
 @pytest.mark.universal_online_stores
 @pytest.mark.parametrize("full_feature_names", [True, False], ids=lambda v: str(v))