Package: feast
OverviewFeast (FeatureStore) is an open source feature store for machine learning. Feast is the fastest path to manage existing infrastructure to productionize analytic data for model training and online inference.Feast allows ML platform teams to:Make features consistently available for training and servingby managing anoffline store(to process historical data for scale-out batch scoring or model training), a low-latencyonline store(to power real-time prediction),and a battle-testedfeature server(to serve pre-computed features online).Avoid data leakageby generating point-in-time correct feature sets so data scientists can focus on feature engineering rather than debugging error-prone dataset joining logic. This ensure that future feature values do not leak to models during training.Decouple ML from data infrastructureby providing a single data access layer that abstracts feature storage from feature retrieval, ensuring models remain portable as you move from training models to serving models, from batch models to realtime models, and from one data infra system to another.Please see ourdocumentationfor more information about the project.📐 ArchitectureThe above architecture is the minimal Feast deployment. Want to run the full Feast on Snowflake/GCP/AWS? Clickhere.🐣 Getting Started1. Install Feastpip install feast2. Create a feature repositoryfeast init my_feature_repo
cd my_feature_repo/feature_repo3. Register your feature definitions and set up your feature storefeast apply4. Explore your data in the web UI (experimental)feast ui5. Build a training datasetfromfeastimportFeatureStoreimportpandasaspdfromdatetimeimportdatetimeentity_df=pd.DataFrame.from_dict({"driver_id":[1001,1002,1003,1004],"event_timestamp":[datetime(2021,4,12,10,59,42),datetime(2021,4,12,8,12,10),datetime(2021,4,12,16,40,26),datetime(2021,4,12,15,1,12)]})store=FeatureStore(repo_path=".")training_df=store.get_historical_features(entity_df=entity_df,features=['driver_hourly_stats:conv_rate','driver_hourly_stats:acc_rate','driver_hourly_stats:avg_daily_trips'],).to_df()print(training_df.head())# Train model# model = ml.fit(training_df)event_timestamp  driver_id  conv_rate  acc_rate  avg_daily_trips
0 2021-04-12 08:12:10+00:00       1002   0.713465  0.597095              531
1 2021-04-12 10:59:42+00:00       1001   0.072752  0.044344               11
2 2021-04-12 15:01:12+00:00       1004   0.658182  0.079150              220
3 2021-04-12 16:40:26+00:00       1003   0.162092  0.309035              9596. Load feature values into your online storeCURRENT_TIME=$(date -u +"%Y-%m-%dT%H:%M:%S")
feast materialize-incremental $CURRENT_TIMEMaterializing feature view driver_hourly_stats from 2021-04-14 to 2021-04-15 done!7. Read online features at low latencyfrompprintimportpprintfromfeastimportFeatureStorestore=FeatureStore(repo_path=".")feature_vector=store.get_online_features(features=['driver_hourly_stats:conv_rate','driver_hourly_stats:acc_rate','driver_hourly_stats:avg_daily_trips'],entity_rows=[{"driver_id":1001}]).to_dict()pprint(feature_vector)# Make prediction# model.predict(feature_vector){"driver_id":[1001],"driver_hourly_stats__conv_rate":[0.49274],"driver_hourly_stats__acc_rate":[0.92743],"driver_hourly_stats__avg_daily_trips":[72]}📦 Functionality and RoadmapThe list below contains the functionality that contributors are planning to develop for Feast.We welcome contribution to all items in the roadmap!Data SourcesSnowflake sourceRedshift sourceBigQuery sourceParquet file sourceAzure Synapse + Azure SQL source (contrib plugin)Hive (community plugin)Postgres (contrib plugin)Spark (contrib plugin)Kafka / Kinesis sources (viapush support into the online store)Offline StoresSnowflakeRedshiftBigQueryAzure Synapse + Azure SQL (contrib plugin)Hive (community plugin)Postgres (contrib plugin)Trino (contrib plugin)Spark (contrib plugin)In-memory / PandasCustom offline store supportOnline StoresSnowflakeDynamoDBRedisDatastoreBigtableSQLiteDragonflyAzure Cache for Redis (community plugin)Postgres (contrib plugin)Cassandra / AstraDB (contrib plugin)Custom online store supportFeature EngineeringOn-demand Transformations (Alpha release. SeeRFC)Streaming Transformations (Alpha release. SeeRFC)Batch transformation (In progress. SeeRFC)StreamingCustom streaming ingestion job supportPush based streaming data ingestion to online storePush based streaming data ingestion to offline storeDeploymentsAWS Lambda (Alpha release. SeeRFC)Kubernetes (Seeguide)Feature ServingPython ClientPython feature serverJava feature server (alpha)Go feature server (alpha)Data Quality Management (SeeRFC)Data profiling and validation (Great Expectations)Feature Discovery and GovernancePython SDK for browsing feature registryCLI for browsing feature registryModel-centric feature tracking (feature services)Amundsen integration (seeFeast extractor)DataHub integration (seeDataHub Feast docs)Feast Web UI (Beta release. Seedocs)🎓 Important ResourcesPlease refer to the official documentation atDocumentationQuickstartTutorialsRunning Feast with Snowflake/GCP/AWSChange Log👋 ContributingFeast is a community project and is still under active development. Please have a look at our contributing and development guides if you want to contribute to the project:Contribution Process for FeastDevelopment Guide for FeastDevelopment Guide for the Main Feast Repository✨ ContributorsThanks goes to these incredible people: