Package: mk-feature-store
OverviewFeast is an open source feature store for machine learning. Feast is the fastest path to productionizing analytic data for model training and online inference.Please see ourdocumentationfor more information about the project.📐 ArchitectureThe above architecture is the minimal Feast deployment. Want to run the full Feast on Snowflake/GCP/AWS? Clickhere.🐣 Getting Started1. Install Feastpip install feast2. Create a feature repositoryfeast init my_feature_repo
cd my_feature_repo3. Register your feature definitions and set up your feature storefeast apply4. Explore your data in the web UI (experimental)5. Build a training datasetfromfeastimportFeatureStoreimportpandasaspdfromdatetimeimportdatetimeentity_df=pd.DataFrame.from_dict({"driver_id":[1001,1002,1003,1004],"event_timestamp":[datetime(2021,4,12,10,59,42),datetime(2021,4,12,8,12,10),datetime(2021,4,12,16,40,26),datetime(2021,4,12,15,1,12)]})store=FeatureStore(repo_path=".")training_df=store.get_historical_features(entity_df=entity_df,features=['driver_hourly_stats:conv_rate','driver_hourly_stats:acc_rate','driver_hourly_stats:avg_daily_trips'],).to_df()print(training_df.head())# Train model# model = ml.fit(training_df)event_timestamp  driver_id  conv_rate  acc_rate  avg_daily_trips
0 2021-04-12 08:12:10+00:00       1002   0.713465  0.597095              531
1 2021-04-12 10:59:42+00:00       1001   0.072752  0.044344               11
2 2021-04-12 15:01:12+00:00       1004   0.658182  0.079150              220
3 2021-04-12 16:40:26+00:00       1003   0.162092  0.309035              9596. Load feature values into your online storeCURRENT_TIME=$(date -u +"%Y-%m-%dT%H:%M:%S")
feast materialize-incremental $CURRENT_TIMEMaterializing feature view driver_hourly_stats from 2021-04-14 to 2021-04-15 done!7. Read online features at low latencyfrompprintimportpprintfromfeastimportFeatureStorestore=FeatureStore(repo_path=".")feature_vector=store.get_online_features(features=['driver_hourly_stats:conv_rate','driver_hourly_stats:acc_rate','driver_hourly_stats:avg_daily_trips'],entity_rows=[{"driver_id":1001}]).to_dict()pprint(feature_vector)# Make prediction# model.predict(feature_vector){"driver_id":[1001],"driver_hourly_stats__conv_rate":[0.49274],"driver_hourly_stats__acc_rate":[0.92743],"driver_hourly_stats__avg_daily_trips":[72]}📦 Functionality and RoadmapThe list below contains the functionality that contributors are planning to develop for FeastItems below that are in development (or planned for development) will be indicated in parentheses.We welcome contribution to all items in the roadmap!Want to influence our roadmap and prioritization? Submit your feedback tothis form.Want to speak to a Feast contributor? We are more than happy to jump on a call. Please schedule a time usingCalendly.Data SourcesSnowflake sourceRedshift sourceBigQuery sourceParquet file sourceSynapse source (community plugin)Hive (community plugin)Postgres (community plugin)Spark (community plugin)Kafka source (withpush support into the online store)HTTP sourceOffline StoresSnowflakeRedshiftBigQuerySynapse (community plugin)Hive (community plugin)Postgres (community plugin)Trino (community plugin)Spark (community plugin)In-memory / PandasCustom offline store supportOnline StoresDynamoDBRedisDatastoreSQLiteAzure Cache for Redis (community plugin)Postgres (community plugin)Custom online store supportBigtable (in progress)CassandraStreamingCustom streaming ingestion job supportPush based streaming data ingestionStreaming ingestion on AWSStreaming ingestion on GCPFeature EngineeringOn-demand Transformations (Alpha release. SeeRFC)Batch transformation (SQL. In progress. SeeRFC)Streaming transformationDeploymentsAWS Lambda (Alpha release. SeeRFC)Kubernetes (Seeguide)Cloud RunKNativeFeature ServingPython ClientREST Feature Server (Python) (Alpha release. SeeRFC)gRPC Feature Server (Java) (See#1497)Push APIJava ClientGo ClientDelete APIFeature Logging (for training)Data Quality Management (SeeRFC)Data profiling and validation (Great Expectations)Training-serving skew detection (in progress)Metric productionDrift detectionFeature Discovery and GovernancePython SDK for browsing feature registryCLI for browsing feature registryModel-centric feature tracking (feature services)Amundsen integration (seeFeast extractor)Feast Web UI (in progress)REST API for browsing feature registryFeature versioning🎓 Important ResourcesPlease refer to the official documentation atDocumentationQuickstartTutorialsRunning Feast with Snowflake/GCP/AWSChange LogSlack (#Feast)👋 ContributingFeast is a community project and is still under active development. Please have a look at our contributing and development guides if you want to contribute to the project:Contribution Process for FeastDevelopment Guide for FeastDevelopment Guide for the Main Feast Repository✨ ContributorsThanks goes to these incredible people: