Package: mk-feature-store-postgres
Feast PostgreSQL SupportThis repo adds PostgreSQL offline and online stores toFeastGet startedInstall feast:pipinstallfeastInstall feast-postgres:pipinstallfeast-postgresCreate a feature repository:feastinitfeature_repocdfeature_repoOnline store:To configure the online store editfeature_store.yamlproject:feature_reporegistry:data/registry.dbprovider:localonline_store:type:feast_postgres.PostgreSQLOnlineStore# MUST be this valuehost:localhostport:5432# Optional, default is 5432database:postgresdb_schema:feature_store# Optional, default is Noneuser:usernamepassword:passwordoffline_store:...When runningfeast apply, ifdb_schemais set then that value will be used when creating the schema, else the name of the schema will be the value inuser. If the schema already exists then no schema is created, but the user must have privileges to create tables and indexes as well as dropping tables and indexes.Offline store:To configure the offline store editfeature_store.yamlproject:feature_reporegistry:data/registry.dbprovider:localonline_store:...offline_store:type:feast_postgres.PostgreSQLOfflineStore# MUST be this valuehost:localhostport:5432# Optional, default it 5432database:postgresdb_schema:my_schemauser:usernamepassword:passwordThe user will need to have privileges to create and drop tables indb_schemasince temp tables will be created when querying for historical values.Registry store:To configure the registry editfeature_store.yamlregistry:registry_store_type:feast_postgres.PostgreSQLRegistryStorepath:feast_registry# This will become the table name for the registryhost:localhostport:5432# Optional, default is 5432database:postgresdb_schema:my_schemauser:usernamepassword:passwordIf the schema does not exists, the user will need to have privileges to create it. If the schema exists, the user will only need privileges to create the table.ExampleStart by setting the values infeature_store.yaml. Then usecopy_from_parquet_to_postgres.pyto create a table and populate it with data from the parquet file that comes with Feast.Thenexample.pycan be used for the feature_store.# This is an example feature definition filefromgoogle.protobuf.duration_pb2importDurationfromfeastimportEntity,Feature,FeatureView,ValueTypefromfeast_postgresimportPostgreSQLSource# Read data from parquet files. Parquet is convenient for local development mode. For# production, you can use your favorite DWH, such as BigQuery. See Feast documentation# for more info.driver_hourly_stats=PostgreSQLSource(query="SELECT * FROM driver_stats",event_timestamp_column="event_timestamp",created_timestamp_column="created",)# Define an entity for the driver. You can think of entity as a primary key used to# fetch features.driver=Entity(name="driver_id",value_type=ValueType.INT64,description="driver id",)# Our parquet files contain sample data that includes a driver_id column, timestamps and# three feature column. Here we define a Feature View that will allow us to serve this# data to our model online.driver_hourly_stats_view=FeatureView(name="driver_hourly_stats",entities=["driver_id"],ttl=Duration(seconds=86400*1),features=[Feature(name="conv_rate",dtype=ValueType.FLOAT),Feature(name="acc_rate",dtype=ValueType.FLOAT),Feature(name="avg_daily_trips",dtype=ValueType.INT64),],online=True,batch_source=driver_hourly_stats,tags={},)Then run:feastapply
feastmaterialize-incremental$(date-u+"%Y-%m-%dT%H:%M:%S")This will create the feature view table and populate if with data from thedriver_statstable that we created in Postgres.