Package: edv-dwh-connector
This project helps to connect to our data warehouse.RequirementsPython3.8.xor later.Quick startSynchronize tagsSynchronize tags from PI server to Data warehouse on period[01/01/2022 05:00:00 - 31/01/2022 05:00:00]can be easily achieved by the code below:fromdatetimeimportdatetimefromedv_dwh_connector.pg_dwhimportPgDwhfromedv_dwh_connector.pi_web_api_clientimportPiWebAPIClientfromedv_dwh_connector.pi.rest.rest_sync_pi_tagsimportRestSyncPITagsfromedv_dwh_connector.utils.periodsimportHourIntervalsfromedv_dwh_connector.pi.sync_pi_dataimportSyncPIDataRestToDwhfromedv_dwh_connector.pi.db.pg_pi_tagimportPgPITags# We should firstly declare the Data warehouse pool connection.dwh=PgDwh.from_connection(name="dwh_db_name",host="dwh_server_name_or_ip",user="dwh_user",password="dwh_password",port=5432)# Next, we should declare the PI server REST API client.client=PiWebAPIClient(base_url="base/url/of/pi/server",username="admin_user",password="admin_password",session_timeout=2.5)# Finally, we can synchronize tags measures.# The code below will automatically synchronize tags (create new tags),# split period provided into hour intervals and synchronize PI measures# on these intervals.SyncPIDataRestToDwh(tags=RestSyncPITags(server_id="F1DSmN2338899MpX8PREOtdbEZ56sypOOOKRZLVNSVi1QSS1ISTGt",# Fake server IDclient=client,codes=['AI162003_SCLD','AI162007_SCLD','AI162014_SCLD'],target=PgPITags(dwh)),periods=HourIntervals(datetime(2022,1,1,5,0,0),datetime(2022,1,31,5,0,0)),client=client,dwh=dwh).synchronize()To store data to a CSV file infact_pi_measureformat, you could do this:SyncPIDataRestToCSVDwhLike(tags=RestSyncPITags(server_id="F1DSmN2338899MpX8PREOtdbEZ56sypOOOKRZLVNSVi1QSS1ISTGt",# Fake server IDclient=client,codes=['AI162003_SCLD','AI162007_SCLD','AI162014_SCLD'],target=PgPITags(dwh)),periods=HourIntervals(datetime(2022,1,1,5,0,0),datetime(2022,1,31,5,0,0)),client=client,file="path/of/csv/file/where/to/store").synchronize()after importingSyncPIDataRestToCSVDwhLikefromedv_dwh_connector.pi.sync_pi_data.
It is very useful when you want to recover data of DWH tablefact_pi_measureon a long period.N.B.You could also fetch on day intervals by using classDayIntervalsinstead ofHourIntervals.
But,HourIntervalscould be faster thanDayIntervalsdepending on the size of data to be imported.Synchronize partially a CSV file for a tagTo synchronize a CSV file with the latest data from interpolated data from DWH, we do like this:dwh=...tag="AI56222_SCDL"CsvWithLatestPIMeasuresDf(path="my/path/data.csv",tag=tag,origin=PgMinuteInterpolatedPIMeasuresDf(tag,dwh)).frame(datetime(2022,1,1,5,0,0),datetime(2022,1,31,5,0,0))Ifdata.csvcontains data for period[01/01/2022 - 24/01/2022], this instruction will only add to the CSV file data of the last week.Read tags and measures from DWHRead tagsTo get all tags, just do this:tags=PgCachedPITags(dwh).items()Read measuresTo read measures of a tag on a period, just do this:tag=...# get PI tag heremeasures=PgCachedPIMeasures(tag,dwh).items(datetime(2022,1,1,5,0,0),datetime(2022,1,31,5,0,0))# or use only tag codemeasures=PgCachedPIMeasures("AI162014_SCLD",dwh).items(datetime(2022,1,1,5,0,0),datetime(2022,1,31,5,0,0))# or use a data frame (by tag or tag code)dt=PgPIMeasuresDf(tag,dwh).frame(datetime(2022,1,1,5,0,0),datetime(2022,1,31,5,0,0))Synchronize blend proposalsWe easily synchronize by this code below:SyncBlendProposals(src=ExcelBlendProposals(file="path/of/excel/file/name",start_date=date.fromisoformat("2022-10-18"),end_date=date.fromisoformat("2022-10-20")),target=PgBlendProposals(dwh)).synchronize()Read blend proposals from DWHblends=PgBlendProposals(dwh).items(date(2022,10,18),datetime(2022,10,20))Development environmentIt is recommended to start by creating a virtual environment. You could do it by following commands:python-mvenv.venvsource.venv/bin/activate
pipinstall-rrequirements.txtN.B.We activate an environment on Windows by executing:.venv\Scripts\activate.batHow to contributePlease readcontributing rules.Fork repository, make changes, send us a pull request. We will review
your changes and apply them to themasterbranch shortly, provided
they don't violate our quality standards. To avoid frustration, before
sending us your pull request please run these commands:shpyqulice.sh# Linuxpyqulice.bat# Windowspytesttests/unit/