Package: wattson-abstract-rtu
Wattson Abstract RTUThis module provides a standardized interface for software-based RTU
implementations for interacting with different type of Power Grid Backends.
These backends either can be physical components, local software implementations,
or distributed Power Simulations for representing complex grids.This module does not provide any simulative functionalities but serves the
sole purpose of enabling exchangable RTU implementations and Power Grid Backends.RTU Abstraction LayerThis document specifies the functionality provided by the RTU abstraction layer, implemented as RTUBackend interface.
The main communication with datapoints attached to the RTU should be handled through this interface.
Since both the simulation and RTU model will change how a RTU can and should be interacted with, some functions need to be implemented
by the inheriting class.
In the current design, it is expected that a class inheriting from the RTU backend implementsallfunctionality specific
to a combination of simulation and RTU model.
We expect a specific RTU model to define a new RTU backend class that inherits from our interface and sets it as attribute of the main RTU object.Datapoints:A RTU is initialised with information about all datapoints attached to it.
Only changing the default cause of transmission for a datapoint is possible at a later point.
The format of datapoints handed over may vary, we require:The datapoint is castable to a tupleThe first four entires accessible as [0:5] represent [coa, ioa, type-ID, cot, related-ioa] (Information Object Adddress, Common Address,
default ASDU-Type-ID, default Cause of Transmission, ioa of related datapoint then identified as (coa, related-ioa))coa, ioa, related-ioa types:intorstr; their values are stored, retrieved, and compared against in a type-sensitive fashion!type-ID, cot type:int; cot in[1,47]; type-ID roughly in[1,127](some values of the interval are undefined)the default cot will be chosen when a query is send and no new cot is givenIfcot == 1, the interface assumes periodic updates are send to the RTU as mentioned in IEC-104the [coa, ioa, type-ID, cot, related-ioa] combination is referred toprimitive datapoint,
its entire model-specific combination ascomplex datapointany usage of the cot in the following is assumed to conform with IEC-104If additional information, like Panda-DataFrame references, are necessary in your model, hand them
over in all entries after the fifth.Model-Specific Adoptions:Retrieving values from/ writing values to datapoints and any further communication is model-dependent
and requires two functions to be implemented.
The typesCOAandIOAare set toUnion[int, str]._build_IO_query(self, coa: COA, ioa: IOA, cot: int = 0, value=None)construct an IO query in your model-specific format.behaviour for coa-ioa combinations not referring to a datapoint attached the RTU is undefinedifcot==0, chooses the cot the backend was initialised withifvalue is None, builds a get-query, otherwise a set-queryResulting limitation: cannot set an IO toNone_send_query(self, query: Any)sends the IO query based on your RTU-grid-simulation-modelreturnNoneif some error occured.non-Nonereturn may either signal correctly sent set-query or return value from get-query.The respective return value is forwarded when retrieving/ setting IOs.Neither of these functions is expected to be called directly by an operator.Pre-Defined Functionality:FunctionsThe RTU backend pre-defines these functions:__init__(self, coa: COA, datapoints: Iterable[Tuple[COA, IOA, int, int, ...]], autostart=False, logger=None, includes_relationship=False)sets up datapoint storagethe datapoints need to be deterministcally castable to tuples in the datapoints sectioncoordination with other devices etc. can be started withautostart=Truethat calls function 2inserts an empty-relationship after index [3] ifincludes_relationship=Falseifincludes_relationship=True, expects the value at [4] to be of type IOA and the
same primitive datatype used for the ioas at the 2nd tuple-indexchecks all relationships on correctness (see function 3)raisesRuntimeErrorif an invalid relationship is stored (see function 3)ends by executing function 2 ifaustostart=Truewait_until_ready(self, timeout: Union[float, None]=None) -> Noneterminates as soon as all model-dependent coordinations & start functions are completedraises aTimeoutErrorif your model-dependent setups have not finished in
the given timeouttimeoutin[s]for numbers, never forces stop iftimeout is Nonemay requires overwriting to conform with your RTU + simulator modelsanitise_check_relationships(self) -> bool:checks if all datapoints storing a relationship link to an attached datapoint
with the same coa and stored relation-ship.raises aRuntimeErrorif an invalid relationship is found_invalid_type_id(self, coa: COA, ioa: IOA, type_id: int) -> Union[bool, None, int]checks if the type_id-argument and the datapoint's default type_id are command-query type_ids (in [45,69])if True, returnsdefaulttype_id == type_idreasoning: restrict allowed command-queries to those specified by the datapointreturnsNonefor unattached datapointreturns0if either the handed over type_id or the type_id stored for the datapoint
are not command-query-type_idsget_IO(self, coa: COA, ioa: IOA, cot: int=0, type_id: int=0) -> Anyretrieve the IO based on the coa-ioa combinationreturnsNoneif the RTU has no such datapoint with a resp. IO attached to it or
a type_id != 0 is given and which is invalid for this datapointifcot==0, chooses default cot storedhas_IO(self, coa: COA, ioa: IOA) -> boolcheck if an IO with the given coa-ioa combination is attached to the RTUset_IO(self, coa: COA, ioa: IOA, value, cot: int=0, type_id: int=0) -> Union[bool, None]sets an IO on a datapointifcot==0build query with default cotreturns the model-dependent query response for attached datapoints andNoneif the RTU does not know such an IO or type_id is given which is not valid for this datapointresulting limitation: cannot differentiate between the return of aNonequery response and a
non-attached IOget_related_IO(self, coa: COA, ioa: IOA, cot: int = 0) -> Anyperformsget_IObut for the datapoint related to the (coa, ioa)-identified datapointalso returnsNoneif no relationship is storedset_related_IO(self, coa: COA, ioa: IOA, value, cot: int = 0) -> Union[bool, None]performsset_IObut for the datapoint related to the (coa, ioa)-identified datapointalso returnsNoneif no relationship is storedget_periodic_ids(self) -> Set[Tuple[COA, IOA]]returns all coa-ioa combinations the RTU expects periodic messages from (initialised withcot==1)get_periodic_data_points(self) -> Set[Tuple[COA, IOA, int, int]]returns all primitive datapoints the RTU expects periodic messages fromget_periodic_ioas(self, coa: COA = -1) -> Set[IOA]- returns all IOAs of periodicly updating datapoints with the given coa
- ifcoa==-1, checks for all datapoints with the backend's coaget_data_point(self, coa: COA, ioa: IOA, with_value=False) -> Union[None, Tuple, Tuple[Tuple, Any]]retrieves the primitive datapoint corresponding to the coa-ioa combinationreturnsNoneif no such datapoint is knownadds the resp. IO ifwith_value=Trueget_related_data_point(self, coa: COA, ioa: IOA, with_value=False) -> Union[None, Tuple, Tuple[Tuple, Any]]performsget_data_pointbut on the related datapoint insteadalso returnsNoneif no relationship is storedchange_cause_of_transmission(self, coa: COA, ioa: IOA, new_cot: int) -> Nonechange the default-cot for a datapoint attachedifnew_cot== 1 or default_cot==1expects that a model-dependent command to the datapoint is send
to update the periodic-update statusdoes not change ifcot not in [1,47]get_ioas(self, coa: COA = -1) -> Set[IOA]retrieve all ioas from datapoints with the given coa attached to the RTUifcoa==-1, checks for the backend's coaget_data_points(self) -> Set(Tuple[COA, IOA, int, int])retrieves all primitive data points attached to the RTU_get_complex_data_point(self, coa: COA, ioa: IOA, with_value=False) -> Union[None, Tuple, Tuple[Tuple, Any]]retrieves the respective complex data pointalso returns the IO ifwith_value=True_get_complex_related_data_point(self, coa: COA, ioa: IOA, with_value=False) -> Union[None, Tuple, Tuple[Tuple, Any]]performs_get_complex_data_pointon the related datapointalso returnsNoneif no relationship is storedThe initialisation function should be overwritten if more objects/ data for the communication between RTU and simulation model needs to be provided
to send or build queries.
For instance, Julian Filter's pandapower model also simulates the communication with ZMQ.
Corresponding wrappers and other metadata necessary to partially access, initialise and control the
client would thus be necessary to be added by the inheriting backend.
This may require overwritingwait_until_ready(...)as well.AttributesThe following attributes are defined upon initialisaton of the interfacedata_store:Dict[COA, Dict[IOA, Tuple[COA, IOA, int, int, IOA, ...]]]stores all complex datapoints for building the queries etc.this format to ensure easy traversal and compability with all known modelsdatapoints:Set[Tuple[COA, IOA, int, int, IOA]]stores all primitive datapointscoa:COAcoa of the RTUstarted:threading.Eventmarks start-up & initialisation of all model-dependent clients etc.logger:Union[sink_logger, logging.Logger]logger to use if wantedif no logger is handed over, a sink_logger that discards all messages is set-up,
this cleans up logging behaviour as no checkup if a logger exists is necessarythesink_loggeronly provides the main logging functions:.critical(msg), .error(msg), .warning(msg), .info(msg), .debug(msg)__inserted_relationship:Boolmarks whether insertion of empty relationships were necessarymostly aimed at debuggingProperties (read-only)logging: If a non-sink logger is attachedLoggingLogging is assumed to be done through theloggingpackage.
The interface assumes the logger is already set up in regards to file handlers, etc. .
The logger is assumed to be separate for each RTU or as it does not always repeat the backend's
COA.
A sink is connected that accepts all main logging messages stated above if no real logger is
handed over.
The interface logs the following data:CRITICALthe backend could not start clients etc. inwait_until_readyin the time-threshold givenWARNINGcallingset_IOorget_IOon an unattached datapointexecutingget_related_IOorset_related_IOon a datapoint without a relationshipsending the query forset_IO,get_IO, or or its related-datapoint versions on an attached datapoint failed for some other
reasonchanging the cot for an unattached datapoint or trying to set it to an invalid valuetrying to set/get IOs with an invalid type_id for the given datapoint or if the corresponding
IO does not lie in the IEC104-range for this type_idINFOwhenever theself.startedstatus changestime [s] it took to setup a client (if applicable)whenever the periodicity of a datapoint was changedDEBUGstr(query)send and its resultRecommended Implementations:A model often needs to be constructable from exported data, e.g., pickle, yaml, or regular xml files.
This is not strictly necessary, but encouraged and should be implemented through a staticfrom_data(...)function.stop(self) -> boolstops all model-specific communicators (zmq-clients, etc.)returns success of this operation (should only fail in special cases or raise
exceptions instead)clears the started Event__del__(self)stops all model-specific communicators (zmq-clients, etc.) safelyTestingWe encourage tests written withpytest.
For several only indirectly or model-independent functions already fully defined by the backend,
test-functions can be imported fromRTU/tests/RTUs.py, denoted by astandardprefix.
They only require the RTU or Backend + the datapoints initialised with as input.
Example tests for a constant and pandapower-backend are defined there as well.A note about pandapowerTo our knowledge, many implementations use pandapower, panda DataFrames, and PPQueries in one way or another.
There exist some test examples and corresponding Backend that use the following DataFrame input format:[coa, ioa, type-ID, cot, pp_table, pp_column, pp_index, others]this does not include relationship-ioas and requires initialising
the backend withincludes_relationship=False!and the following PPQuery format as named tuple:[table, column, index, value]If you make your data frame fit to this format, you should be able to copy the_build_IO_queryimplementation from thePandapowerBackendexample.Model-Compability:The interface requires a python version>=3.5.
Some examples will require a higher python version due to other dependencies. If this is the case, it is clearly marked.Examples are listed in theRTU/Backends.pyfile.
A mini-RTU-simulation with a Backend is given inRTU/examples/RTUs.py.Please let us know if any of the following specifications donothold.the coa, ioa, relationship-ioas are not stored as integers/ strings; the cot/type-ID is not stored as integer in your modelyou believe other functionality is shared between thevastmajority of RTU or simulation models and if implemented in this interface
does not get in the way of those models the functionality is not shared withany other compability problem if this interface would be implemented for your RTU and simulation modelOr if the following any of these threedohold:your model requires the type-ID during IO-query construction/ sendingyour model allows IOs with avalidNonevaluesince the send-query functions etc. returnNonefor various errorsa datapoint identified through (coa, ioa) needs to store a relationship to a datapoint with a different
coa