Office Action 1: Notice of Allowance (2024-09-30)

CLAIMS:
We claim:

1\. A method for detecting vulnerabilities in a cyber-physical system, the method comprising:
defining, by at least one processor, a default state of a cyber-physical system, wherein the default state comprises a plurality of messages, and wherein the default state represents expected functioning of the cyber-physical system;
identifying, by the at least one processor, one or more dependency relationships between one or more of the plurality of messages;
generating, by the at least one processor, an expected response message to an incoming message sent by the cyber-physical system based on the one or more dependency relationships;
fuzzing, by the at least one processor, the expected response message and transmitting the expected response message to the cyber-physical system; and
recording, by the at least one processor, any response from the cyber-physical system to the fuzzed expected response message.

2\. The method of claim 1, wherein the defining further comprises:
obtaining at least one data log from the cyber-physical system, wherein the at least one data log comprises at least some of the plurality of messages.
3\. The method of claim 2, further comprising:
extracting invariants from the at least one data log, wherein the invariants are relationships between entries in the at least one data log that always occur.

4\. The method of claim 2, wherein the defining further comprises:
generating a causal model from the at least one data log, wherein the causal model describes relationships between at least some of the plurality of messages.

5\. The method of claim 4, wherein the generating further comprises:
querying the causal model to determine if the incoming message either is contained within the causal model or has a causal relation with an environmental response in the causal model.

6\. The method of claim 4, further comprising:
utilizing one or more algorithms to infer the causal model, wherein the one or more algorithms comprises the heuristic miner algorithm.

7\. The method of claim 4, wherein the identifying further comprises:
calculating dependency strength for at least one of the one or more dependency relationships.

8\. The method of claim 6, wherein the identifying further comprises:
generating a dependency graph based on the one or more dependency relationships.

9\. The method of claim 7, wherein the generating further comprises:
querying the dependency graph to determine if receipt of the incoming message causes the cyber-physical system to respond in the causal model.

10\. The method of claim 7, further comprising:
analyzing the dependency graph for repeated patterns in the one or more dependency relationships.

11\. The method of claim 1, wherein the generating further comprises:
implementing a lookup table to determine the expected response message for the incoming message.

12\. The method of claim 1, wherein the fuzzing further comprises:
altering contents of the expected response message.

13\. The method of claim 12, wherein the altering further comprises:
randomly manipulating one or more fields of the expected response message.

14\. The method of claim 12, wherein the altering further comprises:
replacing at least one letter in the expected response message with at least one other, different letter.

15\. The method of claim 1, further comprising:
detecting when at least one of the plurality of messages is sent.

16\. The method of claim 1, wherein the fuzzing is performed using artificial intelligence (Al).

17\. A method for fuzzing a system under test (SUT), the method comprising:
receiving, by at least one processor, one or more messages transmitted by a system under test (SUT);
querying, by the at least one processor, a model to determine if the one or more messages is contained within the model;
if the one or more messages is not contained within the model, generating, by the at least one processor, a new state message;
if the one or more messages is contained within the model, querying, by the at least one processor, a dependency graph to determine if receipt of the one or more messages causes a response;
if receipt of the one or more messages causes a response, choosing and sending, by the at least one processor, a valid response to the one or more messages; and
if receipt of the one or more messages does not cause a response, either transmitting an "out of place" message or not transmitting, by the at least one processor, a response.

18\. The method of claim 17, wherein the new state message comprises a first random sequence of letters and/or numbers that is based on a message contained within the model that is similar to the one or more messages, and wherein the "out of place" message comprises a second random sequence of letters and/or numbers, a repetition of the last message transmitted, or a fixed response set by a user.

19\. The method of claim 17, further comprising:
fuzzing the new state message;
fuzzing the valid response; and/or
fuzzing the "out of place" message.

20\. A system for fuzzing a system under test (SUT), the system comprising:
at least one computer comprising at least one processor operatively connected to at least one non-transitory computer readable medium, the at least one non-transitory computer readable medium having computer-executable instructions stored thereon, wherein, when executed by the at least one processor, the computer executable instructions execute operations comprising:
receiving one or more messages transmitted by a system under test (SUT);
querying a model to determine if the one or more messages is contained within the model;
if the one or more messages is not contained within the model, generating a new state message;
if the one or more messages is contained within the model, querying a dependency graph to determine if receipt of the one or more messages causes a response;
if receipt of the one or more messages causes a response, choosing and sending a valid response to the one or more messages, wherein the valid response is contained within the model;
if receipt of the one or more messages does not cause a response, either transmitting an "out of place" message or not transmitting a response, wherein the "out of place" message comprises a random sequence of letters and/or numbers, a repetition of a previous transmitted message, or a fixed response set by a user; and
fuzzing the new state message, fuzzing the valid response, and/or fuzzing the "out of place" message, wherein the valid response is selected from a lookup table that displays one or more output messages that result from a given input message. 25 26 27 28 29

OFFICE ACTION:
Notice of Pre-AIA  or AIA  Status
Claims 1-20 are presented for examination.  The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 9/30/21 has been considered by the examiner.
Allowable Subject Matter
Claims 1-20 are allowed.
The following is an examiner’s statement of reasons for allowance: Independent claim 1 recites inter alia “identifying, by the at least one processor, one or more dependency relationships between one or more of the plurality of messages; and generating, by the at least one processor, an expected response message to an incoming message sent by the cyber-physical system based on the one or more dependency relationships”.  The nearest prior art reference, Ciocarlie (U.S. Patent Publication 2020/0162500) teaches a related invention for testing the security of a cyber-physical system which teaches building a model of the cyber-physical system and uses fuzz testing to generate responses to messages in the system in order to extract information and manipulate the devices in some fashion (paragraphs 0105-0106), but neither Ciocarlie nor any other reference of record discloses determining any sort of dependency relationships among the plurality of messages in the system.  The closest extent to which Ciocarlie approaches this limitation is a dependency relationship graph of the physical devices of the system (Figure 3A, and paragraph 0077), but not the relationship of any messages sent by said physical devices. Dependent claims 2-16 follow from claim 1 and are of consequence allowable.
Independent claims 17 and 20 recite inter alia “a model to determine if the one or more messages is contained within the model; if the one or more messages is not contained within the model” and “if the one or more messages is contained within the model, querying, by the at least one processor, a dependency graph to determine if receipt of the one or more messages causes a response”.  As discussed supra, the Ciocarlie reference is silent regarding any sort of dependency graph; and additionally, the Agrawal reference (U.S. Patent Publication 2021/0081306) which teaches using a graph the encodes inter alia dependency information to generate test cases for the system (e.g. paragraphs 0005-0007, 0034, 0051, 0066, 0068), nevertheless neither Agrawal nor Ciocarlie nor any other reference of record discloses consulting a model of messages of the system under test to determine if a response is required.   Dependent claims 18 & 19 follow from claim 17 are of consequence allowed.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee.  Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure, each of which are in the same basic field of endeavor as the claimed invention:
U.S. Patent 11,397,664 (Lin)
U.S. Patent Publication 2021/0089661 (Rieger)
U.S. Patent Publication 2020/0175171 (Rieger)
PCT Publication WO 2018/084808 (Elovici)



CITED REFERENCE SUMMARIES: Ref US11397664B2: [content unavailable]
Ref US20210089661A1: Claims:
A method for anomaly detection in a cyber-physical system, the method comprising: acquiring a physical feature of the cyber-physical system, the physical feature defining a relationship between a plurality of physical attributes of the cyber-physical system; determining a physical state (PS) error metric, the PS error metric configured to quantify deviation between the relationship defined between the plurality of physical attributes and measurements of one or more of the plurality of physical attributes; and detecting anomalous behavior of the cyber-physical system based, at least in part, on the PS error metric.
acquiring a physical feature of the cyber-physical system, the physical feature defining a relationship between a plurality of physical attributes of the cyber-physical system;
determining a physical state (PS) error metric, the PS error metric configured to quantify deviation between the relationship defined between the plurality of physical attributes and measurements of one or more of the plurality of physical attributes; and
detecting anomalous behavior of the cyber-physical system based, at least in part, on the PS error metric.
The method of claim 1 , wherein the relationship defined by the physical feature corresponds to a physics-based correlational relationship between the plurality of physical attributes.
The method of claim 1 , further comprising receiving measurements of one or more physical attributes of the plurality of physical attributes from a sensor device coupled to a physical process of the cyber-physical system.
The method of claim 3 , further comprising: determining an estimate of a first physical attribute of the plurality of physical attributes based on the relationship between the plurality of physical attributes and measurements of one or more other physical attributes of the plurality of physical attributes; wherein the PS error metric is based, at least in part, on a difference between the estimate of the first physical attribute and a measurement of the first physical attribute.
determining an estimate of a first physical attribute of the plurality of physical attributes based on the relationship between the plurality of physical attributes and measurements of one or more other physical attributes of the plurality of physical attributes;
wherein the PS error metric is based, at least in part, on a difference between the estimate of the first physical attribute and a measurement of the first physical attribute.
The method of claim 4 , further comprising: determining an estimate of a second physical attribute of the plurality of physical attributes based, at least in part, on the relationship between the plurality of physical attributes and the measurement of the first physical attribute; wherein the PS error metric is based, at least in part, on the difference between the estimate of the first physical attribute and the measurement of the first physical attribute and a difference between the estimate of the second physical attribute and a measurement of the second physical attribute.
determining an estimate of a second physical attribute of the plurality of physical attributes based, at least in part, on the relationship between the plurality of physical attributes and the measurement of the first physical attribute;
wherein the PS error metric is based, at least in part, on the difference between the estimate of the first physical attribute and the measurement of the first physical attribute and a difference between the estimate of the second physical attribute and a measurement of the second physical attribute.
The method of claim 1 , further comprising: determining an error threshold for the physical feature, the error threshold based on a deviation between training measurements of the plurality of physical attributes, the training measurements configured to characterize nominal operation of the cyber-physical system; and detecting the anomalous behavior of the cyber-physical system responsive to the PS error metric exceeding the error threshold.
determining an error threshold for the physical feature, the error threshold based on a deviation between training measurements of the plurality of physical attributes, the training measurements configured to characterize nominal operation of the cyber-physical system; and
detecting the anomalous behavior of the cyber-physical system responsive to the PS error metric exceeding the error threshold.
The method of claim 1 , further comprising: deriving a first membership function from one or more fuzzy sets, the one or more fuzzy sets corresponding to nominal operation of the cyber-physical system and comprising training measurements of the plurality of physical attributes, wherein the first membership function is configured to model a PS error distribution, the PS error distribution corresponding to differences between the training measurements of the plurality of physical attributes and the relationship between the plurality of physical attributes defined by the physical feature; acquiring measurements of the plurality of physical attributes; and utilizing the first membership function to determine the PS error metric, the PS error metric configured to quantify a degree to which the acquired measurements of the physical attributes conform to the PS error distribution of the first membership function.
deriving a first membership function from one or more fuzzy sets, the one or more fuzzy sets corresponding to nominal operation of the cyber-physical system and comprising training measurements of the plurality of physical attributes, wherein the first membership function is configured to model a PS error distribution, the PS error distribution corresponding to differences between the training measurements of the plurality of physical attributes and the relationship between the plurality of physical attributes defined by the physical feature;
acquiring measurements of the plurality of physical attributes; and
utilizing the first membership function to determine the PS error metric, the PS error metric configured to quantify a degree to which the acquired measurements of the physical attributes conform to the PS error distribution of the first membership function.
The method of claim 7 , wherein the one or more fuzzy sets further comprise training measurements of a cyber feature, the training measurements of the cyber feature corresponding to nominal operation of an electronic communication network of the cyber-physical system, the method further comprising: deriving a second membership function from the training measurements of the cyber feature; utilizing the second membership function to determine a cyber state (CS) error metric, the CS error metric configured to quantify a degree to which acquired measurements of the cyber feature correspond to the second membership function; and detecting the anomalous behavior of the cyber-physical system based, at least in part, on the PS error metric and the CS error metric.
deriving a second membership function from the training measurements of the cyber feature;
utilizing the second membership function to determine a cyber state (CS) error metric, the CS error metric configured to quantify a degree to which acquired measurements of the cyber feature correspond to the second membership function; and
detecting the anomalous behavior of the cyber-physical system based, at least in part, on the PS error metric and the CS error metric.
An apparatus for monitoring a cyber-physical system, the apparatus comprising: a physical state (PS) monitor configured to acquire measurements of a plurality of physical attributes of the cyber-physical system, the plurality of physical attributes corresponding to a physical feature that defines a correlational relationship between the plurality of physical attributes; a processor; and an anomaly detector configured for operation on the processor, the anomaly detector further configured to: determine a first affinity metric, the first affinity metric configured to quantify a degree to which the measurements of the plurality of physical attributes of the cyber-physical system conform with the correlational relationship between the plurality of physical attributes, and detect anomalous behavior of the cyber-physical system based, at least in part, on a health metric determined for the cyber-physical system, the health metric based, at least in part, on the first affinity metric and a second affinity metric, the second affinity metric configured to quantify a degree to which a cyber feature conforms with nominal behavior of an electronic communication network of the cyber-physical system.
a physical state (PS) monitor configured to acquire measurements of a plurality of physical attributes of the cyber-physical system, the plurality of physical attributes corresponding to a physical feature that defines a correlational relationship between the plurality of physical attributes;
a processor; and
an anomaly detector configured for operation on the processor, the anomaly detector further configured to: determine a first affinity metric, the first affinity metric configured to quantify a degree to which the measurements of the plurality of physical attributes of the cyber-physical system conform with the correlational relationship between the plurality of physical attributes, and detect anomalous behavior of the cyber-physical system based, at least in part, on a health metric determined for the cyber-physical system, the health metric based, at least in part, on the first affinity metric and a second affinity metric, the second affinity metric configured to quantify a degree to which a cyber feature conforms with nominal behavior of an electronic communication network of the cyber-physical system.
determine a first affinity metric, the first affinity metric configured to quantify a degree to which the measurements of the plurality of physical attributes of the cyber-physical system conform with the correlational relationship between the plurality of physical attributes, and
detect anomalous behavior of the cyber-physical system based, at least in part, on a health metric determined for the cyber-physical system, the health metric based, at least in part, on the first affinity metric and a second affinity metric, the second affinity metric configured to quantify a degree to which a cyber feature conforms with nominal behavior of an electronic communication network of the cyber-physical system.
The apparatus of claim 9 , wherein the correlational relationship between the plurality of physical attributes corresponds to a physics-based relationship between the plurality of physical attributes.
The apparatus of claim 9 , wherein the PS monitor is communicatively coupled to a sensor device through the electronic communication network of the cyber-physical system.
The apparatus of claim 9 , wherein: the anomaly detector is further configured to determine an estimate of a first physical attribute of the plurality of physical attributes based on measurements of one or more other physical attributes of the plurality of physical attributes and the correlational relationship between the plurality of physical attributes defined by the physical feature; and the first affinity metric is based, at least in part, on an error between the estimate of the first physical attribute and a measurement of the first physical attribute.
the anomaly detector is further configured to determine an estimate of a first physical attribute of the plurality of physical attributes based on measurements of one or more other physical attributes of the plurality of physical attributes and the correlational relationship between the plurality of physical attributes defined by the physical feature; and
the first affinity metric is based, at least in part, on an error between the estimate of the first physical attribute and a measurement of the first physical attribute.
The apparatus of claim 12 , wherein: the anomaly detector is further configured to determine an estimate of a second physical attribute of the plurality of physical attributes based, at least in part, on the measurement of the first physical attribute and the correlational relationship between the plurality of physical attributes defined by the physical feature; and the first affinity metric is based, at least in part, on the error between the estimate of the first physical attribute and the measurement of the first physical attribute and an error between the estimate of the second physical attribute and the measurement of the second physical attribute.
the anomaly detector is further configured to determine an estimate of a second physical attribute of the plurality of physical attributes based, at least in part, on the measurement of the first physical attribute and the correlational relationship between the plurality of physical attributes defined by the physical feature; and
the first affinity metric is based, at least in part, on the error between the estimate of the first physical attribute and the measurement of the first physical attribute and an error between the estimate of the second physical attribute and the measurement of the second physical attribute.
The apparatus of claim 9 , wherein the anomaly detector is further configured to: derive a first membership function from one or more fuzzy sets, the one or more fuzzy sets comprising training measurements of the plurality of physical attributes, the training measurements configured to characterize nominal operation of the cyber-physical system, wherein the first membership function is configured to model an error distribution between respective training measurements of the plurality of physical attributes and the correlational relationship between the plurality of physical attributes defined by the physical feature; and utilize the first membership function to determine the first affinity metric.
derive a first membership function from one or more fuzzy sets, the one or more fuzzy sets comprising training measurements of the plurality of physical attributes, the training measurements configured to characterize nominal operation of the cyber-physical system, wherein the first membership function is configured to model an error distribution between respective training measurements of the plurality of physical attributes and the correlational relationship between the plurality of physical attributes defined by the physical feature; and
utilize the first membership function to determine the first affinity metric.
The apparatus of claim 14 , further comprising: a cyber state (CS) monitor configured to acquire measurements of the cyber feature; wherein the anomaly detector is further configured to: derive a second membership function from the one or more fuzzy sets, the second membership function configured to model a distribution of the cyber feature under nominal operation of the cyber-physical system; and utilize the second membership function to determine the second affinity metric.
a cyber state (CS) monitor configured to acquire measurements of the cyber feature;
wherein the anomaly detector is further configured to: derive a second membership function from the one or more fuzzy sets, the second membership function configured to model a distribution of the cyber feature under nominal operation of the cyber-physical system; and utilize the second membership function to determine the second affinity metric.
derive a second membership function from the one or more fuzzy sets, the second membership function configured to model a distribution of the cyber feature under nominal operation of the cyber-physical system; and
utilize the second membership function to determine the second affinity metric.
A non-transitory computer-readable storage medium comprising instructions configured to cause a processor to implement operations for anomaly detection in a cyber-physical system, the operations comprising: acquiring a physical feature of the cyber-physical system, the physical feature defining a correlation between a plurality of physical attributes of the cyber-physical system; determining an anomaly detection metric for the cyber-physical system, the anomaly metric based, at least in part, on a first metric configured to quantify a degree to which measurements of the plurality of physical attributes conform to the correlation between the plurality of physical attributes defined by the physical feature; and detecting an anomaly pertaining to the cyber-physical system based, at least in part, on the anomaly detection metric.
acquiring a physical feature of the cyber-physical system, the physical feature defining a correlation between a plurality of physical attributes of the cyber-physical system;
determining an anomaly detection metric for the cyber-physical system, the anomaly metric based, at least in part, on a first metric configured to quantify a degree to which measurements of the plurality of physical attributes conform to the correlation between the plurality of physical attributes defined by the physical feature; and
detecting an anomaly pertaining to the cyber-physical system based, at least in part, on the anomaly detection metric.
The non-transitory computer-readable storage medium of claim 16 , the operations further comprising receiving measurements of one or more of the plurality of physical attributes of the cyber-physical system from one or more sensor devices coupled to a physical process of the cyber-physical system.
The non-transitory computer-readable storage medium of claim 16 , further comprising: determining an estimate of a first physical attribute of the plurality of physical attributes based on a measurement of one or more other physical attributes of the plurality of physical attributes and the correlation between the plurality of physical attributes defined by the physical feature; and determining the first metric based, at least in part, on a difference between the estimate of the first physical attribute and a measurement of the first physical attribute.
determining an estimate of a first physical attribute of the plurality of physical attributes based on a measurement of one or more other physical attributes of the plurality of physical attributes and the correlation between the plurality of physical attributes defined by the physical feature; and
determining the first metric based, at least in part, on a difference between the estimate of the first physical attribute and a measurement of the first physical attribute.
The non-transitory computer-readable storage medium of claim 18 , further comprising: determining an estimate of a second physical attribute of the plurality of physical attributes based, at least in part, on the measurement of the first physical attribute and the correlation between the plurality of physical attributes defined by the physical feature; wherein the first metric is based, at least in part, on the difference between the estimate of the first physical attribute and the measurement of the first physical attribute and a difference between the estimate of the second physical attribute and a measurement of the second physical attribute.
determining an estimate of a second physical attribute of the plurality of physical attributes based, at least in part, on the measurement of the first physical attribute and the correlation between the plurality of physical attributes defined by the physical feature;
wherein the first metric is based, at least in part, on the difference between the estimate of the first physical attribute and the measurement of the first physical attribute and a difference between the estimate of the second physical attribute and a measurement of the second physical attribute.
The non-transitory computer-readable storage medium of claim 16 , further compris
[Truncated to input limit]