Office Action 1: Non-Final Rejection (2024-04-12)
Rejection types: 103 - Obviousness

CLAIMS:
CLAIMS

1\. A method of operation of a robot, the method performed by the robot, the method comprising:
identifying a set of candidate actions that may be performed by the robot;
collecting, for each candidate action of the set of candidate actions, a respective set of ancillary data;
transmitting a request for instructions to a tele-operation system, the tele- operation system communicatively coupled to the robot, the request for the instructions which includes each candidate action of the set of candidate actions and each respective set of ancillary data;
receiving the instructions from the tele-operation system;
executing the instructions; and
updating a control model of the robot, based at least in part on each candidate action of the set of candidate actions, each respective set of ancillary data, and the instructions, to increase a level of autonomy of the robot.

2\. The method of claim 1, the robot comprising one or more sensors, wherein the identifying a set of candidate actions that may be performed by the robot includes sensing, by the one or more sensors, at least one of an external event or an external condition and identifying the set of candidate actions in response to sensing the at least one of an external event or an external condition.

3\. The method of claim 1, further comprising receiving, from the tele- operation system, an objective to be completed by the robot, wherein the identifying a set of candidate actions that may be performed by the robot includes identifying a set of candidate actions that may be performed by the robot in furtherance of the objective.

4\. The method of claim 3, wherein the transmitting a request for instructions to a tele-operation system includes transmitting the request for instructions to the tele-operation system in response to determining the robot is unable to select a candidate action of the set of candidate actions to perform in furtherance of the objective.
5\. The method claim 4, further comprising scoring each candidate action of the set of candidate actions, wherein the scoring each candidate action of the set of candidate actions includes assigning a respective probability to each candidate action, the respective probability being based at least in part on a respective expectation of progress, achievable by each candidate action, towards the objective.

6\. The method of claim 5, further comprising ranking each candidate action in an order of preference, the order of preference being based at least in part on the respective probability of each candidate action.

7\. The method of claim 6, wherein the transmitting a request for instructions to a tele-operation system, the request for the instructions which includes each candidate action of the set of candidate actions and each respective set of ancillary data includes transmitting a request for instructions to a tele-operation system, the request for the instructions which includes each respective probability and the order of preference.

8\. The method of claim 6, further comprising determining whether a respective probability meets a predetermined probability criterion.

9\. The method of claim 6, further comprising determining whether a difference between a first respective probability and a second respective probability meets a predetermined difference criterion.

10\. The method of claim 1, the robot being a semi-autonomous robot characterized by a level of autonomy, the level of autonomy being determined at least in part by the control model, the method further comprising comparing the level of autonomy of the robot to a threshold level of autonomy, the threshold level of autonomy above which the robot is operable to autonomously select a candidate action of the set of candidate actions to perform, wherein the transmitting a request for instructions to a tele-operation system includes transmitting the request for instructions to the tele-operation system in response to determining the level of autonomy of the robot is lower than the threshold level of autonomy.
11\. The method of claim 10, further comprising, in response to determining the level of autonomy of the robot meets or exceeds the threshold level of autonomy, performing one of the candidate actions of the set of candidate actions.

12\. The method of claim 11, further comprising receiving, from the tele- operation system, an objective to be completed by the robot, wherein the identifying a set of candidate actions that may be performed by the robot includes identifying a set of candidate actions that may be performed by the robot in furtherance of the objective.

13\. The method of claim 12, further comprising determining the threshold level of autonomy, wherein the determining the threshold level of autonomy includes assigning a respective probability to each candidate action, the respective probability being based at least in part on a respective expectation of progress, achievable by each candidate action, towards the objective.

14\. The method of claim 13, further comprising ranking each candidate action in an order of preference, the order of preference being based at least in part on the respective probability.

15\. The method of claim 14, wherein the determining the level of autonomy of the robot is lower than the threshold level of autonomy includes determining the respective probability of each candidate action in the set of candidate actions fails to satisfy at least one predetermined criterion.

16\. The method of claim 1, the robot comprising one or more sensors and a data repository, the collecting, for each candidate action of the set of candidate actions, a respective set of ancillary data includes at least one of collecting a first subset of the respective set of ancillary data by the one or more sensors or extracting a second subset of the respective set of ancillary data from the data repository.

17\. The method of claim 1, wherein the transmitting a request for instructions to a tele-operation system includes transmitting a request for the tele-operation system to determine which candidate action the robot should perform.
18\. The method of claim 1, the robot comprising a motion actuator, wherein executing the instructions includes causing, by the motion actuator, the robot to execute a movement.

19\. The method of claim 1, wherein the updating a control model, based at least in part on each candidate action of the set of candidate actions, each respective set of ancillary data, and the instructions includes increasing a level of autonomy of the robot from a first level of autonomy to a second level of autonomy, the second level of autonomy being higher than the first level of autonomy.

20\. The method of claim 1, further comprising identifying, by the robot, an objective to be completed by the robot, wherein the identifying a set of candidate actions that may be performed by the robot includes identifying a set of candidate actions that may be performed by the robot in furtherance of the objective. 29 30 31 32

OFFICE ACTION:
DETAILED ACTION
This Non-Final Office Action is in response to claims filed 4/12/2022.
Claims 1-20 are pending.
Notice of Pre-AIA  or AIA  Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/20/2022 has been considered by the examiner.
Key to Interpreting this Office Action
For readability, all claim language has been underlined.
Citations from prior art are provided at the end of each limitation in parentheses.
Any further explanations that were deemed necessary by the Examiner are provided at the end of each claim limitation.
The Applicant is encouraged to contact the Examiner directly if there are any questions or concerns regarding the current Office Action.
Claim Objections
Claim 7 is objected to because of the following informalities:
Claim 7 recites wherein the transmitting a request for instructions to a tele-operation system, the request for the instructions which includes each candidate action of the set of candidate actions and each respective set of ancillary data includes transmitting a request for instructions to a tele-operation system, the request for the instructions which includes each respective probability and the order of preference (emphasis added). While it may be gleaned that the emphasized portion is further limiting a limitation of independent claim 1 provided as the un-emphasized portion, the claim language does not read well, due to a separate and distinct “request” being transmitted in the emphasized portion, and it cannot be reasonably determined if the “the request for instructions” of the emphasized portion is limiting the separate “request for instructions” in the emphasized portion or the “request for instructions” in the non-emphasized portion. In light of the Applicant’s disclosure, it is recommended to amend this limitation to read more clearly as “wherein the transmitted request for instructions includes each respective probability and the order of preference,” for example, or to provide proper antecedent basis.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA  35 U.S.C. 102 and 103 (or as subject to pre-AIA  35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA  to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over von Reventlow et al. (US 2023/0168670 A1), hereinafter von Reventlow, in view of alternative embodiments described in von Reventlow.
Claim 1
von Reventlow discloses the claimed method of operation of a robot (i.e. robot 1, depicted in Figure 1), the method performed by the robot (see ¶0044, regarding the embodiment in which processing unit 14 may be provided in robot 1), the method comprising:
identifying a set of candidate actions (i.e. action definition candidates) that may be performed by the robot (see ¶0008, regarding the processing unit automatically retrieves action definition candidates from a database according to a task);
collecting, for each candidate action of the set of candidate actions, a respective set of ancillary data (see ¶0008, regarding that the robot includes sensors for sensing the environment and determining the current situation for retrieving action definition candidates from the database; ¶0009-0010, regarding the calculation of a success score for each action definition candidate); “Ancillary data” is interpreted in light of the Applicant’s specification, defined as including sensor data or scores associated with a candidate action (see second paragraph on page 26 of the specification filed 4/12/2022).
transmitting a request for instructions to a tele-operation system, the tele-operation system communicatively coupled to the robot (see ¶0010-0011, regarding that the processing unit sends a request for assistance to a remote operator via a communication interface; ¶0054, with respect to Figure 2, regarding processing unit 14 sends the request for assistance via the communication interface 21 to the remote operation center, defined as including display 18 and input device 19, which communicate via operation center interface 21, in ¶0047), the request for the instructions which includes each candidate action of the set of candidate actions (see ¶0022, regarding that the request includes a plurality of action definition candidates);
receiving the instructions from the tele-operation system (see ¶0011, regarding that the system receives input from a remote operator);
executing the instructions (see ¶0011, regarding the robot will execute an action based on the input provided by the operator in response to the request for assistance); and
updating a control model (i.e. “knowledge base”) of the robot, based at least in part on each candidate action of the set of candidate actions, each respective set of ancillary data, and the instructions, to increase a level of autonomy of the robot (see ¶0011-0012, ¶0026, regarding that the corresponding action definition of the action executed based on remote controlled operation is stored in the database in association with the situation and its success score; ¶0083-0085, regarding that updating the database 17 improves the robot’s capability of autonomously executing a task).
Depending on the application of von Reventlow to the limitation of “ancillary data,” von Reventlow may further teach that the request for the instructions which includes…each respective set of ancillary data, in light of alternative embodiments within von Reventlow.
Specifically, in applying the sensed current situation described in ¶0008 of von Reventlow to the limitation of “ancillary data,” von Reventlow further teaches that a request for instructions includes each respective set of ancillary data (see ¶0066, regarding that all information collected by processing unit 14 from all available sensors is forwarded to the operator and output via display 18 as part of the request; ¶0016, regarding the sensor output is transmitted to the operator for deciding on the next operation of the robot). While von Reventlow particularly teaches the “request” as being associated with the unsegmented task in ¶0066, it would be reasonable to also conclude that the request for the instructions includes each respective set of ancillary data, in light of ¶0016, given that the sensor output is necessary for the operator to decide on the plurality of suggested action definition candidates (see ¶0022).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the request for the instructions of von Reventlow, so as to further include each respective set of ancillary data, in light of ¶0016 and ¶0066 of von Reventlow, with the predictable result of enabling the operator to analyze the situation of the robot in order to decide on one of a plurality of suggested action definition candidates.
In applying the success scores described in ¶0009-0010 of von Reventlow to the limitation of “ancillary data,” von Reventlow further teaches that the request for the instructions includes action definition candidates defined by each respective set of ancillary data (see ¶0022, regarding that the request includes the plurality of action definition candidates with particular success scores for operator selection). Given that the “ancillary data” is a tool used for ranking/selection (see ¶0068-0069), it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the request for the instructions of von Reventlow to further include each respective set of ancillary data, in light of ¶0069 of von Reventlow, with the predictable result of providing the operator with additional information useful in deciding on the next operation.
Claim 2
von Reventlow further discloses that the robot comprising one or more sensors (see Figure 2, depicting microphone 22 and sensors 11, 12 on robot 1), wherein the identifying a set of candidate actions that may be performed by the robot includes sensing, by the one or more sensors, at least one of an external event or an external condition and identifying the set of candidate actions in response to sensing the at least one of an external event or an external condition (see ¶0006, regarding that the task is determined based on information sensed by the robot, where the action definition candidates are retrieved from the database based on the determined task, as described in ¶0008; ¶0057-0060).
Claim 3
von Reventlow further discloses receiving an objective (i.e. task) to be completed by the robot (see ¶0057, regarding that the task is determined through operator input, where a target state is defined by the task and may be changed by the operator, as described in ¶0073), wherein the identifying a set of candidate actions that may be performed by the robot includes identifying a set of candidate actions that may be performed by the robot in furtherance of the objective (see ¶0067, regarding the task is segmented into smaller pieces, each of which may correspond to a single action for which an action definition is stored in database 17).
While von Reventlow teaches the use of operator input, e.g., a smart phone or comparable device, for receiving an objective (see ¶0057), von Reventlow does not specifically teach that the objective is received from the tele-operation system. However, the “tele-operation system” of von Reventlow would be an obvious means of providing operator input, due to input device 19 provided in the remote operation center (see ¶0074, with respect to Figure 2, regarding input device 19 is used by the operator to provide input information and/or control signals for controlling the robot or instructing the use of action definitions).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the objective of von Reventlow to be received from the tele-operation system, in light of ¶0074 of von Reventlow, with the predictable result of defining a task through a known alternative operator input device for controlling a robot.
Claim 4
von Reventlow further discloses that the transmitting a request for instructions to a tele-operation system includes transmitting the request for instructions to the tele-operation system in response to determining the robot is unable to select a candidate action of the set of candidate actions to perform in furtherance of the objective (see ¶0022, regarding that requesting remote controlled operation in a case that the action definition candidates do not have a success score that indicates that the corresponding action may be executed successfully).
Claim 5
von Reventlow further discloses scoring each candidate action of the set of candidate actions (see ¶0009-0010, regarding that a success score is calculated for each retrieved action definition candidate, where the success score is a measure of how likely it is that an action according to an action definition candidate contributes to successfully fulfilling the task), wherein the scoring each candidate action of the set of candidate actions includes assigning a respective probability to each candidate action, the respective probability being based at least in part on a respective expectation of progress, achievable by each candidate action, towards the objective (see ¶0068, regarding that the success score may be a probability).
Claim 6
von Reventlow further discloses ranking each candidate action in an order of preference, the order of preference being based at least in part on the respective probability of each candidate action (see ¶0009-0010, regarding the determination of the action definition candidates with the highest scores for comparison to a threshold; ¶0068, regarding that the success score may be a probability). Due to the preference in selecting the action definition candidates with the highest scores, it is clear that the scores inherently represent a “ranking” of the respective action definition candidate.
Claim 7
von Reventlow is applied to the limitation of “ancillary data,” with respect to the success scores described in ¶0009-0010, as discussed in the second interpretation of “ancillary data” in the rejection of claim 1 above. Specifically, von Reventlow further discloses that the transmitting a request for instructions to a tele-operation system, the request for the instructions which includes each candidate action of the set of candidate actions data and each respective set of ancillary, as discussed in the rejection of claim 1, includes transmitting a request for instructions to a tele-operation system (see ¶0010-0011, regarding that the processing unit sends a request for assistance to a remote operator via a communication interface; ¶0054, with respect to Figure 2, regarding processing unit 14 sends the request for assistance via the communication interface 21 to the remote operation center, defined as including display 18 and input device 19, which communicate via operation center interface 21, in ¶0047).
von Reventlow further teaches the request for the instructions which includes action definition candidates defined by each respective probability and the order of preference (see ¶0022, regarding that the request includes the plurality of action definition candidates with particular success scores for operator selection; ¶0068, regarding that the success score may be a probability). Due to the preference in selecting the action definition candidates with the highest scores (see ¶0010-0011), it is clear that the scores inherently represent an “order of preference” of the respective action definition candidate. Given that the success score (probability) is a tool used for ranking/selection (see ¶0068-0069), it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the request for the instructions of von Reventlow to further include each respective probability and the order of preference, in light of ¶0069 of von Reventlow, with the predictable result of providing the operator with additional information useful in deciding on the next operation.
Claim 8
von Reventlow further discloses determining whether a respective probability meets a predetermined probability criterion (see ¶0068, regarding the success score, defined as a probability, is compared to a threshold to determine if the action is likely to be successfully applied in the current situation).
Claim 9
von Reventlow further discloses determining whether a difference between a first respective probability (i.e. success score for the determined action definition candidates) and a second respective probability (i.e. maximal achievable success score) meets a predetermined difference criterion (i.e. urgency indicator) (see ¶0075, regarding that the difference between the calculated success score for the determined action definition candidates and a maximal achievable success score is compared to an urgency indicator; ¶0068, regarding the success score is a probability).
Claim 10
von Reventlow further discloses the robot being a semi-autonomous robot characterized by a level of autonomy (see ¶0010-0011, regarding the autonomous control of the robot when the success score is above a threshold and the remote operator control of the robot when the success score is below a threshold), the level of autonomy being determined at least in part by the control model (see ¶0011-0012, ¶0026, regarding that the corresponding action definition of the action executed based on remote controlled operation is stored in the database in association with the situation and its success score; ¶0083-0085, regarding that updating the database 17 improves the robot’s capability of autonomously executing a task). “Level of autonomy” may be broadly interpreted, and the success scores of the action definition candidates reasonably reflect a level of autonomy, given that the success scores are the basis for determining manual or autonomous operation of an action (see ¶0010).
von Reventlow further discloses the method further comprising comparing the level of autonomy of the robot to a threshold level of autonomy, the threshold level of autonomy above which the robot is operable to autonomously select a candidate action of the set of candidate actions to perform (see ¶0010, ¶0069, regarding that the robot automatically executes the action associated with an action definition candidate with the highest success score of all evaluated action definition candidates). The “threshold level of autonomy” may be reasonably taught by the threshold used to compare with the success score of an action definition candidate, in which the decision of autonomous operation depends. von Reventlow further discloses that the transmitting a request for instructions to a tele-operation system includes transmitting the request for instructions to the tele-operation system in response to determining the level of autonomy of the robot is lower than the threshold level of autonomy (see ¶0010-0011, regarding that the processing unit sends a request for assistance to a remote operator via a communication interface when the success score is below the threshold).
Claim 11
von Reventlow further teaches in response to determining the level of autonomy of the robot meets or exceeds the threshold level of autonomy, performing one of the candidate actions of the set of candidate actions (see ¶0074, regarding autonomously executing actions with success scores above the threshold).
Claim 12
von Reventlow further discloses receiving an objective (i.e. task) to be completed by the robot (see ¶0057, regarding that the task is determined through operator input, where a target state is defined by the task and may be changed by the operator, as described in ¶0073), wherein the identifying a set of candidate actions that may be performed by the robot includes identifying a set of candidate actions that may be performed by the robot in furtherance of the objective (see ¶0067, regarding the task is segmented into smaller pieces, each of which may correspond to a single action for which an action definition is stored in database 17).
While von Reventlow teaches the use of operator input, e.g., a smart phone or comparable device, for receiving an objective (see ¶0057), von Reventlow does not specifically teach that the objective is received from the tele-operation system. However, the “tele-operation system” of von Reventlow would be an obvious means of providing operator input, due to input device 19 provided in the remote operation center (see ¶0074, with respect to Figure 2, regarding input device 19 is used by the operator to provide input information and/or control signals for controlling the robot or instructing the use of action definitions).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the objective of von Reventlow to be received from the tele-operation system, in light of ¶0074 of von Reventlow, with the predictable result of defining a task through a known alternative operator input device for controlling a robot.
Claim 13
von Reventlow further teaches determining the threshold level of autonomy, wherein the determining the threshold level of autonomy includes assigning a respective probability to each candidate action, the respective probability being based at least in part on a respective expectation of progress, achievable by each candidate action, towards the objective (see ¶0009-0010, regarding that a success score defined as a probability in ¶0068 is calculated for each retrieved action definition candidate, where the success score is a measure of how likely it is that an action according to an action definition candidate contributes to successfully fulfilling the task).
Claim 14
von Reventlow further discloses ranking each candidate action in an order of preference, the order of preference being based at least in part on the respective probability (see ¶0009-0010, regarding the determination of the action definition candidates with the highest scores for comparison to a threshold; ¶0068, regarding that the success score may be a probability). Due to the preference in selecting the action definition candidates with the highest scores, it is clear that the scores (probabilities) inherently represent a “ranking” of the respective action definition candidate.
Claim 15
von Reventlow further teaches that the determining the level of autonomy of the robot is lower than the threshold level of autonomy includes determining the respective probability of each candidate action in the set of candidate actions fails to satisfy at least one predetermined criterion (see ¶0068, regarding that the success score, defined as a probability, is compared to a threshold to determine if the action is likely to be successfully applied in the current situation).
Claim 16
von Reventlow further discloses the robot comprising one or more sensors (see ¶0043, with respect to sensor depicted in Figure 2) and a data repository (see ¶0008-0009, regarding the retrieval of action definition candidates from a database by the processing unit, so as to evaluate and calculate a success score). It is clear that a memory or buffer is used by the processing unit of the robot to perform evaluations and calculations of the action definition candidates. von Reventlow further discloses that the collecting, for each candidate action of the set of candidate actions, a respective set of ancillary data includes at least one of collecting a first subset of the respective set of ancillary data by the one or more sensors or extracting a second subset of the respective set of ancillary data from the data repository (see ¶0008, regarding that the robot includes sensors for sensing the environment and determining the current situation for retrieving action definition candidates from the database; ¶0008-0010, regarding the calculation of a success score for each action definition candidate from a database). As discussed in the rejection of claim 1, “ancillary data” is interpreted in light of the Applicant’s specification, defined as including sensor data or scores associated with a candidate action (see second paragraph on page 26 of the specification filed 4/12/2022).
Claim 17
von Reventlow further discloses that the transmitting a request for instructions to a tele-operation system includes transmitting a request for the tele-operation system to determine which candidate action the robot should perform (see ¶0022, regarding the operator selects one of the suggested action definition candidates in response to the request).
Claim 18
von Reventlow further discloses the robot comprising a motion actuator (see ¶0007, regarding that the robot moves using its drive system; ¶0045, with respect to Figure 2, regarding actuators 16 are controlled by driver 15 of robot 1), wherein executing the instructions includes causing, by the motion actuator, the robot to execute a movement (see ¶0018, regarding that the action definition is any kind of motion that is executed by the robot).
Claim 19
von Reventlow further discloses the updating a control model, based at least in part on each candidate action of the set of candidate actions, each respective set of ancillary data, and the instructions includes increasing a level of autonomy of the robot from a first level of autonomy to a second level of autonomy, the second level of autonomy being higher than the first level of autonomy (see ¶0083-0085, regarding that updating the database 17 improves the robot’s capability of autonomously executing a task). “Levels of autonomy” may be broadly interpreted, and increasing autonomous capability in von Reventlow may reasonably be interpreted as increasing from a “first level” to a “second level,” due to the levels being undefined in the claim.
Claim 20
von Reventlow further discloses identifying, by the robot, an objective (i.e. task) to be completed by the robot (see ¶0006, regarding that the robot determines a task; ¶0057-0059), wherein the identifying a set of candidate actions that may be performed by the robot includes identifying a set of candidate actions that may be performed by the robot in furtherance of the objective (see ¶0067, regarding the task is segmented into smaller pieces, each of which may correspond to a single action for which an action definition is stored in database 17).
Conclusion


NEXT OFFICE ACTION: DETAILED ACTION
This Final Office Action is in response to amendments filed 7/12/2024.
Claims 1-20 have been amended.
Claims 1-20 are pending.
Response to Arguments
Claim Objections
Due to the amendments filed 7/12/2024, the objection of claim 7 has been withdrawn.
Rejections under 35 U.S.C. 103
On page 6 of the Remarks filed 7/12/2024, the Applicant contends that von Reventlow does not teach the amended features that further include elements involving a “fleet of robots,” such that a “control model of a second robot” is further updated based on updates of the control model of the first robot.
The Examiner respectfully disagrees. von Reventlow further teaches additional embodiments in which a plurality of robots may access the same database 17 (see ¶0091) and the processing unit 14 may be included in the robot 1 (see ¶0044), such that updates made to database 17 inherently update the “control models” of each of the plurality of robots. The application of prior art may be overcome by providing amendments that further limit “ancillary data” to be specifically sensor data acquired from sensors onboard the “first robot,” in light of the embodiments described on page 26 of the specification filed 4/12/2022, as well as amendments defining the “control model of the first robot” to be different from the “control model of the second robot,” such that the control models of each robot are not synchronized with a remote database, as taught by von Reventlow, but updated via peer-to-peer communication, as described in the Applicant’s disclosure.
Key to Interpreting this Office Action
For readability, all claim language has been underlined.
Citations from prior art are provided at the end of each limitation in parentheses.
Any further explanations that were deemed necessary by the Examiner are provided at the end of each claim limitation.
The Applicant is encouraged to contact the Examiner directly if there are any questions or concerns regarding the current Office Action.
Claim Objections
Claim 1 is objected to because of the following informalities:
Claim 1 recites updating, by a second robot in the fleet of robots, a control model of the second robot, based at least in part based on the update made to the control model of the first robot, to increase a level of autonomy of the second robot (emphasis added). The emphasized portion of the limitation is repetitive and should instead recite “based at least in part on” for clarity.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA  35 U.S.C. 102 and 103 (or as subject to pre-AIA  35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA  to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over von Reventlow et al. (US 2023/0168670 A1), hereinafter von Reventlow, in view of Ferguson et al. (US 2020/0150674 A1), hereinafter Ferguson.
Claim 1
von Reventlow discloses the claimed method of operation of a fleet of robots (see Figure 3, depicting the method carried out by the system in Figure 2, where processing unit 14 may be arranged in robot 1, as described in ¶0044; ¶0091, regarding the application to a plurality of robots using the same “intelligence”). von Reventlow is applied, in light of the embodiment described in ¶0044, where the robot 1 includes the processing unit 14, such that a plurality of robots (i.e. “fleet of robots”) may access the same database 17, as described in ¶0091.
von Reventlow further discloses that the claimed method comprises identifying, by a first robot in the fleet of robots, a set of candidate actions (i.e. action definition candidates) that may be performed by a first robot (see ¶0067, with respect to steps S4 and S6 of Figure 3, regarding that processing unit 14 retrieves potential action definitions as candidates in step S5 from database 17 based on the segmented task in step S4), and collecting, by the first robot and for each candidate action of the set of candidate actions, a respective set of ancillary data (see ¶0068, with respect to step S6 of Figure 3, regarding that processing unit 14 calculates a success score for each action definition candidate). “Ancilliary data” is interpreted in light of the Applicant’s specification, defined as including scores associated with a candidate action (see third paragraph on page 26 of the specification filed 4/12/2022).
von Reventlow further discloses that the claimed method comprises transmitting, by the first robot, a request for instructions to a tele-operation system, the tele-operation system communicatively coupled to the fleet of robots (see ¶0074, with respect to steps S7 and S11 of Figure 3, regarding that the service robot system sends a request to the operation center when the candidate’s score does not exceed a given threshold, where processing unit 14 sends the request via communication interface 21 to the remote operation center for control by a human operator, as defined in ¶0054), the request for the instructions which includes each candidate action of the set of candidate actions (see ¶0022, regarding that the request includes a plurality of action definition candidates).
von Reventlow does not further disclose that the request for the instructions which includes…each respective set of ancillary data. However, von Reventlow further teaches that the request for the instructions includes action definition candidates defined by each respective set of ancillary data (see ¶0022, regarding that the request includes the plurality of action definition candidates with particular success scores for operator selection). Given that the “ancillary data” is a tool used for ranking/selection (see ¶0068-0069), it would be reasonable to further provide the success scores to the operator as part of the request in step S11 of Figure 3, in light of Ferguson.
Specifically, Ferguson teaches a similar method in which a remote human operator (similar to the tele-operation system taught by von Reventlow) is requested (similar to the request taught by von Reventlow) when an autonomous vehicle (similar to the first robot taught by von Reventlow) encounters particularly difficult situations (see ¶0155), such that the request communicated to the remote human operator includes a risk rating (similar to the ancillary data or success scores taught by von Reventlow) (see ¶0159, with respect to step 508 of Figure 16), defined as based on an estimated probability of success in executing a particular procedure (see ¶0157).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the request for the instructions of von Reventlow to further include each respective set of ancillary data, in light of Ferguson, with the predictable result of increasing the remote operator’s awareness of the particular situation (¶0156 of Ferguson).
von Reventlow further discloses that the claimed method comprises receiving, by the first robot, the instructions from the tele-operation system (see ¶0074, ¶0078, with respect to step S12 of Figure 3, regarding receiving operator input, where the input can be used for immediate execution by the robot 1 or added to the knowledge base of the robot 1, as described in ¶0080-0081), executing, by the first robot, the instructions (see ¶0087, regarding that the additional information added by the operator may include a sequence of executable actions that need to be performed by the robot), and updating a control model (i.e. “knowledge base” or database 17) of the robot, based at least in part on each candidate action of the set of candidate actions, each respective set of ancillary data, and the instructions, to increase a level of autonomy of the first robot (see ¶0079, with respect to step S14 of Figure 3, regarding that the operator adds information to database 17 to make improved evaluation of action definition candidates, reconsider the segmentation of the task and/or selection of action definition candidates, where the updated database 17 improves the robot’s capability of autonomously executing a task, as described in ¶0083-0085; Figure 3, depicting the dependency of step S14 on each candidate and respective success score in step S7).
Given the application of von Reventlow, where the robot 1 includes the processing unit 14 (see ¶0044), and a plurality of robots (i.e. “fleet of robots”) access the same database 17 (see ¶0091), von Reventlow further discloses updating, by a second robot in the fleet of robots, a control model of the second robot, based at least in part based on the update made to the control model of the first robot, to increase a level of autonomy of the second robot (see ¶0079, with respect to step S14 of Figure 3, regarding that the operator adds information to database 17 to make improved evaluation of action definition candidates, reconsider the segmentation of the task and/or selection of action definition candidates, where the updated database 17 improves the robot’s capability of autonomously executing a task, as described in ¶0083-0085). Specifically, when the operator adds information to database 17 for improved execution of a task performed by a particular (“first”) robot, all other (“second”) robots that access the same database inherently update their respective “control models” based on the information added by the operator.
Claim 2
von Reventlow further discloses that the first robot comprising one or more sensors (see Figure 2, depicting microphone 22 and sensors 11, 12 on robot 1), wherein the identifying a set of candidate actions that may be performed by the first robot includes sensing, by the one or more sensors, at least one of an external event or an external condition and identifying the set of candidate actions in response to sensing the at least one of an external event or an external condition (see ¶0006, regarding that the task is determined based on information sensed by the robot, where the action definition candidates are retrieved from the database based on the determined task, as described in ¶0008; ¶0057-0060).
Claim 3
von Reventlow further discloses receiving, by the first robot, an objective (i.e. task) to be completed by the robot (see ¶0057, regarding that the task is determined by the robot through operator input, where a target state is defined by the task and may be changed by the operator, as described in ¶0073; ¶0044, regarding that robot 1 includes the processing unit 14 for communicating with remote locations, such as operation center interface 21, as described in ¶0047), wherein the identifying a set of candidate actions that may be performed by the first robot includes identifying a set of candidate actions that may be performed by the first robot in furtherance of the objective (see ¶0067, regarding the task is segmented into smaller pieces, each of which may correspond to a single action for which an action definition is stored in database 17).
While von Reventlow teaches the use of operator input, e.g., a smart phone or comparable device, for receiving an objective (see ¶0057), von Reventlow does not specifically teach that the objective is received from the tele-operation system. However, the “tele-operation system” of von Reventlow would be an obvious means of providing operator input, due to input device 19 provided in the remote operation center (see ¶0074, with respect to Figure 2, regarding input device 19 is used by the operator to provide input information and/or control signals for controlling the robot or instructing the use of action definitions).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the objective of von Reventlow to be received from the tele-operation system, in light of ¶0074 of von Reventlow, with the predictable result of defining a task through a known alternative operator input device for controlling a robot. Specifically, the operator input provided via an input device described in ¶0057 may be reasonably swapped for the operator input provided by input device 19 described in ¶0074, since both input devices serve the same purpose of providing input directed to controlling the robot. See MPEP 2143(I)(B).
Claim 4
von Reventlow further discloses that the transmitting a request for instructions to a tele-operation system includes transmitting the request for instructions to the tele-operation system in response to determining the first robot is unable to select a candidate action of the set of candidate actions to perform in furtherance of the objective (see ¶0022, regarding that requesting remote controlled operation in a case that the action definition candidates do not have a success score that indicates that the corresponding action may be executed successfully; ¶0068-0069).
Claim 5
von Reventlow further discloses scoring, by the first robot, each candidate action of the set of candidate actions (see ¶0009-0010, regarding that a success score is calculated for each retrieved action definition candidate, where the success score is a measure of how likely it is that an action according to an action definition candidate contributes to successfully fulfilling the task; ¶0068, regarding that the processing unit 14 calculates the success score, where the robot 1 includes the processing unit 14, as described in ¶0044), wherein the scoring each candidate action of the set of candidate actions includes assigning a respective probability to each candidate action, the respective probability being based at least in part on a respective expectation of progress, achievable by each candidate action, towards the objective (see ¶0068, regarding that the success score may be a probability).
Claim 6
von Reventlow further discloses ranking, by the first robot, each candidate action in an order of preference, the order of preference being based at least in part on the respective probability of each candidate action (see ¶0009-0010, regarding the determination of the action definition candidates with the highest scores for comparison to a threshold; ¶0068, regarding that the success score may be a probability and are determined by processing unit 14, where the robot 1 includes the processing unit 14, as described in ¶0044). Due to the preference in selecting the action definition candidates with the highest scores, it is clear that the scores inherently represent a “ranking” of the respective action definition candidate.
Claim 7
von Reventlow, as modified by Ferguson, further teaches that the transmitted request for instructions includes each respective probability and the order of preference, as discussed in the rejection of claim 1, regarding the “request” includes “each respective set of ancillary data.” Specifically, von Reventlow further teaches the “ancillary data” (i.e. success scores) as a probability (see ¶0068), and therefore, due to the preference in selecting the action definition candidates with the highest scores (see ¶0010-0011), it is clear that the scores inherently represent an “order of preference” of the respective action definition candidate.
Claim 8
von Reventlow further discloses determining, by the first robot, whether a respective probability meets a predetermined probability criterion (see ¶0068, regarding the success score, defined as a probability, is compared to a threshold to determine if the action is likely to be successfully applied in the current situation by processing unit 14, where the robot 1 includes the processing unit 14, as described in ¶0044).
Claim 9
von Reventlow further discloses determining, by the first robot, whether a difference between a first respective probability (i.e. success score for the determined action definition candidates) and a second respective probability (i.e. maximal achievable success score) meets a predetermined difference criterion (i.e. urgency indicator) (see ¶0075, regarding that the difference between the calculated success score for the determined action definition candidates and a maximal achievable success score is compared to an urgency indicator, where the success score is a probability, as described in ¶0068; ¶0054, regarding the processing unit 14 comes to the conclusion that autonomously executing the task is possible or not possible and sends the request for assistance to the remote operation center, where the robot 1 includes the processing unit 14, as described in ¶0044).
Claim 10
von Reventlow further discloses the first robot being a semi-autonomous robot characterized by a level of autonomy (see ¶0010-0011, regarding the autonomous control of the robot when the success score is above a threshold and the remote operator control of the robot when the success score is below a threshold), the level of autonomy being determined at least in part by the control model (see ¶0011-0012, ¶0026, regarding that the corresponding action definition of the action executed based on remote controlled operation is stored in the database in association with the situation and its success score; ¶0083-0085, regarding that updating the database 17 improves the robot’s capability of autonomously executing a task). “Level of autonomy” may be broadly interpreted, and the success scores of the action definition candidates reasonably reflect a level of autonomy, given that the success scores are the basis for determining manual or autonomous operation of an action (see ¶0010).
von Reventlow further discloses the method further comprising comparing the level of autonomy of the first robot to a threshold level of autonomy, the threshold level of autonomy above which the first robot is operable to autonomously select a candidate action of the set of candidate actions to perform (see ¶0010, ¶0069, regarding that the robot automatically executes the action associated with an action definition candidate with the highest success score of all evaluated action definition candidates). The “threshold level of autonomy” may be reasonably taught by the threshold used to compare with the success score of an action definition candidate, in which the decision of autonomous operation depends. von Reventlow further discloses that the transmitting, by the first robot, a request for instructions to a tele-operation system includes transmitting the request for instructions to the tele-operation system in response to determining the level of autonomy of the first robot is lower than the threshold level of autonomy (see ¶0010-0011, regarding that the processing unit sends a request for assistance to a remote operator via a communication interface when the success score is below the threshold).
Claim 11
von Reventlow further teaches in response to determining the level of autonomy of the first robot meets or exceeds the threshold level of autonomy, performing, by the first robot, one of the candidate actions of the set of candidate actions (see ¶0074, regarding that the robot autonomously executes actions with success scores above the threshold).
Claim 12
von Reventlow further discloses receiving, by the first robot, an objective (i.e. task) to be completed by the robot (see ¶0057, regarding that the task is determined by the robot through operator input, where a target state is defined by the task and may be changed by the operator, as described in ¶0073; ¶0044, regarding that robot 1 includes the processing unit 14 for communicating with remote locations, such as operation center interface 21, as described in ¶0047), wherein the identifying a set of candidate actions that may be performed by the first robot includes identifying a set of candidate actions that may be performed by the first robot in furtherance of the objective (see ¶0067, regarding the task is segmented into smaller pieces, each of which may correspond to a single action for which an action definition is stored in database 17).
While von Reventlow teaches the use of operator input, e.g., a smart phone or comparable device, for receiving an objective (see ¶0057), von Reventlow does not specifically teach that the objective is received from the tele-operation system. However, the “tele-operation system” of von Reventlow would be an obvious means of providing operator input, due to input device 19 provided in the remote operation center (see ¶0074, with respect to Figure 2, regarding input device 19 is used by the operator to provide input information and/or control signals for controlling the robot or instructing the use of action definitions).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the objective of von Reventlow to be received from the tele-operation system, in light of ¶0074 of von Reventlow, with the predictable result of defining a task through a known alternative operator input device for controlling a robot. Specifically, the operator input provided via an input device described in ¶0057 may be reasonably swapped for the operator input provided by input device 19 described in ¶0074, since both input devices serve the same purpose of providing input directed to controlling the robot. See MPEP 2143(I)(B).
Claim 13
von Reventlow further teaches determining, by the first robot, the threshold level of autonomy, wherein the determining the threshold level of autonomy includes assigning a respective probability to each candidate action, the respective probability being based at least in part on a respective expectation of progress, achievable by each candidate action, towards the objective (see ¶0009-0010, regarding that a success score is calculated for each retrieved action definition candidate, where the success score is a measure of how likely it is that an action according to an action definition candidate contributes to successfully fulfilling the task, where the success score is defined as a probability and calculated by the processing unit 14 in ¶0068, where the robot 1 includes the processing unit 14, as described in ¶0044).
Claim 14
von Reventlow further discloses ranking, by the first robot, each candidate action in an order of preference, the order of preference being based at least in part on the respective probability (see ¶0009-0010, regarding the determination of the action definition candidates with the highest scores for comparison to a threshold; ¶0068, regarding that the success score may be a probability; ¶0053-0054, regarding the calculations and comparisons are performed by processing unit 14, where the robot 1 includes the processing unit 14, as described in ¶0044). Due to the preference in selecting the action definition candidates with the highest scores, it is clear that the scores (probabilities) inherently represent a “ranking” of the respective action definition candidate.
Claim 15
von Reventlow further teaches that the determining the level of autonomy of the first robot is lower than the threshold level of autonomy includes determining the respective probability of each candidate action in the set of candidate actions fails to satisfy at least one predetermined criterion (see ¶0068, regarding that the success score, defined as a probability, is compared to a threshold to determine if the action is likely to be successfully applied in the current situation).
Claim 16
von Reventlow further discloses the first robot comprising one or more sensors (see ¶0043, with respect to sensor depicted in Figure 2) and a data repository (see ¶0008-0009, regarding the retrieval of action definition candidates from a database by the processing unit, so as to evaluate and calculate a success score; ¶0044, regarding that robot 1 includes the processing unit 14). It is clear that a memory or buffer is used by the processing unit of the robot to perform evaluations and calculations of the action definition candidates. von Reventlow further discloses that the collecting, by the first robot and for each candidate action of the set of candidate actions, a respective set of ancillary data includes at least one of collecting a first subset of the respective set of ancillary data by the one or more sensors or extracting a second subset of the respective set of ancillary data from the data repository (see ¶0008, regarding that the robot includes sensors for sensing the environment and determining the current situation for retrieving action definition candidates from the database; ¶0008-0010, regarding the calculation of a success score for each action definition candidate from a database, where processing unit 14 calculates the success scores, as described in ¶0068, and robot 1 includes the processing unit 14, as described in ¶0044). As discussed in the rejection of claim 1, “ancillary data” is interpreted in light of the Applicant’s specification, defined as including scores associated with a candidate action (see third paragraph on page 26 of the specification filed 4/12/2022).
Claim 17
von Reventlow further discloses that the transmitting, by the first robot, a request for instructions to a tele-operation system includes transmitting a request for the tele-operation system to determine which candidate action the first robot should perform (see ¶0022, regarding the operator selects one of the suggested action definition candidates in response to the request).
Claim 18
von Reventlow further discloses the first robot comprising a motion actuator (see ¶0007, regarding that the robot moves using its drive system; ¶0045, with respect to Figure 2, regarding actuators 16 are controlled by driver 15 of robot 1), wherein executing the instructions includes causing, by the motion actuator, the first robot to execute a movement (see ¶0018, regarding that the action definition is any kind of motion that is executed by the robot).
Claim 19
von Reventlow further discloses the updating, by the first robot, a control model, based at least in part on each candidate action of the set of candidate actions, each respective set of ancillary data, and the instructions includes increasing a level of autonomy of the first robot from a first level of autonomy to a second level of autonomy, the second level of autonomy being higher than the first level of autonomy (see ¶0083-0085, regarding that updating the database 17 improves the robot’s capability of autonomously executing a task). “Levels of autonomy” may be broadly interpreted, and increasing autonomous capability in von Reventlow may reasonably be interpreted as increasing from a “first level” to a “second level,” due to the levels being undefined in the claim.
Claim 20
von Reventlow further discloses identifying, by the first robot, an objective (i.e. task) to be completed by the robot (see ¶0006, regarding that the robot determines a task; ¶0057-0059), wherein the identifying a set of candidate actions that may be performed by the first robot includes identifying a set of candidate actions that may be performed by the first robot in furtherance of the objective (see ¶0067, regarding the task is segmented into smaller pieces, each of which may correspond to a single action for which an action definition is stored in database 17).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action.  Accordingly, THIS ACTION IS MADE FINAL.  See MPEP § 706.07(a).  Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action.  In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action.  In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.

CITED REFERENCE SUMMARIES: Ref US20230168670A1: Claims:
A service robot system, comprising a robot, wherein the robot ( 1 ) having a drive system ( 3 ) for moving the robot ( 1 ) to target positions and at least one effector ( 4 ) for manipulating the robot's environment, wherein the service robot system comprises a processing unit ( 14 ) including a task determination unit for determining a task to be executed by the robot and configured to control the drive system ( 3 ) and the at least one effector ( 4 ) according to the task based on action definitions, wherein the processing unit ( 14 ) is configured to automatically retrieve action definition candidates from a database ( 17 ), evaluate the retrieved action definition candidates with respect to a success score indicating a likelihood that an action according to the action definition candidate contributes to successfully fulfilling the task, to execute an action according to the action definition candidate having highest probability equal to or above a predefined threshold, and to send a request for assistance via a communication interface ( 21 ) if the success score is less than a preset threshold.
the robot ( 1 ) having a drive system ( 3 ) for moving the robot ( 1 ) to target positions and at least one effector ( 4 ) for manipulating the robot's environment, wherein the service robot system comprises a processing unit ( 14 ) including a task determination unit for determining a task to be executed by the robot and configured to control the drive system ( 3 ) and the at least one effector ( 4 ) according to the task based on action definitions, wherein the processing unit ( 14 ) is configured to automatically retrieve action definition candidates from a database ( 17 ), evaluate the retrieved action definition candidates with respect to a success score indicating a likelihood that an action according to the action definition candidate contributes to successfully fulfilling the task, to execute an action according to the action definition candidate having highest probability equal to or above a predefined threshold, and to send a request for assistance via a communication interface ( 21 ) if the success score is less than a preset threshold.
The service robot system according to claim 1 , wherein the processing unit ( 14 ) is configured to determine the success score for an action definition candidate based on information on success of an operation of an action according to the respective action definition candidate in the past.
The service robot system according to claim 1 , wherein the processing unit ( 14 ) is configured to segment the task into a sequence of action definitions.
The service robot system according to claim 1 , wherein the robot ( 1 ) is configured to execute an action that has a
The service robot system according to claim 1 , wherein the processing unit ( 14 ) is configured to dynamically predict potential changes of the environment and to adapt actions, or select actions.
The service robot system according to claim 1 , further comprising an operator interface ( 18 , 19 , 20 ) for remotely controlling the robot ( 1 ) to execute an action and/or input information for enhancing the database ( 17 ) by the operator interface in response to the request.
The service robot system according to claim 6 , wherein a definition of the action for remotely controlling is stored in the database ( 17 ) as a new action definition.
The service robot system according to claim 1 , wherein the robot ( 1 ) is equipped with at least one sensor ( 11 , 12 ) for physically sensing the robot's environment, and is configured to transmit output of the sensor to the processing unit ( 14 ) and/or the operator interface ( 18 , 19 , 20 ).
The service robot system according to claim 6 , wherein the operator interface ( 18 , 19 , 20 ) comprises a XR set ( 20 ) including a headset and controls for virtually controlling the robot ( 1 ) to execute an operation before transmitting remote control signals to the robot ( 1 ) for executing an action accordingly.
A service robot comprising a drive system ( 3 ) for moving a robot base ( 2 ) to target positions, wherein an effector ( 4 ) for manipulating a robot's environment and a support structure ( 9 ) are mounted to at least one lifter ( 6 , 7 ), wherein the support structure ( 9 ) is configured to provide a temporary repository ( 10 ) arranged at or movable to a position relative to the effector ( 4 ) which is reachable by the effector ( 4 ) for any possible elevation of the lifter ( 6 ) to which the effector ( 4 ) is mounted.
The service robot according to claim 10 , wherein the robot ( 1 ) comprises at least one sensor ( 11 ) attached to the effector ( 4 ) for sensing a detail of a scene and one additional sensor ( 12 ) sensing the scene.
The service robot according to claim 10 , wherein the robot's effector ( 4 ) comprises a gripping tool and a mechanical and/or an electrical interface for mounting and controlling at least one assistance tool.
A method for improving autonomous execution of tasks of a service robot comprising following steps: sensing environment (S 1 ) of the service robot ( 1 ) with at least one sensor ( 11 , 12 ); determining objects (S 2 ) in the service robot's environment and associating a recognition certainty with the determined object determining a task (S 3 ) to be executed by the service robot ( 1 ); retrieving action definition candidates (S 5 ) from a database ( 17 ); evaluating the retrieved action definitions candidates (S 6 ) to determine a success score for being successfully executed and contributing to successfully fulfilling the task; executing an action (S 8 ) having highest success score equal to or above a preset threshold and to send a request for external assistance via a communication interface if probability of action definition with the highest success score is below the preset threshold.
sensing environment (S 1 ) of the service robot ( 1 ) with at least one sensor ( 11 , 12 );
determining objects (S 2 ) in the service robot's environment and associating a recognition certainty with the determined object
determining a task (S 3 ) to be executed by the service robot ( 1 );
retrieving action definition candidates (S 5 ) from a database ( 17 );
evaluating the retrieved action definitions candidates (S 6 ) to determine a success score for being successfully executed and contributing to successfully fulfilling the task;
executing an action (S 8 ) having highest success score equal to or above a preset threshold and to send a request for external assistance via a communication interface if probability of action definition with the highest success score is below the preset threshold.
The method according to claim 13 , wherein the success score for an action definition candidate is calculated based on information on success of an operation of an action according to the respective action definition candidate in the past.
The method according to claim 13 , wherein the task is segmented (S 4 ) into a plurality of actions to be executed sequentially and/or in parallel, and for each action definition candidates are retrieved (S 5 ).
The method according to claim 13 , wherein the actions having the highest success score equal to or above the preset threshold are executed until an action to be executed next with a success score below the preset threshold is reached and to send the request for external assistance then.
The method according to claim 13 , wherein the request includes at least one action definition candidate.
The method according to claim 13 , wherein the preset threshold for the success score is set individually for an action to be executed depending on one or more tags of one or more object involved in an action to be executed.
The method according to claim 13 , wherein the service robot ( 1 ) is controlled remotely and/or information input by an operator interface is stored in the database ( 17 ) in response to the request.
The method according to claim 19 , wherein a new action definition is created from remote operation of the service robot ( 1 ) and is added to the database ( 17 ).
The method according to claim 13 , wherein information on an entity involved in an action to be executed is read in in response to the request and added to a tag or added as additional tag for the entity.
The method according to claim 13 , wherein the task to be executed is determined (S 3 ) from an instruction received via an operator interface or determined from a current situation encountered by the service robot ( 1 ).
The method according to claim 22 , wherein determination of the task to be executed is based on an association of previously received instructions and their context information extracted from the situation in which the instruction was received.