Office Action 2: Final Rejection (2024-10-25)
Rejection types: 103 - Obviousness

CLAIMS:
Amendments to the Claims

This listing of claims will replace all prior versions of claims in the application:

Listing of Claims:

1\. (Currently Amended) A method of operation of a fleet of robots, the method comprising:
identifying, by a first robot in the fleet of robots, a set of candidate actions that may be performed by a first robot;
collecting, by the first robot and for each candidate action of the set of candidate actions, a respective set of ancillary data;
transmitting, by the first robot, a request for instructions to a tele-operation system, the tele-operation system communicatively coupled to the fleet of robots, the request for the instructions which includes each candidate action of the set of candidate actions and each respective set of ancillary data;
receiving, by the first robot, the instructions from the tele-operation system;
executing, by the first robot, the instructions;
updating, by the first robot, a control model of the first robot, based at least in part on each candidate action of the set of candidate actions, each respective set of ancillary data, and the instructions, to increase a level of autonomy of the firstrobot; and
updatinq, by a second robot in the fleet of robots, a control model of the second robot, based at least in part based on the update made to the control model of the first robot, to increase a level of autonomy of the second robot.

2\. (Currently Amended) The method of claim 1, the first robot comprising one or more sensors, wherein the identifying a set of candidate actions that may be performed by the firstrobot includes sensing, by the one or more sensors, at least one of an external event or an external condition and identifying the set of candidate actions in response to sensing the at least one of an external event or an external condition.

3\. (Currently Amended) The method of claim 1, further comprising receiving, by the first robot and from the tele-operation system, an objective to be completed by the firstrobot, wherein the identifying a set of candidate actions that may be performed by the
first robot includes identifying a set of candidate actions that may be performed by the first robot in furtherance of the objective.

4\. (Currently Amended) The method of claim 3, wherein the transmitting a request for instructions to a tele-operation system includes transmitting the request for instructions to the tele-operation system in response to determining the first robot is unable to select a candidate action of the set of candidate actions to perform in furtherance of the objective.

5\. (Currently Amended) The method claim 4, further comprising scoringJ;>\_y the first robot, each candidate action of the set of candidate actions, wherein the scoring each candidate action of the set of candidate actions includes assigning a respective probability to each candidate action, the respective probability being based at least in part on a respective expectation of progress, achievable by each candidate action, towards the objective.
6\. (Currently Amended) The method of claim 5, further comprising by the first robot, each candidate action in an order of preference, the order of preference being based at least in part on the respective probability of each candidate action.SVG {EMBED HERE} 0.140.537

7\. (Currently Amended) The method of claim 6, wherein the transmitted request for instructions includes each respective probability and the order of preference.

8\. (Currently Amended) The method of claim 6, further comprising determining, by the first robot, whether a respective probability meets a predetermined probability criterion.

9\. (Currently Amended) The method of claim 6, further comprising determining, by the first robot, whether a difference between a first respective probability and a second respective probability meets a predetermined difference criterion.

10\. (Currently Amended) The method of claim 1, the first robot being a semi- autonomous robot characterized by a level of autonomy, the level of autonomy being determined at least in part by the control model, the method further comprising comparing the level of autonomy of the first robot to a threshold level of autonomy, the threshold level of autonomy above which the first robot is operable to autonomously select a candidate action of the set of candidate actions to perform, wherein the transmitting, by the first robot, a request for instructions to a tele-operation system includes transmitting the request for instructions to the tele-operation system in response to determining the level of autonomy of the first robot is lower than the threshold level of autonomy.

11\. (Currently Amended) The method of claim 10, further comprising, in response to determining the level of autonomy of the firstrobot meets or exceeds the threshold level of autonomy, performing, by the first robot, one of the candidate actions of the set of candidate actions.

12\. (Currently Amended) The method of claim 11, further comprising receiving, by the first robot and from the tele-operation system, an objective to be completed by the first robot, wherein the identifying a set of candidate actions that may be performed by thefirst robot includes identifying a set of candidate actions that may be performed by the first robot in furtherance of the objective.

13\. (Currently Amended) The method of claim 12, further comprising determining, by the first robot, the threshold level of autonomy, wherein the determining the threshold level of autonomy includes assigning a respective probability to each candidate action, the respective probability being based at least in part on a respective expectation of progress, achievable by each candidate action, towards the objective.
14\. (Currently Amended) The method of claim 13, further comprising by the first robot, each candidate action in an order of preference, the order of preference being based at least in part on the respective probability.SVG {EMBED HERE} 0.140.537

15\. (Currently Amended) The method of claim 14, wherein the determining the level of autonomy of the first robot is lower than the threshold level of autonomy includes
determining the respective probability of each candidate action in the set of candidate actions fails to satisfy at least one predetermined criterion.

16\. (Currently Amended) The method of claim 1, the first robot comprising one or more sensors and a data repository, the collecting, by the first robot and for each candidate action of the set of candidate actions, a respective set of ancillary data includes at least one of collecting a first subset of the respective set of ancillary data by the one or more sensors or extracting a second subset of the respective set of ancillary data from the data repository.

17\. (Currently Amended) The method of claim 1, wherein the transmittingsby the first robot, a request for instructions to a tele-operation system includes transmitting a request for the tele-operation system to determine which candidate action the first robot should perform.

18\. (Currently Amended) The method of claim 1, the first robot comprising a motion actuator, wherein executing the instructions includes causing, by the motion actuator, the first robot to execute a movement.

19\. (Currently Amended) The method of claim 1, wherein the updating.,\_by the first robot, a control model, based at least in part on each candidate action of the set of candidate actions, each respective set of ancillary data, and the instructions includes increasing a level of autonomy of the first robot from a first level of autonomy to a second level of autonomy, the second level of autonomy being higher than the first level of autonomy.

20\. (Currently Amended) The method of claim 1, further comprising identifying, by the first robot, an objective to be completed by the firstrobot, wherein the identifying a set of candidate actions that may be performed by the firstrobot includes identifying a set of candidate actions that may be performed by the firstrobot in furtherance of the objective.

OFFICE ACTION:
DETAILED ACTION
This Final Office Action is in response to amendments filed 7/12/2024.
Claims 1-20 have been amended.
Claims 1-20 are pending.
Response to Arguments
Claim Objections
Due to the amendments filed 7/12/2024, the objection of claim 7 has been withdrawn.
Rejections under 35 U.S.C. 103
On page 6 of the Remarks filed 7/12/2024, the Applicant contends that von Reventlow does not teach the amended features that further include elements involving a “fleet of robots,” such that a “control model of a second robot” is further updated based on updates of the control model of the first robot.
The Examiner respectfully disagrees. von Reventlow further teaches additional embodiments in which a plurality of robots may access the same database 17 (see ¶0091) and the processing unit 14 may be included in the robot 1 (see ¶0044), such that updates made to database 17 inherently update the “control models” of each of the plurality of robots. The application of prior art may be overcome by providing amendments that further limit “ancillary data” to be specifically sensor data acquired from sensors onboard the “first robot,” in light of the embodiments described on page 26 of the specification filed 4/12/2022, as well as amendments defining the “control model of the first robot” to be different from the “control model of the second robot,” such that the control models of each robot are not synchronized with a remote database, as taught by von Reventlow, but updated via peer-to-peer communication, as described in the Applicant’s disclosure.
Key to Interpreting this Office Action
For readability, all claim language has been underlined.
Citations from prior art are provided at the end of each limitation in parentheses.
Any further explanations that were deemed necessary by the Examiner are provided at the end of each claim limitation.
The Applicant is encouraged to contact the Examiner directly if there are any questions or concerns regarding the current Office Action.
Claim Objections
Claim 1 is objected to because of the following informalities:
Claim 1 recites updating, by a second robot in the fleet of robots, a control model of the second robot, based at least in part based on the update made to the control model of the first robot, to increase a level of autonomy of the second robot (emphasis added). The emphasized portion of the limitation is repetitive and should instead recite “based at least in part on” for clarity.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA  35 U.S.C. 102 and 103 (or as subject to pre-AIA  35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA  to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over von Reventlow et al. (US 2023/0168670 A1), hereinafter von Reventlow, in view of Ferguson et al. (US 2020/0150674 A1), hereinafter Ferguson.
Claim 1
von Reventlow discloses the claimed method of operation of a fleet of robots (see Figure 3, depicting the method carried out by the system in Figure 2, where processing unit 14 may be arranged in robot 1, as described in ¶0044; ¶0091, regarding the application to a plurality of robots using the same “intelligence”). von Reventlow is applied, in light of the embodiment described in ¶0044, where the robot 1 includes the processing unit 14, such that a plurality of robots (i.e. “fleet of robots”) may access the same database 17, as described in ¶0091.
von Reventlow further discloses that the claimed method comprises identifying, by a first robot in the fleet of robots, a set of candidate actions (i.e. action definition candidates) that may be performed by a first robot (see ¶0067, with respect to steps S4 and S6 of Figure 3, regarding that processing unit 14 retrieves potential action definitions as candidates in step S5 from database 17 based on the segmented task in step S4), and collecting, by the first robot and for each candidate action of the set of candidate actions, a respective set of ancillary data (see ¶0068, with respect to step S6 of Figure 3, regarding that processing unit 14 calculates a success score for each action definition candidate). “Ancilliary data” is interpreted in light of the Applicant’s specification, defined as including scores associated with a candidate action (see third paragraph on page 26 of the specification filed 4/12/2022).
von Reventlow further discloses that the claimed method comprises transmitting, by the first robot, a request for instructions to a tele-operation system, the tele-operation system communicatively coupled to the fleet of robots (see ¶0074, with respect to steps S7 and S11 of Figure 3, regarding that the service robot system sends a request to the operation center when the candidate’s score does not exceed a given threshold, where processing unit 14 sends the request via communication interface 21 to the remote operation center for control by a human operator, as defined in ¶0054), the request for the instructions which includes each candidate action of the set of candidate actions (see ¶0022, regarding that the request includes a plurality of action definition candidates).
von Reventlow does not further disclose that the request for the instructions which includes…each respective set of ancillary data. However, von Reventlow further teaches that the request for the instructions includes action definition candidates defined by each respective set of ancillary data (see ¶0022, regarding that the request includes the plurality of action definition candidates with particular success scores for operator selection). Given that the “ancillary data” is a tool used for ranking/selection (see ¶0068-0069), it would be reasonable to further provide the success scores to the operator as part of the request in step S11 of Figure 3, in light of Ferguson.
Specifically, Ferguson teaches a similar method in which a remote human operator (similar to the tele-operation system taught by von Reventlow) is requested (similar to the request taught by von Reventlow) when an autonomous vehicle (similar to the first robot taught by von Reventlow) encounters particularly difficult situations (see ¶0155), such that the request communicated to the remote human operator includes a risk rating (similar to the ancillary data or success scores taught by von Reventlow) (see ¶0159, with respect to step 508 of Figure 16), defined as based on an estimated probability of success in executing a particular procedure (see ¶0157).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the request for the instructions of von Reventlow to further include each respective set of ancillary data, in light of Ferguson, with the predictable result of increasing the remote operator’s awareness of the particular situation (¶0156 of Ferguson).
von Reventlow further discloses that the claimed method comprises receiving, by the first robot, the instructions from the tele-operation system (see ¶0074, ¶0078, with respect to step S12 of Figure 3, regarding receiving operator input, where the input can be used for immediate execution by the robot 1 or added to the knowledge base of the robot 1, as described in ¶0080-0081), executing, by the first robot, the instructions (see ¶0087, regarding that the additional information added by the operator may include a sequence of executable actions that need to be performed by the robot), and updating a control model (i.e. “knowledge base” or database 17) of the robot, based at least in part on each candidate action of the set of candidate actions, each respective set of ancillary data, and the instructions, to increase a level of autonomy of the first robot (see ¶0079, with respect to step S14 of Figure 3, regarding that the operator adds information to database 17 to make improved evaluation of action definition candidates, reconsider the segmentation of the task and/or selection of action definition candidates, where the updated database 17 improves the robot’s capability of autonomously executing a task, as described in ¶0083-0085; Figure 3, depicting the dependency of step S14 on each candidate and respective success score in step S7).
Given the application of von Reventlow, where the robot 1 includes the processing unit 14 (see ¶0044), and a plurality of robots (i.e. “fleet of robots”) access the same database 17 (see ¶0091), von Reventlow further discloses updating, by a second robot in the fleet of robots, a control model of the second robot, based at least in part based on the update made to the control model of the first robot, to increase a level of autonomy of the second robot (see ¶0079, with respect to step S14 of Figure 3, regarding that the operator adds information to database 17 to make improved evaluation of action definition candidates, reconsider the segmentation of the task and/or selection of action definition candidates, where the updated database 17 improves the robot’s capability of autonomously executing a task, as described in ¶0083-0085). Specifically, when the operator adds information to database 17 for improved execution of a task performed by a particular (“first”) robot, all other (“second”) robots that access the same database inherently update their respective “control models” based on the information added by the operator.
Claim 2
von Reventlow further discloses that the first robot comprising one or more sensors (see Figure 2, depicting microphone 22 and sensors 11, 12 on robot 1), wherein the identifying a set of candidate actions that may be performed by the first robot includes sensing, by the one or more sensors, at least one of an external event or an external condition and identifying the set of candidate actions in response to sensing the at least one of an external event or an external condition (see ¶0006, regarding that the task is determined based on information sensed by the robot, where the action definition candidates are retrieved from the database based on the determined task, as described in ¶0008; ¶0057-0060).
Claim 3
von Reventlow further discloses receiving, by the first robot, an objective (i.e. task) to be completed by the robot (see ¶0057, regarding that the task is determined by the robot through operator input, where a target state is defined by the task and may be changed by the operator, as described in ¶0073; ¶0044, regarding that robot 1 includes the processing unit 14 for communicating with remote locations, such as operation center interface 21, as described in ¶0047), wherein the identifying a set of candidate actions that may be performed by the first robot includes identifying a set of candidate actions that may be performed by the first robot in furtherance of the objective (see ¶0067, regarding the task is segmented into smaller pieces, each of which may correspond to a single action for which an action definition is stored in database 17).
While von Reventlow teaches the use of operator input, e.g., a smart phone or comparable device, for receiving an objective (see ¶0057), von Reventlow does not specifically teach that the objective is received from the tele-operation system. However, the “tele-operation system” of von Reventlow would be an obvious means of providing operator input, due to input device 19 provided in the remote operation center (see ¶0074, with respect to Figure 2, regarding input device 19 is used by the operator to provide input information and/or control signals for controlling the robot or instructing the use of action definitions).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the objective of von Reventlow to be received from the tele-operation system, in light of ¶0074 of von Reventlow, with the predictable result of defining a task through a known alternative operator input device for controlling a robot. Specifically, the operator input provided via an input device described in ¶0057 may be reasonably swapped for the operator input provided by input device 19 described in ¶0074, since both input devices serve the same purpose of providing input directed to controlling the robot. See MPEP 2143(I)(B).
Claim 4
von Reventlow further discloses that the transmitting a request for instructions to a tele-operation system includes transmitting the request for instructions to the tele-operation system in response to determining the first robot is unable to select a candidate action of the set of candidate actions to perform in furtherance of the objective (see ¶0022, regarding that requesting remote controlled operation in a case that the action definition candidates do not have a success score that indicates that the corresponding action may be executed successfully; ¶0068-0069).
Claim 5
von Reventlow further discloses scoring, by the first robot, each candidate action of the set of candidate actions (see ¶0009-0010, regarding that a success score is calculated for each retrieved action definition candidate, where the success score is a measure of how likely it is that an action according to an action definition candidate contributes to successfully fulfilling the task; ¶0068, regarding that the processing unit 14 calculates the success score, where the robot 1 includes the processing unit 14, as described in ¶0044), wherein the scoring each candidate action of the set of candidate actions includes assigning a respective probability to each candidate action, the respective probability being based at least in part on a respective expectation of progress, achievable by each candidate action, towards the objective (see ¶0068, regarding that the success score may be a probability).
Claim 6
von Reventlow further discloses ranking, by the first robot, each candidate action in an order of preference, the order of preference being based at least in part on the respective probability of each candidate action (see ¶0009-0010, regarding the determination of the action definition candidates with the highest scores for comparison to a threshold; ¶0068, regarding that the success score may be a probability and are determined by processing unit 14, where the robot 1 includes the processing unit 14, as described in ¶0044). Due to the preference in selecting the action definition candidates with the highest scores, it is clear that the scores inherently represent a “ranking” of the respective action definition candidate.
Claim 7
von Reventlow, as modified by Ferguson, further teaches that the transmitted request for instructions includes each respective probability and the order of preference, as discussed in the rejection of claim 1, regarding the “request” includes “each respective set of ancillary data.” Specifically, von Reventlow further teaches the “ancillary data” (i.e. success scores) as a probability (see ¶0068), and therefore, due to the preference in selecting the action definition candidates with the highest scores (see ¶0010-0011), it is clear that the scores inherently represent an “order of preference” of the respective action definition candidate.
Claim 8
von Reventlow further discloses determining, by the first robot, whether a respective probability meets a predetermined probability criterion (see ¶0068, regarding the success score, defined as a probability, is compared to a threshold to determine if the action is likely to be successfully applied in the current situation by processing unit 14, where the robot 1 includes the processing unit 14, as described in ¶0044).
Claim 9
von Reventlow further discloses determining, by the first robot, whether a difference between a first respective probability (i.e. success score for the determined action definition candidates) and a second respective probability (i.e. maximal achievable success score) meets a predetermined difference criterion (i.e. urgency indicator) (see ¶0075, regarding that the difference between the calculated success score for the determined action definition candidates and a maximal achievable success score is compared to an urgency indicator, where the success score is a probability, as described in ¶0068; ¶0054, regarding the processing unit 14 comes to the conclusion that autonomously executing the task is possible or not possible and sends the request for assistance to the remote operation center, where the robot 1 includes the processing unit 14, as described in ¶0044).
Claim 10
von Reventlow further discloses the first robot being a semi-autonomous robot characterized by a level of autonomy (see ¶0010-0011, regarding the autonomous control of the robot when the success score is above a threshold and the remote operator control of the robot when the success score is below a threshold), the level of autonomy being determined at least in part by the control model (see ¶0011-0012, ¶0026, regarding that the corresponding action definition of the action executed based on remote controlled operation is stored in the database in association with the situation and its success score; ¶0083-0085, regarding that updating the database 17 improves the robot’s capability of autonomously executing a task). “Level of autonomy” may be broadly interpreted, and the success scores of the action definition candidates reasonably reflect a level of autonomy, given that the success scores are the basis for determining manual or autonomous operation of an action (see ¶0010).
von Reventlow further discloses the method further comprising comparing the level of autonomy of the first robot to a threshold level of autonomy, the threshold level of autonomy above which the first robot is operable to autonomously select a candidate action of the set of candidate actions to perform (see ¶0010, ¶0069, regarding that the robot automatically executes the action associated with an action definition candidate with the highest success score of all evaluated action definition candidates). The “threshold level of autonomy” may be reasonably taught by the threshold used to compare with the success score of an action definition candidate, in which the decision of autonomous operation depends. von Reventlow further discloses that the transmitting, by the first robot, a request for instructions to a tele-operation system includes transmitting the request for instructions to the tele-operation system in response to determining the level of autonomy of the first robot is lower than the threshold level of autonomy (see ¶0010-0011, regarding that the processing unit sends a request for assistance to a remote operator via a communication interface when the success score is below the threshold).
Claim 11
von Reventlow further teaches in response to determining the level of autonomy of the first robot meets or exceeds the threshold level of autonomy, performing, by the first robot, one of the candidate actions of the set of candidate actions (see ¶0074, regarding that the robot autonomously executes actions with success scores above the threshold).
Claim 12
von Reventlow further discloses receiving, by the first robot, an objective (i.e. task) to be completed by the robot (see ¶0057, regarding that the task is determined by the robot through operator input, where a target state is defined by the task and may be changed by the operator, as described in ¶0073; ¶0044, regarding that robot 1 includes the processing unit 14 for communicating with remote locations, such as operation center interface 21, as described in ¶0047), wherein the identifying a set of candidate actions that may be performed by the first robot includes identifying a set of candidate actions that may be performed by the first robot in furtherance of the objective (see ¶0067, regarding the task is segmented into smaller pieces, each of which may correspond to a single action for which an action definition is stored in database 17).
While von Reventlow teaches the use of operator input, e.g., a smart phone or comparable device, for receiving an objective (see ¶0057), von Reventlow does not specifically teach that the objective is received from the tele-operation system. However, the “tele-operation system” of von Reventlow would be an obvious means of providing operator input, due to input device 19 provided in the remote operation center (see ¶0074, with respect to Figure 2, regarding input device 19 is used by the operator to provide input information and/or control signals for controlling the robot or instructing the use of action definitions).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the objective of von Reventlow to be received from the tele-operation system, in light of ¶0074 of von Reventlow, with the predictable result of defining a task through a known alternative operator input device for controlling a robot. Specifically, the operator input provided via an input device described in ¶0057 may be reasonably swapped for the operator input provided by input device 19 described in ¶0074, since both input devices serve the same purpose of providing input directed to controlling the robot. See MPEP 2143(I)(B).
Claim 13
von Reventlow further teaches determining, by the first robot, the threshold level of autonomy, wherein the determining the threshold level of autonomy includes assigning a respective probability to each candidate action, the respective probability being based at least in part on a respective expectation of progress, achievable by each candidate action, towards the objective (see ¶0009-0010, regarding that a success score is calculated for each retrieved action definition candidate, where the success score is a measure of how likely it is that an action according to an action definition candidate contributes to successfully fulfilling the task, where the success score is defined as a probability and calculated by the processing unit 14 in ¶0068, where the robot 1 includes the processing unit 14, as described in ¶0044).
Claim 14
von Reventlow further discloses ranking, by the first robot, each candidate action in an order of preference, the order of preference being based at least in part on the respective probability (see ¶0009-0010, regarding the determination of the action definition candidates with the highest scores for comparison to a threshold; ¶0068, regarding that the success score may be a probability; ¶0053-0054, regarding the calculations and comparisons are performed by processing unit 14, where the robot 1 includes the processing unit 14, as described in ¶0044). Due to the preference in selecting the action definition candidates with the highest scores, it is clear that the scores (probabilities) inherently represent a “ranking” of the respective action definition candidate.
Claim 15
von Reventlow further teaches that the determining the level of autonomy of the first robot is lower than the threshold level of autonomy includes determining the respective probability of each candidate action in the set of candidate actions fails to satisfy at least one predetermined criterion (see ¶0068, regarding that the success score, defined as a probability, is compared to a threshold to determine if the action is likely to be successfully applied in the current situation).
Claim 16
von Reventlow further discloses the first robot comprising one or more sensors (see ¶0043, with respect to sensor depicted in Figure 2) and a data repository (see ¶0008-0009, regarding the retrieval of action definition candidates from a database by the processing unit, so as to evaluate and calculate a success score; ¶0044, regarding that robot 1 includes the processing unit 14). It is clear that a memory or buffer is used by the processing unit of the robot to perform evaluations and calculations of the action definition candidates. von Reventlow further discloses that the collecting, by the first robot and for each candidate action of the set of candidate actions, a respective set of ancillary data includes at least one of collecting a first subset of the respective set of ancillary data by the one or more sensors or extracting a second subset of the respective set of ancillary data from the data repository (see ¶0008, regarding that the robot includes sensors for sensing the environment and determining the current situation for retrieving action definition candidates from the database; ¶0008-0010, regarding the calculation of a success score for each action definition candidate from a database, where processing unit 14 calculates the success scores, as described in ¶0068, and robot 1 includes the processing unit 14, as described in ¶0044). As discussed in the rejection of claim 1, “ancillary data” is interpreted in light of the Applicant’s specification, defined as including scores associated with a candidate action (see third paragraph on page 26 of the specification filed 4/12/2022).
Claim 17
von Reventlow further discloses that the transmitting, by the first robot, a request for instructions to a tele-operation system includes transmitting a request for the tele-operation system to determine which candidate action the first robot should perform (see ¶0022, regarding the operator selects one of the suggested action definition candidates in response to the request).
Claim 18
von Reventlow further discloses the first robot comprising a motion actuator (see ¶0007, regarding that the robot moves using its drive system; ¶0045, with respect to Figure 2, regarding actuators 16 are controlled by driver 15 of robot 1), wherein executing the instructions includes causing, by the motion actuator, the first robot to execute a movement (see ¶0018, regarding that the action definition is any kind of motion that is executed by the robot).
Claim 19
von Reventlow further discloses the updating, by the first robot, a control model, based at least in part on each candidate action of the set of candidate actions, each respective set of ancillary data, and the instructions includes increasing a level of autonomy of the first robot from a first level of autonomy to a second level of autonomy, the second level of autonomy being higher than the first level of autonomy (see ¶0083-0085, regarding that updating the database 17 improves the robot’s capability of autonomously executing a task). “Levels of autonomy” may be broadly interpreted, and increasing autonomous capability in von Reventlow may reasonably be interpreted as increasing from a “first level” to a “second level,” due to the levels being undefined in the claim.
Claim 20
von Reventlow further discloses identifying, by the first robot, an objective (i.e. task) to be completed by the robot (see ¶0006, regarding that the robot determines a task; ¶0057-0059), wherein the identifying a set of candidate actions that may be performed by the first robot includes identifying a set of candidate actions that may be performed by the first robot in furtherance of the objective (see ¶0067, regarding the task is segmented into smaller pieces, each of which may correspond to a single action for which an action definition is stored in database 17).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action.  Accordingly, THIS ACTION IS MADE FINAL.  See MPEP § 706.07(a).  Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action.  In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action.  In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.

APPLICANT REBUTTAL: REMARKS
This is a Response to the Office Action mailed April 12, 2024 (the "Office
Action"), in which a three (3) month Shortened Statutory Period for Response has been set, due to expire July 12, 2024. By way of this Response, claims 1-20 are amended. No claims are canceled and no new claims are added. No new matter has been added to the application. Claims 1-20 are pending.
Claim Objections
Claim 7 was objected to due to an informality, for which the Exmainer proposed a re-wording to resolve. Applicant has amended claim 7 in accordance with the Examiner's suggestion and therefore requests that the objection to claim 7 be withdrawn.
Rejections Under 35 USC 103
Claims 1-20 were rejected under 35 USC 103 for being allegedly unpatentable over US 20230168670 (hereafter, "von Reventlow") in view of alternative embodiments described in von Reventlow.
Applicant does not acquiesce to the Examiner's rejections of claims 1-20, but in the interest of expediting allowance of the instant application Applicant has amended independent claim 1 to further distinguish from von Reventflow. Specifically, Applicant has amended the preamble of independent claim 1 to recite a method of operation of a
fleet of robot;
clarified that the original acts recited in original independent claim 1 are performed by a first
robot in the fleet of robots;
and added a further act of
updatin
g
, by a second robot in the fleet of robots, a control model of the second robot, based at least in part based on the update made to the control model of the first robot, to increase a level of autonomy of the second robot.
Support for these features may be found in Applicant's specification at at least paragraphs 0070 and 0121-0122. Based on the foregoing, Applicant has amended dependent claims 2-20 to clarify whether it is the first robot or the second robot that performs the various acts recited.
Applicant respectfully submits that von Reventflow does not teach or suggest all of the features of any of Applicant's amended claims 1-20, and therefore requests that the rejections of claims 1-20 under 35 USC 103 be withdrawn.
Conclusion

NEXT OFFICE ACTION: Acknowledgement
This Notice of Allowance is in response to amendments filed 1/25/2025.
EXAMINER'S AMENDMENT
An examiner’s amendment to the record appears below. Should the changes and/or additions be unacceptable to applicant, an amendment may be filed as provided by 37 CFR 1.312. To ensure consideration of such an amendment, it MUST be submitted no later than the payment of the issue fee.
The application has been amended as follows:
In the sixteenth line of claim 1, the last comma in the sixteenth line of claim 1 has been deleted, such that only one comma is provided after the limitation of “the fleet of robots.”
In the sixth line of claim 16, the limitation of “ancillary” has been deleted, and the limitation of --sensor-- has been inserted.
Reasons for Allowance
Claims 1-20 are allowed.
The following is an examiner’s statement of reasons for allowance:
The closest prior art of record, Duan et al. (CN 105773615 A), taken alone or in combination, does not teach the claimed method of operation of a fleet of robots, the method comprising:
identifying, by a first robot in the fleet of robots, a set of candidate actions that may be performed by a first robot;
collecting, by at least one sensor onboard the first robot and for each candidate action of the set of candidate actions, a respective set of sensor data;
transmitting, by the first robot, a request for instructions to a tele-operation system, the tele-operation system communicatively coupled to the fleet of robots, the request for the instructions which includes each candidate action of the set of candidate actions and each respective set of sensor data;
receiving, by the first robot, the instructions from the tele-operation system;
executing, by the first robot, the instructions;
updating, by the first robot, a first control model of the first robot, based at least in part on each candidate action of the set of candidate actions, each respective set of sensor data, and the instructions, to increase a level of autonomy of the first robot;
sending, by the first robot and directly to a second robot in the fleet of robots, control model update instructions based on the update made to the first control model; and
updating, by the second robot, a second control model of the second robot, based at least in part on the update made to the first control model of the first robot, to increase a level of autonomy of the second robot, wherein the second control model of the second robot is different from the first control model of the first robot.
Specifically, upon further search and consideration of the amendments filed 1/25/2025, Duan et al. has been identified as the closest relevant prior art. Duan et al. teaches a similar method of operation of a fleet of robots (see ¶0081-0086, regarding a particular example in which a robot receives instructions from a robot integrated control platform, where all processes and results are uploaded by the robot to the robot integrated control platform for reference by other robots), the method comprising identifying, by a first robot in the fleet of robots, a set of candidate actions that may be performed by a first robot (see ¶0083, regarding the robot decomposes a task), collecting, by at least one sensor onboard the first robot and for each candidate action of the set of candidate actions, a respective set of sensor data (see ¶0083, regarding that the robot determines conditions, e.g., sound, light, electricity, humidity, etc., defined as being detected by sensors in ¶0053), transmitting, by the first robot, a request for instructions to a tele-operation system (see ¶0083, regarding the robot sends a request to the robot integrated control platform based on the conditions), the tele-operation system communicatively coupled to the fleet of robots (see ¶0011, ¶0017, regarding the communication modules provided for communication between the robot and the integrated control platform, where other robots can access stored actions flows and results via the robot integrated control platform, as described in ¶0065), receiving, by the first robot, the instructions from the tele-operation system (see ¶0084, regarding that instructions are sent to the robot by the robot integrated control platform), executing, by the first robot, the instructions (see ¶0085, regarding that all received instructions are executed by the robot), and updating, by the first robot, a first control model of the first robot, based at least in part on each candidate action of the set of candidate actions, each respective set of sensor data, and the instructions, to increase a level of autonomy of the first robot (see ¶0063-0064, regarding the self-learning module of the robot downloads the action flow of the robot’s task execution and the action results corresponding to the action flow stored in it from the robot integrated control platform, thereby improving the performance of the robot). While Duan et al. teaches that the robot transmits its own stored action flow for completing the task and action results corresponding to the action flow to the robot integrated control platform to be shared with other robots (see ¶0065), Duan et al. does not teach sending, by the first robot and directly to a second robot in the fleet of robots, control model update instructions based on the update made to the first control model, and updating, by the second robot, a second control model of the second robot, based at least in part on the update made to the first control model of the first robot, to increase a level of autonomy of the second robot, wherein the second control model of the second robot is different from the first control model of the first robot, as claimed. Further, Duan et al. does not teach that the request for the instructions includes each candidate action of the set of candidate actions and each respective set of sensor data, as claimed.
Additional relevant prior art identified in the PTO-892 form does not teach the claimed invention, including Boca et al. (WO 2016/172718 A1) that teaches the known technique of transmitting updated models between robot systems (see ¶0040), Rajkumar et al. (US 10,730,181 B1) that teaches the selective sharing of learned information among specific robots in a fleet based on roles (see col. 10, lines 1-8), and von Reventlow et al. (US 2023/0168670 A1) that teaches updating a database by an operator, so as to make improved evaluation of action definition candidates, reconsider segmentation of the task, or selection of action definition candidates that improve a robot’s capability of autonomously executing a task (see ¶0079).
No reasonable combination of prior art can be made to teach the claimed invention. The claimed invention would not have been obvious to one of ordinary skill in the art before the effective filing date.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee.  Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion

CITED REFERENCE SUMMARIES: Ref US20230168670A1: Claims:
A service robot system, comprising a robot, wherein the robot ( 1 ) having a drive system ( 3 ) for moving the robot ( 1 ) to target positions and at least one effector ( 4 ) for manipulating the robot's environment, wherein the service robot system comprises a processing unit ( 14 ) including a task determination unit for determining a task to be executed by the robot and configured to control the drive system ( 3 ) and the at least one effector ( 4 ) according to the task based on action definitions, wherein the processing unit ( 14 ) is configured to automatically retrieve action definition candidates from a database ( 17 ), evaluate the retrieved action definition candidates with respect to a success score indicating a likelihood that an action according to the action definition candidate contributes to successfully fulfilling the task, to execute an action according to the action definition candidate having highest probability equal to or above a predefined threshold, and to send a request for assistance via a communication interface ( 21 ) if the success score is less than a preset threshold.
the robot ( 1 ) having a drive system ( 3 ) for moving the robot ( 1 ) to target positions and at least one effector ( 4 ) for manipulating the robot's environment, wherein the service robot system comprises a processing unit ( 14 ) including a task determination unit for determining a task to be executed by the robot and configured to control the drive system ( 3 ) and the at least one effector ( 4 ) according to the task based on action definitions, wherein the processing unit ( 14 ) is configured to automatically retrieve action definition candidates from a database ( 17 ), evaluate the retrieved action definition candidates with respect to a success score indicating a likelihood that an action according to the action definition candidate contributes to successfully fulfilling the task, to execute an action according to the action definition candidate having highest probability equal to or above a predefined threshold, and to send a request for assistance via a communication interface ( 21 ) if the success score is less than a preset threshold.
The service robot system according to claim 1 , wherein the processing unit ( 14 ) is configured to determine the success score for an action definition candidate based on information on success of an operation of an action according to the respective action definition candidate in the past.
The service robot system according to claim 1 , wherein the processing unit ( 14 ) is configured to segment the task into a sequence of action definitions.
The service robot system according to claim 1 , wherein the robot ( 1 ) is configured to execute an action that has a
The service robot system according to claim 1 , wherein the processing unit ( 14 ) is configured to dynamically predict potential changes of the environment and to adapt actions, or select actions.
The service robot system according to claim 1 , further comprising an operator interface ( 18 , 19 , 20 ) for remotely controlling the robot ( 1 ) to execute an action and/or input information for enhancing the database ( 17 ) by the operator interface in response to the request.
The service robot system according to claim 6 , wherein a definition of the action for remotely controlling is stored in the database ( 17 ) as a new action definition.
The service robot system according to claim 1 , wherein the robot ( 1 ) is equipped with at least one sensor ( 11 , 12 ) for physically sensing the robot's environment, and is configured to transmit output of the sensor to the processing unit ( 14 ) and/or the operator interface ( 18 , 19 , 20 ).
The service robot system according to claim 6 , wherein the operator interface ( 18 , 19 , 20 ) comprises a XR set ( 20 ) including a headset and controls for virtually controlling the robot ( 1 ) to execute an operation before transmitting remote control signals to the robot ( 1 ) for executing an action accordingly.
A service robot comprising a drive system ( 3 ) for moving a robot base ( 2 ) to target positions, wherein an effector ( 4 ) for manipulating a robot's environment and a support structure ( 9 ) are mounted to at least one lifter ( 6 , 7 ), wherein the support structure ( 9 ) is configured to provide a temporary repository ( 10 ) arranged at or movable to a position relative to the effector ( 4 ) which is reachable by the effector ( 4 ) for any possible elevation of the lifter ( 6 ) to which the effector ( 4 ) is mounted.
The service robot according to claim 10 , wherein the robot ( 1 ) comprises at least one sensor ( 11 ) attached to the effector ( 4 ) for sensing a detail of a scene and one additional sensor ( 12 ) sensing the scene.
The service robot according to claim 10 , wherein the robot's effector ( 4 ) comprises a gripping tool and a mechanical and/or an electrical interface for mounting and controlling at least one assistance tool.
A method for improving autonomous execution of tasks of a service robot comprising following steps: sensing environment (S 1 ) of the service robot ( 1 ) with at least one sensor ( 11 , 12 ); determining objects (S 2 ) in the service robot's environment and associating a recognition certainty with the determined object determining a task (S 3 ) to be executed by the service robot ( 1 ); retrieving action definition candidates (S 5 ) from a database ( 17 ); evaluating the retrieved action definitions candidates (S 6 ) to determine a success score for being successfully executed and contributing to successfully fulfilling the task; executing an action (S 8 ) having highest success score equal to or above a preset threshold and to send a request for external assistance via a communication interface if probability of action definition with the highest success score is below the preset threshold.
sensing environment (S 1 ) of the service robot ( 1 ) with at least one sensor ( 11 , 12 );
determining objects (S 2 ) in the service robot's environment and associating a recognition certainty with the determined object
determining a task (S 3 ) to be executed by the service robot ( 1 );
retrieving action definition candidates (S 5 ) from a database ( 17 );
evaluating the retrieved action definitions candidates (S 6 ) to determine a success score for being successfully executed and contributing to successfully fulfilling the task;
executing an action (S 8 ) having highest success score equal to or above a preset threshold and to send a request for external assistance via a communication interface if probability of action definition with the highest success score is below the preset threshold.
The method according to claim 13 , wherein the success score for an action definition candidate is calculated based on information on success of an operation of an action according to the respective action definition candidate in the past.
The method according to claim 13 , wherein the task is segmented (S 4 ) into a plurality of actions to be executed sequentially and/or in parallel, and for each action definition candidates are retrieved (S 5 ).
The method according to claim 13 , wherein the actions having the highest success score equal to or above the preset threshold are executed until an action to be executed next with a success score below the preset threshold is reached and to send the request for external assistance then.
The method according to claim 13 , wherein the request includes at least one action definition candidate.
The method according to claim 13 , wherein the preset threshold for the success score is set individually for an action to be executed depending on one or more tags of one or more object involved in an action to be executed.
The method according to claim 13 , wherein the service robot ( 1 ) is controlled remotely and/or information input by an operator interface is stored in the database ( 17 ) in response to the request.
The method according to claim 19 , wherein a new action definition is created from remote operation of the service robot ( 1 ) and is added to the database ( 17 ).
The method according to claim 13 , wherein information on an entity involved in an action to be executed is read in in response to the request and added to a tag or added as additional tag for the entity.
The method according to claim 13 , wherein the task to be executed is determined (S 3 ) from an instruction received via an operator interface or determined from a current situation encountered by the service robot ( 1 ).
The method according to claim 22 , wherein determination of the task to be executed is based on an association of previously received instructions and their context information extracted from the situation in which the instruction was received.
Ref US20200150674A1: Claims:
An autonomous land vehicle comprising: a land vehicle conveyance system; a sensor system configured to capture sensor information including at least one of: surrounding environment information or vehicle subsystem information; a communication system configured to communicate with a remote human operator management system; at least one processor; and a memory storing instructions which, when executed by the at least one processor, cause the autonomous land vehicle to, autonomously: determine, based on the sensor information, to request a remote human operator, and communicate a request to the remote human operator management system for a remote human operator to assume control of the land vehicle conveyance system.
a land vehicle conveyance system;
a sensor system configured to capture sensor information including at least one of: surrounding environment information or vehicle subsystem information;
a communication system configured to communicate with a remote human operator management system;
at least one processor; and
a memory storing instructions which, when executed by the at least one processor, cause the autonomous land vehicle to, autonomously: determine, based on the sensor information, to request a remote human operator, and communicate a request to the remote human operator management system for a remote human operator to assume control of the land vehicle conveyance system.
determine, based on the sensor information, to request a remote human operator, and
communicate a request to the remote human operator management system for a remote human operator to assume control of the land vehicle conveyance system.
The autonomous land vehicle of claim 1 , wherein the request includes at least a portion of the sensor information captured by the sensor system.
The autonomous land vehicle of claim 1 , wherein in determining to request a remote human operator, the instructions, when executed by the at least one processor, cause the autonomous land vehicle to determine, based on the vehicle subsystem information, that at least one subsystem is not functioning properly.
The autonomous land vehicle of claim 1 , wherein in determining to request a remote human operator, the instructions, when executed by the at least one processor, cause the autonomous land vehicle to determine, based on the sensor information, that a next maneuver cannot be autonomously determined.
The autonomous land vehicle of claim 1 , wherein in determining to request a remote human operator, the instructions, when executed by the at least one processor, cause the autonomous land vehicle to determine, based on the surrounding environment information, that a surrounding environment situation is not recognized.
The autonomous land vehicle of claim 5 , wherein in determining to request a remote human operator, the instructions, when executed by the at least one processor, cause the autonomous land vehicle to: access information representing predetermined situations or predetermined roadways that are to require a request for a remote human operator; and determine, based on the sensor information, that at least one of the predetermined situations or the predetermined roadways has been encountered.
access information representing predetermined situations or predetermined roadways that are to require a request for a remote human operator; and
determine, based on the sensor information, that at least one of the predetermined situations or the predetermined roadways has been encountered.
The autonomous land vehicle of claim 1 , wherein in determining to request a remote human operator, the instructions, when executed by the at least one processor, cause the autonomous land vehicle to determine, based on the surrounding environment information, that a surrounding environment situation should not be autonomously handled.
The autonomous land vehicle of claim 7 , wherein in determining to request a remote human operator, the instructions, when executed by the at least one processor, cause the autonomous land vehicle to determine, based on the vehicle subsystem information, that a situation of at least one subsystem is not recognized.
The autonomous land vehicle of claim 7 , wherein in determining to request a remote human operator, the instructions, when executed by the at least one processor, cause the autonomous land vehicle to: access a predetermined operating geography; and determine, based on the surrounding environment information, that the autonomous land vehicle is outside the predetermined operating geography.
access a predetermined operating geography; and
determine, based on the surrounding environment information, that the autonomous land vehicle is outside the predetermined operating geography.
The autonomous land vehicle of claim 1 , wherein the instructions, when executed by the at least one processor, cause the autonomous land vehicle to determine a risk rating based on the sensor information.
The autonomous land vehicle of claim 10 , wherein in determining a risk rating based on the sensor information, the instructions, when executed by the at least one processor, cause the autonomous land vehicle to determine the risk rating based on estimated safety of autonomous risk mitigation procedures that can be effectuated until a remote human operator assumes control.
The autonomous land vehicle of claim 11 , wherein in determining the risk rating based on estimated safety of autonomous risk mitigation procedures, the instructions, when executed by the at least one processor, cause the autonomous land vehicle to: access the risk mitigation procedures; estimate, based on the sensor information, probability of success in executing the risk mitigation procedures to produce an estimated probability of success; and determine the risk rating based on the estimated probability of success.
access the risk mitigation procedures;
estimate, based on the sensor information, probability of success in executing the risk mitigation procedures to produce an estimated probability of success; and
determine the risk rating based on the estimated probability of success.
The autonomous land vehicle of claim 12 , wherein in estimating the probability of success in executing the risk mitigation procedures, the instructions, when executed by the at least one processor, cause the autonomous land vehicle to estimate the probability of success based on at least one of: road speed limits, current vehicle speed, current speed of surrounding vehicles, number of surrounding vehicles, number of surrounding pedestrians, number of surrounding objects, road width, weather conditions, available time to react, or proximity to surrounding vehicles.
The autonomous land vehicle of claim 10 , wherein the instructions, when executed by the at least one processor, cause the autonomous land vehicle to control the land vehicle conveyance system to mitigate risk while until a remote human operator assumes control.
The autonomous land vehicle of claim 14 , wherein in controlling the land vehicle conveyance system to mitigate risk, the instructions, when executed by the at least one processor, cause the autonomous land vehicle to perform at least one of: park at a curb, parking in a parking lot, travel on roadways in search of parking, or depart from a planned route to a roadway having lower risk.
The autonomous land vehicle of claim 14 , wherein in controlling the land vehicle conveyance system to mitigate risk, the instructions, when executed by the at least one processor, cause the autonomous land vehicle to travel at a slower speed than a travel speed without the risk.
A method of operation of an autonomous land vehicle, comprising: capturing, with a sensor system on the autonomous land vehicle, sensor information including at least one of: surrounding environment information or vehicle subsystem information; determining, based on the sensor information, to request a remote human operator; and communicating a request to a remote human operator management system for a remote human operator to assume control of a conveyance system of the autonomous land vehicle.
capturing, with a sensor system on the autonomous land vehicle, sensor information including at least one of: surrounding environment information or vehicle subsystem information;
determining, based on the sensor information, to request a remote human operator; and
communicating a request to a remote human operator management system for a remote human operator to assume control of a conveyance system of the autonomous land vehicle.
The method of claim 17 , wherein the request includes at least a portion of the sensor information captured by the sensor system.
The method of claim 17 , wherein determining to request the remote human operator includes determining, based on the surrounding environment information, that a surrounding environment situation is not recognized.
The method of claim 19 , wherein determining to request the remote human operator includes: accessing information representing predetermined situations or predetermined roadways that are to require a request for a remote human operator; and determining, based on the sensor information, that at least one of the predetermined situations or the predetermined roadways has been encountered.
accessing information representing predetermined situations or predetermined roadways that are to require a request for a remote human operator; and
determining, based on the sensor information, that at least one of the predetermined situations or the predetermined roadways has been encountered.
The method of claim 17 , wherein determining to request the remote human operator includes determining, based on the surrounding environment information, that a surrounding environment situation should not be autonomously handled.
The method of claim 21 , wherein determining to request the remote human operator includes: accessing information representing a predetermined operating geography; and determining, based on the surrounding environment information, that the autonomous land vehicle is outside the predetermined operating geography.
accessing information representing a predetermined operating geography; and
determining, based on the surrounding environment information, that the autonomous land vehicle is outside the predetermined operating geography.
The method of claim 17 , further comprising: determining a risk rating based on the sensor information.
determining a risk rating based on the sensor information.
The method of claim 23 , wherein determining the risk rating is based on estimated safety of autonomous risk mitigation procedures that can be effectuated until a remote human operator assumes control.
A non-transitory storage media encoded with instructions that, when executed by at least one processor, cause the at least one processor to perform operations including: capturing, with a sensor system on an autonomous land vehicle, sensor information including at least one of: surrounding environment information or vehicle subsystem information; determining, based on the sensor information, to request a remote human
[Truncated to input limit]