Office Action 2: Notice of Allowance (2025-03-04)

CLAIMS:
Amendments to the Claims

This listing of claims replaces all prior versions and listing of claims in this application.

Listing of Claims

OF MARCH 22, 2024

1\. (currently amended) A computer-implemented method comprising the operations of:
obtaining vehicle dimension data describing dimensions of a vehicle;
 causing, based at least in part on the vehicle dimension data, a robotic apparatus to move to a first position about the vehicle to collect data regarding a selected wheel of the vehicle;
determining a first reference point of the vehicle based at least in part on the collected data regarding the selected wheel; and
causing the robotic apparatus to move to a second position about the vehicle based on the determined first reference point.

2\. (original) The method of claim 1, wherein the determined first reference point is a center point of a first wheel of the vehicle.

3\. (currently amended) The method of claim 2, further comprising the operations of:
aligning a coordinate system of the robotic apparatus to the determined first reference point of the first wheel of the vehicle; and
OF MARCH 22, 2024
based on the aligned coordinate system of the robotic apparatus,
 removing, via the robotic apparatus, lug nuts from a first wheel hub of the vehicle .

4\. (currently amended) The method of claim 2, wherein the second position is about a second wheel of the vehicle, the method further comprising the operations of:

determining a second reference point of the second wheel of the vehicle; and
causing the robotic apparatus to move to positions about the second wheel of the vehicle based on the determined second reference point.

5\. (original) The method of claim 4, wherein the determined second reference point is a center point of the second wheel of the vehicle.

6\. (currently amended) The method of claim 5, further comprising the operations of:
aligning a coordinate system of the robotic apparatus to the determined second reference point of the second wheel; and
based on the aligned coordinate system of the robotic apparatus,
removing, via the robotic apparatus, lug nuts from a second wheel hub of the vehicle.
OF MARCH 22, 2024

7\. (currently amended) The method of claim 1, wherein obtaining the vehicle dimension information comprises:
obtaining a vehicle identification number associated with the vehicle;
accessing a database that comprises information related to a plurality of vehicles; and
obtaining \[\[from\]\] the vehicle dimension data from the database based on the vehicle identification number.

8\. (original) The method of claim 7, wherein the vehicle dimension information includes at least one of dimensions of a wheel base of the vehicle and dimensions of a track width of the vehicle.

9\. (original) The method of claim 1, wherein determining a first reference point of the vehicle comprises:
aligning a plane of a digital camera with a plane of a first wheel of the vehicle having a center point; and
moving the digital camera to a position where a center point of the camera is aligned with a center point of the first wheel.

10\. (original) The method of claim 1, wherein determining a first reference point of the first wheel of the vehicle comprises:
obtaining, via one or more cameras, digital imagery of a first wheel of the vehicle, the first wheel having a plurality of lug nuts and a wheel center;
determining, using the digital imagery, a movement offset of a then current position of the robotic apparatus;
OF MARCH 22, 2024
moving the robotic apparatus from the then current position to a new position based on the determined movement offset; and
determining whether the one or more cameras is aligned to the wheel center.

11\. (currently amended) The method of claim 10, wherein the robotic apparatus has a digital camera, and wherein determining a first reference point of the first wheel of the vehicle comprises:

determining one or more movement offset values and moving the robotic apparatus based on the one or more movement offset values until an alignment point of the robotic apparatus is aligned to a center point of the first wheel, the alignment point being a center point of the digital camera.

12\. (currently amended) The method of claim 11, wherein the movement offset comprises \[\[a\]\] coordinate values for an x, y, z, rX, and rY offset, wherein x is an x axis value, wherein y is a y axis value, wherein z is a z axis value, wherein rY is a rotation value about the y axis value, and rX is a rotation value about the x axis value.

13\. (currently amended) The method of claim 11, wherein the digital imagery comprises 2D imagery and a 3D point cloud of the first wheel, and wherein \[\[the\]\] determining, using the digital imagery, the movement offset comprises the operations of
OF MARCH 22, 2024
inputting into a machine learning model the 2D imagery and generating a plurality of bounding boxes associated with the lug nuts of the first wheel and determining a number of the lug nuts of the first wheel;
using the generated plurality of bounding boxes with the 3D point cloud to define \[\[an\]\] x, y, z coordinate information for each of the plurality of lug nuts; and
calculating an average Z distance of the lug nuts.

14\. (currently amended) A system comprising:
a robotic apparatus adapted to perform a maintenance task on a vehicle;
a processing device communicatively coupled to the robotic apparatus, the processing device comprising:a memory storing computer program instructions; and
a processor communicatively coupled to the memory, the processor configured to execute the computer program instructions which, when executed on the processor, cause the processor to perform operations comprising:
obtaining vehicle dimension data describing dimensions of a vehicle;
causing, based at least in part on the vehicle dimension data,
the robotic apparatus to move to a first position about the vehicle to collect data regarding a selected wheel of the vehicle;
OF MARCH 22, 2024
determining a first reference point of the vehicle based at least in part on the collected data regarding the selected wheel; and
causing the robotic apparatus to move to a second position about the vehicle based on the determined first reference point.

15\. (original) The system of claim 14, wherein the determined first reference point is a center point of a first wheel of the vehicle.

16\. (currently amended) The system of claim 15, the operations further comprising:
aligning a coordinate system of the robotic apparatus to the determined first reference point of the first wheel of the vehicle; and
based on the aligned coordinate system of the robotic apparatus,
causing the robotic apparatus to remove lug nuts from a first wheel hub of the vehicle.

17\. (original) The system of claim 14, the operations further comprising:
causing the robotic apparatus to move to a second position about a second wheel of the vehicle;
determining a second reference point of the second wheel of the vehicle; and
causing the robotic apparatus to move to positions about the second wheel of the vehicle based on the determined second reference point.
OF MARCH 22, 2024

18\. (original) The system of claim 17, wherein the determined second reference point is a center point of the second wheel of the vehicle.

19\. (currently amended) The system of claim 18, the operations further comprising:
aligning a coordinate system of the robotic apparatus to the determined second reference point of the second wheel; and
based on the aligned coordinate system of the robotic apparatus,
causing the robotic apparatus to remove lug nuts from a second wheel hub of the vehicle.

20\. (new) A computer-implemented method comprising the operations of:
obtaining vehicle dimension data describing dimensions of a vehicle;
causing a robotic apparatus to move to a first position about the vehicle;
determining a first reference point of the vehicle; and
causing the robotic apparatus to move to a second position about the vehicle based on the determined first reference point;
wherein determining the first reference point of the first wheel of the vehicle comprises:obtaining, via one or more cameras, digital imagery of a first wheel of the vehicle, the first wheel having a plurality of lug nuts and a wheel center;

OF MARCH 22, 2024
determining, using the digital imagery, a movement offset of a then current position of the robotic apparatus;
moving the robotic apparatus from the then current position to a new position based on the determined movement offset; and
determining whether the one or more cameras is aligned to the wheel center, wherein the robotic apparatus is moved from the then current position to the new position until an alignment point of the robotic apparatus is aligned to a center point of the first wheel, wherein the alignment point is a center point of the digital camera;
wherein the movement offset comprises coordinate values for an x, y, z, rX, and rY offset, wherein x is an x axis value, wherein y is a y axis value, wherein z is a z axis value, wherein rY is a rotation value about the y axis value, and rX is a rotation value about the x axis value.

OFFICE ACTION:
Allowable Subject Matter
Claims 1-20 are allowed in light of the amendment filed on 12/23/2024.
Notice of Pre-AIA  or AIA  Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The following is an examiner’s statement of reasons for allowance:
The prior art does not anticipate nor render obvious the claimed subject matter of:
A computer-implemented method comprising the operations of: obtaining vehicle dimension data describing dimensions of a vehicle; causing, based at least in part on the vehicle dimension data, a robotic apparatus to move to a first position about the vehicle to collect data regarding a selected wheel of the vehicle; determining a first reference point of the vehicle based at least in part on the collected data regarding the selected wheel; and causing the robotic apparatus to move to a second position about the vehicle based on the determined first reference point.
No particular single feature of the claim renders the claim as a whole patentable. Only the claim taken as a whole combination is deemed new and unobvious.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee.  Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion

APPLICANT REBUTTAL: REMARKS
This response is in reply to the Non-Final Office Action dated March 22, 2024 (hereinafter the "Action"). Reconsideration is respectfully requested.
Status of Claims
Claims 1-19 were pending. Claims 1, 3, 4, 6-7, 11-14, 16, and 19 are amended herein. Claim 20 is new. Support for the claim amendments and new claim can be found throughout the specification and drawings and at least at paragraphs [0240] to [0250].
Applicant would like to thank the Examiner for indicating claim 12 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. Claims 1-11 and 13-19 are rejected under 35 U.S.C. 103 as being unpatentable over US10,354,460 to Ricci (hereinafter "Ricci").
The Traversal
Applicant traverses the rejections, contending that the cited references fail to meet the requirements necessary to establish a prima facie case of obviousness.
Claim Rejections
Claim 1
Amended claim 1 recites:
REPLY TO NON-FINAL OFFICE ACTION
A computer-implemented method comprising the
operations of:
obtaining vehicle dimension data describing dimensions of
a vehicle;
causing, based at least in part on the vehicle dimension
data, a robotic apparatus to move to a first position about the
vehicle to collect data regarding a selected wheel of the vehicle;
determining a first reference point of the vehicle based at
least in part on the collected data regarding the selected wheel;
and
causing the robotic apparatus to move to a second position
about the vehicle based on the determined first reference point.
Applicant respectfully asserts Ricci fails to disclose the claimed operations.
First, the Office Action alleges Ricci teaches "obtaining a vehicle
dimension data describing dimensions of a vehicle" citing "Fig. 2, 4A, 4B and 6-12 element 100." Applicant is unclear how the Action is suggesting the cited figures and element 100 teach this step, because the figures merely show a diagram (not a method) including a vehicle. Applicant respectfully submits the figures do not show "obtaining vehicle dimension data describing dimensions of a vehicle." Further, the specification is completely silent with respect to obtaining vehicle dimension data describing dimensions of a vehicle. It is clear the reference does not explicitly disclose "obtaining vehicle dimension data describing dimensions of a vehicle." Further this step of the operation is not inherent simply by obtaining a vehicle. Therefore, Ricci fails to disclose "obtaining vehicle dimension data describing dimensions of a vehicle."
REPLY TO NON-FINAL OFFICE ACTION
Ricci also fails to disclose "causing, based at least in part on the
vehicle dimension data, a robotic apparatus to move to a first position about the vehicle to collect data regarding a selected wheel of the vehicle." The Office Action alleges Ricci teaches moving a robotic apparatus to a first position about the vehicle. (Action at 5). However, the alleged robotic apparatus, robotic unit 700 in Ricci, does not move based on vehicle dimension data and the robotic unit 700 does not move to a first position about the vehicle to collect data regarding a selected wheel of the vehicle. The only movement of the robotic unit 700 in Ricci is to "manoeuver charging plate 520 relative to charging panel 608 of vehicle 100." (Ricci at 7:1-2). Therefore, Ricci also fails to disclose "causing, based at least in part on the vehicle dimension data, a robotic apparatus to move to a first position about the vehicle to collect data regarding a selected wheel of the vehicle."
Ricci fails to disclose "determining a first reference point of the vehicle based at least in part on the collected data regarding the selected wheel." The robotic unit 700 in Ricci does not determine a first reference point of the vehicle based at least in part on the collected data regarding the selected wheel. Instead, the robotic unit 700 maneuvers the charging plate 520 to a position relative to the charging panel 608. Thus, the alleged first reference point in Ricci is the charging panel 608, which is not determined on collected data regarding the selected wheel. Therefore, Ricci also fails to disclose "determining a first reference point of the vehicle based at least in part on the collected data regarding the selected wheel."
Ricci fails to disclose "causing the robotic apparatus to move to a
second position about the vehicle based on the determined first reference
REPLY TO NON-FINAL OFFICE ACTION
point." As there is no first reference point based at least in part on the collect, the robotic unit 700 in Ricci cannot teach moving the robotic unit 700 to a second point based on the first reference. Therefore, Ricci does not disclose "causing the robotic apparatus to move to a second position about the vehicle based on the determined first reference point."
The absence of these elements from Ricci marks a difference between the prior art and the claimed invention. To support a rejection on the basis of obviousness the Examiner is required to "explain why the differences between the prior art and the claimed invention would have been obvious to one of ordinary skill in the art" (MPEP, section 2141, III) as well as provide "articulated reasoning with some rational underpinning to support the legal conclusion of obviousness" (KSR International Co. v. Teleflex Inc., 550 U.S. 398, 82 USPQ2d 1385, 1396 (2007)). Applicant contends that the Examiner has not addressed the aforementioned difference between Ricci and the claimed invention, i.e., the Examiner has not explained why the difference between the prior art and the claimed invention would have been obvious to one of ordinary skill in the art, nor has the Examiner provided articulated reasoning with some rational underpinning to support the legal conclusion of obviousness. Therefore, the requirements to establish a prima facie case of obviousness for claim 1 have not been met. Applicant respectfully requests that the rejection of claim 1 be withdrawn.
Applicant respectfully asserts many of the citations in the rejection to Ricci only include figures and element reference numbers. However, the cited figures do not illustrate a method and the element reference numbers are not to a step of a method. Without further explanation, it is unclear to the Applicant how the cited figures and element disclose the claimed steps.
REPLY TO NON-FINAL OFFICE ACTION
If the rejection is maintained, the Applicant respectfully request clarification as to how the cited figures and elements teach the claimed method.
Claim 14
Although not identical, claim 14 includes similar features to those recited in claim 1. Claim 14 comprises a processor configured to perform operations comprising:
obtaining  vehicle  dimension  data  describing
dimensions of a vehicle;
causing, based at least in part on the vehicle
dimension data, the robotic apparatus to move to a first position about the vehicle to collect data regarding a selected wheel of the
vehicle;
determining a first reference point of the vehicle
based at least in part on the collected data regarding the selected
wheel; and
causing the robotic apparatus to move to a second
position about the vehicle based on the determined first reference
point.
Therefore, claim 14 is patentable over Ricci for at least the same reasons as claim 1. Applicant respectfully requests the withdrawal of the rejection of claim 14 as unpatentable over Ricci.
Claim 2-11 and 13-19
Claims 2-11, 13, 15-19 are rejected as obvious over Ricci. However, these claims depend from claim 1 or claim 14, and thus include all of the recitations of claim 1 or claim 14. Therefore, claims 2-11, 13, and 15-19 are
REPLY TO NON-FINAL OFFICE ACTION
patentable over Ricci for at least the same reasons as claim 1 and claim 14. It is well established that if an independent claim is nonobvious under 35 USC 103, then any claim depending therefrom is nonobvious. In re Fine, 5 USPQ2d, 1596 (Fed. Cir. 1988). Applicant requests that the rejection of claims 2-11, 13, and 15-19 as unpatentable over Ricci be withdrawn.
New Claim 21
New claim 21 includes the subject matter of originally filed claim 1 and the allowable subject matter of claim 12 along with the subject matter of intervening claims 10 and 11. Therefore, Applicant contends new claim 21 is allowable over the cited references.
REPLY TO NON-FINAL OFFICE ACTION
Summary
Applicant has shown that the cited references fail to meet the standard for obviousness. Applicant contends that the claims, as amended, are therefore allowable over the cited references and requests that all rejections be withdrawn. Applicant further requests the application passed to issue. An early and favorable decision is earnestly solicited.
Respectfully submitted, Ballard Spahr LLP


CITED REFERENCE SUMMARIES: Ref US20240219191A1: Claims:
A method comprising: collecting sensor data from one or more in-vehicle sensors of a vehicle; processing the sensor data to determine one or more contextual parameter signals associated with the vehicle, one or more passengers of the vehicle, or a combination thereof; determining a context associated with a trip engaged by the vehicle or the one or more passengers based on the one or more contextual parameter signals; determining a routing cost factor based on the context; determining a navigation route, navigation guidance information, or a combination thereof based on the routing cost factor; and providing the navigation route, the navigation guidance information, or a combination thereof as an output.
collecting sensor data from one or more in-vehicle sensors of a vehicle;
processing the sensor data to determine one or more contextual parameter signals associated with the vehicle, one or more passengers of the vehicle, or a combination thereof;
determining a context associated with a trip engaged by the vehicle or the one or more passengers based on the one or more contextual parameter signals;
determining a routing cost factor based on the context;
determining a navigation route, navigation guidance information, or a combination thereof based on the routing cost factor; and
providing the navigation route, the navigation guidance information, or a combination thereof as an output.
The method of claim 1 , wherein the routing cost factor includes an efficiency factor, a resilience factor, a reliability factor, a simplicity of route factor, a safety factor, a point of interest discovery factor, or a combination thereof.
The method of claim 2 , wherein the efficiency factor relates to a power or fuel consumption of the vehicle, a carbon footprint of the vehicle, or a combination thereof.
The method of claim 2 , wherein the resilience factor relates to an availability of one or more back-up routes in case of traffic congestion or an accident.
The method of claim 2 , wherein the reliability factor relates to an ability to reach a destination via a selected route given a current power or fuel level of the vehicle.
The method of claim 2 , wherein the simplicity of route factor relates to a number of road type changes, complex maneuvers, routing decision points, or a combination thereof.
The method of claim 1 , wherein the safety factor relates to a number of vulnerable road users expected to be encountered.
The method of claim 1 , wherein the point of interest discovery factor relates to a number of points of interest expected to be encountered, and wherein the points of interest expected to be encountered relates to at least one designated interest of the one or more passengers.
The method of claim 1 , wherein the one or more contextual parameter signals include a number of the one or more passengers, a weight of the one or more passengers, an age of the one or more passengers, a time of day, or a combination thereof.
The method of claim 1 , further comprising: determining a previous destination, a previous context, or a combination associated with the one or more contextual parameter signals, wherein the routing cost factor is further based on the previous destination, the previous context, or a combination thereof.
determining a previous destination, a previous context, or a combination associated with the one or more contextual parameter signals,
wherein the routing cost factor is further based on the previous destination, the previous context, or a combination thereof.
The method of claim 1 , wherein the navigation route, the navigation guidance information, or a combination thereof is determined further based on one or more multi-task stop options.
An apparatus comprising: at least one processor; and at least one memory including computer program code for one or more programs, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following, collect sensor data from one or more in-vehicle sensors of a vehicle; process the sensor data to determine one or more contextual parameter signals associated with the vehicle, one or more passengers of the vehicle, or a combination thereof; determine a context associated with a trip engaged by the vehicle or the one or more passengers based on the one or more contextual parameter signals; determine a routing cost factor based on the context; determine a navigation route, navigation guidance information, or a combination thereof based on the routing cost factor; and provide the navigation route, the navigation guidance information, or a combination thereof as an output.
at least one processor; and
at least one memory including computer program code for one or more programs,
the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following, collect sensor data from one or more in-vehicle sensors of a vehicle; process the sensor data to determine one or more contextual parameter signals associated with the vehicle, one or more passengers of the vehicle, or a combination thereof; determine a context associated with a trip engaged by the vehicle or the one or more passengers based on the one or more contextual parameter signals; determine a routing cost factor based on the context; determine a navigation route, navigation guidance information, or a combination thereof based on the routing cost factor; and provide the navigation route, the navigation guidance information, or a combination thereof as an output.
collect sensor data from one or more in-vehicle sensors of a vehicle;
process the sensor data to determine one or more contextual parameter signals associated with the vehicle, one or more passengers of the vehicle, or a combination thereof;
determine a context associated with a trip engaged by the vehicle or the one or more passengers based on the one or more contextual parameter signals;
determine a routing cost factor based on the context;
determine a navigation route, navigation guidance information, or a combination thereof based on the routing cost factor; and
provide the navigation route, the navigation guidance information, or a combination thereof as an output.
The apparatus of claim 12 , wherein the routing cost factor includes an efficiency factor, a resilience factor, a reliability factor, a simplicity of route factor, a safety factor, a point of interest discovery factor, or a combination thereof.
The apparatus of claim 12 , wherein the one or more contextual parameter signals include a number of the one or more passengers, a weight of the one or more passengers, an age of the one or more passengers, a time of day, or a combination thereof.
The apparatus of claim 12 , wherein the apparatus is further caused to: determine a previous destination, a previous context, or a combination associated with the one or more contextual parameter signals, wherein the routing cost factor is further based on the previous destination, the previous context, or a combination thereof.
determine a previous destination, a previous context, or a combination associated with the one or more contextual parameter signals,
The apparatus of claim 12 , wherein the navigation route, the navigation guidance information, or a combination thereof is determined further based on one or more multi-task stop options.
A non-transitory computer-readable storage medium carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to perform: collecting sensor data from one or more in-vehicle sensors of a vehicle; processing the sensor data to determine one or more contextual parameter signals associated with the vehicle, one or more passengers of the vehicle, or a combination thereof; determining a context associated with a trip engaged by the vehicle or the one or more passengers based on the one or more contextual parameter signals; determining a routing cost factor based on the context; determining a navigation route, navigation guidance information, or a combination thereof based on the routing cost factor; and providing the navigation route, the navigation guidance information, or a combination thereof as an output.
The non-transitory computer-readable storage medium of claim 17 , wherein the routing cost factor includes an efficiency factor, a resilience factor, a reliability factor, a simplicity of route factor, a safety factor, a point of interest discovery factor, or a combination thereof.
The non-transitory computer-readable storage medium of claim 17 , wherein the one or more contextual parameter signals include a number of the one or more passengers, a weight of the one or more passengers, an age of the one or more passengers, a time of day, or a combination thereof.
The apparatus of claim 17 , wherein the apparatus is caused to further perform: determining a previous destination, a previous context, or a combination associated with the one or more contextual parameter signals, wherein the routing cost factor is further based on the previous destination, the previous context, or a combination thereof.
Ref US20230137789A1: Claims:
A system for locating a center of a lug nut comprising: one or more processors; and a non-transitory computer-readable medium storing a plurality of instructions, which when executed, cause the one or more processors to: obtain a two-dimensional image of a vehicle wheel, the vehicle wheel including a set of lug nuts; determine an initial center point of at least a first lug nut by processing the two-dimensional image using a first machine learning model; obtain a three-dimensional image of at least a portion of the vehicle wheel including the first lug nut; locate a set of edges at various points around a perimeter of the first lug nut by processing the three-dimensional image using a second machine learning model; determine a set of distances between a point within the three-dimensional image that corresponds to the initial center point and the located set of edges; and update the initial center point of the first lug nut based on the determined set of distances.
one or more processors; and
a non-transitory computer-readable medium storing a plurality of instructions, which when executed, cause the one or more processors to:
obtain a two-dimensional image of a vehicle wheel, the vehicle wheel including a set of lug nuts;
determine an initial center point of at least a first lug nut by processing the two-dimensional image using a first machine learning model;
obtain a three-dimensional image of at least a portion of the vehicle wheel including the first lug nut;
locate a set of edges at various points around a perimeter of the first lug nut by processing the three-dimensional image using a second machine learning model;
determine a set of distances between a point within the three-dimensional image that corresponds to the initial center point and the located set of edges; and
update the initial center point of the first lug nut based on the determined set of distances.
The system of claim 1 , wherein updating the initial center point of the first lug nut based on the determined set of distances includes: determining an average distance of the determined set of distances; and updating the center point such that a distance from the updated center point to each of the set of edges corresponds to the determined average distance.
determining an average distance of the determined set of distances; and
updating the center point such that a distance from the updated center point to each of the set of edges corresponds to the determined average distance.
The system of claim 1 , wherein locating the set of edges includes: determining a confidence score for each edge of the set of edges; and retaining only those edges satisfying a predetermined confidence score as part of the located set of edges.
determining a confidence score for each edge of the set of edges; and
retaining only those edges satisfying a predetermined confidence score as part of the located set of edges.
The system of claim 1 , wherein locating the set of edges includes: determining a change of depth between a plurality of points proximate to the various points around the perimeter of the first lug nut; and locating the set of edges at the various points around the perimeter of the first lug nut based on the determined change of depth.
determining a change of depth between a plurality of points proximate to the various points around the perimeter of the first lug nut; and
locating the set of edges at the various points around the perimeter of the first lug nut based on the determined change of depth.
The system of claim 1 , wherein the plurality of instructions, when executed, further cause the one or more processors to: map a set of two-dimensional coordinates representing the initial center point to a set of three-dimensional coordinates representing the point within the three-dimensional image that corresponds to the initial center point.
map a set of two-dimensional coordinates representing the initial center point to a set of three-dimensional coordinates representing the point within the three-dimensional image that corresponds to the initial center point.
The system of claim 1 , wherein the plurality of instructions, when executed, further cause the one or more processors to: direct a robotic apparatus to maneuver to a position based on the updated center point of the first lug nut as part of a sequence of operations to remove the first lug nut from the vehicle wheel.
direct a robotic apparatus to maneuver to a position based on the updated center point of the first lug nut as part of a sequence of operations to remove the first lug nut from the vehicle wheel.
The system of claim 1 , wherein the plurality of instructions, when executed, further cause the one or more processors to: determine one or more dimensions of the first lug nut based on the updated center point and the determined set of distances.
determine one or more dimensions of the first lug nut based on the updated center point and the determined set of distances.
The system of claim 7 , wherein the plurality of instructions, when executed, further cause the one or more processors to: determine a lug nut size based on the determined one or more dimensions of the first lug nut; and direct a robotic apparatus to select a socket corresponding the determined lug nut size as part of a sequence of operations to remove the first lug nut from the vehicle wheel.
determine a lug nut size based on the determined one or more dimensions of the first lug nut; and
direct a robotic apparatus to select a socket corresponding the determined lug nut size as part of a sequence of operations to remove the first lug nut from the vehicle wheel.
A computer-implemented method for locating a center of a lug nut comprising: obtaining a two-dimensional image of a vehicle wheel, the vehicle wheel including a set of lug nuts; determining an initial center point of at least a first lug nut by processing the two-dimensional image using a first machine learning model; obtaining a three-dimensional image of at least a portion of the vehicle wheel including the first lug nut; locating a set of edges at various points around a perimeter of the first lug nut by processing the three-dimensional image using a second machine learning model; determining a set of distances between a point within the three-dimensional image that corresponds to the initial center point and the located set of edges; and updating the initial center point of the first lug nut based on the determined set of distances.
obtaining a two-dimensional image of a vehicle wheel, the vehicle wheel including a set of lug nuts;
determining an initial center point of at least a first lug nut by processing the two-dimensional image using a first machine learning model;
obtaining a three-dimensional image of at least a portion of the vehicle wheel including the first lug nut;
locating a set of edges at various points around a perimeter of the first lug nut by processing the three-dimensional image using a second machine learning model;
determining a set of distances between a point within the three-dimensional image that corresponds to the initial center point and the located set of edges; and
updating the initial center point of the first lug nut based on the determined set of distances.
The method of claim 9 , wherein updating the initial center point of the first lug nut based on the determined set of distances includes: determining an average distance of the determined set of distances; and updating the center point such that a distance from the updated center point to each of the set of edges corresponds to the determined average distance.
The method of claim 9 , wherein locating the set of edges includes: determining a confidence score for each edge of the set of edges; and retaining only those edges satisfying a predetermined confidence score as part of the located set of edges.
The method of claim 9 , wherein locating the set of edges includes: determining a change of depth between a plurality of points proximate to the various points around the perimeter of the first lug nut; and locating the set of edges at the various points around the perimeter of the first lug nut based on the determined change of depth.
The method of claim 9 , further comprising: mapping a set of two-dimensional coordinates representing the initial center point to a set of three-dimensional coordinates representing the point within the three-dimensional image that corresponds to the initial center point.
mapping a set of two-dimensional coordinates representing the initial center point to a set of three-dimensional coordinates representing the point within the three-dimensional image that corresponds to the initial center point.
The method of claim 9 , further comprising: directing a robotic apparatus to maneuver to a position based on the updated center point of the first lug nut as part of a sequence of operations to remove the first lug nut from the vehicle wheel.
directing a robotic apparatus to maneuver to a position based on the updated center point of the first lug nut as part of a sequence of operations to remove the first lug nut from the vehicle wheel.
A non-transitory computer-readable medium storing instructions which, when executed by one or more processors of a system, cause the system to perform operations comprising: obtaining a two-dimensional image of a vehicle wheel, the vehicle wheel including a set of lug nuts; determining an initial center point of at least a first lug nut by processing the two-dimensional image using a first machine learning model; obtaining a three-dimensional image of at least a portion of the vehicle wheel including the first lug nut; locating a set of edges at various points around a perimeter of the first lug nut by processing the three-dimensional image using a second machine learning model; determining a set of distances between a point within the three-dimensional image that corresponds to the initial center point and the located set of edges; and updating the initial center point of the first lug nut based on the determined set of distances.
The non-transitory computer-readable medium of claim 15 , wherein updating the initial center point of the first lug nut based on the determined set of distances includes: determining an average distance of the determined set of distances; and updating the center point such that a distance from the updated center point to each of the set of edges corresponds to the determined average distance.
The non-transitory computer-readable medium of claim 15 , wherein locating the set of edges includes: determining a confidence score for each edge of the set of edges; and retaining only thos
[Truncated to input limit]