Office Action 2: Notice of Allowance (2025-02-20)

CLAIMS:
AMENDMENTS TO THE CLAIMS

Please amend the claims as follows:

1\. (Currently Amended) A system for sensing and control of a vehicle, comprising:
one or more first sensors configured to sense an interior of the vehicle, wherein the one or more first sensors are configured to generate first sensor data based on sensing the interior of the vehicle;
one or more second sensors configured to sense an exterior of the vehicle, wherein the one or more second sensors are configured to generate second sensor data based on sensing the exterior of the vehicle;
a computing device, wherein the computing device is configured to detect a user indication of interest, wherein the computing device is configured to detect the user indication of interest based at least in part on a result of providing the first sensor data as input to an artificial neural network, wherein the computing device is configured to determine an item of interest based at least in part on the detected user indication of interest and the second sensor data;
a vehicle controller, wherein the vehicle controller is configured to receive information about the item of interest, wherein the vehicle controller is configured to control operation of the vehicle based on the information about the item of interest,
wherein the user indication of interest is a gesture,
wherein the computing device is configured to determine the item of interest based at least in part on calculation of a pointing vector,
wherein the computing device is configured to calculate the pointing vector based at least in part on the first sensor data,
wherein the computing device is configured to calculate the pointing vector based at least in part on a plurality of landmarks in the first sensor data, and
wherein the computing device is configured to calculate the pointing vector based at least in part on calculating a best-fit vector for the plurality of landmarks.

2\. (Canceled)
3\. (Canceled)
4\. (Canceled)
5\. (Canceled)
6\. (Currently Amended) The system of claim 1 \[\[5\]\], wherein the plurality of landmarks includes an indication of the location of an index finger of an occupant of the vehicle.
7\. (Original) The system of claim 6, wherein the plurality of landmarks includes an indication of the location of a wrist of an occupant of the vehicle.
8\. (Canceled)
9\. (Currently Amended) The system of claim 1 \[\[8\]\], wherein the computing device is configured to determine the item of interest based at least in part on a geographic location of the vehicle at the time the user indication of interest occurred.

10\. (Currently Amended) The system of claim 1 \[\[4\]\], wherein the computing device is configured to calculate the pointing vector based at least in part on the first sensor data that includes time of flight sensor data.

11\. (Currently Amended) The system of claim 1 \[\[4\]\], wherein the computing device is configured to calculate the pointing vector based at least in part on the first sensor data that includes image sensor data.

12\. (Original) The system of claim 1, wherein the user indication of interest is the utterance of a predefined trigger word.

13\. (Original) The system of claim 12, wherein the computing device is configured to determine the item of interest based at least in part on calculation of a gazing vector.

14\. (Original) The system of claim 13, wherein the computing device is configured to calculate the gazing vector based at least in part on the first sensor data.

15\. (Original) The system of claim 14, wherein the first sensor data includes at least one image that captures the orientation of the head of an occupant of the vehicle.

16\. (Original) The system of claim 15, wherein the computing device is configured to calculate the gazing vector based at least in part on the orientation of the head of the occupant of the vehicle.

17\. (Original) The system of claim 16, wherein the computing device is configured to calculate the gazing vector based at least in part on determining a roll value of the orientation of the head of the occupant of the vehicle, wherein the computing device is configured to calculate the gazing vector based at least in part on determining a pitch value of the orientation of the head of the occupant of the vehicle, wherein the computing device is configured to calculate the gazing vector based at least in part on determining a yaw value of the orientation of the head of the occupant of the vehicle.

18\. (Original) The system of claim 17, wherein the computing device is configured to calculate the item of interest based at least in part on a geographic location of the vehicle at the time the user indication of interest occurred.

19\. (Original) The system of claim 14, wherein the computing device is configured to calculate the gazing vector based at least in part on the first sensor data that includes image data.

20\. (Original) The system of claim 19, wherein the first sensor data includes audio sensor data.

OFFICE ACTION:
DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA  35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art.  The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph:
(A)	the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B)	the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C)	the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Such claim limitation(s) is/are:
“computing device” in claim 1, 9-11, 13-14, and 16-19
“vehicle controller” in claim 1
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
The following are the interpreted corresponding structures found within the specification for some of the above limitations:
“computing device” – includes a transceiver 1102, a coupling 1104, a memory 1106, a processor 1108, a power input 1110, and a data store 1112 ; paragraph [208]
“vehicle controller” – includes a transceiver 1202, a coupling 1204, a memory 1206, a processor 1208, a power input 1210, and a data store 1212 ; paragraph [0216]
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, applicant may:  (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph.
Allowable Subject Matter
Claims 1, 6-7 and 9-20 are allowed.
The following is an examiner’s statement of reasons for allowance:
For independent claim 1, the prior art on record does not teach, describe and/or suggest all the limitations as presented in the claim as a whole and specifically “...a vehicle controller, wherein the vehicle controller is configured to receive information about the item of interest, wherein the vehicle controller is configured to control operation of the vehicle based on the information about the item of interest, wherein the user indication of interest is a gesture, wherein the computing device is configured to determine the item of interest based at least in part on calculation of a pointing vector, wherein the computing device is configured to calculate the pointing vector based at least in part on the first sensor data, wherein the computing device is configured to calculate the pointing vector based at least in part on a plurality of landmarks in the first sensor data, and wherein the computing device is configured to calculate the pointing vector based at least in part on calculating a best-fit vector for the plurality of landmarks.”
The dependent claims are allowable at least based on their dependence from the now allowable independent claims.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion

APPLICANT REBUTTAL: REMARKS
Prior to the present Response, claims 1-20 were pending in the present application. The present amendments: cancel claims 2-5 and 8; amend claims 1, 6, and 9-11. After entry of the present amendments, claims 1, 6-7, and 9-20 will be pending in the present application.
Reconsideration and allowance of the application are respectfully requested.
Claim Objections and Allowable Subject Matter
The Office Action indicates that claims 8, 10, 17, and 18 are objected to as being
dependent on a rejected base claim, but would be allowable if rewritten in independent form. (Office Action, p. 14.) The present amendments incorporate the subject matter of claim 8, along with the intervening dependent claims 2-5, into independent claim 1. As such, it is believe that the claims are in condition for allowance, and reconsideration is respectfully requested.
Re
jections under 35 U.S.C.
Claims 1, 2, 12-16, and 19 are rejected under 35 U.S.C. @ 103 as being unpatentable over U.S. Patent Application Pub. No. 20 15/0362988 ("Yamamoto") in view of U.S. Patent No. 10,913,463 ("Herz"). (Office Action, pp. 2-3.)
Claims 3-7, 9, and 11 are rejected under 35 U.S.C. @ 103 as being unpatentable over Yamamoto, Herz, and U.S. Patent Application Pub. No. 20 10/0274480 ("McCall").
subject matter. As such, reconsideration and withdrawal of these rejections arerequested.
Conclusion


CITED REFERENCE SUMMARIES: Ref US20150362988A1: Claims:
An apparatus, comprising: at least one sensor device configured to sense at least one user indication, wherein the at least one user indication comprises at least one of an indication gesture and an audible indication; at least one processing device communicatively coupled to the at least one sensor device, the at least one processing device configured to: determine at least one occurrence of at least one first user indication; determine a plurality of Points of Interest (POIs) corresponding to the at least one first user indication; determine at least one occurrence of at least one second user indication in response to determining the plurality of POIs; determine at least one narrowed POI from the plurality of POIs in response to determining the at least one occurrence of the at least one second user indication; determine at least one action corresponding to the at least one narrowed POI and at least one of the first and second user indications; and effect performance of the at least one action.
at least one sensor device configured to sense at least one user indication, wherein the at least one user indication comprises at least one of an indication gesture and an audible indication;
at least one processing device communicatively coupled to the at least one sensor device, the at least one processing device configured to: determine at least one occurrence of at least one first user indication; determine a plurality of Points of Interest (POIs) corresponding to the at least one first user indication; determine at least one occurrence of at least one second user indication in response to determining the plurality of POIs; determine at least one narrowed POI from the plurality of POIs in response to determining the at least one occurrence of the at least one second user indication; determine at least one action corresponding to the at least one narrowed POI and at least one of the first and second user indications; and effect performance of the at least one action.
determine at least one occurrence of at least one first user indication;
determine a plurality of Points of Interest (POIs) corresponding to the at least one first user indication;
determine at least one occurrence of at least one second user indication in response to determining the plurality of POIs;
determine at least one narrowed POI from the plurality of POIs in response to determining the at least one occurrence of the at least one second user indication;
determine at least one action corresponding to the at least one narrowed POI and at least one of the first and second user indications; and
effect performance of the at least one action.
The apparatus of claim 1 , wherein the at least one processing device is further configured to effect presentation of the plurality of POIs to a user in response to determining the plurality of POIs, wherein effecting presentation further comprises at least one of effecting a visual representation of the plurality of POIs on a display device and effecting an audible representation of the plurality of POIs.
The apparatus of claim 2 , wherein the at least one processing device is further configured to enable a user to interact with the presentation of the plurality of POIs with the at least one second user indication.
The apparatus of claim 1 , wherein the at least one first user indication comprises at least one indication gesture, and wherein the at least one second user indication comprises at least one audible indication.
The apparatus of claim 1 , wherein the processing device is further configured to determine the plurality of POIs using, at least in part, at least one of environmental data, a time of day, a present location, a known location, information about one or more present users, a user setting, and an historical record of POIs.
The apparatus of claim 1 , wherein the at least one first user indication comprises a coarse user indication and wherein the at least one second user indication comprises a fine indication.
The apparatus of claim 1 , wherein the at least one first user indication comprises both an indication gesture and an audible indication.
The apparatus of claim 1 , wherein the at least one processing device is configured to effect performance of the at least one action further comprises the at least one processing device being configured to at least one of: enable saving a location of the at least one narrowed POI; enable provision of information pertaining to the at least one narrowed POI; enable provision of information pertaining to a present location of a contact in relation to the at least one narrowed POI; identify at least one narrowed POI as a hazard; enable at least one of modification and initiation of a navigation route; and enable operation of at least one barrier operator.
enable saving a location of the at least one narrowed POI;
enable provision of information pertaining to the at least one narrowed POI;
enable provision of information pertaining to a present location of a contact in relation to the at least one narrowed POI;
identify at least one narrowed POI as a hazard;
enable at least one of modification and initiation of a navigation route; and
enable operation of at least one barrier operator.
The apparatus of claim 1 , wherein the at least one processing device is configured to: determine the plurality of POIs by using, at least in part, at least one of location data, compass data, steering wheel angle data, speed data, and CAN bus data.
determine the plurality of POIs by using, at least in part, at least one of location data, compass data, steering wheel angle data, speed data, and CAN bus data.
The apparatus of claim 1 , wherein the at least one processing device is configured to restrict the plurality of POIs to POIs that are located within a geographical region having an outer border.
The apparatus of claim 10 , wherein the at least one processing device is configured to: compare a number of the plurality of POIs to a lower threshold and an upper threshold; and when the number of the plurality POIs is either greater than the upper threshold or lower than the lower threshold, modify the outer border.
compare a number of the plurality of POIs to a lower threshold and an upper threshold; and
when the number of the plurality POIs is either greater than the upper threshold or lower than the lower threshold, modify the outer border.
The apparatus of claim 11 , wherein the at least one processing device is configured to: when the number of the plurality of POIs is less than the lower threshold, enlarge the outer border; and when the number of the plurality of POIs is greater than the upper threshold, contract the outer border.
when the number of the plurality of POIs is less than the lower threshold, enlarge the outer border; and
when the number of the plurality of POIs is greater than the upper threshold, contract the outer border.
The apparatus of claim 12 , wherein the at least one processing device is configured to: determine a default outer border based on, at least in part, a user profile, a current location, a type of search, a threshold value, a border shape, a type of current road, and a time of day.
determine a default outer border based on, at least in part, a user profile, a current location, a type of search, a threshold value, a border shape, a type of current road, and a time of day.
A method, comprising: sensing at least one occurrence of at least one first user indication; determining a plurality of Points of Interest (POIs) corresponding to the at least one first user indication; sensing at least one occurrence of at least one second user indication in response to determining the plurality of POIs; determining at least one narrowed POI from the plurality of POIs in response to determining the at least one occurrence of the at least one second user indication; determining at least one action corresponding to the at least one narrowed POI and at least one of the first and second user indications; and performing the at least one action.
sensing at least one occurrence of at least one first user indication;
determining a plurality of Points of Interest (POIs) corresponding to the at least one first user indication;
sensing at least one occurrence of at least one second user indication in response to determining the plurality of POIs;
determining at least one narrowed POI from the plurality of POIs in response to determining the at least one occurrence of the at least one second user indication;
determining at least one action corresponding to the at least one narrowed POI and at least one of the first and second user indications; and
performing the at least one action.
The method of claim 14 , including presenting the plurality of POIs to a user in response to determining the plurality of POIs using at least one of a visual representation of the plurality of POIs on a display device and an audible representation of the plurality of POIs.
The method of claim 15 , including enabling a user to interact with the presentation of the plurality of POIs with the at least one second user indication.
The method of claim 14 , including restricting the plurality of POIs to POIs that are located within a geographical region having an outer border.
The method of claim 17 , including: comparing a number of the plurality of POIs to a lower threshold and an upper threshold; and when the number of the plurality POIs is either greater than the upper threshold or lower than the lower threshold, modifying the outer border.
comparing a number of the plurality of POIs to a lower threshold and an upper threshold; and
when the number of the plurality POIs is either greater than the upper threshold or lower than the lower threshold, modifying the outer border.
The method of claim 18 , including: when the number of the plurality of POIs is less than the lower threshold, enlarging the outer border; and when the number of the plurality of POIs is greater than the upper threshold, contracting the outer border.
when the number of the plurality of POIs is less than the lower threshold, enlarging the outer border; and
when the number of the plurality of POIs is greater than the upper threshold, contracting the outer border.
The method of claim 19 , including determining a default outer border based on, at least in part, a user profile, a current location, a type of search, a threshold value, a border shape, a type of current road, and a time of day.
The method of claim 14 , wherein the at least one first user indication comprises a coarse user indication and wherein the at least one second user indication comprises a fine indication.
A method, comprising: sensing at least one occurrence of at least one first coarse user indication; determining a plurality of Points of Interest (POIs) corresponding to the at least one first coarse user indication; sensing at least one occurrence of at least one second fine user indication in response to determining the plurality of POIs; determining at least one narrowed POI from the plurality of POIs in response to determining the at least one occurrence of the at least one second fine user indication; performing at least one action corresponding to the at least one narrowed POI.
sensing at least one occurrence of at least one first coarse user indication;
determining a plurality of Points of Interest (POIs) corresponding to the at least one first coarse user indication;
sensing at least one occurrence of at least one second fine user indication in response to determining the plurality of POIs;
determining at least one narrowed POI from the plurality of POIs in response to determining the at least one occurrence of the at least one second fine user indication;
performing at least one action corresponding to the at least one narrowed POI.
Ref US20100274480A1: Claims:
An information system for providing point of interest information to a user in a vehicle, the system comprising: a gesture capture device configured to capture data associated with a user gesture, the user gesture having a direction indicating a desired point of interest; a navigation device configured to provide a location and orientation associated with the vehicle; a processing module coupled to the gesture capture device and the navigation device, the processing module configured to retrieve information about the desired point of interest based on the direction of the user gesture received from the gesture capture device and the location and orientation of the vehicle received from the navigation device; and a display device coupled to the processing module and configured to display the information about the desired point of interest.
a gesture capture device configured to capture data associated with a user gesture, the user gesture having a direction indicating a desired point of interest;
a navigation device configured to provide a location and orientation associated with the vehicle;
a processing module coupled to the gesture capture device and the navigation device, the processing module configured to retrieve information about the desired point of interest based on the direction of the user gesture received from the gesture capture device and the location and orientation of the vehicle received from the navigation device; and
a display device coupled to the processing module and configured to display the information about the desired point of interest.
The information system of claim 1 , wherein the gesture capture device comprises a camera that captures optical data associated with the user gesture.
The information system of claim 1 , wherein the user gesture is a hand signal.
The information system of claim 1 , further comprising a point of interest information database coupled to the processing module, the point of interest database comprising information associated with a plurality of points of interest, including the desired point of interest.
The information system of claim 1 , wherein the processing module includes an optical processor configured to determine the direction of the user gesture.
The information system of claim 1 , further comprising a communication system coupled to the processing module and configured retrieve the information associated with the desired point of interest from an off-board database.
The information system of claim 1 , further comprising an audio device coupled to the processing module and configured to provide audio information to the user about the desired point of interest.
The information system of claim 1 , wherein the processing module is configured to activate the system based on an audio signal.
The information system of claim 8 wherein the processing module is configured to activate the system based on a visual signal.
A method of providing point of interest information to a user in a vehicle, the method comprising: capturing data associated with a user gesture, the user gesture having a direction indicating a desired point of interest; receiving location and orientation of the vehicle from a navigation device; retrieving information about the desired point of interest based on the direction of the user gesture and the location and orientation of the vehicle received from the navigation device; and providing the information about the desired point of interest to the user.
capturing data associated with a user gesture, the user gesture having a direction indicating a desired point of interest;
receiving location and orientation of the vehicle from a navigation device;
retrieving information about the desired point of interest based on the direction of the user gesture and the location and orientation of the vehicle received from the navigation device; and
providing the information about the desired point of interest to the user.
The method of claim 10 , wherein the providing step including displaying visual information about the desired point of interest.
The method of claim 10 , wherein the providing step including providing audio information about the desired point of interest.
The method of claim 10 , wherein the capturing data step includes capturing optical data with a camera.
The method of claim 10 , wherein the capturing data step includes capturing a gesture that includes a hand signal.
The method of claim 10 , wherein the retrieving step includes retrieving the information from an on-board point of interest information database.
The method of claim 10 , wherein the retrieving step includes retrieving the information from an off-board point of interest information database.
The method of claim 10 , further comprising activating the capturing step with an audio signal from the user.
The method of claim 10 , further comprising activating the capturing step with a visual signal from the user.
The method of claim 10 , wherein the providing step includes providing at least one of hours of operation, historical information, or directions associated with the desired point of interest.
An information system for providing point of interest information to a user in a vehicle, the system comprising: a camera configured to capture optical data associated with a user gesture, the user gesture comprising a hand gesture with a direction pointing to a desired point of interest; a navigation device configured to provide a location and orientation associated with the vehicle; a point of interest information database comprising information associated with a plurality of points of interest, including the desired point of interest; a processing module coupled to the gesture capture device, the navigation device, and the point of interest database, the processing module configured to retrieve information about the desired point of interest from the point of interest database based on the direction of the user gesture received from the gesture capture device and the location and orientation of the vehicle received from the navigation device; and a display device coupled to the processing module and configured to display the information about the desired point of interest.
a camera configured to capture optical data associated with a user gesture, the user gesture comprising a hand gesture with a direction pointing to a desired point of interest;
a point of interest information database comprising information associated with a plurality of points of interest, including the desired point of interest;
a processing module coupled to the gesture capture device, the navigation device, and the point of interest database, the processing module configured to retrieve information about the desired point of interest from the point of interest database based on the direction of the user gesture received from the gesture capture device and the location and orientation of the vehicle received from the navigation device; and
the processing module configured to retrieve information about the desired point of interest from the point of interest database based on the direction of the user gesture received from the gesture capture device and the location and orientation of the vehicle received from the navigation device; and
Ref US20140330515A1: Claims:
- 20 . (canceled)
A vehicle-based computer-implemented method for retrieving information associated with a point of interest (POI), the method comprising: accessing location data identifying a current location of the vehicle; analyzing the location data to find a micromapped region; retrieving, from a POI data server, a micromap corresponding to the micromapped region, the micromap including information associated with one or more points of interest inside the micromapped region; receiving a data signal representing an identifying gesture performed by a user inside the vehicle, the identifying gesture oriented in a direction and identifying an object exterior to the vehicle; performing gesture recognition on the data signal to determine a direction vector representing the direction of the identif
[Truncated to input limit]