Office Action 1: Non-Final Rejection (2024-07-05)
Rejection types: 103 - Obviousness

CLAIMS:
CLAIMS I/We claim:
1\. A system for sensing and control of a vehicle, comprising: one or more first sensors configured to sense an interior of the vehicle, wherein the one or more first sensors are configured to generate first sensor data based on sensing the interior of the vehicle; one or more second sensors configured to sense an exterior of the vehicle, wherein the one or more second sensors are configured to generate second sensor data based on sensing the exterior of the vehicle; a computing device, wherein the computing device is configured to detect a user indication of interest, wherein the computing device is configured to detect the user indication of interest based at least in part on a result of providing the first sensor data as input to an artificial neural network, wherein the computing device is configured to determine an item of interest based at least in part on the detected user indication of interest and the second sensor data; a vehicle controller, wherein the vehicle controller is configured to receive information about the item of interest, wherein the vehicle controller is configured to control operation of the vehicle based on the information about the item of interest.
2\. The system of claim 1, wherein the user indication of interest is a gesture.
3\. The system of claim 2, wherein the computing device is configured to determine the item of interest based at least in part on calculation of a pointing vector.
4\. The system of claim 3, wherein the computing device is configured to calculate the pointing vector based at least in part on the first sensor data.
5\. The system of claim 4, wherein the computing device is configured to calculate the pointing vector based at least in part on a plurality of landmarks in the first sensor data.
6\. The system of claim 5, wherein the plurality of landmarks includes an indication of the location of an index finger of an occupant of the vehicle.
7\. The system of claim 6, wherein the plurality of landmarks includes an indication of the location of a wrist of an occupant of the vehicle.
8\. The system of claim 5, wherein the computing device is configured to calculate the pointing vector based at least in part on calculating a best-fit vector for the plurality of landmarks.
9\. The system of claim 8, wherein the computing device is configured to determine the item of interest based at least in part on a geographic location of the vehicle at the time the user indication of interest occurred.
10\. The system of claim 4, wherein the computing device is configured to calculate the pointing vector based at least in part on the first sensor data that includes time of flight sensor data.
11\. The system of claim 4, wherein the computing device is configured to calculate the pointing vector based at least in part on the first sensor data that includes image sensor data.
12\. The system of claim 1, wherein the user indication of interest is the utterance of a predefined trigger word.
13\. The system of claim 12, wherein the computing device is configured to determine the item of interest based at least in part on calculation of a gazing vector.
14\. The system of claim 13, wherein the computing device is configured to calculate the gazing vector based at least in part on the first sensor data.
15\. The system of claim 14, wherein the first sensor data includes at least one image that captures the orientation of the head of an occupant of the vehicle.
16\. The system of claim 15, wherein the computing device is configured to calculate the gazing vector based at least in part on the orientation of the head of the occupant of the vehicle.
17\. The system of claim 16, wherein the computing device is configured to calculate the gazing vector based at least in part on determining a roll value of the orientation of the head of the occupant of the vehicle, wherein the computing device is configured to calculate the gazing vector based at least in part on determining a pitch value of the orientation of the head of the occupant of the vehicle, wherein the computing device is configured to calculate the gazing vector based at least in part on determining a yaw value of the orientation of the head of the occupant of the vehicle.
18\. The system of claim 17, wherein the computing device is configured to calculate the item of interest based at least in part on a geographic location of the vehicle at the time the user indication of interest occurred.
19\. The system of claim 14, wherein the computing device is configured to calculate the gazing vector based at least in part on the first sensor data that includes image data.
20\. The system of claim 19, wherein the first sensor data includes audio sensor data.

OFFICE ACTION:
Notice of Pre-AIA  or AIA  Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 06/08/2022 was filed.  The submission is in compliance with the provisions of 37 CFR 1.97.  Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or non-obviousness.
Claim 1-2, 12-16, and 19 are rejected under 35 U.S.C. 103 as being anticipated by Yamamoto (US Pub. No. 20150362988) in view of Herz (US Pub. No. 10913463).
Regarding Claim 1, Yamamoto discloses A system for sensing and control of a vehicle, comprising: one or more first sensors configured to sense an interior of the vehicle, (Yamamoto, see at least [0017] “visual sensors 104 may be placed within the interior of a vehicle”)
wherein the one or more first sensors are configured to generate first sensor data based on sensing the interior of the vehicle; (Yamamoto, see at least [0017] “visual sensors 104 may be used to detect visual objects both in the inner environment and in the outer environment.”)
one or more second sensors configured to sense an exterior of the vehicle, wherein the one or more second sensors are configured to generate second sensor data based on sensing the exterior of the vehicle; (Yamamoto, see at least [0017] “The visual sensors 104 can also be position such that they have a field of view that is limited to or includes an outer environment 206 that is outside of an inner environment 204 (see FIG. 2). By way of non-limiting example, multiple visual sensors 104 can be coupled to a vehicle to provide a field of view outside of the vehicle (the outer environment in this example) to provide a 360 degree viewing angle of the outer environment, or a partial viewing angle less than 360 degrees.”)
a computing device, wherein the computing device is configured to detect a user indication of interest, (Yamamoto, see at least [0005] “The processing device can be configured to determine an occurrence of a first user indication; determine a plurality of Point of Interest (POI) corresponding to the first user indication”)
wherein the computing device is configured to determine an item of interest based at least in part on the detected user indication of interest and the second sensor data; (Yamamoto, see at least [0005] “The apparatus may further include at least one processing device communicatively coupled to the at least one sensor device. The processing device can be configured to determine an occurrence of a first user indication; determine a plurality of Point of Interest (POI) corresponding to the first user indication;” and [0017] “The user indication recognition system 100 can include multiple sensor devices.”)
a vehicle controller, wherein the vehicle controller is configured to receive information about the item of interest, (Yamamoto, see at least [0051] “the user indication recognition system 100 may be used to control aspects of the larger system (e.g., a vehicle, home-theater, etc.) while the user is in the outer environment 206. For example, a user may be outside of a vehicle and point to the vehicle and state “lock it” or “unlock it.””)
wherein the vehicle controller is configured to control operation of the vehicle based on the information about the item of interest. (Yamamoto, see at least [0030] “Points of Interest (POI) may be present in any given application setting. For example, a POI 210 may be within the inner environment 204 and may relate to a system of a vehicle (such as the infotainment center, navigation, climate control, etc.) with which a user 202 may wish to control, inquire about, or interface with. Additionally, other POIs 212, 214 may exist in the outer environment 206. These may correspond to locations (such as addresses, buildings, parks, rest areas, etc.), businesses (such as stores, gas stations, retail shops, restaurants, hotels, etc.), roads or streets, objects, or any point of interest 212, 214 about which a user may wish to inquire or provide a command.”)
Yamamoto does not explicitly disclose wherein the computing device is configured to detect the user indication of interest based at least in part on a result of providing the first sensor data as input to an artificial neural network,
However, Herz discloses wherein the computing device is configured to detect the user indication of interest based at least in part on a result of providing the first sensor data as input to an artificial neural network, (Herz, see at least [Column 15, Lines 47-51] “functions of the IMD (such as gesture recognition or object recognition) may involve the use of a separate framework or service—e.g., a machine learning framework which supports convolutional neural network models may be invoked by the IMD 810 for object recognition.”)
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Yamamoto with the teachings of Herz. The motivation for doing so would have been to allow for a neural network to be utilized when detecting an indication of interest by a user, as this would allow for gestures to be recognized along with object recognition when sensor data is inputted into a neural network, see Herz [0059].
Regarding Claim 2, Yamamoto discloses The system of claim 1, wherein the user indication of interest is a gesture. (Yamamoto, see at least [0031] “first user indication can include either or both of an indication gesture (for example, a finger point) or an audible indication (for example, a spoken command).”)
Regarding Claim 12, Yamamoto discloses The system of claim 1, wherein the user indication of interest is the utterance of a predefined trigger word. (Yamamoto, see at least [0039] “After analyzing the data stored within the historical data to make a determination as to whether the results of a particular search can be filtered to generate a narrowed set of search results in this manner, the system may automatically filter the results based upon that prior behavior, or can prompt the user to filter the results in accordance with their prior behavior.”)
Regarding Claim 13, Yamamoto discloses The system of claim 12, wherein the computing device is configured to determine the item of interest based at least in part on calculation of a gazing vector. (Yamamoto, see at least [0058] “For example, the processing device 102 may refer to the direction in which the user's eyes are looking or were looking immediately before or after the audible indication. As will be understood by those of skill in the art, any number of audible indications can be utilized which the processing device 102 can be configured to use to distinguish between POIs within the inner 204 or outer environment 206.”)
Regarding Claim 14, Yamamoto discloses The system of claim 13, wherein the computing device is configured to calculate the gazing vector based at least in part on the first sensor data. (Yamamoto, see at least [0058] “For example, the processing device 102 may refer to the direction in which the user's eyes are looking or were looking immediately before or after the audible indication. As will be understood by those of skill in the art, any number of audible indications can be utilized which the processing device 102 can be configured to use to distinguish between POIs within the inner 204 or outer environment 206.” Examiner notes that the audible indication is an example of sensor data collected.)
Regarding Claim 15, Yamamoto discloses The system of claim 14, wherein the first sensor data includes at least one image that captures the orientation of the head of an occupant of the vehicle. (Yamamoto, see at least [0017] “the machine vision system will include at least two visual sensors 104 to be able to capture stereoscopic images. With these stereoscopic images, the processing device 102 can implement various algorithms to determine the contents of one or more captured images and be able to relate the determined images to an action, function, or feature. For example, the machine vision system may be able to capture hand gestures, finger movements, the direction in which a user is pointing or looking, or other macro- or micro-details pertaining to the user.”)
Regarding Claim 16, Yamamoto discloses The system of claim 15, wherein the computing device is configured to calculate the gazing vector (Yamamoto, see at least [0058] “For example, the processing device 102 may refer to the direction in which the user's eyes are looking or were looking immediately before or after the audible indication. As will be understood by those of skill in the art, any number of audible indications can be utilized which the processing device 102 can be configured to use to distinguish between POIs within the inner 204 or outer environment 206.” Examiner notes that the audible indication is an example of sensor data collected.)
based at least in part on the orientation of the head of the occupant of the vehicle. (Yamamoto, see at least [0060] “Such a determination may be made simply by determining one or more particular POIs 210 within the inner environment 204 that the user is specifically addressing or that the user is specifically gesturing toward. Further, such a determination may be made by selecting one or more POIs 212 within an outer boundary 216 (discussed further below) if the outer environment 206 is indicated. However, by one approach, if a directional indication is provided (e.g., a finger point, hand gesture, head nod, eye movement, etc.), the processing device 102 may be able to select a particular one POI or a set of POIs within the outer environment 206.”)
Regarding Claim 18, Yamamoto discloses The system of claim 17, wherein the computing device is configured to calculate the item of interest based at least in part on a geographic location of the vehicle at the time the user indication of interest occurred. (Yamamoto, see at least [0030] “a POI 210 may be within the inner environment 204 and may relate to a system of a vehicle (such as the infotainment center, navigation, climate control, etc.) with which a user 202 may wish to control, inquire about, or interface with. Additionally, other POIs 212, 214 may exist in the outer environment 206. These may correspond to locations (such as addresses, buildings, parks, rest areas, etc.), businesses (such as stores, gas stations, retail shops, restaurants, hotels, etc.), roads or streets, objects, or any point of interest 212, 214 about which a user may wish to inquire or provide a command.”)
Regarding Claim 19, Yamamoto discloses The system of claim 14, wherein the computing device is configured to calculate the gazing vector based at least in part on the first sensor data that includes image data. (Yamamoto, see at least [0017] “the visual sensors 104 may be part of a “machine vision” system that may also include the at least one processing device 102. Typically, the machine vision system will include at least two visual sensors 104 to be able to capture stereoscopic images. With these stereoscopic images, the processing device 102 can implement various algorithms to determine the contents of one or more captured images and be able to relate the determined images to an action, function, or feature. For example, the machine vision system may be able to capture hand gestures, finger movements, the direction in which a user is pointing or looking, or other macro- or micro-details pertaining to the user.”)
Regarding Claim 20, Yamamoto discloses The system of claim 19, wherein the first sensor data includes audio sensor data. (Yamamoto, see at least [0018] “The user indication recognition system 100 may also include or be coupled to one or more audio sensors 106.”)
Claim 3-7, 9, and 11 are rejected under 35 U.S.C. 103 as being anticipated by Yamamoto (US Pub. No. 20150362988) in view of Herz (US Pub. No. 10913463) in further view of McCall (US Pub. No. 20100274480).
Regarding Claim 3, Yamamoto does not explicitly disclose The system of claim 2, wherein the computing device is configured to determine the item of interest based at least in part on calculation of a pointing vector.
However, Mccall discloses wherein the computing device is configured to determine the item of interest based at least in part on calculation of a pointing vector. (Mccall, see at least [0026] “the camera 180 has a field of view 210 that captures the direction vector of the gesture 204 of the user. In the example shown in FIG. 2, the direction vector of the gesture 204 is about 40.degree. relative to the orientation of the vehicle 110 (e.g., angle 222), which results in the gesture 204 having an angle of about 70.degree. relative to true North (e.g., angle 224). Accordingly, this information, along with the location of the vehicle 110, enables the processing module 120 (FIG. 1) to identify the point of interest 202 from a listing of the points of interest in the database 140 (FIG. 1).”)
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Yamamoto with the teachings of Mccall. The motivation for doing so would have been to allow for a direction vetor to be identified because this would allow to enable for the point of interest of a user to be identified in a more accurate manner, see Mccall [0026].
Regarding Claim 4, Yamamoto discloses The system of claim 3, wherein the computing device is configured to calculate the pointing vector based at least in part on the first sensor data. (Yamamoto, see at least [0017] “visual sensors 104 may be used to detect visual objects both in the inner environment and in the outer environment.”)
Yamamoto does not explicitly disclose wherein the computing device is configured to calculate the pointing vector
However, Mccall discloses wherein the computing device is configured to calculate the pointing vector (Mccall, see at least [0026] “the camera 180 has a field of view 210 that captures the direction vector of the gesture 204 of the user. In the example shown in FIG. 2, the direction vector of the gesture 204 is about 40.degree. relative to the orientation of the vehicle 110 (e.g., angle 222), which results in the gesture 204 having an angle of about 70.degree. relative to true North (e.g., angle 224). Accordingly, this information, along with the location of the vehicle 110, enables the processing module 120 (FIG. 1) to identify the point of interest 202 from a listing of the points of interest in the database 140 (FIG. 1).”)
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Yamamoto with the teachings of Mccall. The motivation for doing so would have been to allow for a direction vetor to be identified because this would allow to enable for the point of interest of a user to be identified in a more accurate manner, see Mccall [0026].
Regarding Claim 5, Yamamoto discloses The system of Claim 4, based at least in part on a plurality of landmarks in the first sensor data. (Yamamoto, see at least [0017] “a POI 210 may be within the inner environment 204 and may relate to a system of a vehicle (such as the infotainment center, navigation, climate control, etc.) with which a user 202 may wish to control, inquire about, or interface with. Additionally, other POIs 212, 214 may exist in the outer environment 206. These may correspond to locations (such as addresses, buildings, parks, rest areas, etc.), businesses (such as stores, gas stations, retail shops, restaurants, hotels, etc.), roads or streets, objects, or any point of interest 212, 214 about which a user may wish to inquire or provide a command..”)
Yamamoto does not explicitly disclose wherein the computing device is configured to calculate the pointing vector
However, Mccall discloses wherein the computing device is configured to calculate the pointing vector (Mccall, see at least [0026] “the camera 180 has a field of view 210 that captures the direction vector of the gesture 204 of the user. In the example shown in FIG. 2, the direction vector of the gesture 204 is about 40.degree. relative to the orientation of the vehicle 110 (e.g., angle 222), which results in the gesture 204 having an angle of about 70.degree. relative to true North (e.g., angle 224). Accordingly, this information, along with the location of the vehicle 110, enables the processing module 120 (FIG. 1) to identify the point of interest 202 from a listing of the points of interest in the database 140 (FIG. 1).”)
Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Yamamoto with the teachings of Mccall. The motivation for doing so would have been to allow for a direction vetor to be identified because this would allow to enable for the point of interest of a user to be identified in a more accurate manner, see Mccall [0026].
Regarding Claim 6, Yamamoto discloses The system of claim 5, wherein the plurality of landmarks (Yamamoto, see at least [0017] “a POI 210 may be within the inner environment 204 and may relate to a system of a vehicle (such as the infotainment center, navigation, climate control, etc.) with which a user 202 may wish to control, inquire about, or interface with. Additionally, other POIs 212, 214 may exist in the outer environment 206. These may correspond to locations (such as addresses, buildings, parks, rest areas, etc.), businesses (such as stores, gas stations, retail shops, restaurants, hotels, etc.), roads or streets, objects, or any point of interest 212, 214 about which a user may wish to inquire or provide a command..”)
includes an indication of the location of an index finger of an occupant of the vehicle. (Yamamoto, see at least [0017] “the visual sensors 104 provide suitable resolution to capture and allow determination of, for example, a direction in which a user's eyes are directed, head movements, and even fine and/or quick finger movements”)
Regarding Claim 7, Yamamoto discloses The system of claim 6, wherein the plurality of landmarks (Yamamoto, see at least 0037] and [0017] “a POI 210 may be within the inner environment 204 and may relate to a system of a vehicle (such as the infotainment center, navigation, climate control, etc.) with which a user 202 may wish to control, inquire about, or interface with. Additionally, other POIs 212, 214 may exist in the outer environment 206. These may correspond to locations (such as addresses, buildings, parks, rest areas, etc.), businesses (such as stores, gas stations, retail shops, restaurants, hotels, etc.), roads or streets, objects, or any point of interest 212, 214 about which a user may wish to inquire or provide a command..”)
includes an indication of the location of a wrist of an occupant of the vehicle. (Yamamoto, see at least [0035] “For example, a coarse user indication may be a finger point, a hand directional indication, or a head nod. However, a fine user indication may constitute much finer movement such as moving a finger up or down slightly (possibly while keeping the arm steady).”)
Regarding Claim 9, Yamamoto discloses The system of claim 8, wherein the computing device is configured to determine the item of interest based at least in part on a geographic location of the vehicle at the time the user indication of interest occurred. (Yamamoto, see at least [0019] “The user indication recognition system 100 may also include or be coupled to one or more telemetry sensors 108. Non-limiting examples of telemetry sensors 108 can include weather sensors, traffic condition sensors (in a vehicular setting), accelerometers, compasses, or other sensors configured to measure a configuration, attitude, position, speed, velocity, orientation, steering wheel angle (in a vehicular setting), and the like. In various approaches, a given application setting may already include multiple telemetry sensors of varying types as may be required in that particular setting, which sensors may be made available to the user indication recognition system 100 as needed or requested.”)
Regarding Claim 11, Yamamoto discloses The system of claim 4, wherein the computing device is configured to calculate the pointing vector based at least in part on the first sensor data that includes image sensor data. (Yamamoto, see at least [0017] “The visual sensors 104 may be used to detect visual objects both in the inner environment and in the outer environment. As described, the visual sensors 104 may be part of a “machine vision” system that may also include the at least one processing device 102. Typically, the machine vision system will include at least two visual sensors 104 to be able to capture stereoscopic images. With these stereoscopic images, the processing device 102 can implement various algorithms to determine the contents of one or more captured images and be able to relate the determined images to an action, function, or feature. For example, the machine vision system may be able to capture hand gestures, finger movements, the direction in which a user is pointing or looking, or other macro- or micro-details pertaining to the user.”)
Allowable Subject Matter
Claims 8 and 10 are objected as being dependent upon a rejected base claim.
Claim 17 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 18 is rejected based upon an objected claim.
Relevant Art
The prior art made of record and not relied upon are considered pertinent to the applicant’s disclosure:
US8818716 – A user, such as the driver of a vehicle, to retrieve information related to a point of interest (POI) near the vehicle by pointing at the POI or performing some other gesture to identify the POI. Gesture recognition is performed on the gesture to generate a target region that includes the POI that the user identified. After generating the target region, information about the POI can be retrieved by querying a server-based POI service with the target region or by searching in a micromap that is stored locally. The retrieved POI information can then be provided to the user via a display and/or speaker in the vehicle. This process beneficially allows a user to rapidly identify and retrieve information about a POI near the vehicle without having to navigate a user interface by manipulating a touchscreen or physical buttons.
US 20180078444 – This invention described a Spatial Recognition Device (SRD) based on a mounted high precision measurement sensor that enables the user to detect and navigate around objects and obstacles like traditional white cane would, but without contact, and also allow the user to identify the location and shape of objects to enable identifying such obstacles and objects. The device can also provide route guidance through a GPS and collision avoidance through onboard and/or cloud based or hybrid computational systems, accelerometers and other sensors. The device enables sensing ranges from approximately one inch to a maximum range dependent on the high precision measurement sensor used in the device. This device would not be mounted on a “white cane” or similar assistive device.
Conclusion

APPLICANT REBUTTAL: REMARKS
Prior to the present Response, claims 1-20 were pending in the present application. The present amendments: cancel claims 2-5 and 8; amend claims 1, 6, and 9-11. After entry of the present amendments, claims 1, 6-7, and 9-20 will be pending in the present application.
Reconsideration and allowance of the application are respectfully requested.
Claim Objections and Allowable Subject Matter
The Office Action indicates that claims 8, 10, 17, and 18 are objected to as being
dependent on a rejected base claim, but would be allowable if rewritten in independent form. (Office Action, p. 14.) The present amendments incorporate the subject matter of claim 8, along with the intervening dependent claims 2-5, into independent claim 1. As such, it is believe that the claims are in condition for allowance, and reconsideration is respectfully requested.
Re
jections under 35 U.S.C.
Claims 1, 2, 12-16, and 19 are rejected under 35 U.S.C. @ 103 as being unpatentable over U.S. Patent Application Pub. No. 20 15/0362988 ("Yamamoto") in view of U.S. Patent No. 10,913,463 ("Herz"). (Office Action, pp. 2-3.)
Claims 3-7, 9, and 11 are rejected under 35 U.S.C. @ 103 as being unpatentable over Yamamoto, Herz, and U.S. Patent Application Pub. No. 20 10/0274480 ("McCall").
subject matter. As such, reconsideration and withdrawal of these rejections arerequested.
Conclusion

NEXT OFFICE ACTION: DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA  35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art.  The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph:
(A)	the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B)	the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C)	the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Such claim limitation(s) is/are:
“computing device” in claim 1, 9-11, 13-14, and 16-19
“vehicle controller” in claim 1
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
The following are the interpreted corresponding structures found within the specification for some of the above limitations:
“computing device” – includes a transceiver 1102, a coupling 1104, a memory 1106, a processor 1108, a power input 1110, and a data store 1112 ; paragraph [208]
“vehicle controller” – includes a transceiver 1202, a coupling 1204, a memory 1206, a processor 1208, a power input 1210, and a data store 1212 ; paragraph [0216]
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, applicant may:  (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph.
Allowable Subject Matter
Claims 1, 6-7 and 9-20 are allowed.
The following is an examiner’s statement of reasons for allowance:
For independent claim 1, the prior art on record does not teach, describe and/or suggest all the limitations as presented in the claim as a whole and specifically “...a vehicle controller, wherein the vehicle controller is configured to receive information about the item of interest, wherein the vehicle controller is configured to control operation of the vehicle based on the information about the item of interest, wherein the user indication of interest is a gesture, wherein the computing device is configured to determine the item of interest based at least in part on calculation of a pointing vector, wherein the computing device is configured to calculate the pointing vector based at least in part on the first sensor data, wherein the computing device is configured to calculate the pointing vector based at least in part on a plurality of landmarks in the first sensor data, and wherein the computing device is configured to calculate the pointing vector based at least in part on calculating a best-fit vector for the plurality of landmarks.”
The dependent claims are allowable at least based on their dependence from the now allowable independent claims.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion

CITED REFERENCE SUMMARIES: Ref US20150362988A1: Claims:
An apparatus, comprising: at least one sensor device configured to sense at least one user indication, wherein the at least one user indication comprises at least one of an indication gesture and an audible indication; at least one processing device communicatively coupled to the at least one sensor device, the at least one processing device configured to: determine at least one occurrence of at least one first user indication; determine a plurality of Points of Interest (POIs) corresponding to the at least one first user indication; determine at least one occurrence of at least one second user indication in response to determining the plurality of POIs; determine at least one narrowed POI from the plurality of POIs in response to determining the at least one occurrence of the at least one second user indication; determine at least one action corresponding to the at least one narrowed POI and at least one of the first and second user indications; and effect performance of the at least one action.
at least one sensor device configured to sense at least one user indication, wherein the at least one user indication comprises at least one of an indication gesture and an audible indication;
at least one processing device communicatively coupled to the at least one sensor device, the at least one processing device configured to: determine at least one occurrence of at least one first user indication; determine a plurality of Points of Interest (POIs) corresponding to the at least one first user indication; determine at least one occurrence of at least one second user indication in response to determining the plurality of POIs; determine at least one narrowed POI from the plurality of POIs in response to determining the at least one occurrence of the at least one second user indication; determine at least one action corresponding to the at least one narrowed POI and at least one of the first and second user indications; and effect performance of the at least one action.
determine at least one occurrence of at least one first user indication;
determine a plurality of Points of Interest (POIs) corresponding to the at least one first user indication;
determine at least one occurrence of at least one second user indication in response to determining the plurality of POIs;
determine at least one narrowed POI from the plurality of POIs in response to determining the at least one occurrence of the at least one second user indication;
determine at least one action corresponding to the at least one narrowed POI and at least one of the first and second user indications; and
effect performance of the at least one action.
The apparatus of claim 1 , wherein the at least one processing device is further configured to effect presentation of the plurality of POIs to a user in response to determining the plurality of POIs, wherein effecting presentation further comprises at least one of effecting a visual representation of the plurality of POIs on a display device and effecting an audible representation of the plurality of POIs.
The apparatus of claim 2 , wherein the at least one processing device is further configured to enable a user to interact with the presentation of the plurality of POIs with the at least one second user indication.
The apparatus of claim 1 , wherein the at least one first user indication comprises at least one indication gesture, and wherein the at least one second user indication comprises at least one audible indication.
The apparatus of claim 1 , wherein the processing device is further configured to determine the plurality of POIs using, at least in part, at least one of environmental data, a time of day, a present location, a known location, information about one or more present users, a user setting, and an historical record of POIs.
The apparatus of claim 1 , wherein the at least one first user indication comprises a coarse user indication and wherein the at least one second user indication comprises a fine indication.
The apparatus of claim 1 , wherein the at least one first user indication comprises both an indication gesture and an audible indication.
The apparatus of claim 1 , wherein the at least one processing device is configured to effect performance of the at least one action further comprises the at least one processing device being configured to at least one of: enable saving a location of the at least one narrowed POI; enable provision of information pertaining to the at least one narrowed POI; enable provision of information pertaining to a present location of a contact in relation to the at least one narrowed POI; identify at least one narrowed POI as a hazard; enable at least one of modification and initiation of a navigation route; and enable operation of at least one barrier operator.
enable saving a location of the at least one narrowed POI;
enable provision of information pertaining to the at least one narrowed POI;
enable provision of information pertaining to a present location of a contact in relation to the at least one narrowed POI;
identify at least one narrowed POI as a hazard;
enable at least one of modification and initiation of a navigation route; and
enable operation of at least one barrier operator.
The apparatus of claim 1 , wherein the at least one processing device is configured to: determine the plurality of POIs by using, at least in part, at least one of location data, compass data, steering wheel angle data, speed data, and CAN bus data.
determine the plurality of POIs by using, at least in part, at least one of location data, compass data, steering wheel angle data, speed data, and CAN bus data.
The apparatus of claim 1 , wherein the at least one processing device is configured to restrict the plurality of POIs to POIs that are located within a geographical region having an outer border.
The apparatus of claim 10 , wherein the at least one processing device is configured to: compare a number of the plurality of POIs to a lower threshold and an upper threshold; and when the number of the plurality POIs is either greater than the upper threshold or lower than the lower threshold, modify the outer border.
compare a number of the plurality of POIs to a lower threshold and an upper threshold; and
when the number of the plurality POIs is either greater than the upper threshold or lower than the lower threshold, modify the outer border.
The apparatus of claim 11 , wherein the at least one processing device is configured to: when the number of the plurality of POIs is less than the lower threshold, enlarge the outer border; and when the number of the plurality of POIs is greater than the upper threshold, contract the outer border.
when the number of the plurality of POIs is less than the lower threshold, enlarge the outer border; and
when the number of the plurality of POIs is greater than the upper threshold, contract the outer border.
The apparatus of claim 12 , wherein the at least one processing device is configured to: determine a default outer border based on, at least in part, a user profile, a current location, a type of search, a threshold value, a border shape, a type of current road, and a time of day.
determine a default outer border based on, at least in part, a user profile, a current location, a type of search, a threshold value, a border shape, a type of current road, and a time of day.
A method, comprising: sensing at least one occurrence of at least one first user indication; determining a plurality of Points of Interest (POIs) corresponding to the at least one first user indication; sensing at least one occurrence of at least one second user indication in response to determining the plurality of POIs; determining at least one narrowed POI from the plurality of POIs in response to determining the at least one occurrence of the at least one second user indication; determining at least one action corresponding to the at least one narrowed POI and at least one of the first and second user indications; and performing the at least one action.
sensing at least one occurrence of at least one first user indication;
determining a plurality of Points of Interest (POIs) corresponding to the at least one first user indication;
sensing at least one occurrence of at least one second user indication in response to determining the plurality of POIs;
determining at least one narrowed POI from the plurality of POIs in response to determining the at least one occurrence of the at least one second user indication;
determining at least one action corresponding to the at least one narrowed POI and at least one of the first and second user indications; and
performing the at least one action.
The method of claim 14 , including presenting the plurality of POIs to a user in response to determining the plurality of POIs using at least one of a visual representation of the plurality of POIs on a display device and an audible representation of the plurality of POIs.
The method of claim 15 , including enabling a user to interact with the presentation of the plurality of POIs with the at least one second user indication.
The method of claim 14 , including restricting the plurality of POIs to POIs that are located within a geographical region having an outer border.
The method of claim 17 , including: comparing a number of the plurality of POIs to a lower threshold and an upper threshold; and when the number of the plurality POIs is either greater than the upper threshold or lower than the lower threshold, modifying the outer border.
comparing a number of the plurality of POIs to a lower threshold and an upper threshold; and
when the number of the plurality POIs is either greater than the upper threshold or lower than the lower threshold, modifying the outer border.
The method of claim 18 , including: when the number of the plurality of POIs is less than the lower threshold, enlarging the outer border; and when the number of the plurality of POIs is greater than the upper threshold, contracting the outer border.
when the number of the plurality of POIs is less than the lower threshold, enlarging the outer border; and
when the number of the plurality of POIs is greater than the upper threshold, contracting the outer border.
The method of claim 19 , including determining a default outer border based on, at least in part, a user profile, a current location, a type of search, a threshold value, a border shape, a type of current road, and a time of day.
The method of claim 14 , wherein the at least one first user indication comprises a coarse user indication and wherein the at least one second user indication comprises a fine indication.
A method, comprising: sensing at least one occurrence of at least one first coarse user indication; determining a plurality of Points of Interest (POIs) corresponding to the at least one first coarse user indication; sensing at least one occurrence of at least one second fine user indication in response to determining the plurality of POIs; determining at least one narrowed POI from the plurality of POIs in response to determining the at least one occurrence of the at least one second fine user indication; performing at least one action corresponding to the at least one narrowed POI.
sensing at least one occurrence of at least one first coarse user indication;
determining a plurality of Points of Interest (POIs) corresponding to the at least one first coarse user indication;
sensing at least one occurrence of at least one second fine user indication in response to determining the plurality of POIs;
determining at least one narrowed POI from the plurality of POIs in response to determining the at least one occurrence of the at least one second fine user indication;
performing at least one action corresponding to the at least one narrowed POI.
Ref US20100274480A1: Claims:
An information system for providing point of interest information to a user in a vehicle, the system comprising: a gesture capture device configured to capture data associated with a user gesture, the user gesture having a direction indicating a desired point of interest; a navigation device configured to provide a location and orientation associated with the vehicle; a processing module coupled to the gesture capture device and the navigation device, the processing module configured to retrieve information about the desired point of interest based on the direction of the user gesture received from the gesture capture device and the location and orientation of the vehicle received from the navigation device; and a display device coupled to the processing module and configured to display the information about the desired point of interest.
a gesture capture device configured to capture data associated with a user gesture, the user gesture having a direction indicating a desired point of interest;
a navigation device configured to provide a location and orientation associated with the vehicle;
a processing module coupled to the gesture capture device and the navigation device, the processing module configured to retrieve information about the desired point of interest based on the direction of the user gesture received from the gesture capture device and the location and orientation of the vehicle received from the navigation device; and
a display device coupled to the processing module and configured to display the information about the desired point of interest.
The information system of claim 1 , wherein the gesture capture device comprises a camera that captures optical data associated with the user gesture.
The information system of claim 1 , wherein the user gesture is a hand signal.
The information system of claim 1 , further comprising a point of interest information database coupled to the processing module, the point of interest database comprising information associated with a plurality of points of interest, including the desired point of interest.
The information system of claim 1 , wherein the processing module includes an optical processor configured to determine the direction of the user gesture.
The information system of claim 1 , further comprising a communication system coupled to the processing module and configured retrieve the information associated with the desired point of interest from an off-board database.
The information system of claim 1 , further comprising an audio device coupled to the processing module and configured to provide audio information to the user about the desired point of interest.
The information system of claim 1 , wherein the processing module is configured to activate the system based on an audio signal.
The information system of claim 8 wherein the processing module is configured to activate the system based on a visual signal.
A method of providing point of interest information to a user in a vehicle, the method comprising: capturing data associated with a user gesture, the user gesture having a direction indicating a desired point of interest; receiving location and orientation of the vehicle from a navigation device; retrieving information about the desired point of interest based on the direction of the user gesture and the location and orientation of the vehicle received from the navigation device; and providing the information about the desired point of interest to the user.
capturing data associated with a user gesture, the user gesture having a direction indicating a desired point of interest;
receiving location and orientation of the vehicle from a navigation device;
retrieving information about the desired point of interest based on the direction of the user gesture and the location and orientation of the vehicle received from the navigation device; and
providing the information about the desired point of interest to the user.
The method of claim 10 , wherein the providing step including displaying visual information about the desired point of interest.
The method of claim 10 , wherein the providing step including providing audio information about the desired point of interest.
The method of claim 10 , wherein the capturing data step includes capturing optical data with a camera.
The method of claim 10 , wherein the capturing data step includes capturing a gesture that includes a hand signal.
The method of claim 10 , wherein the retrieving step includes retrieving the information from an on-board point of interest information database.
The method of claim 10 , wherein the retrieving step includes retrieving the information from an off-board point of interest information database.
The method of claim 10 , further comprising activating the capturing step with an audio signal from the user.
The method of claim 10 , further comprising activating the capturing step with a visual signal from the user.
The method of claim 10 , wherein the providing step includes providing at least one of hours of operation, historical information, or directions associated with the desired point of interest.
An information system for providing point of interest information to a user in a vehicle, the system comprising: a camera configured to capture optical data associated with a user gesture, the user gesture comprising a hand gesture with a direction pointing to a desired point of interest; a navigation device configured to provide a location and orientation associated with the vehicle; a point of interest information database comprising information associated with a plurality of points of interest, including the desired point of interest; a processing module coupled to the gesture capture device, the navigation device, and the point of interest database, the processing module configured to retrieve information about the desired point of interest from the point of interest database based on the direction of the user gesture received from the gesture capture device and the location and orientation of the vehicle received from the navigation device; and a display device coupled to the processing module and configured to display the information about the desired point of interest.
a camera configured to capture optical data associated with a user gesture, the user gesture comprising a hand gesture with a direction pointing to a desired point of interest;
a point of interest information database comprising information associated with a plurality of points of interest, including the desired point of interest;
a processing module coupled to the gesture capture device, the navigation device, and the point of interest database, the processing module configured to retrieve information about the desired point of interest from the point of interest database based on the direction of the user gesture received from the gesture capture device and the location and orientation of the vehicle received from the navigation device; and
the processing module configured to retrieve information about the desired point of interest from the point of interest database based on the direction of the user gesture received from the gesture capture device and the location and orientation of the vehicle received from the navigation device; and
Ref US20140330515A1: Claims:
- 20 . (canceled)
A vehicle-based computer-implemented method for retrieving information associated with a point of interest (POI), the method comprising: accessing location data identifying a current location of the vehicle; analyzing the location data to find a micromapped region; retrieving, from a POI data server, a micromap corresponding to the micromapped region, the micromap including information associated with one or more points of interest inside the micromapped region; receiving a data signal representing an identifying gesture performed by a user inside the vehicle, the identifying gesture oriented in a direction and identifying an object exterior to the vehicle; performing gesture recognition on the data signal to determine a direction vector representing the direction of the identif
[Truncated to input limit]