Office Action 2: Notice of Allowance (2024-09-04)

CLAIMS:
AMENDMENTS TO THE CLAIMS

Applicant submits below a complete listing of the current claims, including marked-upclaim(s) with insertions indicated by underlining and deletions indicated by strikeouts and/or double bracketing. This listing of claims replaces all prior versions, and listings, of claims in the application.

Listing of the Claims:

1-20. (Canceled)219

21\. (Currently Amended) An augmented reality system, comprising:at least one image sensor comprising:
a plurality of pixel cells, each pixel cell of the plurality of pixel cells comprising a light-sensitive component;
first circuitry coupled to light-sensitive components of pixel cells of the plurality of pixel cells and configurable to provide, as output signals from at least some pixel cells of the plurality of pixel cells, indications of light intensity; and
second circuitry coupled to light-sensitive components of pixel cells of the plurality of pixel cells and configurable to provide, as output signals from at least some pixel cells of the plurality of pixel cells, event-based indications of change in a light property based on the change in the light property exceeding a threshold; and
at least one processor configured to:for a first image data processing technique, receive, from the first circuitry, indications of light intensity; and
for a second image data processing technique, receive, from the second circuitry, event-based indications of change in the light property.

22\. (Previously Presented) The augmented reality system of claim 21, wherein the at least one processor is further configured to:

perform the first image data processing technique, the first image data processing technique comprising generating or updating a world model using the indications of light intensity.

23\. (Previously Presented) The augmented reality system of claim 21, wherein the at least one processor is further configured to:perform the first image data processing technique using indications of light intensity for a full image frame.

24\. (Previously Presented) The augmented reality system of claim 21, wherein the at least one processor is further configured to:perform the first image data processing technique using indications of light intensity captured using a rolling shutter mode.

25\. (Previously Presented) The augmented reality system of claim 21, wherein the at least one processor is further configured to:perform the first image data processing technique using indications of light intensity captured using a global shutter mode.

26\. (Previously Presented) The augmented reality system of claim 21, wherein:the at least one image sensor comprises optical components disposed in light paths at least some pixel cells of the plurality of pixel cells;
the optical components are configured to convert an angle of incident light into light intensity; and
the at least one processor is further configured to perform the first image data processing technique, the first image data processing technique comprising generating passive depth information using the indications of light intensity.

27\. (Previously Presented) The augmented reality system of claim 21, wherein the augmented reality system comprises a wearable headset.

28\. (Currently amended) Non-transitory computer-readable medium comprising computer- executable instructions, configured for execution on at least one processor of an augmented reality system comprising at least one image sensor comprising a plurality of pixel cells, each pixel cell of the plurality of pixel cells comprising a light-sensitive component; first circuitry coupled to light- sensitive components of pixel cells of the plurality of pixel cells and configurable to provide, as output signals from at least some pixel cells of the plurality of pixel cells, indications of light intensity; and second circuitry coupled to light-sensitive components of pixel cells of the plurality of pixel cells and configurable to provide, as output signals from at least some pixel cells of the plurality of pixel cells, indications of change in a light property, the computer-executable instructions configured to, when executed, control the at least one processor to:set an output mode of the at least one image sensor based on an image data processing technique to be performed, wherein:the at least one image sensor is set to operate in a first mode in which the first circuitry provides, as output signals from at least some pixel cells of the plurality of pixel cells, indications of light intensity to perform a first image data processing technique;
the at least one image sensor is set to operate in a second mode in which the second circuitry provides, as output signals from at least some pixel cells of the plurality of pixel cells, indications of change in a light property to perform a second image data processing technique;
receive for processing, based on an image processing technique to be performed, indications of light intensity from the first circuitry or indications of change in the light property from the second circuitry, such that:for the\[\[a\]\] first image data processing technique, the at least one processor receives,
from the first circuitry, indications of light intensity; andfor the\[\[a\]\] second image data processing technique, the at least one processor receives, from the second circuitry, indications of change in the light property.

29\. (Previously Presented) The non-transitory computer-readable medium of claim 28, wherein the computer-executable instructions further comprise computer-executable instructions configured to, when executed control the at least one processor to:perform the first image data processing technique, wherein the first image data processing technique comprises tracking an object or pose using the indications of change in the light property.

30\. (Previously Presented) The non-transitory computer-readable medium of claim 28, wherein the computer-executable instructions further comprise computer-executable instructions configured to, when executed control the at least one processor to:perform the second image data processing technique using the indications of change in the light property for an image patch.

31\. (Previously Presented) The non-transitory computer-readable medium of claim 28, wherein the computer-executable instructions further comprise computer-executable instructions configured to, when executed control the at least one processor to:perform the first image data processing technique using indications of light intensity received at a first data rate; and
perform the second image data processing technique using indications of change in the light property received at a second data rate.

32\. (Previously Presented) The non-transitory computer-readable medium of claim 28, wherein the computer-executable instructions further comprise computer-executable instructions configured to, when executed control the at least one processor to:perform the first image data processing technique using indications of light intensity in response to an indication to operate in a first power consumption mode; and
perform the second image data processing technique using indications of change in the light property in response to an indication to operate in a second power consumption mode.

33\. (Currently Amended) A method of operating an augmented reality system, comprising at least one image sensor comprising a plurality of pixel cells, each pixel cell of the plurality of pixel cells comprising a light-sensitive component, first circuitry coupled to light-sensitive components of the plurality of pixel cells and configurable to provide, as output signals from at least some pixel cells of the plurality of pixel cells, indications of light intensity; and second circuitry coupled to light-sensitive components of the plurality of pixel cells and configurable to provide, as output signals from at least some pixel cells of the plurality of pixel cells, indications of change in a light property, the method comprising:controlling an output mode of the at least one image sensor such that the at least one image sensor operates in:a first mode in which the first circuitry provides, as output signals from at least some pixel cells of the plurality of pixel cells, indications of light intensity; or a second mode in which the second circuitry provides, as output signals from at least some pixel cells of the plurality of pixel cells, indications of change in a light property;
using at least one processor to, for a first image data processing technique, receive from the first circuitry, indications of light intensity; and
using the at least one processor to for a second image data processing technique, receive, from the second circuitry, indications of change in the light property.

34\. (Previously Presented) The method of claim 33, further comprising:using the at least one processor to perform the first image data processing technique using indications of light intensity for a full image frame.

35\. (Previously Presented) The method of claim 33, further comprising:using the at least one processor to perform the first image data processing technique using indications of light intensity captured using a rolling shutter mode.

36\. (Previously Presented) The method of claim 33, further comprising:

using the at least one processor to perform the first image data processing technique using indications of light intensity captured using a global shutter mode.

37\. (Previously Presented) The method of claim 33, further comprising:using the at least one processor to perform the second image data processing technique, the first image data processing technique comprising tracking an object or pose using the indications of change in the light property.

38\. (Previously Presented) The method of claim 33, further comprising:using the at least one processor to perform the second image data processing technique using the indications of change in the light property for an image patch.

39\. (Previously Presented) The method of claim 33, further comprising:using the at least one processor to perform the first image data processing technique using indications of light intensity received at a first data rate; and
using the at least one processor to perform the second image data processing technique using indications of change in the light property received at a second data rate.

40\. (Previously Presented) The method of claim 33, further comprising:using the at least one processor to perform the first image data processing technique using indications of light intensity in response to an indication to operate in a first power consumption mode; and
using the at least one processor to perform the second image data processing technique using indications of change in the light property in response to an indication to operate in a second power consumption mode.

OFFICE ACTION:
DETAILED ACTION
Notice of Pre-AIA  or AIA  Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see page nine, first paragraph filed 07/03/2024, with respect to claims 21, 28, and 33 have been fully considered and are persuasive.  The rejection of claims 21, 28, and 33 has been withdrawn.
Allowable Subject Matter
Claims 21 – 40 are allowed.
The following is an examiner’s statement of reasons for allowance:
(Per amendments filed on 07/03/2024)
With respect to claim 21 the prior art discloses An augmented reality system, comprising:
at least one image sensor comprising:
a plurality of pixel cells, each pixel cell of the plurality of pixel cells comprising a light-sensitive component;
first circuitry coupled to light-sensitive components of pixel cells of the plurality of pixel cells and configurable to provide, as output signals from at least some pixel cells of the plurality of pixel cells, indications of light intensity;
and second circuitry coupled to light-sensitive components of pixel cells of the plurality of pixel cells and configurable to provide, as output signals from at least some pixel cells of the plurality of pixel cells.
However, the prior art does not teach or fairly suggest event-based indications of change in a light property based on the change in the light property exceeding a threshold;
and at least one processor configured to:
for a first image data processing technique, receive, from the first circuitry, indications of light intensity;
and for a second image data processing technique, receive, from the second circuitry, event-based indications of change in the light property.
With respect to claim 28 the prior art discloses Non-transitory computer-readable medium comprising computer-executable instructions, configured for execution on at least one processor of an augmented reality system comprising at least one image sensor comprising a plurality of pixel cells, each pixel cell of the plurality of pixel cells comprising a light-sensitive component;
first circuitry coupled to light-sensitive components of pixel cells of the plurality of pixel cells and configurable to provide, as output signals from at least some pixel cells of the plurality of pixel cells, indications of light intensity;
and second circuitry coupled to light-sensitive components of pixel cells of the plurality of pixel cells and configurable to provide, as output signals from at least some pixel cells of the plurality of pixel cells, indications of change in a light property, the computer-executable instructions configured to, when executed, control the at least one processor to.
However, the prior art does not teach or fairly suggest set an output mode of the at least one image sensor based on an image data processing technique to be performed, wherein:
the at least one image sensor is set to operate in a first mode in which the first circuitry provides, as output signals from at least some pixel cells of the plurality of pixel cells, indications of light intensity to perform a first image data processing technique;
the at least one image sensor is set to operate in a second mode in which the second circuitry provides, as output signals from at least some pixel cells of the plurality of pixel cells, indications of change in a light property to perform a second image data processing technique;
receive for processing, based on an image processing technique to be performed, indications of light intensity from the first circuitry or indications of change in the light property from the second circuitry, such that:
for the first image data processing technique, the at least one processor receives, from the first circuitry, indications of light intensity;
and for the second image data processing technique, the at least one processor receives, from the second circuitry, indications of change in the light property.
With respect to claim 33 the prior art discloses A method of operating an augmented reality system, comprising at least one image sensor comprising a plurality of pixel cells, each pixel cell of the plurality of pixel cells comprising a light-sensitive component, first circuitry coupled to light-sensitive components of the plurality of pixel cells and configurable to provide, as output signals from at least some pixel cells of the plurality of pixel cells, indications of light intensity;
and second circuitry coupled to light-sensitive components of the plurality of pixel cells and configurable to provide, as output signals from at least some pixel cells of the plurality of pixel cells, indications of change in a light property, the method comprising.
However, the prior art does not teach or fairly suggest controlling an output mode of the at least one image sensor such that the at least one image sensor operates in:
a first mode in which the first circuitry provides, as output signals from at least some pixel cells of the plurality of pixel cells, indications of light intensity;
or a second mode in which the second circuitry provides, as output signals from at least some pixel cells of the plurality of pixel cells, indications of change in a light property;
using at least one processor to, for a first image data processing technique, receive from the first circuitry, indications of light intensity;
and using the at least one processor to for a second image data processing technique, receive, from the second circuitry, indications of change in the light property.
Dependent claims 22 – 27, 29 – 32, and 34 - 40 are allowable for at least the reason that they depend on allowable independent claims 21, 28, and 33.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee.  Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion

APPLICANT REBUTTAL: REMARKS
In response to the Office Action dated April 8, 2024, Applicant respectfully requests
reconsideration. Claims 21-40 were previously pending in this application. No claims are canceled or added herein. Claims 21, 28, and 33 are amended herein. As a result, claims 21-40 are pending for examination with claims 21, 28, and 33 being independent claims. No new matter has been added.
Allowable Sub
j
ect Matter
Applicant thanks the Examiner for the indication that claims 26, 32, and 40 include
allowable subject matter and would be allowable if rewritten in independent form including all of
the limitations of the base claim and any intervening claims.
Interview Summary
Applicant thanks Examiner Monk for the courtesies extended in granting and holding the
telephone interview of for July 2, 2024. The participants of the interview were Examiner Monk and Applicant's representative Samuel L. Doskocil (Reg. No. 80,312), The interview was pursuant toand consistent with the interview agenda emailed to Examiner Monk on July 1, 2024. The substance of the interview is further summarized by the remarks herein.
During the interview, the participants discussed the Office Action of April 8, 2024,
including the portions of Sheffield and Liu cited in the rejections of independent claims 21, 28, and 33.
Applicant's representative discussed proposed amendments to claim 21 to recite circuitry configurable to output "event-based" indications of change in a light property based on the change exceeding a "threshold."
The Examiner indicated that the proposed amendment to claim 21 was likely to overcome the rejection of record, and that further searching and consideration would be performed. Claim 21 is presented herein consistent with the proposed amendment discussed during the interview.
The Examiner confirmed that the statement on page 17 of the Office Action that claim 6 includes allowable subject matter should read that claim 26 includes allowable subject matter.
Rej
ections Under 35 U.S.C.
§
Independent claims 21, 28, and 33 are rejected under 35 U.S.C. §103 as allegedly being
obvious over U.S. Patent No. 10,235,797 "Sheffield" in view of U.S. Publication No. 2018/0376082 "Liu." Reconsideration is respectfully requested. The proposed combination of Sheffield and Liu would not have satisfied each and every limitation of any of the claims.
A.
Claim 21
The proposed combination of Sheffield and Liu would not have satisfied each and every limitation of claim 21. Amended claim 21 recites, among other limitations:
"second circuitry coupled to light-sensitive components of pixel cells of the
plurality of pixel cells and configurable to provide, as output signals from at least some pixel cells of the plurality of pixel cells, event-based indications of change in a light property based on the change in the light property exceeding
a threshold."
The proposed combination of Sheffield and Liu would not have satisfied at least the above- emphasized limitations of claim 21.
Sheffield does not describe circuitry providing output of "event-based indications" nor does Sheffield describe such circuitry providing output "based on the change in the light property exceeding a threshold." Rather, Sheffield describes a step 306 to "detect a change in reflectance values" (Sheffield, 11:4-25), where step 306 is part of a process 300 that is "implemented as code" such as "one or more computer programs" (Sheffield, 10:13-30). Sheffield thus does not describe circuitry providing output of "event-based indications" nor providing output "based on the change in the light property exceeding a threshold," because Sheffield simply uses a computer program to identify changes in already-output images. As such, Sheffield does not describe "second circuitry coupled to light-sensitive components of pixel cells of the plurality of pixel cells and configurable to provide, as output signals from at least some pixel cells of the plurality of pixel cells, event-based indications of change in a light property exceeding a threshold," as recited by claim 21.
Liu does not describe circuitry providing output of "event-based indications" nor does Liu describe such circuitry providing output "based on the change in the light property exceeding a threshold." Rather, Liu describe two intensity output modes, stating that in "the two different
measurement modes, the pixel cell may be able to generate digital output that tracks the intensity of incident light" (emphasis added, Liu, [0036]). Accordingly, Liu does not describe "second circuitry coupled to light-sensitive components of pixel cells of the plurality of pixel cells and configurable to provide, as output signals from at least some pixel cells of the plurality of pixel cells, event-based indications of change in a light property based on the change in the light property exceeding a threshold," as recited by claim 21.
As neither of Sheffield or Liu describes "second circuitry coupled to light-sensitive
components of pixel cells of the plurality of pixel cells and configurable to provide, as output signals from at least some pixel cells of the plurality of pixel cells, event-based indications of change in a light property based on the change in the light property exceeding a threshold," as recited by 21, the proposed combination of these cited references would not have satisfied at least these limitations.
For at least these reasons, independent claim 21, and each claim that depends therefrom,
patentably distinguishes over the proposed combination of Sheffield and Liu and is believed to be in condition for allowance. Accordingly, it is respectfully requested that the rejections under 35 U.S.C. § 103 of independent claim 21, and each claim that depends therefrom, be withdrawn.
B.
Claim 28
The proposed combination of Sheffield and Liu would not have satisfied each and every limitation of claim 28. Amended claim 28 recites, among other limitations:
"set an output mode of the at least one image sensor based on an image
data processing technique to be performed, wherein:
the at least one image sensor is set to operate in a first mode in which the first circuitry provides, as output signals from at least some pixel cells of the plurality of pixel cells, indications of light intensity to perform a first image data processing technique; the at least one image sensor is set to operate in a second mode in which the second circuitry provides, as output signals from at least some pixel cells of the plurality of pixel cells, indications of change in a light property to perform a second image data processing technique."
The proposed combination of Sheffield and Liu would not have satisfied at least the above- emphasized limitations of claim 28.
Sheffield does not describe "set an output mode of the at least one image sensor" where in a first mode these is output of "indications of light intensity" and in a second mode there is output of "indications of change in a light property." Sheffield does not describe two output modes of image sensors, nor does Sheffield describe setting image sensors in one of two modes. Rather, as previously discussed with respect to claim 21, Sheffield simply describes using a computer program to identify changes in already-output images (Sheffield, 11:4-25). As such, Sheffield does not describe "set an output mode of the at least one image sensor" where in a first mode these is output of "indications of light intensity" and in a second mode there is output of "indications of change in a light property," as recited by claim 28.
Liu also does not describe "set an output mode of the at least one image sensor" where in a first mode these is output of "indications of light intensity" and in a second mode there is output of "indications of change in a light property." Rather, Liu describes a first mode to "measure the light intensity based on the charges stored in the capacitor when the capacitor is electrically coupled with the photodiode" and a second mode to measure the intensity based on "charges stored in the capacitor when the capacitor is electrically isolated from the photodiode" (Liu, [0036]). However, Liu describes that both modes output light intensity, stating that in each of "the two different measurement modes, the pixel cell may be able to generate digital output that tracks the intensity of incident light" (emphasis added, Liu, [0036]). Liu uses two modes in order to "track[] the intensity of incident light, when the intensity exceeds the saturation limit of the photodiode," (Liu, [0036]). Accordingly, Liu does not describe "set an output mode of the at least one image sensor" where in a first mode these is output of "indications of light intensity" and in a second mode there is output of "indications of change in a light property," as recited by claim 28.
As neither of Sheffield or Liu describes "set an output mode of the at least one image
sensor" where in a first mode these is output of "indications of light intensity" and in a second mode there is output of "indications of change in a light property," as recited by claim 28, the proposed combination of these cited references would not have satisfied at least these limitations.
For at least these reasons, independent claim 28, and each claim that depends therefrom,
patentably distinguishes over the proposed combination of Sheffield and Liu and is believed to be in
condition for allowance. Accordingly, it is respectfully requested that the rejections under 35 U.S.C. § 103 of independent claim 28, and each claim that depends therefrom, be withdrawn.
C.
Claim 33
The proposed combination of Sheffield and Liu would not have satisfied each and every limitation of claim 33. Amended claim 33 recites, among other limitations:
"controlling an output mode of the at least one image sensor such that the
at least one image sensor operates in:
a first mode in which the first circuitry provides, as output signals from at least some pixel cells of the plurality of pixel cells, indications of light intensity; or
a second mode in which the second circuitry provides, as output signals from at least some pixel cells of the plurality of pixel cells, indications of change in a light property."
The proposed combination of Sheffield and Liu would not have satisfied at least the above- emphasized limitations of claim 33.
Neither of Sheffield or Liu describes "controlling an output mode of the at least one image sensor" to operate in a first mode with "indications of light intensity" or a second mode with "indications of change in a light property" As neither of Sheffield or Liu describes these limitations, the proposed combination of the cited references would not have satisfied at least these limitations.
For at least these reasons, independent claim 33, and each claim that depends therefrom,
patentably distinguishes over the proposed combination of Sheffield and Liu and is believed to be in condition for allowance. Accordingly, it is respectfully requested that the rejections under 35 U.S.C. § 103 of independent claim 33 and each claim that depends therefrom, be withdrawn.
General Comments on Dependent Claims
Since each of the dependent claims depends from a base claim that is believed to be in
condition for allowance, for the sake of brevity, Applicant believes that it is unnecessary at this time to argue the further distinguishing features of the dependent claims. However, Applicant does not necessarily concur with the interpretation of the previously presented dependent claims as set forth in the Office Action, nor does Applicant concur that the basis for rejection of any of the previously presented dependent claims is proper. Therefore, Applicant reserves the right to specifically address the further patentability of the dependent claims in the future.
CONCLUSION