Office Action 1: Non-Final Rejection (2024-09-20)
Rejection types: 101 - Patent-Eligible Subject Matter, 102 - Lack of Novelty, 103 - Obviousness, 112 - Written Description/Enablement

CLAIMS:
CLAIMS

1\. A method (400) for assisting a medical procedure on a patient (106), wherein the method (400) comprises, under the control of a computing device (130):
acquiring (406), by the computing device (130), a luminescence image (205F)
of a field of view (103) containing a region of interest (109) for the medical procedure of the patient (106), the region of interest (109) comprising at least one target body (112) of the medical procedure containing a luminescence substance, and one or more foreign objects (115-124) different from the region of interest (109), the luminescence image (205F) comprising a plurality of luminescence values representative of a luminescence light being emitted by the luminescence substance at corresponding luminescence locations of the field of view (103), acquiring (408), by the computing device (130), an auxiliary image (205R) of the field of view (103), the auxiliary image (205R) comprising a plurality of auxiliary values representative of an auxiliary light, different from the luminescence light, being received from corresponding auxiliary locations of the field of view (103), identifying (410-424), by the computing device (130), an auxiliary informative region (210Ri) of the auxiliary image (205R) representative of the region of interest (109) without the foreign objects (115-124) according to a content of the auxiliary image (205R), identifying (428), by the computing device (130), a luminescence informative region (210Fi) of the luminescence image (205F) corresponding to the auxiliary informative region (210Ri), generating (432-466), by the computing device (130), a processed luminescence image (220Fi,225Fi) by processing the luminescence image (205F) limited to the luminescence informative region (210Fi), the processing of the luminescence image (205F) being based on the luminescence values of the luminescence informative region (210Fi) for facilitating an identification of a representation of the target body (112) therein, and
outputting (468-470), by the computing device (130), an output image based on the processed luminescence image (220Fi,225Fi).

2\. The method (400) according to claim 1, wherein the method (400) is for assisting a surgical procedure on the patient (106), said outputting (468-470) the output image comprising:
displaying (470), by the computing device (130), the output image substantially in real-time with said acquiring (406) the luminescence image (205F).

3\. The method (400) according to claim 2, wherein the region of interest is a surgical cavity (109) of the patient (106).

4\. The method (400) according to any claim from 1 to 3, wherein at least part of the foreign objects (115-124) overlaps the region of interest (109).

5\. The method (400) according to any claim from 1 to 4, wherein the foreign objects (115-124) comprise one or more medical instruments (115), one or more hands (118), one or more medical tools (121), one or more body-parts (124) of the patient (106) not of interest for the medical procedure and/or background materials (125).

6\. The method (400) according to any claim from 1 to 5, wherein the auxiliary image is a reflectance image (205R), the auxiliary light is a visible light and the auxiliary values are representative of the visible light being reflected by the corresponding auxiliary locations of the field of view (103) illuminated by a white light.

7\. The method (400) according to any claim from 1 to 6, wherein the luminescence substance is a luminescence agent being pre-administered to the patient (106) before performing the method (400).

8\. The method (400) according to any claim from 1 to 7, wherein said identifying (410-424) the auxiliary informative region (210Ri) comprises:
segmenting (414-424), by the computing device (130), the auxiliary image (205R) semantically into the auxiliary informative region (210Ri) corresponding to at least one region of interest class of the region of interest (109) and an auxiliary non- informative region (210Rn) corresponding to one or more foreign object classes of the foreign objects (115-124).

9\. The method (400) according to claim 8, wherein said segmenting (410-424) the auxiliary image (205R) comprises:
segmenting (424), by the computing device (130), the auxiliary image (205R) semantically with a neural network.

10\. The method (400) according to claim 8, wherein said segmenting (410-424) the auxiliary image (205R) comprises:
determining (418), by the computing device (130), one or more feature maps for corresponding features of the auxiliary image (205R), each of the feature maps comprising corresponding feature values of the auxiliary locations, and
segmenting (420-422), by the computing device (130), the auxiliary image (205R) semantically by applying a classification algorithm to the feature values of the feature maps.

11\. The method (400) according to any claim from 1 to 10, wherein said identifying (410-424) the auxiliary informative region (210Ri) comprises, when the auxiliary image (205R) comprises one or more disconnected portions completely surrounded by the auxiliary informative region (210Ri):
assigning (422), by the computing device (130), the disconnected portions to the auxiliary informative region (210Ri).

12\. The method (400) according to any claim from 1 to 11, wherein the method (400) comprises:
pre-processing (410), by the computing device (130), the auxiliary image (205R) before said identifying (412-424) the auxiliary informative region (210Ri) by applying a histogram equalization to the auxiliary image (205R) in response to a brightness indicator of the auxiliary image (205R) being comprised between a
darkness threshold, indicative of a feasibility of said identifying (410-424) the auxiliary informative region (210Ri), and a brightness threshold, higher than the darkness threshold.

13\. The method (400) according to any claim from 1 to 12, wherein said identifying (410-424) the auxiliary informative region (210Ri) comprises: identifying (414-424), by the computing device (130), the auxiliary informative region (210Ri) further according to a content of the luminescence image (205F).

14\. The method (400) according to any claim from 1 to 13, wherein said generating (432-466) the processed luminescence image (220Fi,225Fi) comprises:
auto-scaling (436-446), by the computing device (130), the luminescence informative region (210Fi) according to the luminescence values of the luminescence informative region (210Fi).

15\. The method (400) according to claim 14, wherein said auto-scaling (436- 446) the luminescence informative region (210Fi) comprises:
determining (436), by the computing device (130), a luminescence range of the luminescence values of the luminescence informative region (210Fi), and
converting (438-446), by the computing device (130), the luminescence values of the luminescence informative region (210Fi) according to a mapping function mapping the luminescence range to a display range for displaying the luminescence
image (205F).

16\. The method (400) according to any claim from 1 to 13, wherein said generating (432-466) the processed luminescence image (220Fi,225Fi) comprises:
thresholding (448-464), by the computing device (130), the luminescence informative region (210Fi) according to the luminescence values of the luminescence
informative region (210Fi) thereby partitioning the luminescence informative region (210Fi) into a target segment representative of the target body (112) and a non-target segment representative of a rest of the region of interest (109) different from the target body (112), and wherein said outputting (468-470) the output image comprises:
outputting (470), by the computing device (130), the output image by highlighting the target segment with respect to the non-target segment.

17\. The method (400) according to claim 16, wherein said thresholding (448- 464) the luminescence informative region (210Fi) comprises:
determining (448), by the computing device (130), a threshold value according to a statistical distribution of the luminescence values of the luminescence informative (210Fi), and
assigning (450-460), by the computing device (130), each of the luminescence locations of the luminescence informative region (210Fi) to the target segment or to
the non-target segment according to a comparison of the corresponding luminescence value with the threshold value.

18\. The method (400) according to claim 16 or 17, wherein said thresholding (448-464) the luminescence informative region (210Fi) comprises:
calculating (462), by the computing device (130), one or more target statistical
parameters of the luminescence values of the target segment and/or one or more non- target statistical parameters of the luminescence values of the non-target segment, and
updating (464), by the computing device (130), the luminescence values of the target segment each according to the target statistical parameters and/or the non-target statistical parameters.

19\. The method (400) according to any claim from 1 to 18, wherein the method (400) comprises:
determining (426), by the computing device (130), corresponding optical values, of at least one optical parameter relating to the luminescence light, for the auxiliary locations of the auxiliary image (205F) limited to the auxiliary informative
region (210Ri) according to a content thereof, and wherein said generating (432-466) the processed luminescence image (220Fi,225Fi) comprises:
equalizing (432), by the computing device (130), the luminescence values of the luminescence informative region (210Fi) according to the optical values.

20\. The method (400) according to any claim from 1 to 19, wherein the luminescence substance is a fluorescence substance, the luminescence image being a fluorescence image (205F) and the luminescence values being representative of a fluorescence light emitted by the fluorescence substance at the corresponding luminescence locations illuminated by an excitation light thereof.

21\. A computer program (300) configured for causing a computing device (130) to perform the method (400) according to any claim from 1 to 20 when the computer program (300) is executed on the computing device (130).

22\. A computer program product comprising one or more computer readable storage media having program instructions collectively stored on the readable storage media, the program instructions readable by a computing device to cause the computing device to perform the method of any claim from 1 to 20.

23\. A computing device (130) comprising means (300) configured for performing the steps of the method (400) according to any claim from 1 to 20.

24\. An imaging system (100) comprising:
the computing device (130) of claim
23, an illumination unit (133,139) for applying an excitation light adapted to exciting the luminescence substance to the field of view (103), and
an acquisition unit (145-157) for acquiring the luminescence image (205F) and
the auxiliary image (205R).

25\. A medical procedure on a patient comprising:
imaging a field of view containing a region of interest for the medical procedure of the patient, the region of interest comprising at least one target body of the medical procedure containing a luminescence substance, and one or more foreign
objects different from the region of interest according to the method of any claim from 1 to 20 to output the output image, and
performing the medical procedure with an assistance of the output image.

26\. The medical procedure according to claim 25, wherein the medical procedure comprises:
administering a luminescence agent containing the luminescence substance to the patient. 5 10 15 20 25

OFFICE ACTION:
DETAILED ACTION
This office action is in response to the communication received on 08/19/2022 concerning application no. 17/801,218 filed on 08/19/2022.
Claims 1-26 are pending.
Notice of Pre-AIA  or AIA  Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA  35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art.  The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph:
(A)	the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B)	the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C)	the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.  Such claim limitation(s) is/are:
“an illumination unit for applying an excitation light…” in claim 24: Paragraph 0028 teaches “Starting from the imaging probe 127, it has an illumination unit (for illuminating the field of view 103)… In the illumination unit, an excitation light source 133 and a white light source 136 generate an excitation light and a white light, respectively.” Paragraph 0118 teaches “In an embodiment, the imaging system comprises an illumination unit for applying an excitation light adapted to exciting the luminescence substance to the field of view. However, the illumination unit may be of any type (for example, based on laser, LEDs, UV/halogen/Xenon lamps, providing the white light as well or not, and so on).”
“an acquisition unit for acquiring…” in claim 24: Paragraph 0028 teaches “In the acquisition unit, collection optics 145 collect light from the field of view 103 (in an epi-illumination geometry).” Paragraph 0119 teaches “However, the acquisition unit may be of any type (for example, based on any number and type of lenses, wave guides, mirrors, CCD, ICCD, EMCCD, CMOS, InGaAs or PMT sensors, and so on).”
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, applicant may:  (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA  35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 23 (Recites “A computing device comprising means configured for performing the steps of the method according to claim 1”) is rejected under 35 U.S.C. 112(a) or pre-AIA  35 U.S.C. 112, first paragraph, because the claim purports to invoke 35 U.S.C. 112(f) or pre-AIA  35 U.S.C. 112, sixth paragraph, but fails to recite a combination of elements as required by that statutory provision and thus cannot rely on the specification to provide the structure, material or acts to support the claimed function.  As such, the claim recites a function that has no limits and covers every conceivable means for achieving the stated function, while the specification discloses at most only those means known to the inventor.  Accordingly, the disclosure is not commensurate with the scope of the claim.
The following is a quotation of 35 U.S.C. 112(b):
(b)  CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-26 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA  35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 is indefinite for the following reasons:
Lines 26, recite “facilitating an identification of a representation of the target body therein”. This claim element is indefinite. It is unclear if this identification of the representation of the target body is the same as the identification of the auxiliary informative region of the auxiliary image representative of the region of interest (ROI) in lines 16-17. Line 6 of the claim establishes that “the region of interest comprising at least one target body”. Given this, the identification is happening of a target body that is associated to the ROI. Furthermore, the lines 19-21 then establish the luminescence region is corresponding to that auxiliary informative region. It is unclear if the element in question is establishing a different identification.
Applicant is encouraged to clarify the identification relationship to the other identifying steps.
Claim 2 is indefinite for the following reasons:
Line 2, recite “a surgical procedure”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the “surgical procedure” is the same as the “medical procedure” established in claim 1 or is a separate and distinct feature.
For purposes of examination, the Office is considering them to be the same.
Claim 5 is indefinite for the following reasons:
Lines 3, recite “one or more medical tools”. This claim element is indefinite. It is unclear what the difference is between “medical instruments” as established in line 2 and the “medical tools” in the element in question. An instrument is defined as “a mechanical tool or implement, especially one used for delicate or precision work”.1
For purposes of examination, the Office is considering them to be the same.
Claim 21 is indefinite for the following reasons:
Line 1-2, recite “a computing device”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the “computing device” is the same as the “computing device” established in claim 1 or is a separate and distinct feature.
For purposes of examination, the Office is considering them to be the same.
Claim 22 is indefinite for the following reasons:
Lines 3, recite “a computing device”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the “computing device” is the same as the “computing device” established in claim 1 or is a separate and distinct feature.
For purposes of examination, the Office is considering them to be the same.
Claim 23 is indefinite for the following reasons:
Line 1, recite “a computing device”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the “computing device” is the same as the “computing device” established in claim 1 or is a separate and distinct feature.
For purposes of examination, the Office is considering them to be the same.
Claim 25 is indefinite for the following reasons:
Line 1, recite “A medical procedure”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the “medical procedure” is the same as the “medical procedure” established in claim 1 or is a separate and distinct feature.
For purposes of examination, the Office is considering them to be the same.
Line 1, recite “a patient”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the “patient” is the same as the “patient” established in claim 1 or is a separate and distinct feature.
For purposes of examination, the Office is considering them to be the same.
Line 2, recite “a field of view”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the “field of view” is the same as the “field of view” established in claim 1 or is a separate and distinct feature.
For purposes of examination, the Office is considering them to be the same.
Line 2, recite “a region of interest”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the “region of interest” is the same as the “region of interest” established in claim 1 or is a separate and distinct feature.
For purposes of examination, the Office is considering them to be the same.
Line 4, recite “a luminescence substance”. This claim element is indefinite. It would be unclear to one with ordinary skill in the art if the “luminescence substance” is the same as the “luminescence substance” established in claim 1 or is a separate and distinct feature.
For purposes of examination, the Office is considering them to be the same.
Claims that are not discussed above but are cited to be rejected under 35 U.S.C. 112(b) are also rejected because they inherit the indefiniteness of the claims they respectively depend upon.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 21 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.  The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the claim recites the limitation “A computer program” This limitation as drafted, under broadest reasonable interpretation, covers a transitory form of signal transmission. MPEP 2106.03 recites “When the BRI encompasses transitory forms of signal transmission, a rejection under 35 U.S.C. 101 as failing to claim statutory subject matter would be appropriate.”
Claim 22 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.  The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the claim recites the limitation “A computer program product comprising one or more computer readable storage media having program instructions collectively stored on the readable storage media” This limitation as drafted, under broadest reasonable interpretation, covers a transitory form of signal transmission. MPEP 2106.03 recites “When the BRI encompasses transitory forms of signal transmission, a rejection under 35 U.S.C. 101 as failing to claim statutory subject matter would be appropriate.”
Applicant is encouraged to limit the medium to a non-transitory computer-readable storage medium.
Claims 1-26 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a method for assisting a medical procedure on a patient and therefore, is a method.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “identifying an auxiliary informative region of the auxiliary image representative of the region of interest without the foreign objects according to a content of the auxiliary image, identifying a luminescence informative region of the luminescence image corresponding to the auxiliary informative region, generating a processed luminescence image by processing the luminescence image limited to the luminescence informative region, the processing of the luminescence image being based on the luminescence values of the luminescence informative region for facilitating an identification of a representation of the target body therein”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to the identification of a region related to an ROI on an auxiliary image and the assessment of that same region on the luminescence image and the generation of a processed image based on the luminescence values for further identification of a representation of the target body. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “acquiring, by the computing device, a luminescence image of a field of view containing a region of interest for the medical procedure of the patient, the region of interest comprising at least one target body of the medical procedure containing a luminescence substance, and one or more foreign objects different from the region of interest , the luminescence image comprising a plurality of luminescence values representative of a luminescence light being emitted by the luminescence substance at corresponding luminescence locations of the field of view, acquiring, by the computing device, an auxiliary image of the field of view, the auxiliary image comprising a plurality of auxiliary values representative of an auxiliary light, different from the luminescence light, being received from corresponding auxiliary locations of the field of view, and outputting, by the computing device, an output image based on the processed luminescence image” and “by the computing device”. Acquisition of the luminescence and the auxiliary images are data gathering steps which is a form of a pre-solution insignificant activity. The output of the image that has been processed is a display step which merely amounts to a post-solution insignificant activity. The use of computing device does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.  For these reasons, there is no inventive concept in the claim. In light of the above, claim 1 is ineligible.
Claim 2 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 1 rejection.
Claim 2 recites the following elements: “wherein the method is for assisting a surgical procedure on the patient, said outputting the output image comprising: displaying, by the computing device, the output image substantially in real-time with said acquiring the luminescence image”. This claim element is a mere displaying step which amounts to a post-solution insignificant activity. The use of a computing device does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception. This post-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 2 is ineligible.
Claim 3 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 2 rejection.
Claim 3 recites the following elements: “wherein the region of interest is a surgical cavity of the patient”. This claim element is a mere data gathering step which amounts to a pre-solution insignificant activity. This pre-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 3 is ineligible.
Claim 4 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 1 rejection.
Claim 4 recites the following elements: “wherein at least part of the foreign objects overlaps the region of interest”. This claim element is a mere data gathering step which amounts to a pre-solution insignificant activity. This pre-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 4 is ineligible.
Claim 5 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 1 rejection.
Claim 5 recites the following elements: “wherein the foreign objects comprise one or more medical instruments, one or more hands, one or more medical tools, one or more body parts of the patient not of interests for the medical procedure and/or background materials”. This claim element is a mere data gathering step which amounts to a pre-solution insignificant activity. This pre-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 5 is ineligible.
Claim 6 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 1 rejection.
Claim 6 recites the following elements: “wherein the auxiliary image is a reflectance image, the auxiliary light is a visible light and the auxiliary values are representative of the visible light being reflected by the corresponding auxiliary locations of the field of view illuminated by a white light”. This claim element is a mere data gathering step which amounts to a pre-solution insignificant activity. This pre-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 6 is ineligible.
Claim 7 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 1 rejection.
Claim 1 recites the following elements: “wherein the luminescence substance is a luminescence agent being pre-administered to the patient before performing the method”. This claim element is a mere data gathering step which amounts to a pre-solution insignificant activity. This pre-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 7 is ineligible.
Claim 8 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a method and therefore, is a method.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “segmenting the auxiliary image semantically into the auxiliary informative region corresponding to at least one region of interest class of the region of interest and an auxiliary non-informative region corresponding to one or more foreign object classes of the foreign objects”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to the segmentation of the regions of the image to correspond to a ROI or to a foreign object. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “by the computing device”. The use of a computing device does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.  For these reasons, there is no inventive concept in the claim. In light of the above, claim 8 is ineligible.
Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a method and therefore, is a method.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “segmenting the auxiliary image semantically”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to segmentation of an auxiliary image semantically. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “with a neural network” and “by the computing device”. The use of a computing device and a neural network does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.  For these reasons, there is no inventive concept in the claim. In light of the above, claim 9 is ineligible.
Claim 10 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a method and therefore, is a method.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “determining one or more feature maps for corresponding features of the auxiliary image, each of the feature maps comprising corresponding feature values of the auxiliary locations, and segmenting the auxiliary image semantically by applying a classification algorithm to the feature values of the feature maps”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to the determination of feature maps for corresponding features of locations in an image and segmenting of the image based on a procedure of classification. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “by the computing device”. The use of a computing device does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.  For these reasons, there is no inventive concept in the claim. In light of the above, claim 10 is ineligible.
Claim 11 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a method and therefore, is a method.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “assigning the disconnected portions to the auxiliary informative region”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to assigning portions to the region that has been identifies. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “by the computing device”. The use of a computing device does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.  For these reasons, there is no inventive concept in the claim. In light of the above, claim 11 is ineligible.
Claim 12 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a method and therefore, is a method.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “pre-processing the auxiliary image before said identifying the auxiliary informative region by applying a histogram equalization to the auxiliary image in response to a brightness indicator of the auxiliary image being comprised between a darkness threshold, indicative of a feasibility of said identifying the auxiliary informative region, and a brightness threshold, higher than the darkness threshold”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to pre-processing according to histogram equalization that is regard to brightness values that are thresholded for the identification of regions. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “by the computing device”. The use of a computing device does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.  For these reasons, there is no inventive concept in the claim. In light of the above, claim 12 is ineligible.
Claim 13 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a method and therefore, is a method.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “identifying the auxiliary informative region further according to a content of the luminescence image”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to the identification of the region according to the content of a luminescence image. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “by the computing device”. The use of a computing device does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.  For these reasons, there is no inventive concept in the claim. In light of the above, claim 13 is ineligible.
Claim 14 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a method and therefore, is a method.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “auto-scaling the luminescence informative region according to the luminescence values of the luminescence informative region”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to scaling the luminescence information regions according to luminescence values value. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “by the computing device”. The use of a computing device does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.  For these reasons, there is no inventive concept in the claim. In light of the above, claim 14 is ineligible.
Claim 15 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a method and therefore, is a method.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “determining a luminescence range of the luminescence values of the luminescence informative region, and converting the luminescence values of the luminescence informative region according to a mapping function mapping the luminescence range to a display range for displaying the luminescence image”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to the determination for luminescence range and the conversion of the luminescence values according to a mapping function. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “by the computing device”. The use of a computing device does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.  For these reasons, there is no inventive concept in the claim. In light of the above, claim 15 is ineligible.
Claim 16 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a method and therefore, is a method.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “thresholding the luminescence informative region according to the luminescence values of the luminescence informative region thereby partitioning the luminescence informative region into target segment representative of the target body and a non-target segment representative of a rest of the region of interest different from the target body”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to thresholding the region according to luminescence image to differentiate between target and non-target regions. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “wherein said outputting the output image comprises: outputting, by the computing device, the output image by highlighting the target segment with respect to the non-target segment” and “by a computing device”. Outputting the highlighted target is a display step which merely amounts to a post-solution insignificant activity. The use of a computing device does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.  For these reasons, there is no inventive concept in the claim. In light of the above, claim 16 is ineligible.
Claim 17 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a method and therefore, is a method.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “determining a threshold value according to a statistical distribution of the luminescence values of the luminescence informative, and assigning each of the luminescence locations of the luminescence informative region to the target segment or to the non-target segment according to a comparison of the corresponding luminescence value with the threshold value”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to the determination of threshold value according to a statistical distribution and the assignment  of the locations to be a target or non-target according to thresholding. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “by the computing device”. The use of a computing device does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception. For these reasons, there is no inventive concept in the claim. In light of the above, claim 17 is ineligible.
Claim 18 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a method and therefore, is a method.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “calculating one or more target statistical parameters of the luminescence values of the target segment and/or one or more non-target statistical parameters of the luminescence values of the non-target segment, and updating the luminescence values of the target segment each according to the target statistical parameters and/or the non-target statistical parameters”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to the calculation of the target or non-target parameters and updating the luminescence valuers according to the parameters. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “by the computing device”. The use of a computing device does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.  For these reasons, there is no inventive concept in the claim. In light of the above, claim 18 is ineligible.
Claim 19 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a method and therefore, is a method.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “determining corresponding optical values, of at least one optical parameter relating to the luminescence light, for the auxiliary locations of the auxiliary image limited to the auxiliary informative region according to a content thereof, and wherein said generating the processed luminescence image comprises: equalizing the luminescence values of the luminescence informative region according to the optical values”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to the determination of optical values that are used in the generation of the processed luminescence image that equalizes the luminescence values according to the optical values. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “by the computing device”. The use of a computing device does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.  For these reasons, there is no inventive concept in the claim. In light of the above, claim 19 is ineligible.
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 1 rejection.
Claim 20 recites the following elements: “wherein the luminescence substance is a fluorescence substance, the luminescence image being a fluorescence image and the luminesce values being representation of a fluorescence light emitted by the fluorescence substance at the corresponding luminescence locations illuminate by an excitation light thereof”. This claim element is a mere data gathering step which amounts to a pre-solution insignificant activity. This pre-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 20 is ineligible.
Claim 21 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 1 rejection.
Claim 21 recites the following elements: “A computer program configured for causing a computing device to perform the method according to claim 1 when the computer program is executed on the computing device”. The use of a computer program that causes the computing device to perform the method does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception. This extra-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 21 is ineligible.
Claim 22 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 1 rejection.
Claim 22 recites the following elements: “A computer program product comprising one or more computer readable storage media having program instructions collectively stored on the readable storage media, the program instructions readable by a computing device to cause the computing device to perform the method of claim 1”. The use of a computer product that causes the computing device to perform the method does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception. This extra-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 22 is ineligible.
Claim 23 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 1 rejection.
Claim 23 recites the following elements: “A computing device comprising means configured for performing the steps of the method according to claim 1”. The use of computing device to perform the method does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception. This extra-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 23 is ineligible.
Claim 24 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 23 rejection.
Claim 24 recites the following elements: “the computing device of claim 23, an illumination unit for applying an excitation light adapted to exciting the luminescence substance to the field of view, and an acquisition unit for acquiring the luminescence image and the auxiliary image”. This claim element is a mere data gathering step which amounts to a pre-solution insignificant activity. The use of a computing device does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception. This extra-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 24 is ineligible.
Claim 25 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Statutory Category: Yes - The claims recite a method and therefore, is a method.
Step 2A, Prong 1, Judicial Exception: Yes - The claim recites the limitation “performing the medical procedure with an assistance of the output image”.
This limitation, as drafted, is a process step that, under its broadest reasonable interpretation, covers the performance of the limitation in the mind as it is regarding a concept relating to the performance a subsequent observation and treatment plan according to the output image. That is, nothing in the claim element precludes the step from practically being performed in the mind and/or being performed with the aid of a pen and paper. Accordingly, the claim recites a mental process-type abstract idea.
Step 2A, Prong 2, Integrated into Practical Application: No - The claim recites the following additional elements: “imaging a field of view containing a region of interest for the medical procedure of the patient, the region of interest comprising at least one target body of the medical procedure containing a luminescence substance, and one or more foreign objects different from the region of interest according to the method of claim 1 to output the output image”. Imaging the field of view is a data gathering step which is a form of a pre-solution insignificant activity. The use of a display or a computing device does not integrate the judicial exception into a practical application as it is merely used to perform the judicial exception.
These additional elements, taken individually or in combination, merely amount to insignificant pre/post-solution activities and do not integrate the judicial exception into a practical application. This claim is therefore directed to an abstract idea.
Step 2B, Inventive Concept: No - Similarly to Step 2A Prong 2, the additional claim elements merely recite insignificant extra-solution activities, which do not amount to significantly more than the judicial exception.  For these reasons, there is no inventive concept in the claim. In light of the above, claim 25 is ineligible.
Claim 26 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 and Step 2A, Prong 1, Judicial Exception are discussed above in the claim 25 rejection.
Claim 26 recites the following elements: “wherein the medical procedure comprises: administering a luminescence agent containing the luminescence substance to the patient”. This claim element is a mere data gathering step which amounts to a pre-solution insignificant activity. This pre-solution insignificant activity does not integrate the judicial exception into a practical application nor does it contain an inventive step. In light of above, claim 26 is ineligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA  35 U.S.C. 102 and 103 (or as subject to pre-AIA  35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 5-7, 13, and 20-26 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kishima (PGPUB No. US 2019/0046041).
Regarding claim 1, Kishima teaches a method for assisting a medical procedure on a patient, wherein the method comprises, under the control of a computing device:
acquiring, by the computing device (Paragraph 0036 teaches that the image processing apparatus is constituted of hardware elements like a CPU, read only memory (ROM), and random access memory (RAM)), a luminescence image of a field of view containing a region of interest for the medical procedure of the patient (Paragraph 0050 teaches the capture of the entire surgical field in the fluorescence. This includes the ROI. Fig. 1 shows the display of the fluorescence area and the ROI), the region of interest comprising at least one target body of the medical procedure containing a luminescence substance (Paragraph 0050 teaches the capture of the entire surgical field in the fluorescence. This includes cancerous portions. Paragraph 0040 teaches the application of a contrast medium to the surgical area. Fig. 1 shows the peripheral area of the ROI around the fluorescence area), and one or more foreign objects different from the region of interest (Paragraph 0056 teaches that needles and clamps are within the field of view. Fig. 1 shows peripheral areas in the overall image field of view, the surgical area, and the ROI that is not the fluorescent area), the luminescence image comprising a plurality of luminescence values representative of a luminescence light being emitted by the luminescence substance at corresponding luminescence locations of the field of view (Paragraph 0047 teaches that the pixel-by-pixel display of the fluorescence is performed. A pixel is a unit that is indicative of an intensity value),
acquiring, by the computing device, an auxiliary image of the field of view, the auxiliary image comprising a plurality of auxiliary values representative of an auxiliary light, different from the luminescence light, being received from corresponding auxiliary locations of the field of view (Paragraph 0032 teaches that the visible light capturing system is capturing video camera information. Fig. 1 shows that it is capturing the same image storing area as the fluorescence system. Paragraph 0088-89 teaches the illimitation of visible light via a visible light LED. The display shows the surgical area),
identifying, by the computing device, an auxiliary informative region of the auxiliary image representative of the region of interest without the foreign objects according to a content of the auxiliary image (Paragraph 0032 teaches that the video camera can capture the surgical field and provide an ROI while the operator is considering which area to remove. Fig. 1 does not show the presence of the needle in the display or the surgical area),
identifying, by the computing device, a luminescence informative region of the luminescence image corresponding to the auxiliary informative region (Paragraph 0032 teaches that the video camera can capture the surgical field and provide an ROI while the operator is considering which area to remove),
generating, by the computing device, a processed luminescence image by processing the luminescence image limited to the luminescence informative region, the processing of the luminescence image being based on the luminescence values of the luminescence informative region for facilitating an identification of a representation of the target body therein (Paragraph 0047 teaches that the pixel-by-pixel display of the fluorescence is performed. A pixel is a unit that is indicative of an intensity value. Paragraph 0013 teaches that the cutting out unit can allow the user to select a partial area of the fluorescence image and cut out the selected area of the ROI and allow for it to be displayed), and
outputting, by the computing device, an output image based on the processed luminescence image (Paragraph 0013 teaches that the cutting out unit can allow the user to select a partial area of the fluorescence image and cut out the selected area of the ROI and allow for it to be displayed).
Regarding claim 5, Kishima teaches the method in claim 1, as discussed above.
Kishima further teaches a method, wherein the foreign objects comprise one or more medical instruments, one or more hands, one ore more medical tools, one or more body parts of the patient not of interests for the medical procedure and/or background materials (Paragraph 0056 teaches that needles and clamps are within the field of view. Fig. 1 shows peripheral areas in the overall image field of view, the surgical area, and the ROI that is not the fluorescent area).
Regarding claim 6, Kishima teaches the method in claim 1, as discussed above.
Kishima further teaches a method, wherein the auxiliary image is a reflectance image, the auxiliary light is a visible light and the auxiliary values are representative of the visible light being reflected by the corresponding auxiliary locations of the field of view illuminated by a white light (Paragraph 0032 teaches that the video camera can capture the surgical field and provide an ROI while the operator is considering which area to remove. Paragraph 0032 teaches that the visible light capturing system is capturing video camera information. Fig. 1 shows that it is capturing the same image storing area as the fluorescence system. Paragraph 0088-89 teaches the illimitation of visible light via a visible light LED. The display shows the surgical area).
Regarding claim 7, Kishima teaches the method in claim 1, as discussed above.
Kishima further teaches a method, wherein the luminescence substance is a luminescence agent being pre-administered to the patient before performing the method (Paragraph 0040 teaches the delivery of the fluorescent contrast agent in advance for the surgical area).
Regarding claim 13, Kishima teaches the method in claim 1, as discussed above.
Kishima further teaches a method, wherein said identifying the auxiliary informative region comprises:
identifying, by the computing device, the auxiliary informative region further according to a content of the luminescence image (Paragraph 0006 teaches that the visible light and the fluorescence images can be superimposed based on the shared zoom and field of view matching).
Regarding claim 20, Kishima teaches the method in claim 1, as discussed above.
Kishima further teaches a method, wherein the luminescence substance is a fluorescence substance, the luminescence image being a fluorescence image and the luminesce values being representation of a fluorescence light emitted by the fluorescence substance at the corresponding luminescence locations illuminate by an excitation light thereof (Paragraph 0040 teaches the delivery of the fluorescent contrast agent in advance for the surgical area. Abstract teaches the image of fluorescence being obtained by applying excitation light to the observation target area to which a fluorescent reagent is added in advance).
Regarding claim 21, Kishima teaches the method in claim 1, as discussed above.
Kishima further teaches a computer program configured for causing a computing device to perform the method according to claim 1 when the computer program is executed on the computing device (Paragraph 0002 teaches that the disclosure is regarding a fluorescence navigation surgery system for performing fluorescence navigation surgery as surgery, an image processing apparatus used therefor, an image processing method, an information processing program, and a fluorescence observation system).
Regarding claim 22, Kishima teaches the method in claim 1, as discussed above.
Kishima further teaches a computer program product comprising one or more computer readable storage media having program instructions collectively stored on the readable storage media, the program instructions readable by a computing device to cause the computing device to perform the method of claim 1 (Paragraph 0002 teaches that the disclosure is regarding a fluorescence navigation surgery system for performing fluorescence navigation surgery as surgery, an image processing apparatus used therefor, an image processing method, an information processing program, and a fluorescence observation system. Paragraph 0045 teaches the use of an HDD).
Regarding claim 23, Kishima teaches the method in claim 1, as discussed above.
Kishima further teaches a computing device comprising means configured for performing the steps of the method according to claim 1 (Paragraph 0002 teaches that the disclosure is regarding a fluorescence navigation surgery system for performing fluorescence navigation surgery as surgery, an image processing apparatus used therefor, an image processing method, an information processing program, and a fluorescence observation system).
Regarding claim 24, Kishima teaches the computing device in claim 23, as discussed above.
Kishima further teaches an imaging system comprising:
the computing device of claim 23 (Paragraph 0002 teaches that the disclosure is regarding a fluorescence navigation surgery system for performing fluorescence navigation surgery as surgery, an image processing apparatus used therefor, an image processing method, an information processing program, and a fluorescence observation system),
an illumination unit for applying an excitation light adapted to exciting the luminescence substance to the field of view (Paragraph 0034 teaches that the LED is used as an excitation light source), and
an acquisition unit for acquiring the luminescence image and the auxiliary image (Fig. 1 shows fluorescence navigation surgery system with the visible light capturing system 300 and the fluorescence capturing system 200).
Regarding claim 25, Kishima teaches the method in claim 1, as discussed above.
Kishima further teaches a medical procedure on a patient comprising: imaging a field of view containing a region of interest for the medical procedure of the patient, the region of interest comprising at least one target body of the medical procedure containing a luminescence substance, and one or more foreign objects different from the region of interest according to the method of claim 1 to output the output image (Fig. 1 shows fluorescence navigation surgery system with the visible light capturing system 300 and the fluorescence capturing system 200. Paragraph 0050 teaches the capture of the entire surgical field in the fluorescence. This includes the ROI. Fig. 1 shows the display of the fluorescence area and the ROI. Paragraph 0050 teaches the capture of the entire surgical field in the fluorescence. This includes cancerous portions. Paragraph 0040 teaches the application of a contrast medium to the surgical area. Fig. 1 shows the peripheral area of the ROI around the fluorescence area. Paragraph 0056 teaches that needles and clamps are within the field of view. Fig. 1 shows peripheral areas in the overall image field of view, the surgical area, and the ROI that is not the fluorescent area), and
performing the medical procedure with an assistance of the output image (Paragraph 0014 teaches that the display is to facilitate a surgical operation).
Regarding claim 26, Kishima teaches the method in claim 25, as discussed above.
Kishima further teaches a method, administering a luminescence agent containing the luminescence substance to the patient (Paragraph 0040 teaches the delivery of the fluorescent contrast agent in advance for the surgical area).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA  35 U.S.C. 102 and 103 (or as subject to pre-AIA  35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2-3 are rejected under 35 U.S.C. 103 as being unpatentable over Kishima (PGPUB No. US 2019/0046041) in view of Zuzak et al. (PGPUB No. US 2010/0056928).
Regarding claim 2, Kishima teaches the method in claim 1, as discussed above.
While Kishima teaches the display of the images (Fig. 1), Kishima is silent regarding a method, wherein the method is for assisting a surgical procedure on the patient, said outputting the output image comprising:
displaying, by the computing device, the output image substantially in real-time with said acquiring the luminescence image.
In an analogous imaging field of endeavor, regarding fluorescent-based surgical performance, Zuzak teaches a method, wherein the method is for assisting a surgical procedure on the patient, said outputting the output image comprising:
displaying, by the computing device, the output image substantially in real-time with said acquiring the luminescence image (Paragraph 0026 teaches the performance of real-time analysis of fluorescent targets and can utilize surgical microscopes. Paragraph 0028 teaches the implementation for the disclosure in surgical procedures. Paragraph 0040-41 teaches real-time display and analysis).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Kishima with Zuzak’s teaching of real-time display of image data. This modified method would allow the user to provide real time processed images based on matching the reflectance of each pixel in an image to known reflectance spectra with a reduced required number of frames (Paragraph 0002 of Zuzak). Furthermore, the modification allows for implementation in cholecystectomy and can be used in treatment of gallstones (Paragraphs 0003-04 of Zuzak).
Regarding claim 3, modified Kishima teaches the method in claim 2, as discussed above.
Kishima further teaches a method, wherein the region of interest is a surgical cavity of the patient (Paragraph 0009 teaches that the area to be removed with the scalpel can be marked).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Kishima (PGPUB No. US 2019/0046041) in view of Benaron (PGPUB No. US 2004/0010192).
Regarding claim 4, Kishima teaches the method in claim 1, as discussed above.
However, Kishima is silent regarding a method, wherein at least part of the foreign objects overlaps the region of interest.
In an analogous imaging field of endeavor, regarding fluorescent-based surgical performance, Benaron teaches a method, wherein at least part of the foreign objects overlaps the region of interest (Fig. 11B shows the instrument and above the lesion. Paragraph 0147 teaches that the area is injected with a dye. Paragraph 0088 teaches that the dye can be fluorescent).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Kishima with Benaron’s teaching of foreign objects that overlap with the ROI. This modified method would allow the user to improve signal-to-noise ratio (SNR) while imaging the target tissue (Paragraph 0004 of Benaron). Furthermore, the modification allows for operating invasive cancer surgery or guided tumor biopsy, without requiring a darkened operating theater (Paragraph 0001 of Benaron).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Kishima (PGPUB No. US 2019/0046041) in view of Snyder (PGPUB No. US 2020/0237445).
Regarding claim 8, Kishima teaches the method in claim 1, as discussed above.
However, Kishima is silent regarding a method, wherein said identifying the auxiliary informative region comprises:
segmenting, by the computing device, the auxiliary image semantically into the auxiliary informative region corresponding to at least one region of interest class of the region of interest and an auxiliary non-informative region corresponding to one or more foreign object classes of the foreign objects.
In an analogous imaging field of endeavor, regarding visible light-based surgical performance, Snyder teaches a method, wherein said identifying the auxiliary informative region comprises:
segmenting, by the computing device, the auxiliary image semantically into the auxiliary informative region corresponding to at least one region of interest class of the region of interest and an auxiliary non-informative region corresponding to one or more foreign object classes of the foreign objects (Paragraph 0080 teaches that the anatomy can be segmented from the tracking devices. Paragraph 0150 teaches that the real time positioning can be performed from the portions of the image that are segmented. This can be of the anatomy and the instruments. Paragraph 0048 teaches that the imaging can be performed from a camera that images in the visible light spectra).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Kishima with Snyder’s teaching of segmentation of the ROI and the instruments of a visible light image. This modified method would allow the user to track and navigate a system for multiple procedures (0002 of Snyder). Furthermore, the modification allows for accurate imaging and assessment of the image data of the patient position (Paragraphs 0047-48 of Snyder).
Claims 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over Kishima (PGPUB No. US 2019/0046041) in view of Snyder (PGPUB No. US 2020/0237445) further in view of Dascalu (PGPUB No. US 2019/0231249).
Regarding claim 9, modified Kishima teaches the method in claim 8, as discussed above.
However, the combination of Kishima and Snyder is silent regarding a method, wherein said segmenting the auxiliary image comprises:
segmenting, by the computing device, the auxiliary image semantically with a neural network.
In an analogous imaging field of endeavor, regarding camera-based medical procedures for cancer study and treatment, Dascalu teaches a method, wherein said segmenting the auxiliary image comprises:
segmenting, by the computing device, the auxiliary image semantically with a neural network (Paragraph 0134 teaches the use of a CNN to segment the lesions).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Kishima and Snyder with Dascalu’s teaching of using a neural network for segmentations. This modified method would allow the user to increase the precision of diagnosis (Paragraph 0002 of Dascalu). Furthermore, the modification improves detection sensitivity and specificity (Paragraph 0007 of Dascalu).
Regarding claim 10, modified Kishima teaches the method in claim 8, as discussed above.
However, the combination of Kishima and Snyder is silent regarding a method, wherein said segmenting the auxiliary image comprises:
determining, by the computing device, one or more feature maps for corresponding features of the auxiliary image, each of the feature maps comprising corresponding feature values of the auxiliary locations, and
segmenting, by the computing device, the auxiliary image semantically by applying a classification algorithm to the feature values of the feature maps.
In an analogous imaging field of endeavor, regarding camera-based medical procedures for cancer study and treatment, Dascalu teaches a method, wherein said segmenting the auxiliary image comprises:
determining, by the computing device, one or more feature maps for corresponding features of the auxiliary image, each of the feature maps comprising corresponding feature values of the auxiliary locations (Paragraph 0033 teaches the generation of a feature map. Fig. 2shows a camera. Paragraph 0098 teaches the implementation of the data acquisition in the visible light spectra), and
segmenting, by the computing device, the auxiliary image semantically by applying a classification algorithm to the feature values of the feature maps (Paragraph 0033 teaches the determination of a feature map based on the deep learning classifier that is used to determine the patterns and using a clustering algorithm for the extraction. Paragraph 0058 teaches that the classification method includes cluster analysis).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combination of Kishima and Snyder with Dascalu’s teaching of segmentation with the use of feature maps. This modified method would allow the user to increase the precision of diagnosis (Paragraph 0002 of Dascalu). Furthermore, the modification improves detection sensitivity and specificity (Paragraph 0007 of Dascalu).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Kishima (PGPUB No. US 2019/0046041) in view of Ikehara (PGPUB No. US 2019/0247126).
Regarding claim 11, Kishima teaches the method in claim 1, as discussed above.
However, Kishima is silent regarding a method, wherein said identifying the auxiliary informative region comprises, when the auxiliary image comprises one or more disconnected portions completely surrounded by the auxiliary informative region:
assigning, by the computing device, the disconnected portions to the auxiliary informative region.
In an analogous imaging field of endeavor, regarding image analysis, Ikehara teaches a method, wherein said identifying the auxiliary informative region comprises, when the auxiliary image comprises one or more disconnected portions completely surrounded by the auxiliary informative region:
assigning, by the computing device, the disconnected portions to the auxiliary informative region (Paragraph 0089 teaches that the spots that are visible in the visible image can be assessed. See Fig. 7A that shows the regions to be separately located).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Kishima with Ikehara’s teaching of assigning the disconnected portions on the auxiliary information. This modified method would allow the user to motion track the targets of interest (Paragraph 0089 of Ikehara). Furthermore, the modification the motion tracking is accurate and with greater image quality as it is compatible with fluorescent imaging (Paragraph 0023 of Ikehara).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Kishima (PGPUB No. US 2019/0046041) in view of Meier et al. (PGPUB No. US 2007/0058862).
Regarding claim 12, Kishima teaches the method in claim 1, as discussed above.
However, Kishima is silent regarding a method, wherein the method comprises:
pre-processing, by the computing device, the auxiliary image before said identifying the auxiliary informative region by applying a histogram equalization to the auxiliary image in response to a brightness indicator of the auxiliary image being comprised between a darkness threshold, indicative of a feasibility of said identifying the auxiliary informative region, and a brightness threshold, higher than the darkness threshold.
In an analogous imaging field of endeavor, regarding image analysis, Meier teaches a method, wherein the method comprises:
pre-processing, by the computing device, the auxiliary image before said identifying the auxiliary informative region by applying a histogram equalization to the auxiliary image in response to a brightness indicator of the auxiliary image being comprised between a darkness threshold, indicative of a feasibility of said identifying the auxiliary informative region, and a brightness threshold, higher than the darkness threshold (Paragraphs 0015-16 teaches the use of histogram equalization to assess the regions by comparing it to a brightness threshold. The low reflectivity regions fall within a first region. The high reflectivity regions fall into a second region. These varying regions are providing enhanced contrast with respect to one another).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Kishima with Meier’s teaching of assessment of image data based on histogram equalization and thresholding. This modified method would allow the user to improve the contrast between the varying regions of interest (Paragraph 0015 of Meier). Furthermore, the modification provides reliable edge detection of multiple objects of interest within a single image (Abstract of Meier).
Claims 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Kishima (PGPUB No. US 2019/0046041) in view of Dana et al. (PGPUB No. US 2017/0055822).
Regarding claim 14, Kishima teaches the method in claim 1, as discussed above.
However, Kishima is silent regarding a method, wherein said generating the processed luminescence image comprises:
auto-scaling by the computing device, the luminescence informative region according to the luminescence values of the luminescence informative region.
In an analogous imaging field of endeavor, regarding image analysis, Dana teaches a method, wherein said generating the processed luminescence image comprises:
auto-scaling by the computing device, the luminescence informative region according to the luminescence values of the luminescence informative region (Paragraph 0012 teaches that the mapping can be performed based on the fluorophore value in relation to a predetermined range. The mapping can be according to hue saturation value).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Kishima with Dana’s teaching of mapping according to fluorophore values. This modified method would allow the user to assess the potential of disease inflictions based on color coding and mapping (Paragraph 0012 of Dana). Furthermore, the modification provides quantitative determination of a particular color content in an image or a portion of the image (Paragraph 0004 of Dana).
Regarding claim 15, Kishima teaches the method in claim 14, as discussed above.
However, Kishima is silent regarding a method, wherein said auto- scaling the luminescence informative region comprises:
determining, by the computing device, a luminescence range of the luminescence values of the luminescence informative region, and
converting, by the computing device, the luminescence values of the luminescence informative region according to a mapping function mapping the luminescence range to a display range for displaying the luminescence image.
In an analogous imaging field of endeavor, regarding image analysis, Dana teaches a method, wherein said auto- scaling the luminescence informative region comprises:
determining, by the computing device, a luminescence range of the luminescence values of the luminescence informative region (Paragraph 0012 teaches that the mapping can be performed based on the fluorophore value in relation to a predetermined range. The mapping can be according to hue saturation value), and
converting, by the computing device, the luminescence values of the luminescence informative region according to a mapping function mapping the luminescence range to a display range for displaying the luminescence image (Paragraph 0012 teaches that the mapping can be performed based on the fluorophore value in relation to a predetermined range. The mapping can be according to hue saturation value).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Kishima with Dana’s teaching of mapping according to fluorophore values. This modified method would allow the user to assess the potential of disease inflictions based on color coding and mapping (Paragraph 0012 of Dana). Furthermore, the modification provides quantitative determination of a particular color content in an image or a portion of the image (Paragraph 0004 of Dana).
Claims 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Kishima (PGPUB No. US 2019/0046041) in view of Kishima (PGPUB No. US 2011/0006220; hereinafter referred to as “Kishima ‘220”).
Regarding claim 16, Kishima teaches the method in claim 1, as discussed above.
However, Kishima is silent regarding a method, wherein said generating the processed luminescence image comprises:
thresholding, by the computing device, the luminescence informative region according to the luminescence values of the luminescence informative region thereby partitioning the luminescence informative region into target segment representative of the target body and a non-target segment representative of a rest of the region of interest different from the target body,
and wherein said outputting the output image comprises:
outputting, by the computing device, the output image by highlighting the target segment with respect to the non-target segment.
In an analogous imaging field of endeavor, regarding image analysis, Kishima ‘220 teaches a method, wherein said generating the processed luminescence image comprises:
thresholding, by the computing device, the luminescence informative region according to the luminescence values of the luminescence informative region thereby partitioning the luminescence informative region into target segment representative of the target body and a non-target segment representative of a rest of the region of interest different from the target body (Paragraph 0089 teaches that the luminescent sports are thresholded to ascertain the shape and area that they occupy. This is according to the ratio relationship as defined by the axial relationship. Paragraph 0090 teaches that the targets are then classified according to the selection via thresholding),
and wherein said outputting the output image comprises:
outputting, by the computing device, the output image by highlighting the target segment with respect to the non-target segment (Paragraph 0088 teaches that the markers are overlapped and mark the biological sample. Fig. 5 shows the outline of the sample).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Kishima with Kishima ‘220’s teaching of thresholding and displaying the outline of the target. This modified method would allow the user to accurately assess and compute the luminescent spots according to a cellular level (Paragraph 0113 of Kishima ‘220). Furthermore, the modification cell-based classification is precise (Paragraph 0090 of Kishima ‘220).
Regarding claim 17, Kishima teaches the method in claim 16, as discussed above.
However, Kishima is silent regarding a method, wherein said thresholding the luminescence informative region comprises:
determining, by the computing device, a threshold value according to a statistical distribution of the luminescence values of the luminescence informative, and
assigning, by the computing device, each of the luminescence locations of the luminescence informative region to the target segment or to the non-target segment according to a comparison of the corresponding luminescence value with the threshold value.
In an analogous imaging field of endeavor, regarding image analysis, Kishima ‘220 teaches a method, wherein said thresholding the luminescence informative region comprises:
determining, by the computing device, a threshold value according to a statistical distribution of the luminescence values of the luminescence informative (Paragraph 0089 teaches that the luminescent sports are thresholded to ascertain the shape and area that they occupy. This is according to the ratio relationship as defined by the axial relationship. Paragraph 0090 teaches that the targets are then classified according to the selection via thresholding), and
assigning, by the computing device, each of the luminescence locations of the luminescence informative region to the target segment or to the non-target segment according to a comparison of the corresponding luminescence value with the threshold value (Paragraph 0089 teaches that the luminescent sports are thresholded to ascertain the shape and area that they occupy. This is according to the ratio relationship as defined by the axial relationship. Paragraph 0090 teaches that the targets are then classified according to the selection via thresholding. Paragraph 0088 teaches that the markers are overlapped and mark the biological sample. Fig. 5 shows the outline of the sample. Paragraph 0083 teaches that the thresholding is according to the luminance to ascertain the target).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Kishima with Kishima ‘220’s teaching of thresholding and classifying the target. This modified method would allow the user to accurately assess and compute the luminescent spots according to a cellular level (Paragraph 0113 of Kishima ‘220). Furthermore, the modification cell-based classification is precise (Paragraph 0090 of Kishima ‘220).
Regarding claim 18, Kishima teaches the method in claim 16, as discussed above.
However, Kishima is silent regarding a method, wherein said thresholding the luminescence informative region comprises:
calculating, by the computing device, one or more target statistical parameters of the luminescence values of the target segment and/or one or more non-target statistical parameters of the luminescence values of the non-target segment, and
updating, by the computing device, the luminescence values of the target segment each according to the target statistical parameters and/or the non-target statistical parameters.
In an analogous imaging field of endeavor, regarding image analysis, Kishima ‘220 teaches a method, wherein said thresholding the luminescence informative region comprises:
calculating, by the computing device, one or more target statistical parameters of the luminescence values of the target segment and/or one or more non-target statistical parameters of the luminescence values of the non-target segment (Paragraph 0089 teaches that the luminescent sports are thresholded to ascertain the shape and area that they occupy. This is according to the ratio relationship as defined by the axial relationship. Paragraph 0090 teaches that the targets are then classified according to the selection via thresholding), and
updating, by the computing device, the luminescence values of the target segment each according to the target statistical parameters and/or the non-target statistical parameters (Paragraph 0089 teaches that the luminescent sports are thresholded to ascertain the shape and area that they occupy. This is according to the ratio relationship as defined by the axial relationship. Paragraph 0090 teaches that the targets are then classified according to the selection via thresholding. Paragraph 0081 teaches that the data can be updated).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Kishima with Kishima ‘220’s teaching of thresholding and classifying the target with updating. This modified method would allow the user to accurately assess and compute the luminescent spots according to a cellular level (Paragraph 0113 of Kishima ‘220). Furthermore, the modification cell-based classification is precise (Paragraph 0090 of Kishima ‘220).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Kishima (PGPUB No. US 2019/0046041) in view of Themelis (PGPUB No. US 2017/0237958).
Regarding claim 19, modified Kishima teaches the method in claim 1, as discussed above.
However, Kishima is silent regarding a method, wherein the method comprises:
determining, by the computing device, corresponding optical values, of at least one optical parameter relating to the luminescence light, for the auxiliary locations of the auxiliary image limited to the auxiliary informative region according to a content thereof, and wherein said generating the processed luminescence image comprises:
equalizing, by the computing device, the luminescence values of the luminescence informative region according to the optical values.
In an analogous imaging field of endeavor, regarding image analysis, Themelis teaches a method, wherein the method comprises:
determining, by the computing device, corresponding optical values, of at least one optical parameter relating to the luminescence light, for the auxiliary locations of the auxiliary image limited to the auxiliary informative region according to a content thereof (Paragraph 0069 teaches that the images of the visible light and the fluorescence light are adjusted according to their correlation and patterns and cropping, shifting, stretching, and/or rotating of the images can be performed to align them to one another. Paragraph 0014 teaches that the processing is according to the intensity values of the two images), and wherein said generating the processed luminescence image comprises:
equalizing, by the computing device, the luminescence values of the luminescence informative region according to the optical values (Paragraph 0069 teaches that the images of the visible light and the fluorescence light are adjusted according to their correlation and patterns and cropping, shifting, stretching, and/or rotating of the images can be performed to align them to one another. Paragraph 0014 teaches that the processing is according to the intensity values of the two images).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Kishima with Themelis’s teaching of equalizing the values of the two images such that they correspond. This modified method would allow the user to align the images to one another and ensure that the display of the images is consistent with one another (Paragraph 0069 of Themelis). Furthermore, the modification improves the quality of images displayed to the user (Paragraph 0003 of Themelis).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Sawada (PGPUB No. US 2005/0276475): Teaches the assessment of luminescence intensity and saturation and autoscaling.
Liang et al. (PGPUB No. US 2018/0225820): Teaches the use of classification to assess and generate feature maps. Teaches machine learning.
Godavarty (PGPUB No. US 2019/0008387): Teaches the segmentation of image data for targets and foreign bodies and non-targets.
Zhu et al. (PGPUB No. US 2016/0324420): Teaches thresholding data to obtain a highlight of a target that is then displayed.
Itou et al. (US Patent No. 12,036,059): Teaches the use of classification to assess and generate feature maps. Teaches machine learning.
Singer et al. (US Patent No. 5,866,331): Teaches autoscaling.
Landon et al. (US Patent No. 11,259,874): Teaches the use of classification to assess and generate feature maps. Teaches machine learning.

APPLICANT REBUTTAL: REMARKS
After entry of this Response, claims 1-7, 9-20, and 22-26 are pending in the patent application. Claims 8 and 21 have been cancelled without prejudice or disclaimer, claims 1- 2, 5, 9-13, 16, 19, 22-23, and 25 are currently amended, and no claims have been newly added. Applicant's attorney respectfully requests reconsideration of, and favorable action regarding, claims 1-7, 9-20, and 22-26 in view of this Response, which adds no new matter to the patent application.
Claim Interpretation of "Illumination Unit" and "Ac
q
uisition Unit" Under 35 U.S.C.
112(f)
Claim 24
Not only is the examiner's interpretation of "an illumination unit" as a "generic
placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier" incorrect, but Applicant does not understand how the examiner can so interpret "an illumination unit" in view of the structural description of "an illumination unit" in the patent application. FIG. 1 and paragraph [0028] of the patent application, which paragraph the examiner cited and quoted only partially, recites:
[0028] Starting from the imaging probe 127, it has an illumination unit (for
illuminating the field of view 103) and an acquisition unit for acquiring
images in the field of view (103) that comprise the following components. In
the illumination unit, an excitation light source 133 and a white light source
136 generate an excitation light and a white light, respectively. The excitation
light has wavelength and energy suitable to excite the fluorophores of the fluorescence agent (such as of Near Infra-Red, or NIR, type), whereas the
white light appears substantially colorless to the human eye (such as
containing all the wavelengths of the spectrum that is visible to the human eye at equal intensity). Corresponding delivery optics 139 and delivery optics 142 deliver the excitation light and the white light, respectively, to the (same) field
of view 103.
"Illumination unit" by itself recites sufficient structure to avoid a "means plus
function" interpretation because the recitation of "for applying an excitation light adapted to exciting the luminescence substance" inherently describes a light (structure) for generating a specified type of light (a light that can excite the luminescence substance).
But even assuming, arguendo, that "illumination unit," by itself, does not recite
sufficient structure to avoid a "means plus function" interpretation, paragraph [0028] of the patent application discloses structural "components" (e.g., excitation light source 133, white light source 136, delivery optics 139 and delivery optics 142) of the "illumination unit" sufficient to avoid a means-plus-function interpretation of "illumination unit" under 35 U.S.C. § 112(f).
A similar analysis and reasoning applies to the claim-24 term "acquisition unit,"
which is in FIG. 1 of the patent application, and about which paragraph [0028] of the patent application states:
In the acquisition unit, collection optics 145 collect light from the field of view
103 (in an epi-illumination geometry).
In view of the preceding analysis and reasoning, Applicant requests the examiner to interpret the claim-24 terms "illumination unit" and "acquisition unit" under standard claim- interpretation guidelines, not under 35 U.S.C. § 112(f).
Rej
ection of Claim 23 Under 35 U.S.C.
&
112(a)
Without acquiescing to the propriety of this rejection, Applicant has amended claim 23 in a manner that overcomes this rejection as set forth below.
The examiner rejected claim 23 as allegedly failing to recite a combination of elements supporting the claimed function.
Claim 23 has been amended to remove "means," and to specify the elements used to perform the corresponding steps of claim 1 (see claim 1 as amended, "A method for imaging a region of interest of a patient for a medical procedure performed thereon, wherein the method comprises, under the control of a computing device," see the patent application, p. 11, lines 16-22, "With reference now to FIG.3, the main software components are shown that may be used to implement the solution according to an embodiment of the present disclosure. ... The software components 300 are typically stored in the mass memory and loaded (at least in part) into the working memory of the central unit of the imaging system when the programs are running," and see the patent application p. 44, lines 16-17, "However, the computing device may be of any type (for example, a central unit of an imaging system, a separate computer and so on).").
Particularly, claim 23 has been amended to recite "a luminescence drive for ..." (see the patent application, p. 11, line 29 through page 12, line 2, "A fluorescence drive 305 drives a fluorescence unit of the imaging system (comprising the excitation light source and the fluorescence camera) dedicated to acquiring the fluorescence images of the field of view suitable illuminated for this purpose," and see the patent application, p. 33, lines 1-5, "However, the luminescence substance may be of any extrinsic/intrinsic or exogenous/endogenous type (for example, any luminescence agent, any natural luminescence component, based on any luminescence phenomenon, such as fluorescence, phosphorescence, chemiluminescence, bio-luminescence, induced Raman-radiation, and so on).").
Claim 23 has been amended further to recite "an auxiliary drive for ..." (see the
patent application, p. 12, lines 12-15, "Likewise, a reflectance drive 315 drives a reflectance unit of the imaging system (comprising the white light source and the reflectance camera) dedicated to acquiring the reflectance images of the field of view suitable illuminated for this purpose," and p. 34, lines 1-3, "the auxiliary light may be of any type different from the luminescence light of the luminescence image (for example, visible light, IR light, Ultra- Violet (UV) light, other luminescence light at different wavelength and so on).").
Claim 23 has been amended also to recite "a segmenter for ..." (see the patent
application, p. 13, lines 2-4, "A segmenter 335 segments the prepared reflectance images (semantically) into their informative regions and non-informative regions;").
Claim 23 has been amended to recite "an adapter for ..." (see the patent application, p. 13, line 31 through page 14, line 2, "An adapter 355 optionally adapts the reflectance segmentation masks and the reflectance equalization maps to the fluorescence images, so as to equalize their sizes and to synchronize them.").
Claim 23 has been amended to recite "a processor for ..." (see the patent application, p. 14, lines 17-19, "A processor 370 (post-)processes the fluorescence images limited to their informative regions for facilitating the identification of the representation of the tumor therein (for example, by auto-scaling and/or thresholding them).").
Claim 23 has been amended further to recite "an output drive for ..." (see the patent application, p. 15, lines 6-8, "A monitor drive 390 drives the monitor of the imaging system to display the output images (substantially in real-time during the surgical procedure)," and see the patent application, p. 35, lines 14-18, "However, the output image may be of any type (for example, the same processed luminescence image, the processed luminescence image
combined/overlaid with the reflectance image, and so on) and it may be output in any way (for example, displayed, printed, transmitted remotely, in real-time or off-line, and so on).").
Rej
ection of Claims 1-26 Under 35 U.S.C.
&
112(b)
Without acquiescing to the propriety of this rejection, Applicant has amended some of claims 1-26 in a manner that overcomes this rejection for all of claims 1-26.
The examiner rejected claim 1 as allegedly being indefinite with respect to the identifications specified in section 11 of the Office Action.
Claim 1 has been amended to recite
"wherein the processed luminescence ima
ge
provides
for facilitating an id
enti
fication of
a
f
urther
representation of the target body
there
in
facilitatin
g
an identification thereo
f,
"
so as to clarify that the processed luminescence image provides a further representation of the target body different from the one in the luminescence image, for example, resulting from its auto-scaling (see the patent application, p. 9, line 26 through p. 10, line 20) or thresholding (see p. 10, line 21 through p. 11, line 8) and then the target body is identified in this further representation thereof.
The examiner rejected claim 2 as being indefinite with respect to the "surgical procedure."
Claim 2 has been amended to recite "wherein
t
he me
thod is for assisting
the medical procedure is
a surgical procedure
on t
he
pati
e
nt,
" (see the patent application, p. 32, lines 2-4, "However, the method may be used for assisting any medical procedure (for example, a surgical procedure, a diagnostic procedure, a therapeutic procedure and so on).").
The examiner rejected claim 5 as allegedly being indefinite with respect to the "medical tools."
Claim 5 has been amended to recite "wherein the foreign objects comprise one or more medical
inst
rume
nts
im
p
lements
,
one or more hands,
on
e
or mor
e me
dical tools,
" so as to use a common word for both instruments and tools (see Gemini, "What is an implement? An implement is a tool or instrument used to carry out a task or achieve a specific purpose.").
The examiner rejected claim 21 as being indefinite with respect to the "computing device."
This rejection has been rendered moot by the cancellation of claim 21.
The examiner rejected claim 22 as allegedly being indefinite with respect to the "computing device."
Claim 22 has been amended to clarify the presence of a single computing device (see the patent application, p. 11, lines 16-22, "With reference now to FIG.3, the main software components are shown that may be used to implement the solution according to an embodiment of the present disclosure... The software components 300 are typically stored in the mass memory and loaded (at least in part) into the working memory of the central unit of the imaging system when the programs are running," and p. 44, lines 16-17, "However, the computing device may be of any type (for example, a central unit of an imaging system, a separate computer and so on).").
The examiner objected to claim 23 as allegedly being indefinite with respect to the "computing device."
Claim 23 has been amended to clarify the presence of a single computing device (see above).
The examiner objected to claim 25 as being indefinite with respect to features recited in claim 1.
Claim 25 has been amended to clarify the presence of a single instance of each feature.
Rej
ection of Claims 1-26 Under 35 U.S.C.
Without acquiescing to the propriety of this rejection, Applicant has amended some of claims 1-26 in a manner that overcomes this rejection for all of claims 1-26.
The examiner rejected claim 21 as allegedly being directed to a non-statutory subject matter.
This rejection has been rendered moot by the cancellation of claim 21.
The examiner rejected claim 22 as allegedly being directed to a non-statutory subject matter as encompassing a transitory form of signal transmission.
Claim 22 has been amended to recite "A computer program product comprising one or more
non-transitor
y
computer readable storage media" (see the patent application, p. 43, lines 24-26, "The storage medium is any tangible medium (different from transitory signals per se) that may retain and store instructions for use by the computing device.").
The examiner rejected claim 1 as allegedly being directed to an abstract idea due to at least some recitations of claim 1 allegedly being capable of being performed in the human mind.
Claim 1 has been amended to recite "A method for
assisting
imaging
a
region of interest of a patient for
a medical procedure
performed thereon
,
wherein the method comprises, under the control of a computing device: acquiring, by the computing device, a luminescence image of a field of view containing the region of interes
t for th
e
medical proc
e
dure of the pati
en
t,
" (see the patent application, p. 15, lines 15-16, "Starting from FIG.4A-FIG.4B, their activity diagram represents an exemplary process that may be used to imaging a patient with a method 400.").
This amendment clarifies that claim 1, as a whole, integrates all the recited features in a practical application (i.e., medical imaging) that is not directed to a judicial exception.
In addition, claim 1 has been amended to recite "
id
entify
ings
egmenting,
by the
computing device,
the auxiliar
y
values of the auxiliar
y
image semantically into at least one region of interest class defining
an auxiliary informative region
of th
e
auxiliary image
representative of the region of interest without the foreign objects
according to a content of the auxilia
r
y image
and one or more foreign ob
j
ect classes defining a
non-informative region representative of
the
foreign objects
,
"
(see the patent application, p. 18, lines 7-11, "In this specific case, the semantic segmentation is aimed at partitioning the reflectance image into the informative region representing the surgical cavity without the foreign objects and the non-informative region representing the foreign objects (i.e., surgical instruments, hands, surgical tools, surrounding body-parts and/or background materials)").
Claim 1 has been amended further to recite "identifying, by the computing device,
the
luminescence values of the luminescence image defining
a luminescence informative region
of the luminescence image
corresponding to
the auxiliar
y
values of
the auxiliary informative region
of
the auxiliar
y
image
,
"
(see the patent application, p. 21, lines 8-12, "In any case, the adapter adds the fluorescence segmentation mask and the fluorescence equalization map (equal to the reflectance segmentation mask and to the reflectance equalization map, respectively, possibly adapted to the fluorescence image) to the corresponding repositories.").
Claim 1 also has been amended to recite "the
processing of th
ep
rocessed
luminescence image
comprising a plurality of
processed luminescence values
being based on the luminescence values of the luminescence informative regions" (see the patent application, p. 22, line 25 through page 23, line 10, "If the label is asserted (meaning that the pixel belongs to the informative region), the processor at block 442 converts the corresponding
pixel value by applying the informative mapping function and adds it to the same cell of a (new) auto-scaled fluorescence image (in a corresponding temporary variable)" and p. 23, lines 13-31, "If the label is asserted (meaning that the pixel belongs to the informative region), the processor at block 454 compares the corresponding pixel value with the threshold value. If the pixel value is (possibly strictly) higher than the threshold value (i.e., it belongs to the target segment), the processor at block 456 copies it to the same cell of a (new) thresholded fluorescence image (in a corresponding temporary variable). Conversely, if the pixel value is (possibly strictly) lower than the threshold value (i.e., it belongs to the non-target segment), the processor at block 458 resets the pixel value in the same cell of the thresholded fluorescence image to zero (so as to mask it).").
Claims 22-23 and 25 have been amended in a similar manner.
The remaining claims are directed to statutory subject matter by virtue of their dependencies from an independent claim directed to statutory subject matter.
Claim Re
j
ections under
102 and
The examiner rejected claims 1, 5-7, 13, and 20-26 under 35 U.S.C. § 102(a)(1) as allegedly being anticipated by U.S. 2019/0046041 to Kishima.
The examiner rejected claims 2-3 under 35 U.S.C. § 103 as allegedly being unpatentable over Kishima in view of U.S. 2010/0056928 to Zuzak.
The examiner rejected claim 4 under 35 U.S.C. § 103 as allegedly being unpatentable over Kishima in view of U.S. 2004/0010192 to Benaron.
The examiner rejected claim 8 under 35 U.S.C. § 103 as allegedly being unpatentable over Kishima in view of U.S. 2020/0237445 to Snyder.
The examiner rejected claims 9-10 under 35 U.S.C. § 103 as allegedly being unpatentable over Kishima in view of Snyder further in view of U.S. 2019/0231249 to Dascalu.
The examiner rejected claim 11 under 35 U.S.C. § 103 as allegedly being unpatentable over Kishima in view of U.S. 2019/0247126 to Ikehara.
The examiner rejected claim 12 under 35 U.S.C. § 103 as allegedly being unpatentable over Kishima in view of U.S. 2007/0058862 to Meier.
The examiner rejected claims 14-15 under 35 U.S.C. § 103 as allegedly being unpatentable over Kishima in view of U.S. 2017/0055822 to Dana.
The examiner rejected claims 16-18 under 35 U.S.C. § 103 as allegedly being unpatentable over Kishima in view of U.S. 2011/0006220 to Kishima (Kishima '220).
The examiner rejected claim 19 under 35 U.S.C. § 103 as allegedly being unpatentable over Kishima in view of U.S. 2017/0237958 to Themelis.
Regarding claim 1, Kishima et al. proposes acquiring and storing a fluorescence
image of an entire surgical field SA in which an operator is not interested, for example, for confirming complete removal of a cancer (see Kishima's paragraph 0009, "there is a need to confirm, during or after the surgery, whether or not cancer cells that have not been removed exist") or for a possible medical malpractice suit (see Kishima's paragraph 0059 "Therefore, it is possible to prepare for a market of countermeasures against a medical malpractice suit, for example"); however, an ROI area of the fluorescence image in which the operator is actually interested is enlarged (by digital zooming) and the ROI area is then displayed (see Kishima's paragraph 0059, "The fluorescence navigation surgery system according to this embodiment is capable of enlarging an image of the area ROI in which the operator is interested by digital zooming in response to a user's request and displaying the image, and capturing and storing an image of the entire surgical filed SA with a pixel resolution in which a tumor is not missed. Therefore, it is possible to capture an image of an area in which the operator is not interested, which is not captured in the typical system, and to store the image."). The fluorescence image shows only a fluorescence area FA wherein a fluorescent contrast medium collects (see Kashima's paragraph 0048, "As shown in FIG. 2, on the monitor for fluorescence image observation 16, the area ROI including the fluorescence area FA, in which the operator is interested, is enlarged and displayed, for example"). At the same time, a visible light image of the surgical field SA is acquired (see Kashima's paragraph 0032, "The video camera for surgical field observation 1 is capable of capturing an image of the entire surgicalfield SA"), wherein the fluorescence area FA is instead not visible (see Kashima's paragraph 0033, "the fluorescence area FA is not displayed on the monitor for surgicalfield observation 3").
In a first embodiment of Kishima, the fluorescence image (of the ROI area) is
displayed on a monitor 16 (see Kishima's paragraph 0047, "display a fluorescence image of only the area ROI ... on the monitor for fluorescence image observation 16") and the visible light image (of the surgical field SA) is displayed on another monitor 3 (see Kishima's paragraph 0033, "the entire surgical field SA is displayed on the monitor for surgical field
observation 3"). In a second embodiment, the visible light image of the ROI area is captured by digital zooming (see Kishima's paragraph 0076, "performing optical zooming with the optical zoom lens 2 to capture an image of only the area ROI in which the operator is interested"); the fluorescence image and the visible light image (both of them of the ROI area) are then superimposed and displayed on a single monitor 16B (see Kishima's paragraph 0073, "two images are superimposed and the superimposed image is observed on a monitor in the second embodiment. It should be noted that the second embodiment is the same as the first embodiment in that the area ROI in which the operator is interested is displayed"). A
third embodiment simply differs in that the fluorescence image and the visual light image are captured from the same optical axis (see Kishima's paragraph 0085, "the image is captured from the same optical axis direction").
In contrast, claim 1 is directed to the computing device automatically segmenting the auxiliary image into the auxiliary informative region (representative of the region of interest without the foreign objects) and the non-informative region (representative of the foreign objects).
Conversely, in Kishima et al. the ROI area is selected manually by the operator (see Kishima's paragraph 0021, "cause a user to select at least a partial area of the observation target area and to capture a visible light image of the selected area as a ROI area"). In any case, to the contrary of the subject matter recited in claim 1, in Kishima et al. the ROI area is expressly selected to comprise the foreign objects as well (see Kishima's paragraph 0032, "the video camera for surgical field observation 1 needs to capture an image of the operator's handling of a needle accurately").
Moreover, claim 1 is directed to the computing device generating the processed luminescence image by processing the luminescence image limited to the luminescence informative region, so as to provide a different representation of the target body, thus facilitating its identification.
Conversely, in Kishima et al., the fluorescence image and the visual light image are simply aligned and then superimposed (see Kishima's paragraph 0021, "generate a superimposed image by aligning a position of the visible light image with a position of the cut fluorescence image and superimposing the images"). Therefore, the representation of the fluorescence area FA remains the same as in the fluorescence image.
Starting from Kishima et al. and even taking into account Snyder at al. (being cited by the examiner against claim 8, the subject matter of which is now substantially incorporated into claim 1) it is not possible to attain the proposed solution.
In fact, Snyder et al. simply mentions the use of segmentation for identifying corresponding elements in images.
However, Snyder et al. is completely silent about any identification of a luminescence informative region of a luminescence image corresponding to an auxiliary informative region of an auxiliary image (resulting from its segmentation), let alone the generation of any processed luminescence image by processing the luminescence image limited to the luminescence informative region so as to provide a different representation of the target body facilitating its identification.
In any case, Snyder et al. teaches away from the subject matter recited in claim 1. In fact, in relation to claim 1, the auxiliary informative region (determined on the auxiliary image to represent the region of interest without the foreign objects) is transferred to the luminescence image so as to process it by taking into account only the representation of the region of interest for the medical procedure (i.e., the surgical cavity), instead disregarding the representation of the foreign objects around and/or overlapping it (comprising the surgical instruments/tools).
Conversely, Snyder et al. is aimed at tracking instruments/tools (see Snyder's
paragraph 0055, "According to various embodiments, the navigation system 26 can be used to track the instrument 68 relative to the patient 30") used during a surgical procedure (see Snyder's paragraph 0038, "The tool 68 may be any appropriate tool such as a drill, forceps, or other tool operated by the user 72"). Therefore, contrary to the subject matter recited in claim 1 and in other claims, in Snyder et al. it is the instruments/tools that are displayed (see Snyder's paragraph 0042, "When navigating the instrument 68, a position of the instrument 68 can be illustrated relative to image data acquired of the patient 30 on a display device 84").
Claims 22-23 and 25 are patentable for reasons similar to those set forth above in conjunction with claim 1, and the dependent claims are patentable at least by virtue of the dependencies from an independent claim.
Conclusion

NEXT OFFICE ACTION: DETAILED ACTION
This office action is in response to the communication received on 01/21/2025 concerning application no. 17/801,218 filed on 08/19/2022.
Notice of Pre-AIA  or AIA  Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
EXAMINER’S AMENDMENT
An examiner’s amendment to the record appears below. Should the changes and/or additions be unacceptable to applicant, an amendment may be filed as provided by 37 CFR 1.312. To ensure consideration of such an amendment, it MUST be submitted no later than the payment of the issue fee.
Authorization for this examiner’s amendment was given in an interview with Bryan Santarelli (Reg. No. 39,564) on 02/05/2025.
Claim 1, in its entirety, has been amended as follows:
A method for imaging a region of interest of a patient fora medical procedure performed thereon, wherein the method comprises, under the control of a computing device:
acquiring, by the computing device, a luminescence image of a field of view containing the region of interest, the region of interest comprising at least one target body of the medical procedure containing a luminescence substance, and one or more foreign objects different from the region of interest, the luminescence image comprising a plurality of luminescence values representative of a luminescence light being emitted by the luminescence substance at corresponding luminescence locations of the field of view,
acquiring, by the computing device, an auxiliary image of the field of view, the auxiliary image comprising a plurality of auxiliary values representative of an auxiliary light, different from the luminescence light, being received from corresponding auxiliary locations of the field of view,
segmenting, by the computing device, the auxiliary values of the auxiliary image semantically into at least one region of interest class defining an auxiliary informative region representative of the region of interest without the foreign objects and one or more foreign object classes defining a non-informative region representative of the foreign objects,
identifying, by the computing device, the luminescence values of the luminescence image defining a luminescence informative region corresponding to the auxiliary values of the auxiliary informative region of the auxiliary image,
generating, by the computing device, a processed luminescence image by processing the luminescence image limited to the luminescence informative region, the processed luminescence image comprising a plurality of processed luminescence values being based on the luminescence values of the luminescence informative region, wherein the processed luminescence image provides a further representation of the target body facilitating an identification thereof, and
displaying, by the computing device an output image base on the processed luminescence image substantially in real-time with said acquiring the luminescence image.
Claim 2, in its entirety, has been amended as follows:
The method according to claim 1, wherein the medical procedure is a surgical procedure
Claim 16, in its entirety, has been amended as follows:
The method according to claim 1:
wherein said generating the processed luminescence image comprises:
thresholding, by the computing device, the luminescence informative region according to the luminescence values of the luminescence informative region thereby partitioning the luminescence informative region into a target segment representative of the target body and a non-target segment representative of a rest of the region of interest different from the target body, and
wherein said displaying
displaying
Claim 22, in its entirety, has been amended as follows:
A computer program product comprising one or more non-transitory computer readable storage media having program instructions collectively stored on the readable storage media, the program instructions readable by a computing device to cause the computing device to perform a method for imaging a region of interest of a patient for a medical procedure performed thereon, wherein the method comprises:
acquiring a luminescence image of a field of view containing the region of interest, the region of interest comprising at least one target body of the medical procedure containing a luminescence substance, and one or more foreign objects different from the region of interest, the luminescence image comprising a plurality of luminescence values representative of a luminescence light being emitted by the luminescence substance at corresponding luminescence locations of the field of view,
acquiring an auxiliary image of the field of view, the auxiliary image comprising a plurality of auxiliary values representative of an auxiliary light, different from the luminescence light, being received from corresponding auxiliary locations of the field of view,
segmenting the auxiliary values of the auxiliary image semantically into at least one region of interest class defining an auxiliary informative region representative of the region of interest without the foreign objects and one or more foreign object classes defining a non-informative region representative of the foreign objects,
identifying the luminescence values of the luminescence image defining a luminescence informative region corresponding to the auxiliary values of the auxiliary informative region of the auxiliary image,
generating a processed luminescence image by processing the luminescence image limited to the luminescence informative region, the processed luminescence image comprising a plurality of processed luminescence values being based on the luminescence values of the luminescence informative region, wherein the processed luminescence image provides a further representation of the target body facilitating an identification thereof, and
displaying substantially in real-time with said acquiring the luminescence image.
Claim 23, in its entirety, has been amended as follows:
A computing device for imaging a region of interest of a patient for a medical procedure performed thereon, wherein the computing device comprises:
a luminescence drive for acquiring a luminescence image of a field of view containing the region of interest, the region of interest comprising at least one target body of the medical procedure containing a luminescence substance, and one or more foreign objects different from the region of interest, the luminescence image comprising a plurality of luminescence values representative of a luminescence light being emitted by the luminescence substance at corresponding luminescence locations of the field of view,
an auxiliary drive for acquiring an auxiliary image of the field of view, the auxiliary image comprising a plurality of auxiliary values representative of an auxiliary light, different from the luminescence light, being received from corresponding auxiliary locations of the field of view,
a segmenter for segmenting the auxiliary values of the auxiliary image semantically into at least one region of interest class defining an auxiliary informative region representative of the region of interest without the foreign objects and one or more foreign object classes defining a non-informative region representative of the foreign objects,
an adapter for identifying the luminescence values of the luminescence image defining a luminescence informative region corresponding to the auxiliary values of the auxiliary informative region of the auxiliary image,
a processor for generating a processed luminescence image by processing the luminescence image limited to the luminescence informative region, the processed luminescence image comprising a plurality of processed luminescence values being based on the luminescence values of the luminescence informative region, wherein the processed luminescence image provides a further representation of the target body facilitating an identification thereof, and
an output drive for displaying substantially in real-time with said acquiring the luminescence image.
Claim 25, in its entirety, has been amended as follows:
A medical procedure for performing on a patient, the medical procedure comprising:
acquiring, by a computing device, a luminescence image of a field of view containing the region of interest, the region of interest comprising at least one target body of the medical procedure containing a luminescence substance, and one or more foreign objects different from the region of interest, the luminescence image comprising a plurality of luminescence values representative of a luminescence light being emitted by the luminescence substance at corresponding luminescence locations of the field of view,
acquiring, by the computing device, an auxiliary image of the field of view, the auxiliary image comprising a plurality of auxiliary values representative of an auxiliary light, different from the luminescence light, being received from corresponding auxiliary locations of the field of view,
segmenting, by the computing device, the auxiliary values of the auxiliary image semantically into at least one region of interest class defining an auxiliary informative region representative of the region of interest without the foreign objects and one or more foreign object classes defining a non-informative region representative of the foreign objects,
identifying, by the computing device, the luminescence values of the luminescence image defining a luminescence informative region corresponding to the auxiliary values of the auxiliary informative region of the auxiliary image,
generating, by the computing device, a processed luminescence image by processing the luminescence image limited to the luminescence informative region, the processed luminescence image comprising a plurality of processed luminescence values being based on the luminescence values of the luminescence informative region, wherein the processed luminescence image provides a further representation of the target body facilitating an identification thereof,
displaying substantially in real-time with said acquiring the luminescence image, and
performing the medical procedure with an assistance of the output image.
Reasons for Allowance
Claims 1-7, 9-20, and 22-26 are allowed over the prior art.
The following is an examiner’s statement of reasons for allowance:
With regard to independent claims 1, 22-23, and 25, the prior art fails to anticipate and/or render obvious, either solely or in combination, a method and corresponding non-transitory computer-readable storage medium, computing device, and medical procedure for the acquiring of a luminescence image field of view that includes a region of interest with a target body that has a luminescence substance that defines luminescence values into the luminescence image, acquiring an auxiliary image that is operably different from the luminescence image and is segmented to classes for the segmentation of region of interest and foreign objects, identifying the luminescence values in the luminescence image to correspond to auxiliary values in the auxiliary image, generating a processed luminescence that has the processed luminescence for the representation of the target body for further identification, and the display of the output image substantially in real time and based on the processed luminescence image.
Paragraphs 0026-29 of the specification specifically describe the system’s consideration of luminescence to be with respect to fluorescence that is generated based on fluorophores and acquired by a fluorescence camera. Paragraph 0026 teaches that “these foreign objects may comprise one or more surgical instruments 115 (such as a scalpel), one or more hands 118 (such as of the surgeon), one or more surgical tools 121 (such as a gauze), one or more surrounding body-parts 124 (such as skin around the surgical cavity 109) and/or one or more background materials 125 (such as an operating table).”
Etsuro et al. (JP 2020156860) teaches the superimposition of fluorescent and visible image data for the highlighting of a region of interest. However, the reference is silent regarding segmenting the auxiliary image according to classifying of the region of interest and foreign objects.
Claims 2-7, 9-20, 24, and 26 are allowed at least by virtue of their dependency upon an allowable base claim.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee.  Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion

CITED REFERENCE SUMMARIES: Ref US20170055822A1: Claims:
A computer-implemented method for determining a score representing an amount of staining of the cornea, the method comprising: obtaining a digital image of the cornea stained with a tracer material; receiving a selection of a portion of the image; processing, by a processing device, the selection to exclude areas with one or more artifacts to define an evaluation area; for each of a plurality of pixels within the evaluation area: determining a plurality of color components that are based on a Cartesian color space, determining, from the color components, a hue value in a polar coordinate based color space, and determining an amount of staining of the cornea as a function of the hue value; and assigning a score to the evaluation area based on the amount of staining calculated for the plurality of pixels.
obtaining a digital image of the cornea stained with a tracer material;
receiving a selection of a portion of the image;
processing, by a processing device, the selection to exclude areas with one or more artifacts to define an evaluation area;
for each of a plurality of pixels within the evaluation area: determining a plurality of color components that are based on a Cartesian color space, determining, from the color components, a hue value in a polar coordinate based color space, and determining an amount of staining of the cornea as a function of the hue value; and
determining a plurality of color components that are based on a Cartesian color space,
determining, from the color components, a hue value in a polar coordinate based color space, and
determining an amount of staining of the cornea as a function of the hue value; and
assigning a score to the evaluation area based on the amount of staining calculated for the plurality of pixels.
The method of claim 1 , wherein the selection is received via an adjustable control overlaid on the digital image of the cornea.
The method of claim 1 , wherein processing the selection further comprises: dividing the selection into a plurality of zones; analyzing each of the zones to detect a presence of the one or more artifacts; and modifying the selection to exclude zones in which the presence of the one or more artifacts are detected to define the evaluation area.
dividing the selection into a plurality of zones;
analyzing each of the zones to detect a presence of the one or more artifacts; and
modifying the selection to exclude zones in which the presence of the one or more artifacts are detected to define the evaluation area.
The method of claim 3 , wherein the one or more artifacts include one of a specular reflection and a confocal reflection, optionally detected using a flooding algorithm.
The method of claim 1 , wherein determining the amount of staining further comprises: mapping an angle corresponding to the hue value to a scalar value within a predetermined range using a parabolic curve; and determining the amount of staining as a product of the scalar value and at least one component of the polar coordinate based color space that is different from the hue.
mapping an angle corresponding to the hue value to a scalar value within a predetermined range using a parabolic curve; and
determining the amount of staining as a product of the scalar value and at least one component of the polar coordinate based color space that is different from the hue.
The method of claim 1 , wherein the tracer material is a fluorophore.
The method of claim 8 , wherein the fluorophore is fluorescein, and the digital image is acquired in the presence of blue cobalt light.
The method of claim 1 , wherein the score indicates a degree of greenness of the image of the cornea.
The method of claim 1 , wherein the Cartesian color space is an RGB color space.
The method of claim 1 , wherein the polar coordinate based color space is an HSV color space.
The method of claim 1 , further comprising storing the score and an association of the score with the digital image.
The method of claim 1 , wherein the digital images represent corneas afflicted with a corneal epithelial disease.
A system for determining a score representing an amount of staining of the cornea, the system comprising: an imaging system configured to obtain a digital image of the cornea stained with a tracer material; and a score calculator module configured to: receive a selection of a portion of the image via a user interface, process the selection to exclude areas with one or more artifacts to define an evaluation area, for each of a plurality of pixels within the evaluation area: determine a plurality of color components that are based on a Cartesian color space, determine, from the color components, a hue value in a polar coordinate based color space, and determine an amount of staining of the cornea as a function of the hue value, and assign a score to the evaluation area based on the amount of staining calculated for the plurality of pixels.
an imaging system configured to obtain a digital image of the cornea stained with a tracer material; and
a score calculator module configured to: receive a selection of a portion of the image via a user interface, process the selection to exclude areas with one or more artifacts to define an evaluation area, for each of a plurality of pixels within the evaluation area: determine a plurality of color components that are based on a Cartesian color space, determine, from the color components, a hue value in a polar coordinate based color space, and determine an amount of staining of the cornea as a function of the hue value, and assign a score to the evaluation area based on the amount of staining calculated for the plurality of pixels.
receive a selection of a portion of the image via a user interface,
process the selection to exclude areas with one or more artifacts to define an evaluation area,
for each of a plurality of pixels within the evaluation area: determine a plurality of color components that are based on a Cartesian color space, determine, from the color components, a hue value in a polar coordinate based color space, and determine an amount of staining of the cornea as a function of the hue value, and
determine a plurality of color components that are based on a Cartesian color space,
determine, from the color components, a hue value in a polar coordinate based color space, and
determine an amount of staining of the cornea as a function of the hue value, and
assign a score to the evaluation area based on the amount of staining calculated for the plurality of pixels.
The system of claim 16 , wherein the user interface includes an adjustable control overlaid on the digital image of the cornea.
The system of claim 16 , wherein the score calculator module is further configured to: divide the selection into a plurality of zones; analyze each of the zones to detect a presence of the one or more artifacts; and modify the selection to exclude zones in which the presence of the one or more artifacts are detected to define the evaluation area.
divide the selection into a plurality of zones;
analyze each of the zones to detect a presence of the one or more artifacts; and
modify the selection to exclude zones in which the presence of the one or more artifacts are detected to define the evaluation area.
The system of claim 16 , wherein the one or more artifacts include one of a specular reflection and a confocal reflection.
A computer readable storage device having encoded thereon computer readable instructions, which when executed by a processor, cause a processor to perform operations comprising: obtaining a digital image of the cornea stained with a tracer material; receiving a selection of a portion of the image; processing the selection to exclude areas with one or more artifacts to define an evaluation area; for each of a plurality of pixels within the evaluation area: determining a plurality of color components that are based on a Cartesian color space, determining, from the color components, a hue value in a polar coordinate based color space, and determining an amount of staining of the cornea as a function of the hue value; and assigning a score to the evaluation area based on the amount of staining calculated for the plurality of pixels.
processing the selection to exclude areas with one or more artifacts to define an evaluation area;
- 30 . (canceled)
A method of determining severity of an eye condition, the method comprising: determining a score for a cornea of a subject using the method of claim 1 ; and determining the severity of the condition based on the score.
determining a score for a cornea of a subject using the method of claim 1 ; and
determining the severity of the condition based on the score.
The method of claim 31 , wherein the score being above a reference score indicates that the subject has a disease.
A method of monitoring efficacy of a treatment for a condition of the cornea of a subject, the method comprising: determining a first score for the cornea of the subject; administering one or more treatments to the eye of the subject; determining a second score for the cornea of the subject, wherein the first and second scores are determined by the method of claim 1 ; and comparing the first and second scores, wherein a decrease in the score indicates that the treatment was effective, and no change or an increase in the score indicates that the treatment was ineffective.
determining a first score for the cornea of the subject;
administering one or more treatments to the eye of the subject;
determining a second score for the cornea of the subject,
wherein the first and second scores are determined by the method of claim 1 ; and
comparing the first and second scores,
wherein a decrease in the score indicates that the treatment was effective, and no change or an increase in the score indicates that the treatment was ineffective.
A method of monitoring progression of a condition of the cornea of a subject, the method comprising: determining a first score for the cornea; and determining a second score for the cornea, wherein the first and second scores are determined by the method of claim 1 ; and comparing the first and second scores, wherein a decrease from the first score to the second score indicates that the condition is improving, no change between the first and second scores indicate that the condition is stable, and an increase from the first score to the second score indicates that the condition is worsening.
determining a first score for the cornea; and
determining a second score for the cornea, wherein the first and second scores are determined by the method of claim 1 ; and
wherein a decrease from the first score to the second score indicates that the condition is improving, no change between the first and second scores indicate that the condition is stable, and an increase from the first score to the second score indicates that the condition is worsening.
The method of claim 34 , wherein the condition is selected from a group consisting of dry eye syndrome, corneal epithelial disease, ocular graft-versus-host disease, and Sjogren's syndrome.
Ref US20190046041A1: Claims:
A surgical navigation system for the use during surgical operation, comprising: a surgical microscope including: a light source device configured to apply excitation light and visible light to an observation target of a living body; a fluorescent imaging device configured to capture a fluorescent image of the observation target, having a first pixel resolution; and a visible light imaging device configured to capture a visible light image of the observation target, the imaging device having a red, green and blue (RGB) Bayer pattern and generating a color image by a demosaicing process; a display having a second pixel resolution different from the first pixel resolution; and circuitry configured to: receive selecting information about a partial area of the observation target as a region of interest (ROI); generate a superimposed image by aligning a position of the fluorescent image with a position of the visible light image; and control the display to display the superimposed image in the ROI.
a surgical microscope including: a light source device configured to apply excitation light and visible light to an observation target of a living body; a fluorescent imaging device configured to capture a fluorescent image of the observation target, having a first pixel resolution; and a visible light imaging device configured to capture a visible light image of the observation target, the imaging device having a red, green and blue (RGB) Bayer pattern and generating a color image by a demosaicing process; a display having a second pixel resolution different from the first pixel resolution; and
a light source device configured to apply excitation light and visible light to an observation target of a living body;
a fluorescent imaging device configured to capture a fluorescent image of the observation target, having a first pixel resolution; and
a visible light imaging device configured to capture a visible light image of the observation target, the imaging device having a red, green and blue (RGB) Bayer pattern and generating a color image by a demosaicing process;
a display having a second pixel resolution different from the first pixel resolution; and
circuitry configured to: receive selecting information about a partial area of the observation target as a region of interest (ROI); generate a superimposed image by aligning a position of the fluorescent image with a position of the visible light image; and control the display to display the superimposed image in the ROI.
receive selecting information about a partial area of the observation target as a region of interest (ROI);
generate a superimposed image by aligning a position of the fluorescent image with a position of the visible light image; and
control the display to display the superimposed image in the ROI.
Ref US20200237445A1: Claims:
A method of performing a navigated procedure relative to a subject, comprising: accessing a first image data of at least a region of interest of the subject, wherein the first image data includes at least a first segmented portion and a second segmented portion; tracking a first tracking device, wherein the first tracking device is connected to a first portion of the subject; tracking a second tracking device, wherein the second tracking device is connected to a second portion of the subject; associating the first tracking device with the first segmented portion in the first image data; associating the second tracking device with the second segmented portion in the first image data; displaying the first image with a display device; and updating the first image, including changing a position of the first segmented portion or the second segmented portion based upon tracking the first tracking device connected to the first portion and tracking the second tracking device connected to the second portion.
accessing a first image data of at least a region of interest of the subject, wherein the first image data includes at least a first segmented portion and a second segmented portion;
tracking a first tracking device, wherein the first tracking device is connected to a first portion of the subject;
tracking a second tracking device, wherein the second tracking device is connected to a second portion of the subject;
associating the first tracking device with the first segmented portion in the first image data;
associating the second tracking device with the second segmented portion in the first image data;
displaying the first image with a display device; and
updating the first image, including changing a position of the first segmented portion or the second segmented portion based upon tracking the first tracking device connected to the first portion and tracking the second tracking device connected to the second portion.
The method of claim 1 , further comprising: segmenting the image data to determine at least a boundary of the first segmented portion and the second segmented portion.
segmenting the image data to determine at least a boundary of the first segmented portion and the second segmented portion.
The method of claim 1 , wherein associating the first tracking device with the first segmented portion in the first image data and associating the second tracking device with the second segmented portion in the first image data, comprises: inputting into a navigation system the identification of the first tracking device with the first segmented portion; and inputting into the navigation system the identification of the second tracking device with the second segmented portion.
inputting into a navigation system the identification of the first tracking device with the first segmented portion; and
inputting into the navigation system the identification of the second tracking device with the second segmented portion.
The method of claim 1 , wherein associating the first tracking device with the first segmented portion in the first image data and associating the second tracking device with the second segmented portion in the first image data, comprises: determining by executing instructions with a navigation system a position of the first tracking device relative to a known reference device and the first segmented portion; and determining by executing instructions with the navigation system a position of the second tracking device relative to the known reference device and the second segmented portion.
determining by executing instructions with a navigation system a position of the first tracking device relative to a known reference device and the first segmented portion; and
determining by executing instructions with the navigation system a position of the second tracking device relative to the known reference device and the second segmented portion.
The method of claim 1 , further comprising: forming a bore in the first portion of the subject with an instrument; and disengaging the first tracking device from the instrument within the bore to connect the first tracking device to the first portion of the subject.
forming a bore in the first portion of the subject with an instrument; and
disengaging the first tracking device from the instrument within the bore to connect the first tracking device to the first portion of the subject.
The method of claim 1 , further comprising: determining a greater confidence of tracking between the first tracking device and the second tracking device; tracking an instrument; and selecting the determined greater confidence of tracking between the first tracking device and the second tracking device as a reference for the tracking the instrument.
determining a greater confidence of tracking between the first tracking device and the second tracking device;
tracking an instrument; and
selecting the determined greater confidence of tracking between the first tracking device and the second tracking device as a reference for the tracking the instrument.
The method of claim 1 , further comprising: determining a real time position of a third portion of the subject; updating the first image to include a real time position of the third portion of the subject.
determining a real time position of a third portion of the subject;
updating the first image to include a real time position of the third portion of the subject.
The method of claim 7 , wherein determining the real time position of the third portion of the subject, comprises: interpolating between the first portion and the second portion based on the tracking of the first tracking device and the second tracking device.
interpolating between the first portion and the second portion based on the tracking of the first tracking device and the second tracking device.
The method of claim 8 , wherein determining the real time position of the third portion of the subject, comprises: accessing a biomechanical model of the 
[Truncated to input limit]