Office Action 2: Final Rejection (2024-07-16)
Rejection types: 103 - Obviousness, 112 - Written Description/Enablement

CLAIMS:
Listing of Claims:

AMENDMENTS TO THE CLAIMSThis listing of claims will replace all prior versions, and listings of claims in the application.

1\. (Currently Amended) A computer-implemented method in which one or more processing devices perform operations comprising:
storing a three-dimensional (3D) virtual object that corresponds to a real-world object, the 3D virtual object including a stored user-configured superimposition of an image area showing a stored portion of the real-world object on a face of a 3D shape;
storing a virtual reset that includes information about the 3D virtual object and a position of the 3D virtual object in the virtual reset; and
presenting the virtual reset, the presentation showing the 3D virtual object at the position.
2\. (Currently Amended) The computer-implemented method of claim 1, further comprising:
generating the 3Dvirtual object, wherein generating the 3Dvirtual object comprises:receiving, via a user interface of a device, a selection of the 3D shape that includes a plurality of faces including the face;
receiving, via the user interface of the device, a first indication of an input indicating properties for the 3D shape;
receiving, via the user interface of the device, a second indication of an image showing a portion of \[\[a\]\] the real-world object;
determining an area of the image that corresponds to the portion of the real-world object;
associating, in the 3Dvirtual object, the area of the image with \[\[a\]\] theface of the plurality of faces of the 3D shape so the area of the image is superimposed on the face of the 3D virtual object; and
associating, in the 3Dvirtual object or in metadata of the virtual object, the properties with the 3Dvirtual object.

3\. (Currently Amended) The computer-implemented method of claim 2, wherein associating, in the 3Dvirtual object, the area of the image with the face of the plurality of faces of the 3D shape comprises:
presenting, via the user interface, the image imposed on the face, wherein boundaries of the portion of the real-world object in the image do not correspond with boundaries of the face;
providing, via the user interface, an editing interface object selectable to enable editing of the image; and
editing the image, responsive to receiving inputs to the editing interface object, so that the boundaries of the portion of the real-world object in the image correspond to the boundaries of the face.
4\. (Original) The computer-implemented method of claim 2, wherein the properties comprise one or more of dimensions or a weight.
5\. (Original) The computer-implemented method of claim 2, further comprising:
presenting, via the user interface, a set of three-dimensional shapes including the three- dimensional shape and one or more other three-dimensional shapes.
6\. (Original) The computer-implemented method of claim 5, wherein the set of three-dimensional shapes include one or more of a cube, a rectangular prism, a sphere, a spheroid, a cylinder, a cone, or a pyramid.
7\. (Currently Amended) The computer-implemented method of claim 1, further comprising generating the 3D virtual reset, wherein generating the 3D virtual reset comprises:
presenting, at a user interface, the 3D virtual object in the 3D virtual reset at a first position;
receiving, via the user interface, an indication of a user selection to \[\[an\]\] edit \[\[to\]\] the 3D virtual object in the 3D virtual reset;
updating the presentation of the 3D virtual reset by showing the edit; and
storing the 3D virtual reset by including, in the 3D virtual reset, information about the 3D virtual object and information about the edit.

8\. (Original) The computer-implemented method of claim 7, wherein the information about the 3D virtual object comprises the 3D virtual object or a link to the 3D virtual object.

9\. (Original) The computer-implemented method of claim 7, wherein the edit comprises one or more of a change in position of the 3D virtual object in the 3D virtual reset from the first position to a second position or a rotation of the 3D virtual object.

10\. (Original) The computer-implemented method of claim 7, wherein the edit comprises a change to the 3D virtual object comprising one or more of changes to images of one or more faces of the 3D virtual object or a resizing of the 3D virtual object.

11\. (Original) The computer-implemented method of claim 7, further comprising:
storing the edited 3D virtual object at a storage location in a data storage unit, wherein the information about the 3D virtual object and the information about the edit include the edited 3D virtual object or a link to the storage location of the edited 3D virtual object.

12\. (Original) The computer-implemented method of claim 7, further comprising:
storing the edit at a storage location in a data storage unit, wherein the information about the 3D virtual object includes the 3D virtual object or a link to the 3D virtual object and the information about the edit includes the edit or a link to the storage location of the edit.

13\. (Currently Amended) The computer-implemented method of claim 1, further comprising:
responsive to receiving an input in \[\[the\]\] an augmented reality environment, moving the virtual reset within the augmented reality environment so that a location within the augmented reality environment of the virtual reset corresponds to a physical location in a physical environment of a physical reset to be assembled.

14\. (Currently Amended) The computer-implemented method of claim 1, wherein presenting the virtual reset comprises presenting the virtual \[\[rest\]\] reset in at least one of: a virtual reality environment or an augmented reality environment.

15\. (Currently Amended) The computer-implemented method of claim 2, wherein the stored user-configured superimposition of the image area is obtained based on user interactions with one or more user interface objects that cause the image corresponding to the image area to be manipulated.

16\. (Currently Amended) A system comprising:
a processor; and
a non-transitory computer readable medium storing computer-readable program instructions that, when executed by the processor, cause the system to:storing a three-dimensional (3D) virtual object that corresponds to a real-world object, the 3D virtual object including a stored user-configured superimposition of an image area showing a stored portion of the real-world object on a face of a 3D shape;
storing a virtual reset that includes information about the 3D virtual object and a position of the 3D virtual object in the virtual reset; and
presenting the virtual reset, the presentation showing the 3D virtual object at the position.

17\. (Currently Amended) The system of claim 16, the operations further comprising:
generating the 3Dvirtual object, wherein generating the 3Dvirtual object comprises:receiving, via a user interface of a device, a selection of the 3D shape that includes a plurality of faces including the face;
receiving an input indicating properties for the 3D shape;
receiving an image showing a portion of \[\[a\]\] the real-world object;
determining an area of the image that corresponds to the portion of the real-world object;
associating, in the 3Dvirtual object, the area of the image with a user-specified face of the plurality of faces of the 3D shape; and
associating, in the 3Dvirtual object or in metadata of the 3Dvirtual object, the properties with the 3Dvirtual object.

18 (Original) The system of claim 16, the operations further comprising generating the 3D virtual reset, wherein generating the 3D virtual reset comprises:
presenting, at a user interface, the 3D virtual object in the 3D virtual reset at a first position;
receiving, via the user interface, an edit to the 3D virtual object in the 3D virtual reset;
updating the presentation of the 3D virtual reset by showing the edit; and
storing the 3D virtual reset by including, in the 3D virtual reset, information about the 3D virtual object and information about the edit.

19\. (Currently Amended) The system of claim 16,the operations further comprising:
presenting a user interface object for performing manipulations to the image area showing the stored portion of the real-world object, wherein the stored user-configured superimposition of the image area on the face of the 3D shape is generated based on the manipulations including at least one of: resizing, rotating, or cropping.

20\. (Currently Amended) A non-transitory computer-readable medium having program code that is stored thereon, the program code executable by one or more processing devices for performing operations comprising:
storing a three-dimensional (3D) virtual object that corresponds to a real-world object, the 3D virtual object including a stored-user configured superimposition of an image area showing a stored portion of the real-world object on a face of a 3D shape;
storing a virtual reset that includes information about the 3D virtual object and a position of the 3D virtual object in the virtual reset; and
presenting the virtual reset, the presentation showing the 3D virtual object at the position.

OFFICE ACTION:
Notice of Pre-AIA  or AIA  Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
See 112(a) rejection below:
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA  35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA  35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
In claims 1, 16 and 20 the amended sections “… stored user-configured superimposition … stored portion … ” as the subject matter not properly described in the application as filed.
What is the stored user-configured?
How does a user generate the stored user-configured superimposition (interpreted as a virtual model)?
What are the factors of an object involved in the stored user-configured?
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-13, 15-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Geisner et al., US 2013/0083064 A1, hereinafter Geisner, and further in view of  Rolion et al., US 2021/0248823 A1, hereinafter Rolion.
Claim 1.
A computer-implemented method in which one or more processing devices perform operations comprising:
storing a three-dimensional (3D) virtual object that corresponds to a real-world object, the 3D virtual object including a stored user-configured superimposition of an image area showing a stored portion of the real-world object on a face of a 3D shape (see [0035] discloses personal A/V apparatus 8 includes an optical see-through, augmented reality display device as a near-eye, augmented reality display device 2 in communication with a companion processing module 4 via a wire 6 in this example or wirelessly in other examples. In this embodiment, head mounted display device 2 is in the shape of eyeglasses in a frame 115, with a display optical system 14 for each eye in which image data is projected into a user's eye to generate a display of the image data while a user also sees through the display optical systems 14 for an actual direct view of the real world. See [0006-0007] for storing the 3D object, also see [0051]);
storing a virtual reset that includes information about the 3D virtual object and a position of the 3D virtual object in the virtual reset (see [0092] discloses the virtual data engine 195 performs translation, rotation, and scaling operations for display of the virtual data at the correct size and perspective. A virtual data position may be dependent upon, a position of a corresponding object, real or virtual, to which it is registered. The virtual data engine 195 can update the scene mapping engine about the positions of the virtual objects processed.); and
presenting the virtual reset, the presentation showing the 3D virtual object at the position (see [0041], [0051], [0127] and [0092]).
Geisner does not explicitly specify a stored user-configured superimposition of an image area.
However, Rolion teaches at [0012] placement of a three-dimensional virtual object in the registered virtual world, such that the virtual object is substantially overlaid, in the virtual world, on the real object in the real world. Also at [0032] discloses, a user can select a virtual object that has the shape closest to that of the real object. The three-dimensional virtual objects from the library of three-dimensional virtual objects may be meshes that are well-known to a person skilled in the art, or three-dimensional shapes associated with parameters. By way of illustration, a cylindrical three-dimensional shape having a circular base can be associated with a diameter parameter and a height parameter.
Thus, it would have been obvious to one of ordinary skill in the art to combine the teachings of Rolion into Geisner in order to provide the 3D virtual object including the stored user-configured superimposition of an image area showing a stored portion of the real-world object on a face of a 3D shape, see [0103] discloses an object can be selected so as to be of the same shape as the real object, or at least a shape approximate thereto.
Claim 2.
The computer-implemented method of claim 1, further comprising:
generating the 3D virtual object, wherein generating the 3D virtual object comprises: receiving, via a user interface of a device, a selection of the 3D shape that includes a plurality of faces including the face (see [0070] discloses 0070] The scene mapping engine 306 is first described. A 3D mapping of the display field of view of the augmented reality display can be determined by the scene mapping engine 306 based on captured image data, and captured depth data if available, for the display field of view. The 3D mapping includes 3D space positions or position volumes for objects. A 3D space is a volume of space occupied by the object. Depending on the precision desired, the 3D space can match the 3D shape of the object or be a less precise volume like a bounding shape like a bounding box or bounding ellipse around an object. A 3D space position represents position coordinates for the volume or 3D space. In other words the 3D space position identifies how much space an object occupies and where in the display field of view that occupied space is.);
receiving, via the user interface of the device, a first indication of an input indicating properties for the 3D shape (see [0100] FIG. 3C illustrates some examples of data fields in an object physical properties data set 320N stored for a real object, which includes data values detected or otherwise determined based on captured data of the real object, or for a specific virtual object. Also see [0070], [0079] and [0099]);
receiving, via the user interface of the device, a second indication of an image showing a portion of a real-world object (see [0041], [0051] and [0092]);
determining an area of the image that corresponds to the portion of the real-world object (see [0079], [0092] and [0099]);
associating, in the 3D virtual object, the area of the image with the face of the plurality of faces of the 3D shape (see [0083] discloses a set of reference properties for an object is an object reference data set as stored in one or more object reference data sets 318.) so the area of the image is superimposed on the face of the 3D virtual object; and
associating, in the 3D virtual object or in metadata of the virtual object, the properties with the 3D virtual object (see [0099] and 0100]).
Claim 3.
The computer-implemented method of claim 2, wherein associating, in the 3D virtual object, the area of the image with the face of the plurality of faces of the 3D shape comprises: presenting, via the user interface, the image imposed on the face, wherein boundaries of the portion of the real-world object in the image do not correspond with boundaries of the face; providing, via the user interface, an editing interface object selectable to enable editing of the image; and editing the image, responsive to receiving inputs to the editing interface object, so that the boundaries of the portion of the real-world object in the image correspond to the boundaries of the face. (see [0093] discloses in one example implementation of updating the 3D display, the virtual data engine 195 renders the previously created three dimensional model of the display field of view including depth data for both virtual and real objects in a Z-buffer. The real object boundaries in the Z-buffer act as references for where the virtual objects are to be three dimensionally positioned in the display as the image generation unit 120 displays just the virtual objects as the display device is an optical see-through display device. For a virtual object, the virtual data engine 195 has a target space position of where to insert the virtual object. In some examples, the virtual object target position is registered to a position of a real world object, and in other examples, the virtual object is independent of a particular real object. Also see 0092-0093], [0099] and [0129]).
Claim 4.
The computer-implemented method of claim 2, wherein the properties comprise one or more of dimensions or a weight. (see [0080], [0083], [0100] and [0099]).
Claim 5.
The computer-implemented method of claim 2, further comprising: presenting, via the user interface, a set of three-dimensional shapes including the three-dimensional shape and one or more other three-dimensional shapes. (see 0092-0093], [0099]).
Claim 6.
The computer-implemented method of claim 5, wherein the set of three-dimensional shapes include one or more of a cube, a rectangular prism, a sphere, a spheroid, a cylinder, a cone, or a pyramid. (See [0105] discloses a physical damage indicator may be updated based on shape changes to a container like broken glass or an indentation in a cylindrical can shape. Also see [0099]).
Claim 7.
The computer-implemented method of claim 1, further comprising generating the 3D virtual reset, wherein generating the 3D virtual reset comprises:
presenting, at a user interface, the 3D virtual object in the 3D virtual reset at a first position (see [0041], [0051] and [0092]);
receiving, via the user interface, an indication of a user selection to edit the 3D virtual object in the 3D virtual reset (see [0119], [0092-0093] and [0129]) and also see Rolion at [0032] discloses a user can select a virtual object that has the shape closest to that of the real object;
updating the presentation of the 3D virtual reset by showing the edit (see [0104] also see [0119], [0092-0093] and [0129]); and
storing the 3D virtual reset by including, in the 3D virtual reset, information about the 3D virtual object and information about the edit. (see [0119], [0092-0093], [0104], [0123] and [0129]);
Claim 8.
The computer-implemented method of claim 7, wherein the information about the 3D virtual object comprises the 3D virtual object or a link to the 3D virtual object. (See [0041], [0051], [0092] and [0099]).
Claim 9.
The computer-implemented method of claim 7, wherein the edit comprises one or more of a change in position of the 3D virtual object in the 3D virtual reset from the first position to a second position or a rotation of the 3D virtual object. See [0120], [0122] and [0129].
Claim 10.
The computer-implemented method of claim 7, wherein the edit comprises a change to the 3D virtual object comprising one or more of changes to images of one or more faces of the 3D virtual object or a resizing of the 3D virtual object. (see [0070], [0079], [0092-0093], [0099] and [0129]).
Claim 11.
The computer-implemented method of claim 7, further comprising: storing the edited 3D virtual object at a storage location in a data storage unit, wherein the information about the 3D virtual object and the information about the edit include the edited 3D virtual object or a link to the storage location of the edited 3D virtual object. See [0070], [0079], [0092-0093], [0099] and [0129].
Claim 12.
The computer-implemented method of claim 7, further comprising: storing the edit at a storage location in a data storage unit, wherein the information about the 3D virtual object includes the 3D virtual object or a link to the 3D virtual object and the information about the edit includes the edit or a link to the storage location of the edit. See [0070], [0079], [0092-0093], [0099] and [0129].
Claim 13.
The computer-implemented method of claim 1, further comprising: responsive to receiving an input in an augmented reality environment, moving the virtual reset within the augmented reality environment so that a location within the augmented reality environment of the virtual reset corresponds to a physical location in a physical environment of a physical reset to be assembled. See [0041], [0051], [0070], [0079], [0092-0093], [0099] and [0129].
Claim 15.
The computer-implemented method of claim 2, wherein the stored user-configured superimposition of the image area is obtained based on user interactions with one or more user interface objects that cause the image corresponding to the image area to be manipulated. (see [0032], [0041], [0051] and [0099].
Claims 16 and 20 are rejected with similar reasons as set forth in claim 1, above.
Claim 17 is rejected with similar reasons as set forth in claim 2, above.
Claim 18.
The system of claim 16, the operations further comprising generating the 3D virtual reset, wherein generating the 3D virtual reset comprises: presenting, at a user interface, the 3D virtual object in the 3D virtual reset at a first position (see [0041], [0051] and [0092]); receiving, via the user interface, an edit to the 3D virtual object in the 3D virtual reset (see [0092-0093], [0099], [0119] and [0129]); updating the presentation of the 3D virtual reset by showing the edit (see [0092-0093], [0099], [0104], [0123] and [0129]); and storing the 3D virtual reset by including, in the 3D virtual reset, information about the 3D virtual object and information about the edit. see [0092-0093], [0099], [0104], [0119] and [0129]).
Claims 14 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Geisner, Rolion and further in view of Williams et al., US 2012/0223943 A1, hereinafter Williams.
Claim 14.
Geisner does not explicitly disclose the computer-implemented method of claim 1, wherein presenting the virtual reset comprises presenting the virtual reset in a virtual reality environment or an augmented reality environment.
Rolion illustrates in fig. 5 a marker and information but may not present the virtual reset of the claimed invention.
However, Williams teaches the virtual reality environment at [0034] and fig. 2 step 210.
Thus, it would have been obvious to one of ordinary skill in the art to combine the teachings of Williams the virtual reality environment into teachings of Geisner and Rolion, as it would allow the system to be completely virtual when generating the reset.
Claim 19.
Geisner does not disclose (see the underlined areas) the system of claim 16 the operations further comprising: presenting a user interface object for performing manipulations to the image area showing the stored portion of the real-world object, wherein the stored user-configured superimposition of the image area on the face of the 3D shape is generated based on the manipulations including at least one of: resizing, rotating, or cropping.
However, Williams discloses at [0033] the effect of moving a product 700 from a first shelf to a second shelf may not be fully appreciated on a typical computer monitor. At [0051] discloses by combining the advantages of a 3D physical model in terms of ease of arrangement and comprehension of the overall store layout with the detail and point of view of the virtual model, a user is given the ability to make layout changes in context of the overall store and see in real time a visual view of the store with product-level detail. Also see [0058].
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action.  Accordingly, THIS ACTION IS MADE FINAL.  See MPEP § 706.07(a).  Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action.  In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action.  In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.

APPLICANT REBUTTAL: REMARKS/ARGUMENTS
Upon entry of the present amendment, claims 1, 3, 5-16, and 19-23 will be pending in this application. Claims 1, 3, 5, 7, 9, 12, 13, 15, 16, and 20 have been amended. Claims 2, 4, and 17 have been canceled. New claims 21-23 have been added. No new matter is added. Based on the following remarks, Applicant respectfully requests reconsideration and allowance of the pending claims.
Claims 1-20 stand rejected under 35 U.S.C. § 112(a) First Paragraph as allegedly failing to comply with the written description requirement. Applicant respectfully disagrees.
Support for claims 1, 16, and 20 can be found in at least FIGS. 4A-4E and paragraphs [0081] - [0082] which state "[r]esponsive to receiving a selection of one of the interface objects 410, 411, or 412, the modeling system 130 can activate or cause the camera device 213 to enable a user to capture a facial image of a corresponding face of a physical object 201X or can display one or more user interface objects that enable the user to access an image file to upload as the facial image. FIG. 4D depicts an illustration of a user interface for generating a virtual object, including a facial image imposed to a face selected via the user interface of FIG. 4C ... User interface object 419 enables saving of the modified facial image subsequent to application of image modification operations instructed via interface objects 415, 416, 417, and 418." (Specification at [0081] - [0082], Emphasis Added).
35 U.S. C. §103
Claims 1-13, 15-18, and 20 stand rejected under 35 U.S.C. § 103 as allegedly being
unpatentable over Geisner et al., U.S. Pat. App. Pub. No. 2013/0083064 (hereinafter, "Geisner"), and further in view of Rolion et al., U.S. Pat. App. Pub. No. 2021/0248823 (hereinafter, "Rolion").
Claims 14 and 19 stand rejected under 35 U.S.C. § 103 as allegedly being unpatentable over Geisner, Rolion, and further in view of Williams et al., U.S. Pat. App. Pub. No. 2012/0223943 (hereinafter, "Williams").
Claim 1
Without commenting on the merits of the rejections and in an effort to move prosecution forward, claim 1 has been amended to recite elements of dependent claim 2 and narrowed elements of dependent claim 4. Amended claim 1 now recites:
generating a three dimensional (3D) virtual object, wherein generating the
3D virtual object comprises:
receiving, via a user interface of a device, a selection of a 3D shape that
includes a plurality offaces including the face;
receiving, via the user interface of the device, afirst indication of an input
indicating properties for the 3D shape;
receiving, via the user interface of the device, a second indication of an
image showing a portion of a real-world object;
determining an area of the image that corresponds to the portion of the
real-world object;
associating, in the 3D virtual object, the area of the image with the face
of the plurality offaces of the 3D shape so the area of the image is superimposed
on the face of the 3D virtual object; and
associating, in the 3D virtual object or in metadata of the virtual object,
the properties with the 3D virtual object, wherein the properties comprise
dimensions and a weight,
among other elements. (Amended Claim 1). For the reasons discussed during the
Examiner Interview, Applicant believes that amended claim 1 moots the pending rejection and is in condition for allowance.
New Claims 21-23
New claims 21-23 have been added. Support for new claim 21 can be found in at least
paragraphs [0085] - [0086] of the Specification. Support for new claims 22 and 23 can be found in at least paragraphs [0059], [0077], and [0099] of the Specification.
CONCLUSION

NEXT OFFICE ACTION: Notice of Pre-AIA  or AIA  Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection.  Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114.  Applicant's submission filed on 8/19/2024 has been entered.
Allowable Subject Matter
Claims 1, 3, 5-16, 18-23 allowed.
The following is an examiner’s statement of reasons for allowance: the cited prior arts do not teach or suggest the features recited in claim 1 and similarly in claims 16 , 20, as follows: “ … receiving, via the user interface of the device, a second indication of an image showing a portion of a real-world object: determining an area of the image that corresponds to the portion of the real-world object: associating, in the 3D virtual object, the area of the image with the face of the plurality of faces of the 3D shape so the area of the image 1s superimposed on the face of the 3D virtual object; and associating, in the 3D virtual object or in metadata of the virtual object, the properties with the 3D virtual object, wherein the properties comprise dimensions and a weight; storing the 3D virtual object that corresponds to the real- world object, the 3D virtual object including a stored user-configured superimposition of the area of the image showing a stored portion of the real-world object on a face of the 3D shape; storing a virtual reset that includes information about the 3D virtual object and a position of the 3D virtual object in the virtual reset; and presenting the virtual reset, the presentation showing the 3D virtual object at the position”.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee.  Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”

CITED REFERENCE SUMMARIES: Ref US20130147839A1: Claims:
An augmented reality providing system comprising an information processing terminal having an imaging unit and a display unit and an information processing apparatus which are communicable with each other, wherein the information processing terminal comprises: a request unit sending to the information processing apparatus a request, including image information of contents of imaging in the imaging unit, for obtaining product information indicating a product which can be displayed on a shelf which can be determined from the image information; and a display control unit displaying product information, included in a reply sent from the information processing apparatus in response to the request, on the display unit in a manner of overlaying on image information imaged by the imaging unit, and the information processing apparatus comprises: a determination unit determining a shelf from the image information included in the request and determining a size of an empty space of the shelf; a selection unit selecting product information of products smaller than the size of the empty space of the shelf determined by the determination unit, from a storage unit storing a plurality of sets of product information indicating a product and size information indicating a size of the product in association with each other; and a reply unit sending to the information processing terminal a reply including product information selected in the selection unit.
the information processing terminal comprises:
a request unit sending to the information processing apparatus a request, including image information of contents of imaging in the imaging unit, for obtaining product information indicating a product which can be displayed on a shelf which can be determined from the image information; and
a display control unit displaying product information, included in a reply sent from the information processing apparatus in response to the request, on the display unit in a manner of overlaying on image information imaged by the imaging unit, and
the information processing apparatus comprises:
a determination unit determining a shelf from the image information included in the request and determining a size of an empty space of the shelf;
a selection unit selecting product information of products smaller than the size of the empty space of the shelf determined by the determination unit, from a storage unit storing a plurality of sets of product information indicating a product and size information indicating a size of the product in association with each other; and
a reply unit sending to the information processing terminal a reply including product information selected in the selection unit.
The augmented reality providing system according to claim 1 , wherein the storage unit stores display rule information identifying a product allowed to be displayed on the shelf, and the selection unit selects product information of a product identified as allowed to be displayed on the shelf from the display rule information out of the product information of products smaller than the size of the empty space of the shelf.
the storage unit stores display rule information identifying a product allowed to be displayed on the shelf, and
the selection unit selects product information of a product identified as allowed to be displayed on the shelf from the display rule information out of the product information of products smaller than the size of the empty space of the shelf.
The augmented reality providing system according to claim 2 , wherein the display rule information includes category information of the product allowed to be displayed on the shelf, and the selection unit selects product information of a product belonging to the category information out of the product information of products smaller than the size of the empty space of the shelf.
the display rule information includes category information of the product allowed to be displayed on the shelf, and
the selection unit selects product information of a product belonging to the category information out of the product information of products smaller than the size of the empty space of the shelf.
The augmented reality providing system according to claim 2 , wherein the display rule information includes brand information of the product allowed to be displayed on the shelf, and the selection unit selects product information of a product of a brand corresponding to the brand information out of the product information of products smaller than the size of the empty space of the shelf.
the display rule information includes brand information of the product allowed to be displayed on the shelf, and
the selection unit selects product information of a product of a brand corresponding to the brand information out of the product information of products smaller than the size of the empty space of the shelf.
The augmented reality providing system according to claim 2 , wherein the display rule information includes product information of the product allowed to be displayed on the shelf, and the selection unit selects product information which matches the product information included in the display rule information out of the product information of products smaller than the size of the empty space of the shelf.
the display rule information includes product information of the product allowed to be displayed on the shelf, and
the selection unit selects product information which matches the product information included in the display rule information out of the product information of products smaller than the size of the empty space of the shelf.
The augmented reality providing system according to claim 1 , wherein the storage unit stores information indicating unit sales records of every product, and the selection unit preferentially selects product information of a product of which the unit sales record is relatively high out of the product information of products smaller than the size of the empty space of the shelf.
the storage unit stores information indicating unit sales records of every product, and
the selection unit preferentially selects product information of a product of which the unit sales record is relatively high out of the product information of products smaller than the size of the empty space of the shelf.
The augmented reality providing system according to claim 1 , wherein the storage unit stores information indicating unit sales records of every product, and the product information stored in the storage unit include product image information representing products, and the determination unit identifies products displayed on the shelf from the image information included in the request and the product image information stored in the storage unit, determines a product of which the unit sales record is relatively low from the products displayed on the shelf, and determines a size of a space which is generated when the determined product is removed from the shelf as the size of the empty space of the shelf.
the product information stored in the storage unit include product image information representing products, and
the determination unit identifies products displayed on the shelf from the image information included in the request and the product image information stored in the storage unit, determines a product of which the unit sales record is relatively low from the products displayed on the shelf, and determines a size of a space which is generated when the determined product is removed from the shelf as the size of the empty space of the shelf.
The augmented reality providing system according to claim 1 , wherein when it is determined that there are a plurality of empty spaces on a same level of the shelf from the image information included in the request, the determination unit determines a size obtained by adding up the plurality of empty spaces as the size of the empty space of the shelf.
when it is determined that there are a plurality of empty spaces on a same level of the shelf from the image information included in the request, the determination unit determines a size obtained by adding up the plurality of empty spaces as the size of the empty space of the shelf.
The augmented reality providing system according to claim 1 , wherein the storage unit stores end information indicating when a sale ends for every product, the information processing apparatus further comprises a creation unit creating, when it is determined based on the end information that there is product information of a product of which sale ends when a preset time passes in the product information selected in the selection unit, instruction information indicating an instruction related to the end of sale of the product, the reply unit sends the instruction information to the information processing terminal when the instruction information is created in the creation unit, and the display control unit displays the instruction information on the display unit.
the storage unit stores end information indicating when a sale ends for every product,
the information processing apparatus further comprises a creation unit creating, when it is determined based on the end information that there is product information of a product of which sale ends when a preset time passes in the product information selected in the selection unit, instruction information indicating an instruction related to the end of sale of the product,
the reply unit sends the instruction information to the information processing terminal when the instruction information is created in the creation unit, and
the display control unit displays the instruction information on the display unit.
The augmented reality providing system according to claim 1 , wherein the storage unit stores display rule information identifying a product allowed to be displayed on the shelf and product image information representing the product, the information processing apparatus further comprises a creation unit creating, when it is identified that there is a product which is not allowed to be displayed on the shelf from the display rule information and from the image information included in the request and the product image information stored in the storage unit, indication information indicating an existence of the product which is not allowed to be displayed on the shelf, the reply unit sends the indication information to the information processing terminal when the indication information is created in the creation unit, and the display control unit displays the indication information on the display unit.
the storage unit stores display rule information identifying a product allowed to be displayed on the shelf and product image information representing the product,
the information processing apparatus further comprises a creation unit creating, when it is identified that there is a product which is not allowed to be displayed on the shelf from the display rule information and from the image information included in the request and the product image information stored in the storage unit, indication information indicating an existence of the product which is not allowed to be displayed on the shelf,
the reply unit sends the indication information to the information processing terminal when the indication information is created in the creation unit, and
the display control unit displays the indication information on the display unit.
An information processing apparatus, comprising: a determination unit determining a shelf from image information of contents of imaging in an imaging unit sent from an information processing terminal having the imaging unit and a display unit and determining a size of an empty space of the shelf; a selection unit selecting product information of products smaller than the size of the empty space of the shelf determined by the determination unit, from a storage unit storing a plurality of sets of product information indicating a product and size information indicating a size of the product in association with each other; and a reply unit sending to the information processing terminal a reply including product information selected in the selection unit.
a determination unit determining a shelf from image information of contents of imaging in an imaging unit sent from an information processing terminal having the imaging unit and a display unit and determining a size of an empty space of the shelf;
An information processing terminal having an imaging unit and a display unit, the terminal comprising: a determination unit determining a shelf from image information of contents of imaging in the imaging unit and determining a size of an empty space of the shelf; a request unit sending a request, including information of the size of the empty space of the shelf determined in the determination unit, for obtaining product information indicating a product which can be displayed on the shelf to an information processing apparatus selecting product information of products smaller than the size of the empty space of the shelf from a storage unit storing a plurality of sets of product information indicating a product and size information indicating a size of the product in association with each other; and a display control unit displaying product information, included in a reply sent from the information processing apparatus in response to the request, on the display unit in a manner of overlaying on image information imaged by the imaging unit.
a determination unit determining a shelf from image information of contents of imaging in the imaging unit and determining a size of an empty space of the shelf;
a request unit sending a request, including information of the size of the empty space of the shelf determined in the determination unit, for obtaining product information indicating a product which can be displayed on the shelf to an information processing apparatus selecting product information of products smaller than the size of the empty space of the shelf from a storage unit storing a plurality of sets of product information indicating a product and size information indicating a size of the product in association with each other; and
a display control unit displaying product information, included in a reply sent from the information processing apparatus in response to the request, on the display unit in a manner of overlaying on image information imaged by the imaging unit.
An augmented reality providing method in an augmented reality providing system including an information processing terminal having an imaging unit and a display unit and an information processing apparatus which are communicable with each other, the method comprising: sending by the information processing terminal a request including image information of contents of imaging in the imaging unit to the information processing apparatus; determining by the information processing apparatus a shelf from the image information included in the request and determining a size of an empty space of the shelf; selecting by the information processing apparatus product information of products smaller than the size of the empty space of the shelf determined in the determining step, from a storage unit storing a plurality of sets of product information indicating a product and size information indicating a size of the product in association with each other; sending by the information processing apparatus a reply including product information selected in the selecting step to the information processing terminal; and displaying by the information processing terminal product information, included in a reply sent from the information processing apparatus in response to the request, on the display unit in a manner of overlaying on image information imaged by the imaging unit.
sending by the information processing terminal a request including image information of contents of imaging in the imaging unit to the information processing apparatus;
determining by the information processing apparatus a shelf from the image information included in the request and determining a size of an empty space of the shelf;
selecting by the information processing apparatus product information of products smaller than the size of the empty space of the shelf determined in the determining step, from a storage unit storing a plurality of sets of product information indicating a product and size information indicating a size of the product in association with each other;
sending by the information processing apparatus a reply including product information selected in the selecting step to the information processing terminal; and
displaying by the information processing terminal product information, included in a reply sent from the information processing apparatus in response to the request, on the display unit in a manner of overlaying on image information imaged by the imaging unit.
An information processing method, comprising: determining a shelf from image information of contents of imaging in an imaging unit sent from an information processing terminal having the imaging unit and a display unit and determining a size of an empty space of the shelf; selecting product information of products smaller than the size of the empty space of the shelf determined in the determining step, from a storage unit storing a plurality of sets of product information indicating a product and size information indicating a size of the product in association with each other; and sending to the information processing terminal a reply including product information selected in the selecting step.
determining a shelf from image information of contents of imaging in an imaging unit sent from an information processing terminal having the imaging unit and a display unit and determining a size of an empty space of the shelf;
selecting product information of products smaller than the size of the empty space of the shelf determined in the determining step, from a storage unit storing a plurality of sets of product information indicating a product and size information indicating a size of the product in association with each other; and
sending to the information processing terminal a reply including product information selected in the selecting step.
An information processing method executed by an information processing terminal having an imaging unit and a display unit, the method comprising: determining a shelf from image information of contents of imaging in the imaging unit and determining a size of an empty space of the shelf; sending a request, including information of the size of the empty space of the shelf determined in the determining step, for obtaining product information indicating a product which can be displayed on the shelf to an information processing apparatus selecting product information of products smaller than the size of the empty space of the shelf from a storage unit storing a plurality of sets of product information indicating a product and size information indicating a size of the product in association with each other; and displaying product information, included in a reply sent from the information processing apparatus in response to the request, on the display unit in a manner of overlaying on image information imaged by the imaging unit.
determining a shelf from image information of contents of imaging in the imaging unit and determining a size of an empty space of the shelf;
sending a request, including information of the size of the empty space of the shelf determined in the determining step, for obtaining product information indicating a product which can be displayed on the shelf to an information processing apparatus selecting product information of products smaller than the size of the empty space of the shelf from a storage unit storing a plurality of sets of product information indicating a product and size informat
[Truncated to input limit]