Office Action 5: Notice of Allowance (2024-11-19)

CLAIMS:
IN THE CLAIMS

Please amend the claims are shown below to include underlined sections and delete strikeout or double bracketed ("\[\[ \]\]") sections.

Listing of Claims:

1\. (Currently amended) A hand-held controller comprising:a main body, wherein the main body has a touch surface;
a first extension extending from the main body;
a second extension extending from the main body;
a first group of buttons;
a second group of directional buttons;
a light bar located with respect to the main body, wherein the light bar is located along an edge of the touch surface to be visible to a camera fitted to a lower region of a head-mounted display (HMD), wherein the light bar extends between the first group of buttons and the second group of directional buttons, wherein the light bar is configured to emit light to be detected by the camera; and
a light indicator located with respect to the main body, wherein the light indicator is configured to emit light to be detected by the camera, wherein the lights emitted by the light bar and the light indicator facilitate a determination of a plurality of orientations of the main body, the first extension, and the second extension with reference to the HMD,
wherein the light bar has a plurality of edges, wherein the plurality of orientations are determined based on whether each of the plurality of edges is above or below a horizontal level that passes through the light indicator, wherein the plurality of orientations include a first orientation and a second orientation,
wherein the plurality of edges include a first edge and a second edge, wherein the first orientation of the first and second extensions is achieved when the plurality of edges are below the horizontal level, wherein the second orientation of the first and second extensions is achieved when the first edge is above the horizontal level and the second edge is below the horizontal level.
2\. (Previously presented) The hand-held controller of claim 1, wherein the hand-held controller has an edge, wherein the light bar is located along the edge of the hand-held Attorney Docket No. SONYP360.C1 2

controller to be visible to the camera fitted to the lower region of the HMD, wherein the light indicator is located at a distance from the light bar to be visible to the camera.
3\. (Previously presented) The hand-held controller of claim 1, wherein the touch surface is configured to determine movements of fingers of a user, wherein the light bar includes a plurality of light sources, wherein the plurality of light sources are located at pre-determined locations with reference to the touch surface.
4\. (Previously presented) The hand-held controller of claim 1, wherein the light bar is an infrared light emitter.
5\. (Previously presented) The hand-held controller of claim 1, wherein the light bar is configured to face the camera to be in a field-of-view facing downward from the camera.
6\. (Previously presented) The hand-held controller of claim 1, further comprising a touch panel situated between the first extension and the second extension, wherein the touch panel has a curvature, wherein the light bar is a string of light sources located along an edge of the main body and is situated above the touch panel.
7\. (Previously presented) The hand-held controller of claim 1, further comprising a battery configured to provide power to the light bar and the light indicator, wherein the light indicator is a light emitting diode.
8\. (Currently amended) The hand-held controller of claim 1, wherein the camera has a co-ordinate system having an x-axis, a y-axis, and a z-axis, wherein the first and second orientations are determined with respect to the x, y, and z axes of the co-ordinate system.
9\. (Previously presented) The hand-held controller of claim 1, further comprising a haptic feedback module and a touch panel, wherein the main body includes a first joystick and a second joystick that are configured to be moved to generate input data for processing by a computing device, wherein the touch panel is located between the first and second joysticks, wherein the haptic feedback module is configured to provide tactile feedback to a user.

10\. (Currently amended) The hand-held controller of claim 1, wherein the lights are detected by an additional camera located at the bottom surface of the HMD to determine the first and second orientations of the main body, the first extension, and the second extension with reference to the additional camera.

11\. (Previously presented) The hand-held controller of claim 1, wherein the light bar is located below the touch surface to be visible to the camera.

12\. (Currently amended) A head-mounted display comprising:a top surface;
a front surface located adjacent to the top surface;
a bottom surface located adjacent to the front surface;
a band coupled to the top surface; and
a camera located at the bottom surface, wherein the camera is configured to capture images of lights emitted from a light bar and a light indicator of a hand-held controller to facilitate a determination of a plurality of orientations of the hand-held controller with reference to the camera, wherein the light bar extends between a first group of buttons of the hand-held controller and a second group of directional buttons of the hand-held controller,
wherein the light bar has a plurality of edges, wherein the plurality of orientations are determined based on whether each of the plurality of edges is above or below a horizontal level that passes through the light indicator, wherein the plurality of orientations include a first orientation and a second orientation,
wherein the plurality of edges include a first edge and a second edge, wherein the first orientation of the first and second extensions is achieved when the plurality of edges are below the horizontal level, wherein the second orientation of the first and second extensions is achieved when the first edge is above the horizontal level and the second edge is below the horizontal level.

13\. (Original) The head-mounted display of claim 12, further including a haptic feedback device configured to vibrate the top, front, and bottom surfaces to provide tactile feedback to a head of a user.

14\. (Currently amended) The head-mounted display of claim 12, further including an additional camera located at the bottom surface, wherein the additional camera is configured to capture images of the lights from the light bar and the light indicator to facilitate a determination of the first and second orientations of the hand-held controller with reference to the additional camera.

15\. (Original) The head-mounted display of claim 12, further comprising:an eye mask having the top surface, the front surface, the bottom surface, and a frame, wherein the frame is fitted to the bottom surface, wherein the band is a headband coupled to the frame and a portion of the top surface.

16\. (Previously presented) The head-mounted display of claim 12, wherein the hand-held controller has a touch surface, wherein the light bar is located below the touch surface and along an edge of the touch surface to be visible to the camera.

17\. (Currently amended) A system comprising:a head-mounted display (HMD) including:
a top surface;
a front surface located adjacent to the top surface;
a bottom surface located adjacent to the front surface;
a band coupled to the top surface; and
a camera located at the bottom surface;
a hand-held controller including:
a main body;
a first extension extending from the main body;
a second extension extending from the main body;
a first group of buttons;
a second group of directional buttons;
a light bar located with respect to the main body, wherein the light bar extends between the first group of buttons and the second group of directional buttons, wherein the light bar is configured to emit light to be detected by the camera of the HMD; anda light indicator located with respect to the main body, wherein the light indicator is configured to emit light to be detected by the camera of the HMD, wherein Attorney Docket No. SONYP360.C1 5

the lights emitted by the light bar and the light indicator facilitate a determination of a plurality of orientations of the main body, the first extension, and the second extension with reference to the HMD,wherein the light bar has a plurality of edges, wherein the plurality of orientations are determined based on whether each of the plurality of edges is above or below a horizontal level that passes through the light indicator, wherein the plurality of orientations include a first orientation and a second orientation,
wherein the plurality of edges include a first edge and a second edge, wherein the first orientation of the first and second extensions is achieved when the plurality of edges are below the horizontal level, wherein the second orientation of the first and second extensions is achieved when the first edge is above the horizontal level and the second edge is below the horizontal level.

18\. (Previously presented) The system of claim 17, wherein the hand-held controller has an edge, wherein the light bar is located along the edge to be visible to the camera of the lower region of the HMD with the main body, wherein the light indicator is located at a distance from the light bar to be visible to the camera.

19\. (Currently amended) The system of claim 17, wherein the HMD includes an additional camera located at the bottom surface, wherein the additional camera is configured to capture images of the lights from the light bar and the light indicator to facilitate a determination of the first and second orientations of the hand-held controller with reference to the additional camera.

20\. (Previously presented) The system of claim 17, wherein the main body has a touch surface, wherein the light bar is located below the touch surface and along an edge of the touch surface to be visible to the camera of the HMD.

21.-26. (Canceled)2225

OFFICE ACTION:
DETAILED ACTION
Notice of Pre-AIA  or AIA  Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
As a result of the After Final proposed amendment on October 31, 2024, claims 1-20 are pending. Claims 1, 8, 10, 12, 14, 17 and 19 are amended. Claims 21-26 are canceled or were previously canceled.  This amendment is entered as it puts the application in condition for allowance.
Allowable Subject Matter
Claims 1-20 are allowed.
The following is an examiner’s statement of reasons for allowance:
Regarding independent claim 1, 12, or 17, the prior art of record does not disclose a hand-held controller, a head-mounted display, or a system comprising:
wherein the light bar has a plurality of edges, wherein the plurality of orientations are determined based on whether each of the plurality of edges is above or below a horizontal level that passes through the light indicator, wherein the plurality of orientations include a first orientation and a second orientation,
wherein the plurality of edges include a first edge and a second edge, wherein the first orientation of the first and second extensions is achieved when the plurality of edges are below the horizontal level, wherein the second orientation of the first and second extensions is achieved when the first edge is above the horizontal level and the second edge is below the horizontal level.
The prior art of record does not teach nor suggest the additional structural features of apparatus claims 1, 12 17, in addition to the other structural aspects of the claims, comprising wherein the light bar has a plurality of edges, wherein the plurality of orientations are determined based on whether each of the plurality of edges is above or below a horizontal level that passes through the light indicator, wherein the plurality of orientations include a first orientation and a second orientation, wherein the plurality of edges include a first edge and a second edge, wherein the first orientation of the first and second extensions is achieved when the plurality of edges are below the horizontal level, wherein the second orientation of the first and second extensions is achieved when the first edge is above the horizontal level and the second edge is below the horizontal level.
Since the previously cited Ohishima reference teaches tracking the position of the mallet in 2-D space (See Detailed Description, [0191-0230], discussing XY coordinates), it does not teach nor suggest orientations when the first edge is above the horizontal level and the second is below the horizontal level, and thereby would teach away from a modification to arrive at the features of the independent claims. Thus, claims 1, 12 and 17 are free of and unobvious over the prior art and are allowed.
Additional prior art is cited on the PTO-892 as relevant to the Field of invention but do not teach nor suggest the features of claims 1-20. The remaining claims are dependent off of claims 1, 12 or 17 and are allowed as result of their dependencies.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee.  Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion

APPLICANT REBUTTAL: REMARKS
Claims 1-20 are pending. Claims 1-20 and 23-25 are rejected. Claims 1, 8, 10, 12, 14, 17, and 19 are amended. Claim 26 is objected to. Claims 23-26 are canceled. No new matter has been added.
Claim Rejection - 35 USC § 103
Claims 1-8, 10-12, 14-20, and 23-25 are rejected under 35 U.S.C. §103 as being
unpatentable over Samples et al., (US 2018/0330521 Al) ("Samples") in view of Ohishima et al., ( US 2002/0084974 Al) ("Ohishima"). This rejection is respectfully traversed. However, to expedite the prosecution of the present patent application, claim 1 is amended to include the recitations similar to that of allowable claim 26. Also, claim 12 is amended to include the recitations similar to that of allowable claim 26 and claim 17 is amended to include the recitations similar to that of allowable claim 26. Hence, the independent claims 1, 12, and 17 are submitted to be in condition for allowance.
Claims 23-25 are canceled. Claims 2-8 and 10-11 depend from independent claim 1. Also, claims 14-16 depend from independent claim 12 and claims 18-20 depend from independent claim 17. Hence, for the reasons set forth above, the dependent claims 2-8, 10- 11, 14-16, and 18-20 are submitted to be in condition for allowance.
As such, it is respectfully requested that the section 103 rejection of claims 1-8, 10-12, 14-20, and 23-25 be withdrawn.
Claim Rejection - 35 USC § 103
Claims 9 and 13 are rejected under 35 U.S.C. §103 as being unpatentable over Samples, in view of Ohishima, further in view of Mallinson, United States Patent Application No. US 2016/0025978 Al. This rejection is respectfully traversed.
Claim 9 depends from independent claim 1 and claim 13 depends from independent claim 12. The independent claims 1 and 12 are submitted to be in condition for allowance.
Thus, for the reasons set forth above, the dependent claims 9 and 13 are submitted to be in condition for allowance.
As such, it is respectfully requested that the section 103 rejection of claims 9 and 13 be withdrawn.
Allowable Subject Matter
Claim 26 is objected to. Applicant thanks the Examiner for the indication of allowable subject matter. Claim 26 has been canceled.
Conclusion


CITED REFERENCE SUMMARIES: Ref US20150258432A1: Claims:
A method for enhancing detection of a user's hand relative to a head-mounted display (HMD), comprising: sensing a disrupted portion of energy, when disrupted, by a plurality of sensors integrated within a pad device and arranged in an array, wherein the disrupted portion of the energy is generated when the hand of the user is placed proximate to the pad device, wherein the plurality of sensors that sense the disrupted portion of the energy produce an energy image that mirrors a current position of the hand; repeating the sensing continuously to produce a stream of energy images, the stream of energy images showing changes of the position of the hand while the sensing is being repeated; and communicating the stream of energy images to a game console for processing each of the energy images to produce a model of the hand and movement of the model of the hand, wherein the model of the hand is at least partially rendered as a virtual hand in a virtual environment that is displayed in the HMD.
sensing a disrupted portion of energy, when disrupted, by a plurality of sensors integrated within a pad device and arranged in an array, wherein the disrupted portion of the energy is generated when the hand of the user is placed proximate to the pad device, wherein the plurality of sensors that sense the disrupted portion of the energy produce an energy image that mirrors a current position of the hand;
repeating the sensing continuously to produce a stream of energy images, the stream of energy images showing changes of the position of the hand while the sensing is being repeated; and
communicating the stream of energy images to a game console for processing each of the energy images to produce a model of the hand and movement of the model of the hand, wherein the model of the hand is at least partially rendered as a virtual hand in a virtual environment that is displayed in the HMD.
The method of claim 1 , further comprising emitting the energy using a plurality of energy emitters, the energy emitters integrated within the pad device and arranged in an array.
The method of claim 1 , wherein communicating the stream of energy images to the game console to produce the model of the hand comprises communicating the stream of energy images to the game console for facilitating a determination of a position and an orientation of the hand, wherein the position of the hand is determined at least partially based on intensities of the disrupted portion of the energy within the energy images.
The method of claim 1 , wherein communicating the stream of energy images to the game console to produce the model of the hand comprises communicating the stream of energy images to the game console for facilitating a determination of a position and an orientation of the hand, wherein another position and another orientation of the hand is determined using a camera connected to the game console or using a camera in the HMD or using a camera in the pad device or using a camera in an environment in which the pad device is located.
The method of claim 1 , wherein communicating the stream of energy images to the game console to produce the model of the hand comprises communicating electrical signals to the game console for determining a position and an orientation of the hand, wherein the position and orientation of the hand is to be determined with respect to an xyz co-ordinate system, wherein a surface of the pad device defines a reference for the xyz co-ordinate system.
The method of claim 1 , wherein the pad device includes markers that are located on a surface of the pad device, wherein the virtual hand is displayed on a display device of the HMD, wherein the HMD is to be worn on a head of the user and has a camera, wherein the camera is for capturing occlusion of one or more of the markers to facilitate a determination of a position and an orientation of the hand of the user.
The method of claim 1 , wherein the energy includes light energy, said method further comprising emitting the light energy using a plurality of infrared light emitters, the infrared light emitters integrated within the pad device and arranged in an array on the pad device.
The method of claim 1 , wherein sensing the disrupted portion of the energy is performed when the user is interacting with the virtual environment via the HMD.
The method of claim 1 , wherein sensing the disrupted portion of the energy is performed when the user is interacting with the virtual environment via the HMD, wherein the energy includes light energy, or ultrasonic energy, or electromagnetic energy, or a combination of two or more thereof.
The method of claim 1 , wherein the pad device has a surface on which the sensors are located, wherein the surface is for orientation towards the hand and for receiving a shadow of the hand when the hand interacts with an energy volume above the pad device, wherein the shadow defines intensities of the disrupted portion when disrupted by the user's hand.
The method of claim 1 , further comprising emitting the energy using a plurality of energy emitters, the energy emitters integrated within the pad device and arranged in an array, wherein the pad device is configured for receiving touch-less interaction from the hand, wherein the energy emitted extends to an energy boundary above the pad device, wherein the disrupted portion of the energy is generated when the hand is located within the energy boundary.
The method of claim 1 , wherein the model of the virtual hand is defined by a position of the virtual hand, or an orientation of the virtual hand, or a shape of the virtual hand, or a color of the virtual hand, or a boundary of the virtual hand, or a texture of the virtual hand, or a combination of two or more thereof, wherein the boundary of the virtual hand is defined by pixels of an image that form an outline of the virtual hand.
The method of claim 1 , wherein sensing the disrupted portion comprises sensing energy disrupted by a first finger joint of a finger of the hand and sensing energy disrupted by a second finger joint of the finger for facilitating a determination of a position of the finger of the user, the determination of the position performed using a line that couples a position of the first finger joint to a finger of the second finger joint.
The method of claim 1 , wherein the virtual hand is for display in the HMD when worn on a head of the user, or is for display on a television that is located in a real-world environment in which the user is located.
The method of claim 1 , wherein the stream of energy images is generated at a frame rate of the sensors, the frame rate being a rate of generation of frames of the energy images, wherein the frame rate is a frequency at which the disrupted portion of the energy is sensed or captured by the sensors.
The method of claim 1 , further comprising emitting the energy using a plurality of energy emitters, the energy emitters integrated within the pad device and arranged in an array, wherein the energy emitted is for creating an energy volume above the pad device, wherein the hand is placed proximate to the pad device to interact with and disrupt the energy volume.
The method of claim 16 , wherein the energy volume includes an electromagnetic energy field.
The method of claim 1 , wherein the movement of the model of the hand defines movement of one or more images of fingers of the hand, wherein the virtual hand is enabled to interact with a virtual object in the virtual environment, wherein the interaction is defined by an object model for the virtual object, the object model is defined for pre-defined movement or actions within the virtual environment.
A pad device for enhancing detection of a user's hand relative to a head-mounted display (HMD), comprising: a plurality of sensors for sensing a disrupted portion of energy, the sensors arranged in an array on a surface of the pad device, wherein the disrupted portion of the energy is generated when the hand of the user is placed proximate to the pad device, wherein the sensors are configured to sense the disrupted portion of the energy for producing an energy image that mirrors a current position of the hand, wherein the sensors are further configured to repeat the sensing continuously to produce a stream of energy images, the stream of energy images showing changes of the position of the hand while the sensing is being repeated; and a communication device coupled to the sensors, the communication device for communicating the stream of energy images to a game console for processing each of the energy images to produce a model of the hand and movement of the model of the hand, wherein the model of the hand is at least partially rendered as a virtual hand in a virtual environment that is displayed in the HMD.
a plurality of sensors for sensing a disrupted portion of energy, the sensors arranged in an array on a surface of the pad device, wherein the disrupted portion of the energy is generated when the hand of the user is placed proximate to the pad device, wherein the sensors are configured to sense the disrupted portion of the energy for producing an energy image that mirrors a current position of the hand,
wherein the sensors are further configured to repeat the sensing continuously to produce a stream of energy images, the stream of energy images showing changes of the position of the hand while the sensing is being repeated; and
a communication device coupled to the sensors, the communication device for communicating the stream of energy images to a game console for processing each of the energy images to produce a model of the hand and movement of the model of the hand, wherein the model of the hand is at least partially rendered as a virtual hand in a virtual environment that is displayed in the HMD.
The pad device of claim 19 , further comprising a plurality of emitters for emitting the energy, the energy emitters integrated within the pad device and arranged in an array on the surface of the pad device.
A system comprising: a pad device including: a plurality of sensors located on a surface of the pad device for receiving an energy portion upon interaction of a body part with energy emitted, wherein the sensors are configured to generate an energy image from the energy portion by sensing the energy portion when the body part is at a current position, wherein the sensors are further configured to repeat the sensing continuously to produce a stream of energy images, the stream of energy images showing changes of the position of the hand while the sensing is being repeated; a pad communication device coupled to the sensors for communicating the stream of energy images; a game console coupled to the pad device, the game console including: a game communication device coupled to the pad communication device for receiving the stream of the energy images from the pad communication device; a game processor coupled to the game communication device for determining a movement of the body part based on the stream of energy images, wherein the game processor is for determining a state of a virtual object in a virtual environment based on the movement of the body part, wherein the game communication device is for sending data regarding the state of the virtual object; and a head-mounted display (HMD) coupled to the game console, the HMD comprising: an HMD communication device coupled to the game communication device for receiving the data regarding the state of the virtual image from the game communication device; a processor coupled to the HMD communication device for displaying the virtual object having the state in an image.
a pad device including: a plurality of sensors located on a surface of the pad device for receiving an energy portion upon interaction of a body part with energy emitted, wherein the sensors are configured to generate an energy image from the energy portion by sensing the energy portion when the body part is at a current position, wherein the sensors are further configured to repeat the sensing continuously to produce a stream of energy images, the stream of energy images showing changes of the position of the hand while the sensing is being repeated; a pad communication device coupled to the sensors for communicating the stream of energy images;
a plurality of sensors located on a surface of the pad device for receiving an energy portion upon interaction of a body part with energy emitted,
wherein the sensors are configured to generate an energy image from the energy portion by sensing the energy portion when the body part is at a current position, wherein the sensors are further configured to repeat the sensing continuously to produce a stream of energy images, the stream of energy images showing changes of the position of the hand while the sensing is being repeated;
a pad communication device coupled to the sensors for communicating the stream of energy images;
a game console coupled to the pad device, the game console including: a game communication device coupled to the pad communication device for receiving the stream of the energy images from the pad communication device; a game processor coupled to the game communication device for determining a movement of the body part based on the stream of energy images, wherein the game processor is for determining a state of a virtual object in a virtual environment based on the movement of the body part, wherein the game communication device is for sending data regarding the state of the virtual object; and
a game communication device coupled to the pad communication device for receiving the stream of the energy images from the pad communication device;
a game processor coupled to the game communication device for determining a movement of the body part based on the stream of energy images,
wherein the game processor is for determining a state of a virtual object in a virtual environment based on the movement of the body part,
wherein the game communication device is for sending data regarding the state of the virtual object; and
a head-mounted display (HMD) coupled to the game console, the HMD comprising: an HMD communication device coupled to the game communication device for receiving the data regarding the state of the virtual image from the game communication device; a processor coupled to the HMD communication device for displaying the virtual object having the state in an image.
an HMD communication device coupled to the game communication device for receiving the data regarding the state of the virtual image from the game communication device;
a processor coupled to the HMD communication device for displaying the virtual object having the state in an image.
The system of claim 21 , wherein the game processor is for determining a position of the virtual object with reference to a reference point defined in the image as being proportional to the position of the body part with respect a reference defined by the surface of the pad device, wherein the game processor is for determining an orientation of the virtual object with reference to the reference point defined in the image as being proportional to an orientation of the body part with respect to the reference defined by the surface of the pad device.
wherein the game processor is for determining a position of the virtual object with reference to a reference point defined in the image as being proportional to the position of the body part with respect a reference defined by the surface of the pad device,
wherein the game processor is for determining an orientation of the virtual object with reference to the reference point defined in the image as being proportional to an orientation of the body part with respect to the reference defined by the surface of the pad device.
The system of claim 21 , wherein the pad device includes a plurality of light emitters for emitting light towards the HMD, wherein the HMD includes a plurality of sensors for sensing the light emitted by the light emitters to facilitate determination of a position of the hand of the user.