System, method, and apparatus for an interactive container

An interactive container creation method, apparatus and system. The method includes creating a list, deploying the list to at least one device, calibrating and identifying touch areas, identifying at least one of an asset and a shape to be defined as a touch area, identifying the x,y axis of each point for a predetermined number of points for each of the at least one of asset or shape, and creating a touch area based of the identified x,y axis.

FIELD OF THE INVENTION

The disclosure relates to systems, apparatus and methods for creating and operating interactive containers. More specifically, this disclosure relates to creating and operating interactive containers that relate to any assets that are projected, printed, displayed, etc.

BACKGROUND OF THE INVENTION

It has become more common from assets of different origin or type to communicate and cause an activity based on such interaction. For example, it has become common for users to utilize their portable devices to control various products in their home and/or office made by different manufacturers. The selection of the assets and its interaction can be customizable and variable. Therefore, it is desirable to be able to simulate such interactions and to be able to customize it. In addition, some assets may be susceptible to tampering. Thus, it is beneficial to display an interactive image, printout, etc. of such assets. Therefore, there is a need for an improved system, apparatus and method for creating and operating interactive container(s).

SUMMARY OF THE INVENTION

Embodiments described herein relate to an interactive container creation method, apparatus and system. The method includes creating a list, deploying the list to at least one device, calibrating and identifying touch areas, identifying at least one of an asset and a shape to be defined as a touch area, identifying the x,y axis of each point for a predetermined number of points for each of the at least one of asset or shape, and creating a touch area based of the identified x,y axis.

DETAILED DESCRIPTION

In the descriptions that follow, like parts are marked throughout the specification and drawings with the same numerals, respectively. The drawing figures are not necessarily drawn to scale and certain figures may be shown in exaggerated or generalized form in the interest of clarity and conciseness.

Any combination of one or more computer readable media may be utilized. The computer readable media may be a computer readable signal medium, any type of memory or a computer readable storage medium. For example, a computer readable storage medium may be, but not limited to, an electronic, magnetic, optical, electromagnetic, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium would include, but are not limited to: a portable computer diskette, a hard disk, a random access memory (“RAM”), a read-only memory (“ROM”), an erasable programmable read-only memory (“EPROM” or Flash memory), an appropriate optical fiber with a repeater, a portable compact disc read-only memory (“CD-ROM”), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. Thus, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

Computer program code for carrying out operations utilizing a processor for aspects of the present disclosure may be written in any combination of one or more programming languages, markup languages, style sheets and JavaScript libraries, including but not limited to Windows Presentation Foundation (WPF), HTML/CSS, XAML, and JQuery, C, Basic, *Ada, Python, C++, C #, Pascal, *Arduino. Additionally, operations can be carried out using any variety of compiler available.

FIG. 1is an embodiment illustrating a flow diagram of a method100for creating at least one interactive container. The method100starts at step102and proceeds to step104. At step102, the method100creates a list. The list may contain images, assets, attributes, WISPSs, rules, menus, etc. A WISP in this application relates to a shell that defines the rules and the interaction between the assets and/or containers. In an embodiment, the creation of the list is performed at a remote location or on a cloud. In other embodiments, the creation of the list is performed on the same device operating the interaction between the assets, menus, and/or containers. In such embodiments, the deployment step would not be necessary.

At step106, the method100deploys at least one list to a device that is operating the interaction between the assets, menus, and/or containers. In one embodiment, the deployment may occur on several devices that may or may not be at the same location. The device(s) may be at the same location as the container being operated. In one embodiment, the axis location, i.e. x, y, x, location of the assets may be incorporated into the list at the list creation time or it may be determined on the device controlling the interaction, i.e., a device located at the same location as the container. The device controlling the interaction may learn the location of the assets, it may display the assets, or it may scan for characteristics to learn their location. In one embodiment, a list may already exist and only changes, omissions and/or additions are deployed, rather than the entire list. Furthermore, the deployment may be initiated/conducted manually or it may be automatic.

At step108, the method100calibrates assets subjects in the container and/or identifies the touch areas. During the calibration process, the method100may perform projection mapping for every container to ensure that the display matches the physical space. In one embodiment, the method100uses image training during calibration to detect a known image, item, logo, etc.

In other embodiments, a person manually calibrates the system by shifting from point to point identifying the touch area and triggering a new touch area when the current touch area is done and another touch area exists and needs to be identified by the system. Whereas, during an automatic calibration, the system automatically identifies a predetermined number of points per touch area relating to assets and/or shapes. In another embodiment, a calibration stream is cropped to where only areas of interest are calibrated. Only calibrating areas of interest results in a more accurate and more efficient calibration. The calibration process is better described inFIG. 2. Method100ends at step110.

FIG. 2is an embodiment illustrating a flow diagram of a method200for calibrating at least one interactive container. Method200starts at step202and proceeds to step204, wherein the method202detects an asset or shape displayed that needs to be defined as a touch area. At step206, the method200identifies a predetermined number of points relating to the asset or shape where each point is defined by its x, y axis. At step208, the method200determines if there are more assets or shapes to be identified as touch areas. If there are more assets or shapes to be identified as touch areas, the method200returns to step204. Otherwise, the method200ends at step210.

For example, a projector displays a pre-determined shape over a touch area not identified yet. Using a camera, the method identifies the x, y axis for each point in a pre-determined number of points relating to the asset or displayed shape. Once the axis is identified, the method200proceeds to the next asset or shape in the container. The method200may perform such function on a single container or multiple containers. The method200may utilize asset identification, display recognition, shape recognition, light, exposure, contrast, RGB difference, infrared, etc. to determine the areas that need to be identified as touch areas. When all touch areas are identified, the camera and/or method are capable of identifying the touch areas and identify the corresponding rule, menu, activity etc. relating to the touch area.

FIG. 3is a block diagram illustrating an embodiment of an apparatus300of interactive containers. In this embodiment, the apparatus300has two containers302A and302B, where container302A has two menus/attributes304A and304B. Container302B has a single menu/attributes304C. Each of the menu/attribute's304A,304B and304C has a WISP/Rules306A,306B and306C, respectively. Each of the WISP/Rules306A,306B and306C has assets308A,308B and308C, respectively.

A single interactive apparatus300may include any number of containers that may or may not communicate and/or interact. As such, in one embodiment, interacting with one container may cause a change in another container. Containers create an interactive experience using the menus/attributes and WISP/rules relating to assets. The menu/attributes are options at an instance, which may be a default instance or options that come about due to an interaction or touch on or around a menu item or attribute presented. A container may contain any number of menus/attributes306, which may interact or stand alone. Attributes may be audio, video, image, change in display, etc. WISP/rules are the interactive active mask over a touch area that triggers a menu or attribute due to a pre-determined activity. Assets may be pre-determined object or person, printouts of objects, displayed items, images, video, an identified object or person, and the like.

In one embodiment, a weighted average may be used. In such an embodiment, a new object/asset is added to a container. The weighted average method adds the object/asset incrementally over time where the accounting of the new item increases in percentile in relation to the whole picture over time. Such a method insures that the item is truly added, allows of real-time reaction to change in a container, and allows for a realistic change over time.

FIG. 4is a block diagram illustrating an embodiment of an interactive system400relating to at least one interactive container. In this embodiment, the system400includes a processor402, memory/storage medium404, a calibrator406, a touch detector408, a touch listener410, an analytics module412and an I/O414. The memory404include deployed data404A, touch area data404B, analytics data404C and the likes.

Even though all these items are shown to be in the same system400, yet, they may be distributed in multiple systems that may or may not be in the same location. In one embodiment, a cloud may communicate with the systems400to deploy items from remote locations, such as, the deployed data404A.

The touch detector408detects touch and its related information, which includes identifying coordinate related to a touch area. In one embodiment, the touch detector408may distinguish between a hover and a touch, where the distinction relates to the z axis of the touch. If the hand or object is closer to the object or further from a camera or system then it is a touch. If the hand or object is further from the object or closer to a camera or system then it is hover. In one embodiment, the touch detector may identify different types of touch based on thresholds, such as time, proximity, color of the object doing the touch, based on a sequence of touches, etc. The touch detection408may refine the recognition of a touch by performing the method ofFIG. 5, which will be described herein below. In another embodiment, the touch detector may crop areas to where only areas of interest are detected, resulting in a touch detection that is more accurate and more efficient.

The touch listener410reads the coordinates determined by the touch detector and determines if the touch occurred in a touch area identified during calibration. The touch listener410determines the type of reaction or no reaction to take place based on the deployed data, the location of the touch and sometime the type of touch. In some cases, the touch listener410may facilitate a zoom in/out or a drag based on the determination of the type of touch. Touch listener may determine that there are no persons and/or no touch for a predetermined time or sense a person walk away and initiate a default display or a predetermined activity.

The analytics module412is designed to collect data and/or measure characteristics related to a predetermined object, person, movement, lack of movement, etc. for example, the analytics module412may identify a person, follow a path of a person, follow selections of a person, duration of a touch, lack of touch, list a person's activity, determine gender, personal characteristics, traffic, dwell time, etc.

FIG. 5is an embodiment illustrating a flow diagram of a method500for refining touch recognition. The method500starts at step502and proceeds to step504. At step504the method500creates a baseline depth area using multi-frames from a depth camera. At step506, the method500creates a moving average of a real-time area from the depth camera. At step508, the method500determines the difference between the baseline and the moving average. At step510, the method500determines if the difference is less than a pre-determined threshold. If it is less, then the method500proceeds to step512and looks at the surrounding pixels to determine if the event is a touch or noise. If the surrounding pixels have the same z-axis depth, the event is a touch, and the method500proceeds to step514. In one embodiment, the radius of the surrounding pixels changes based on the depth of the camera. If the difference is greater than the threshold, then determine that the event is a touch, at step514. If the surrounding pixels have different z-axis, then the method500proceeds to step516. At step516, the method500determines that the event is not a touch. From steps514and516, the method500proceeds to step518where it ends.

FIG. 6A-Care diagrams depicting an embodiment of an interactive container. InFIG. 6A, a container is shown that displays a car engine with its mechanics and electronics. InFIG. 6B, a touch is detected activating a touch area. InFIG. 6C, the touch results in the display of information related to the touch area. In other embodiments, such a touch may result in an engine sound, a menu display, a video activation etc.

It will be appreciated by those skilled in the art that changes could be made to the embodiments described above without departing from the broad inventive concept. It is understood, therefore, that this disclosure is not limited to the particular embodiments herein, but it is intended to cover modifications within the spirit and scope of the present disclosure as defined by the appended claims.