User input passive device for use with an interactive display device

A user input passive device for interaction with a touchscreen of an interactive display device includes a housing including: a conductive shell and a non-conductive supporting surface coupled to the conductive shell. The user input further includes an impedance circuit having a desired impedance at a desired frequency, a first conductive plate mounted on the non-conductive supporting surface, electrically isolated from the conductive shell, and a second conductive plate mounted on the non-conductive supporting surface, electrically isolated from the conductive shell and the first conductive plate. The first terminal of the impedance circuit is coupled to the first conductive plate and a second terminal of the impedance circuit is coupled to the second conductive plate. When the user input passive device is used with the touchscreen, a perimeter of the conductive shell, and the first and second conductive plates are in close proximity to an interactive surface of the touchscreen.

Not Applicable.

Not Applicable.

BACKGROUND OF THE INVENTION

Technical Field of the Invention

This invention relates to computer systems and more particularly to interaction with a touch screen of a computing device.

Description of Related Art

Computers include user interfaces to receive data from a user and to output data to a user. A common user interface is a graphical user interface (GUI) that provides images, or icons, for various types of data input (e.g., select a file, edit a word, type a character, draw a picture, look at a photo, format a document, etc.). In an example, the user selects an icon by manipulating a mouse to align a cursor with an icon and then “selects” the icon. In another example, the user selects an icon by touching a touch screen interface with the user's finger or with a user input device. User input devices may be passive or active. Active devices provide power gain to a circuit whereas passive devices do not provide power gain to a circuit and do not transmit stimulus signals. For example, a traditional capacitive pen is a user input passive device made of conductive material, contains no battery, and interacts with a touch screen in the same manner as a user's finger.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1is a schematic block diagram of an embodiment of an interactive display device10having a touch screen12, which may further include a personalized display area18to form an interactive touch screen display (also referred to herein as an interactive surface). Personalized display area18may extend to all of touch screen12or a portion as shown. Further, touch screen12may include multiple personalized display areas18(e.g., for multiple users, functions, etc.). The interactive display device10, which will be discussed in greater detail with reference to one or more ofFIGS. 2-3, may be a portable computing device and/or a fixed computing device. A portable computing device may be a social networking device, a gaming device, a cell phone, a smart phone, a digital assistant, a digital music player, a digital video player, a laptop computer, a handheld computer, a tablet, a video game controller, and/or any other portable device that includes a computing core.

A fixed computing device may be a computer (PC), an interactive white board, an interactive table top, an interactive desktop, an interactive display, a computer server, a cable set-top box, vending machine, an Automated Teller Machine (ATM), an automobile, a satellite receiver, a television set, a printer, a fax machine, home entertainment equipment, a video game console, and/or any type of home or office computing equipment. An interactive display functions to provide users with an interactive experience (e.g., touch the screen to obtain information, be entertained, etc.). For example, a store provides interactive displays for customers to find certain products, to obtain coupons, to enter contests, etc.

Here, the interactive display device10is implemented as an interactive table top. An interactive table top is an interactive display device10that has a touch screen display for interaction with users but also functions as a usable table top surface. For example, the interactive display device10may include one or more of a coffee table, a dining table, a bar, a desk, a conference table, an end table, a night stand, a cocktail table, a podium, and a product display table.

As an interactive table top, the interactive display device10has interactive functionality and well as non-interactive functionality. For example, interactive objects14(e.g., a finger, a user input passive device, a user input active device, a pen, tagged objects, etc.) interact with the touch screen12to communicate data with interactive display device10. A user input passive device for interaction with the interactive display device10will be discussed in greater detail with reference to one or more ofFIGS. 5-32.

Additionally, non-interactive objects16(e.g., a coffee mug, books, magazines, a briefcase, an elbow, etc.) may also be placed on the interactive display device10that are not intended to communicate data with the interactive display device10. The interactive display device10is able to recognize objects, distinguish between interactive and non-interactive objects, and adjust the personalized display area18accordingly. For example, if a coffee mug is placed in the center of the personalized display area18, the interactive display device10recognizes the object, recognizes that it is a non-interactive object16and shifts the personalized display over such that the coffee mug is no longer obstructed the user's view of the personalized display area18. Detecting objects on the interactive display device10and adjusting personalized displays accordingly will be discussed in greater detail with reference to one or more ofFIGS. 36-44.

Further, the interactive display device10supports interactions from multiple users having differing orientations around the table top. For example, the interactive display device10is a dining table where each user's presence around the table triggers personalized display areas18with correct orientation (e.g., a sinusoidal signal is generated when a user sits in a chair at the table and the signal is communicated to the interactive display device10, the user is using/wearing a unique device having a particular frequency detected by the interactive display device10, etc.). As another example, the use of a game piece triggers initiation of a game and the correct personalized display areas18are generated in accordance with the game (e.g., detection of an air hockey puck and/or striker segments the display area into a player1display zone and a player2display zone). Generation of personalized display areas18will be discussed in greater detail with reference to one or more ofFIGS. 45-48.

FIG. 2is a schematic block diagram of an embodiment of an interactive display device10that includes a core control module40, one or more processing modules42, one or more main memories44, cache memory46, a video graphics processing module48, a display50, an Input-Output (I/O) peripheral control module52, one or more input interface modules, one or more output interface modules, one or more network interface modules60, and one or more memory interface modules62. A processing module42is described in greater detail at the end of the detailed description of the invention section and, in an alternative embodiment, has a direction connection to the main memory44. In an alternate embodiment, the core control module40and the I/O and/or peripheral control module52are one module, such as a chipset, a quick path interconnect (QPI), and/or an ultra-path interconnect (UPI).

Each of the main memories44includes one or more Random Access Memory (RAM) integrated circuits, or chips. For example, a main memory44includes four DDR4 (4thgeneration of double data rate) RAM chips, each running at a rate of 2,400 MHz. In general, the main memory44stores data and operational instructions most relevant for the processing module42. For example, the core control module40coordinates the transfer of data and/or operational instructions from the main memory44and the memory64-66. The data and/or operational instructions retrieve from memory64-66are the data and/or operational instructions requested by the processing module or will most likely be needed by the processing module. When the processing module is done with the data and/or operational instructions in main memory, the core control module40coordinates sending updated data to the memory64-66for storage.

The memory64-66includes one or more hard drives, one or more solid state memory chips, and/or one or more other large capacity storage devices that, in comparison to cache memory and main memory devices, is/are relatively inexpensive with respect to cost per amount of data stored. The memory64-66is coupled to the core control module40via the I/O and/or peripheral control module52and via one or more memory interface modules62. In an embodiment, the I/O and/or peripheral control module52includes one or more Peripheral Component Interface (PCI) buses to which peripheral components connect to the core control module40. A memory interface module62includes a software driver and a hardware connector for coupling a memory device to the I/O and/or peripheral control module52. For example, a memory interface62is in accordance with a Serial Advanced Technology Attachment (SATA) port.

The core control module40coordinates data communications between the processing module(s)42and a network, or networks, via the I/O and/or peripheral control module52, the network interface module(s)60, and a network card68or70. A network card68or70includes a wireless communication unit or a wired communication unit. A wireless communication unit includes a wireless local area network (WLAN) communication device, a cellular communication device, a Bluetooth device, and/or a ZigBee communication device. A wired communication unit includes a Gigabit LAN connection, a Firewire connection, and/or a proprietary computer wired connection. A network interface module60includes a software driver and a hardware connector for coupling the network card to the I/O and/or peripheral control module52. For example, the network interface module60is in accordance with one or more versions of IEEE 802.11, cellular telephone protocols, 10/100/1000 Gigabit LAN protocols, etc.

The core control module40coordinates data communications between the processing module(s)42and input device(s) via the input interface module(s) and the I/O and/or peripheral control module52. An input device includes a keypad, a keyboard, control switches, a touchpad, a microphone, a camera, etc. An input interface module includes a software driver and a hardware connector for coupling an input device to the I/O and/or peripheral control module52. In an embodiment, an input interface module is in accordance with one or more Universal Serial Bus (USB) protocols.

The core control module40coordinates data communications between the processing module(s)42and output device(s) via the output interface module(s) and the I/O and/or peripheral control module52. An output device includes a speaker, etc. An output interface module includes a software driver and a hardware connector for coupling an output device to the I/O and/or peripheral control module52. In an embodiment, an output interface module is in accordance with one or more audio codec protocols.

The processing module42communicates directly with a video graphics processing module48to display data on the display50. The display50includes an LED (light emitting diode) display, an LCD (liquid crystal display), and/or other type of display technology. The display has a resolution, an aspect ratio, and other features that affect the quality of the display. The video graphics processing module48receives data from the processing module42, processes the data to produce rendered data in accordance with the characteristics of the display, and provides the rendered data to the display50.

The display50includes the touch screen12(e.g., and personalized display area18), a plurality of drive-sense circuits (DSC), and a touch screen processing module82. The touch screen12includes a plurality of sensors (e.g., electrodes, capacitor sensing cells, capacitor sensors, inductive sensor, etc.) to detect a proximal touch of the screen. For example, when a finger or pen touches the screen, capacitance of sensors proximal to the touch(es) are affected (e.g., impedance changes). The drive-sense circuits (DSC) coupled to the affected sensors detect the change and provide a representation of the change to the touch screen processing module82, which may be a separate processing module or integrated into the processing module42.

The touch screen processing module82processes the representative signals from the drive-sense circuits (DSC) to determine the location of the touch(es). This information is inputted to the processing module42for processing as an input. For example, a touch represents a selection of a button on screen, a scroll function, a zoom in-out function, etc.

FIG. 3is a schematic block diagram of another embodiment of an interactive display device10that includes the touch screen12, the drive-sense circuits (DSC), the touch screen processing module81, a display83, electrodes85, the processing module42, the video graphics processing module48, and a display interface93. The display83may be a small screen display (e.g., for portable computing devices) or a large screen display (e.g., for fixed computing devices). In general, a large screen display has a resolution equal to or greater than full high-definition (HD), an aspect ratio of a set of aspect ratios, and a screen size equal to or greater than thirty-two inches. The following table lists various combinations of resolution, aspect ratio, and screen size for the display83, but it is not an exhaustive list.

The display83is one of a variety of types of displays that is operable to render frames of data87into visible images. For example, the display is one or more of: a light emitting diode (LED) display, an electroluminescent display (ELD), a plasma display panel (PDP), a liquid crystal display (LCD), an LCD high performance addressing (HPA) display, an LCD thin film transistor (TFT) display, an organic light emitting diode (OLED) display, a digital light processing (DLP) display, a surface conductive electron emitter (SED) display, a field emission display (FED), a laser TV display, a carbon nanotubes display, a quantum dot display, an interferometric modulator display (IMOD), and a digital microshutter display (DMS). The display is active in a full display mode or a multiplexed display mode (i.e., only part of the display is active at a time).

The touch screen12includes integrated electrodes85that provide the sensors the touch sense part of the touch screen display. The electrodes85are distributed throughout the display area or where touch screen functionality is desired. For example, a first group of the electrodes are arranged in rows and a second group of electrodes are arranged in columns.

The electrodes85are comprised of a transparent conductive material and are in-cell or on-cell with respect to layers of the display. For example, a conductive trace is placed in-cell or on-cell of a layer of the touch screen display. The transparent conductive material, which is substantially transparent and has negligible effect on video quality of the display with respect to the human eye. For instance, an electrode is constructed from one or more of: Indium Tin Oxide, Graphene, Carbon Nanotubes, Thin Metal Films, Silver Nanowires Hybrid Materials, Aluminum-doped Zinc Oxide (AZO), Amorphous Indium-Zinc Oxide, Gallium-doped Zinc Oxide (GZO), and poly polystyrene sulfonate (PEDOT).

In an example of operation, the processing module42is executing an operating system application89and one or more user applications91. The user applications91includes, but is not limited to, a video playback application, a spreadsheet application, a word processing application, a computer aided drawing application, a photo display application, an image processing application, a database application, a gaming application, etc. While executing an application91, the processing module generates data for display (e.g., video data, image data, text data, etc.). The processing module42sends the data to the video graphics processing module48, which converts the data into frames of video87.

The video graphics processing module48sends the frames of video87(e.g., frames of a video file, refresh rate for a word processing document, a series of images, etc.) to the display interface93. The display interface93provides the frames of data87to the display83, which renders the frames of data87into visible images.

While the display83is rendering the frames of data87into visible images, the drive-sense circuits (DSC) provide sensor signals to the electrodes85. When the screen is touched by a pen or device, signals on the electrodes85proximal to the touch (i.e., directly or close by) are changed. The DSCs detect the change for effected electrodes and provide the detected change to the touch screen processing module81.

The touch screen processing module81processes the change of the effected electrodes to determine one or more specific locations of touch and provides this information to the processing module42. Processing module42processes the one or more specific locations of touch to determine if an operation of the application is to be altered. For example, the touch is indicative of a pause command, a fast forward command, a reverse command, an increase volume command, a decrease volume command, a stop command, a select command, a delete command, etc.

If the signals received from a device include embedded data, the touch screen processing module81interprets the embedded data and provides the resulting information to the processing module42. If, interactive display device10is not equipped to process embedded data, the device still communicates with the interactive display device10using the change to the signals on the effected electrodes (e.g., increase magnitude, decrease magnitude, phase shift, etc.).

FIGS. 4A-4Bare schematic block diagrams of embodiments of a touch screen electrode pattern that includes rows of electrodes85-rand columns of electrodes85-c. Each row of electrodes85-rand each column of electrodes85-cincludes a plurality of individual conductive cells (e.g., capacitive sense plates) (e.g., light gray squares for rows, dark gray squares for columns) that are electrically coupled together. The size of a cell depends on the desired resolution of touch sensing. For example, a cell size may be 1 millimeter by 1 millimeter to 5 millimeters by 5 millimeters to provide adequate touch sensing for cell phones and tablets. Making the cells smaller improves touch resolution and will typically reduce touch sensor errors (e.g., touching a “w” by an “e” is displayed). While the cells are shown to be square, they may be of any polygonal shape, diamond, or circular shape.

The cells for the rows and columns may be on the same layer or on different layers. InFIG. 4A, the cells for the rows and columns are shown on different layers. InFIG. 4B, the cells for the rows and columns are shown on the same layer. The electric coupling between the cells is done using vias and running traces (e.g., wire traces) on another layer. Note that the cells are on one or more ITO layers of a touch screen, which includes a touch screen display.

FIG. 5is a schematic block diagram of an embodiment of a touch screen system86that includes a user input passive device88in close proximity to a touch screen12(e.g., interactive surface of the interactive display device10).FIG. 5depicts a front, cross sectional view of the user input passive device88(also referred to herein as the passive device88) that includes conductive plates98-1and98-2coupled to an impedance circuit96. The user input passive device88may include a plurality of conductive (i.e., electrically conductive) plates and impedance circuits.

The impedance circuit96and the conductive plates98-1and98-2cause an impedance and/or frequency effect on electrodes85when in close proximity to an interactive surface of the touch screen12(e.g., the passive device88is close to or in direct contact with the touch screen12) that is detectable by the touch screen12. As an alternative, conductive plates98-1and98-2may be a dielectric material. Dielectric materials generally increase mutual capacitance whereas conductive materials typically decrease mutual capacitance. The touch screen is operable to detect either or both effect. The user input passive device88will be discussed in greater detail with reference to one or more ofFIGS. 6-25.

FIGS. 6A-6Bare schematic block diagrams of embodiments of a touch screen system86that include a simplified depiction of the touch screen12as a touch screen electrode pattern that includes rows of electrodes85-rand columns of electrodes85-cand a simplified depiction of the user input passive device88with a transparent housing for ease of viewing the bottom surface.

The row electrodes85-r(light gray squares) and the column electrodes85-c(dark gray squares) of the touch screen12are on different layers (e.g., the rows are layered above the columns). A mutual capacitance is created between a row electrode and a column electrode.

The user input passive device88includes a housing that includes a shell102(e.g., conductive, non-conductive, dielectric, etc.), a non-conductive supporting surface (not shown), a plurality of impedance circuits, and a plurality of conductive plates. The plurality of conductive plates are mounted on the non-conductive supporting surface such that the shell102and the plurality of conductive plates are electrically isolated from each other and able to affect the touch screen12surface. The impedance circuits and the conductive plates that may be arranged in a variety of patterns (e.g., equally spaced, staggered, diagonal, etc.). The size of the conductive plates varies depending on the size of the electrode cells and the desired impedance and/or frequency change to be detected.

One or more of the plurality of impedance circuits and plurality of conductive plates cause an impedance and/or frequency effect when the user input passive device88is in close proximity to an interactive surface of the touch screen12(e.g., the passive device88is resting on or near the touch screen12). The impedance and/or frequency effects detected by the touch screen12are interpreted as device identification, orientation, one or more user functions, one or more user instructions, etc.

InFIG. 6A, the user input passive device88includes impedance circuits Z1-Z3and conductive plates P1-P6. Each of the conductive plates P1-P6are larger than each electrode of the touch screen12in order to affect multiple touch screen electrodes per plate. For example, a conductive plate may be 2-10 times larger than an electrode. In this example, the conductive plates are shown having approximately four times the area of an electrode (e.g., an electrode is approximately 5 millimeters by 5 millimeters and a conductive plate is approximately 10 millimeters by 10 millimeters). With multiple electrodes affected per plate, the impedance and/or frequency effect caused by a particular plate can be better identified by the touch screen12.

InFIG. 6B, the user input passive device88includes impedance circuits Z1-Z6and conductive plates P1-P12. In the example ofFIG. 6B, each conductive plate is approximately the same size as an electrode. Each conductive plate may be the same size as an electrode or smaller than an electrode. While less electrodes are affected per plate than in the example ofFIG. 6A, multiple electrodes are affected (e.g., relative impedance changes and/or direct impedance changes) in a particular pattern recognizable to the touch screen12. The user input passive device88will be discussed in greater detail with reference to one or more ofFIGS. 7A-25.

FIGS. 7A-7Bare cross section schematic block diagrams of examples of capacitance of a touch screen12with no contact with a user input passive device88. The electrode85sare positioned proximal to dielectric layer92, which is between a cover dielectric layer90and the display substrate94. InFIG. 7A, the row electrodes85-r1and85-r2are on a layer above the column electrodes85-c1and85-c2. InFIG. 7B, the row electrodes85-rand the column electrodes85-care on the same layer. Each electrode85has a self-capacitance, which corresponds to a parasitic capacitance created by the electrode with respect to other conductors in the display (e.g., ground, conductive layer(s), and/or one or more other electrodes).

For example, row electrode85-r1has a parasitic capacitance Cp2, column electrode85-c1has a parasitic capacitance Cri, row electrode85-r2has a parasitic capacitance Cp4, and column electrode85-c2has a parasitic capacitance Cp3. Note that each electrode includes a resistance component and, as such, produces a distributed R-C circuit. The longer the electrode, the greater the impedance of the distributed R-C circuit. For simplicity of illustration the distributed R-C circuit of an electrode will be represented as a single parasitic self-capacitance.

As shown, the touch screen12includes a plurality of layers90-94. Each illustrated layer may itself include one or more layers. For example, dielectric layer90includes a surface protective film, a glass protective film, and/or one or more pressure sensitive adhesive (PSA) layers. As another example, the second dielectric layer92includes a glass cover, a polyester (PET) film, a support plate (glass or plastic) to support, or embed, one or more of the electrodes85-c1,85-c2,85-r1, and85-r2(e.g., where the column and row electrodes are on different layers), a base plate (glass, plastic, or PET), an ITO layer, and one or more PSA layers. As yet another example, the display substrate94includes one or more LCD layers, a back-light layer, one or more reflector layers, one or more polarizing layers, and/or one or more PSA layers.

A mutual capacitance (Cm_1and Cm_2) exists between a row electrode and a column electrode. When no touch and/or device is present, the self-capacitances and mutual capacitances of the touch screen12are at a nominal state. Depending on the length, width, and thickness of the electrodes, separation from the electrodes and other conductive surfaces, and dielectric properties of the layers, the self-capacitances and mutual capacitances can range from a few pico-Farads to 10's of nano-Farads.

Touch screen12includes a plurality of drive sense circuits (DSCs). The DSCs are coupled to the electrodes and detect changes for affected electrodes. The DSC functions as described in co-pending patent application entitled, “DRIVE SENSE CIRCUIT WITH DRIVE-SENSE LINE”, having a serial number of Ser. No. 16/113,379, and a filing date of Aug. 27, 2018.

FIG. 8is a schematic block diagram of an example of capacitance of a touch screen system86that includes the touch screen12and a user input passive device88in contact with the touch screen12. In this example, the user input passive device88is in contact (or within a close proximity) with an interactive surface of the touch screen12but there is no human touch on the user input passive device88.

The user input passive device88includes impedance circuit96, conductive plates98-1and98-2, a non-conductive supporting surface100, and a conductive shell102. The conductive shell102and non-conductive supporting surface shell100together form a housing for the user input passive device88. The housing has an outer shape corresponding to at least one of: a computing mouse, a game piece, a cup, a utensil, a plate, and a coaster. The conductive shell102may alternatively be a non-conductive or dielectric shell. When the shell102is non-conductive, a human touch does not provide a path to ground and does not affect both self-capacitance and mutual capacitance of the sensor electrodes85. In that example, only mutual capacitance changes from the conductive plates are detected by touch screen12when the user input passive device88is in close proximity to the touch screen12surface. Because additional functionality exists when the shell is conductive, the shell102is referred to as conductive shell102in the remainder of the examples.

The conductive plates98-1and98-2and the conductive shell102are in contact with the touch screen12's interactive surface. The non-conductive supporting surface100electrically isolates the conductive shell102, the conductive plate98-1, and the conductive plate98-2. The impedance circuit96connects the conductive plate98-1and the conductive plate98-2and has a desired impedance at a desired frequency. The impedance circuit96is discussed with more detail with reference toFIGS. 15A-15F.

The user input passive device88is capacitively coupled to one or more sensor electrodes85proximal to the contact. The sensor electrodes85may be on the same or different layers as discussed with reference toFIGS. 7A-7B. Because the conductive plates98-1and98-2and the conductive shell102are electrically isolated, when a person touches the conductive shell102of the passive device88, the person provides a path to ground such that the conductive shell102affects both the mutual capacitance and the self-capacitance.

When the passive device88is not touched by a person (as shown here), there is no path to ground and the conductive shell102only affects the mutual capacitance. The conductive plates98-1and98-2do not have a path to ground regardless of a touch and thus only affect mutual capacitance when the passive device is touched or untouched. Because the contact area of the conductive plates98-1and98-2is much larger than the conductive shell102, the mutual capacitance change(s) detected is primarily due to the conductive plates98-1and98-2and the effect of the impedance circuit96not the conductive shell102.

As an example, when the user input passive device88is resting on the touch screen12with no human touch, the user input passive device88is capacitively coupled to the touch screen12of the touch screen system86via capacitance Cd1and Cd2(e.g., where Cd1and Cd2are with respect to a row and/or a column electrode). Depending on the area of the conductive plates98-1and98-2, the effect of the impedance circuit96, and the dielectric layers90-92, the capacitance of Cd1or Cd2is in the range of 1 to 2 pico-Farads. The values of Cd1and Cd2affect mutual capacitances Cm_1and Cm_2. For example, Cd1and Cd2may raise or lower the value of Cm_1and Cm_2by approximately 1 pico-Farad. Examples of the mutual capacitance changes caused by the passive device88will be discussed in more detail with reference toFIGS. 16A-25.

In this cross-sectional view, two conductive plates and one impedance circuit are shown. However, the passive device88may include multiple sets of conductive plates where each set is connected by an impedance circuit. The various sets of conductive plates can have different impedance effects on the electrodes of the touch screen which can correspond to different information and/or passive device functions.

Drive-sense circuits (DSC) are operable to detect the changes in mutual capacitance and/or other changes to the electrodes and interpret their meaning. For example, by detecting changes in mutual capacitance and/or by detecting characteristics of the impedance circuit96(e.g., a sweep for resonant frequency of an impedance circuit96), the DSCs of the touch screen12determines the presence, identification (e.g., of a particular user), and/or orientation of the user input passive device88.

FIG. 9is a schematic block diagram of another example of capacitance of a touch screen system86that includes the touch screen12and a user input passive device88in contact with the touch screen12. In this example, the user input passive device88is in contact (or within a close proximity) with the touch screen12and there is a human touch on the conductive shell102of the user input passive device88. When a person touches the conductive shell102of the passive device88, the person provides a path to ground such that the conductive shell102affects both the mutual capacitance and the self-capacitance. Here, parasitic capacitances Cp1, Cp2, Cp3, and Cp4are shown as affected by CHB (the self-capacitance change caused by the human body).

Drive-sense circuits (DSC) are operable to detect the changes in self capacitance and/or other changes to the electrodes and interpret their meaning. For example, by detecting changes in self capacitance along with mutual capacitance changes, the DSCs of the touch screen12determines that the user input passive device88is on the touch screen12and that it is in use by a user. While the user input passive device88continues to be touched (e.g., the self-capacitance change is detected), mutual capacitance changes may indicate different functions. For example, without a touch, a mutual capacitance changes caused by the conductive plates ID the passive device. With a touch, the mutual capacitance change caused by the conductive plates can indicate a selection, an orientation, and/or any user initiated touch screen function.

In an embodiment where the conductive shell102is not conductive, a person touching the passive device does not provide a path to ground and a touch only minimally affects mutual capacitance.

FIG. 10is a schematic block diagram of another example of capacitance of a touch screen system86that includes the touch screen12and a user input passive device88in contact with the touch screen12. In this example, the user input passive device88is in contact (or in close proximity) with the touch screen12and there is a human touch on the conductive shell102of the user input passive device88.

When a person touches the conductive shell102of the passive device88, the person provides a path to ground such that the conductive shell102affects both the mutual capacitance and the self-capacitance. Here, parasitic capacitances Cp1, Cp2, Cp3, and Cp4are shown as affected by CHB (the self-capacitance change caused by the human body).

Further, in this example, the conductive shell includes a switch mechanism (e.g., switch104) on the conductive shell102of the passive device88housing. When a user presses (or otherwise engages/closes) the switch104, the impedance circuit is adjusted (e.g., the impedance circuit Zx is connected to Z1in parallel). Adjusting the impedance circuit causes a change to Cd1and Cd2thus affecting the mutual capacitances Cm_1and Cm_2. The change in impedance can indicate any number of functions such as a selection, a right click, erase, highlight, select, etc.

While one switch is shown here, multiple switches can be included where each impedance caused by an open and closed switch represents a different user function. Further, gestures or motion patterns can be detected via the impedance changes that corresponding to different functions. For example, a switch can be touched twice quickly to indicate a double-click. As another example, the switch can be pressed and held down for a period of time to indicate another function (e.g., a zoom). A pattern of moving from one switch to another can indicate a function such as a scroll.

FIG. 11is a schematic block diagram of another example of capacitance of a touch screen system86that includes the touch screen12and a user input passive device75in contact with the touch screen12. The user input passive device75includes conductive plates98-1and98-2, and a non-conductive layer77. The non-conductive layer77electrically isolates conductive plates98-1and98-2from each other.

In this example, the user input passive device75is in contact (or within a close proximity) with the touch screen12and there is a human touch directly on the conductive plate98-1of the user input passive device75. When a person touches a conductive plate of the passive device75, the person provides a path to ground such that the conductive plates affect both the mutual capacitance and the self-capacitance of the sensor electrodes85. With conductive plates98-1and98-2capacitively coupled (e.g., Cd1and Cd2) to sensor electrodes85, mutual capacitances Cm_1and Cm_2are affected and parasitic capacitances Cp1, Cp2, Cp3, and Cp4are affected by CHB (the self-capacitance change caused by the human body).

Drive-sense circuits (DSC) are operable to detect the changes in self and mutual capacitance and/or other changes to the electrodes and interpret their meaning. For example, by detecting changes in self capacitance along with mutual capacitance changes, the DSCs of the touch screen12determines that the user input passive device75is on the touch screen12and that it is in use by a user. While the user input passive device75continues to be touched (e.g., the self-capacitance change is detected), mutual capacitance changes may indicate different functions. For example, without a touch, a mutual capacitance changes caused by the conductive plates ID the passive device. With a touch, the mutual capacitance change caused by the conductive plates can indicate a selection, an orientation, and/or any user initiated touch screen function.

While two conductive plates are shown here, the user input passive device75may include one or more conductive plates, where touches to the one or more conductive plates can indicate a plurality of functions. For example, a touch to both conductive plates98-1and98-2may indicate a selection, a touch to conductive plate98-1may indicate a right click, touching conductive plates in a particular pattern and/or sequence may indicate a scroll, etc. The user input passive device75may further include a scroll wheel in contact with one or more conductive plates, conductive pads on one or more surfaces of the device, conductive zones for indicating various functions, etc. As such, any number of user functions including traditional functions of a mouse and/or trackpad can be achieved passively.

FIG. 12is a cross section schematic block diagram of an example of capacitance of a touch screen12with no contact with a user input passive device88.FIG. 12is similar to the example ofFIG. 7Bexcept one row electrode85-rand one column electrode85-cof the touch screen12are shown on the same layer. The electrode85sare positioned proximal to dielectric layer92, which is between a cover dielectric layer90and the display substrate94.

Each electrode85has a self-capacitance, which corresponds to a parasitic capacitance created by the electrode with respect to other conductors in the display (e.g., ground, conductive layer(s), and/or one or more other electrodes).

As shown, the touch screen12includes a plurality of layers90-94. Each illustrated layer may itself include one or more layers. For example, dielectric layer90includes a surface protective film, a glass protective film, and/or one or more pressure sensitive adhesive (PSA) layers. As another example, the second dielectric layer92includes a glass cover, a polyester (PET) film, a support plate (glass or plastic) to support, or embed, one or more of the electrodes85-cand85-r(e.g., where the column and row electrodes are on different layers), a base plate (glass, plastic, or PET), an ITO layer, and one or more PSA layers. As yet another example, the display substrate94includes one or more LCD layers, a back-light layer, one or more reflector layers, one or more polarizing layers, and/or one or more PSA layers.

A mutual capacitance (Cm_0) exists between a row electrode and a column electrode. When no touch and/or device is present, the self-capacitances and mutual capacitances of the touch screen12are at a nominal state. Depending on the length, width, and thickness of the electrodes, separation from the electrodes and other conductive surfaces, and dielectric properties of the layers, the self-capacitances and mutual capacitances can range from a few pico-Farads to 10's of nano-Farads.

Touch screen12includes a plurality of drive sense circuits (DSCs). The DSCs are coupled to the electrodes and detect changes for affected electrodes.

FIGS. 13A-13Bare schematic block diagrams of examples of capacitance of a touch screen system86that includes the touch screen12and a user input passive device88in contact with the touch screen12. In this example, the user input passive device88is in contact (or within a close proximity) with an interactive surface of the touch screen12but there is no human touch on the user input passive device88.FIGS. 13A-13Boperate similarly to the example ofFIG. 8except that only one row electrode85-rand one column electrodes85-care shown on a same layer of the touch screen12.

As shown inFIG. 13A, the user input passive device88includes impedance circuit96(Z1), conductive plates98-1and98-2(P1and P2), a non-conductive supporting surface100, and a conductive shell102. The conductive shell102and non-conductive supporting surface shell100together form a housing for the user input passive device88. The housing has an outer shape corresponding to at least one of: a computing mouse, a game piece, a cup, a utensil, a plate, and a coaster.

The conductive plates98-1and98-2and the conductive shell102are in contact with the touch screen12's interactive surface. The non-conductive supporting surface100electrically isolates the conductive shell102, the conductive plate98-1, and the conductive plate98-2. The impedance circuit96connects the conductive plate98-1and the conductive plate98-2and has a desired impedance at a desired frequency. The impedance circuit96is discussed with more detail with reference toFIGS. 15A-15F.

The user input passive device88is capacitively coupled to one or more rows and/or column electrodes proximal to the contact. Because the conductive plates98-1and98-2and the conductive shell102are electrically isolated, when a person touches the conductive shell102of the passive device88, the person provides a path to ground such that the conductive shell102affects both the mutual capacitance and the self-capacitance.

When the passive device88is not touched by a person (as shown here), there is no path to ground and the conductive shell102only affects the mutual capacitance. The conductive plates98-1and98-2do not have a path to ground regardless of a touch and thus only affect mutual capacitance when the passive device is touched or untouched. Because the contact area of the conductive plates98-1and98-2is much larger than the conductive shell102, the mutual capacitance change detected is primarily due to the conductive plates98-1and98-2and the effect of the impedance circuit96not the conductive shell102.

As an example, when the user input passive device88is resting on the touch screen12with no human touch, the user input passive device88is capacitively coupled to the touch screen12of the touch screen system86via capacitance Cd1and Cd2(e.g., where Cd1and Cd2are with respect to a row and/or a column electrode). Depending on the area of the conductive plates98-1and98-2, the effect of the impedance circuit96, and the dielectric layers90-92, the capacitance of Cd1or Cd2is in the range of 1 to 2 pico-Farads. The values of Cd1and Cd2affect mutual capacitance Cm_0(created between the column and row electrode on the same layer). For example, Cd1and Cd2may raise or lower the value of Cm_0by approximately 1 pico-Farad.

In this cross-sectional view, two conductive plates and one impedance circuit are shown. However, the passive device88may include multiple sets of conductive plates where each set is connected by an impedance circuit. The various sets of conductive plates can have different impedance effects on the electrodes of the touch screen which can correspond to different information and/or passive device functions.

Drive-sense circuits (DSCs1-2) are operable to detect the changes in mutual capacitance and/or other changes to the electrodes and interpret their meaning. One DSC per row and one DSC per column are affected in this example. For example, by detecting changes in mutual capacitance and/or by detecting characteristics of the impedance circuit96(e.g., a sweep for resonant frequency of an impedance circuit96), the DSCs of the touch screen12determines the presence, identification (e.g., of a particular user), and/or orientation of the user input passive device88.

FIG. 13Bshows a simplified circuit diagram representation ofFIG. 13A. The capacitances Cd1and Cd2of the user input passive device88are coupled to the touch screen12such that the mutual capacitance Cm_0between column and row electrodes85is affected. However, with no human touch, there is no path to ground. Therefore, the collective parasitic capacitances Cp2and Cp1remain substantially unchanged. DSC1may detect changes to one row and DSC2may detect changes to one column. Thus, DSC1and DSC2are operable to sense a mutual capacitance change to Cm_0.

FIGS. 14A-14Bare schematic block diagrams of another example of capacitance of a touch screen system86that includes the touch screen12and a user input passive device88in contact with the touch screen12. In this example, the user input passive device88is in contact (or within a close proximity) with the touch screen12and there is a human touch on the conductive shell102of the user input passive device88.FIGS. 14A and 14Boperate similarly toFIG. 9except electrodes85-rand85-care shown on the same layer of the touch screen12.

When a person touches the conductive shell102of the passive device88, the person provides a path to ground such that the conductive shell102affects both the mutual capacitance and the self-capacitance. Here, parasitic capacitances Cp1and Cp2are shown as affected by CHB (the self-capacitance change caused by the human body).

Drive-sense circuits (DSCs1-2) are operable to detect the changes in self capacitance and/or other changes to the electrodes and interpret their meaning. For example, by detecting changes in self capacitance along with mutual capacitance changes, the DSCs of the touch screen12determines that the user input passive device88is on the touch screen12and that it is in use by a user. While the user input passive device88continues to be touched (e.g., the self-capacitance change is detected), mutual capacitance changes may indicate different functions. For example, without a touch, a mutual capacitance change IDs the passive device. With a touch, the mutual capacitance change can indicate a selection, an orientation, and/or any user initiated touch screen function.

FIG. 14Bshows a simplified circuit diagram representation ofFIG. 14A. The capacitances Cd1and Cd2of the user input passive device88are coupled to the touch screen12such that the mutual capacitance Cm_0between column and row electrodes85is affected. With a human touch there is path to ground. Therefore, the collective parasitic capacitances Cp2and Cp1are affected by CHB (the self-capacitance change caused by the human body). DSC1may detect changes to one row and DSC2may detect changes to one column. Thus, DSC1and DSC2are operable to sense a mutual capacitance change to Cm_0as well as the effect of CHB on Cp2and Cp1.

FIGS. 15A-15Fare schematic block diagrams of examples of the impedance circuit96. InFIG. 15Athe impedance circuit96is a parallel tank (LC) circuit (e.g., an inductor and a capacitor connected in parallel). In resonance, (i.e., operating at resonant frequency) a parallel tank circuit experiences high impedance and behaves like an open circuit allowing minimal current flow.

InFIG. 15B, the impedance circuit96is a series tank (LC) circuit (e.g., an inductor and a capacitor connected in series). In resonance, a series tank circuit experiences low impedance and behaves like a short circuit allowing maximum current flow.

FIGS. 16A-16Bare schematic block diagrams of examples of mutual capacitance changes to electrodes85with a parallel tank circuit as the impedance circuit96. The parallel tank circuit96includes an inductor and a capacitor connected in parallel. The user input passive device is capacitively coupled to the touch screen12of the touch screen system86via capacitance Cd1and Cd2. In this example, row and column electrodes are on different layers and the capacitance of each of Cd1is Cd2is 2 pico-Farads. The values of Cd1and Cd2affect mutual capacitances Cm_1and Cm_2. Without any contact, the capacitance of each of Cm_1and Cm_2are 2 pico-Farad in this example.

As shown inFIG. 16A, when the parallel tank circuit96is out of resonance (i.e., operating at any frequency besides resonant frequency), the parallel tank circuit96has low impedance allowing current to flow. Thus, out of resonance, Cm_1is connected in parallel to a series combination of Cd1and Cd2and Cm_2is connected in parallel to a series combination of Cd1and Cd2. Therefore, out of resonance, Cm_1and Cm_2go from 2 pico-Farads to 3 pico-Farads.

FIGS. 17A-17Bare schematic block diagrams of examples of mutual capacitance changes to electrodes85with a series tank circuit as the impedance circuit96. The series tank circuit96includes an inductor and a capacitor connected in series. The user input passive device is capacitively coupled to the touch screen12of the touch screen system86via capacitance Cd1and Cd2. In this example, row and column electrodes are on different layers and the capacitance of each of Cd1is Cd2is 2 pico-Farads. The values of Cd1and Cd2affect mutual capacitances Cm_1and Cm_2. Without any contact, the capacitance of each of Cm_1and Cm_2are 2 pico-Farad in this example.

As shown inFIG. 17A, when the series tank circuit96is out of resonance (i.e., operating at any frequency besides resonant frequency) the series tank circuit96has high impedance restricting current flow. Thus, out of resonance, Cm_1and Cm_2experience minimal change from Cd1and Cd2. Therefore, out of resonance, Cm_1and Cm_2stay at 2 pico-Farads.

As shown inFIG. 17B, when the series tank circuit96is in resonance (i.e., operating at resonant frequency), the series tank circuit96has low impedance allowing current to flow. Thus, Cm_1is connected in parallel to a series combination of Cd1and Cd2and Cm_2is connected in parallel to a series combination of Cd1and Cd2. Therefore, in resonance, Cm_1and Cm_2go from 2 pico-Farads to 3 pico-Farads.

FIGS. 18A-18Bare examples of detecting mutual capacitance change.FIG. 18Adepicts a graph of frequency versus mutual capacitances Cm_1and Cm_2from the example ofFIGS. 16A-16Bwhere the impedance circuit is a parallel tank circuit. In this example, the touch screen12does a frequency sweep. At all frequencies besides the resonant frequency of the parallel tank circuit, Cm_1and Cm_2will be 3 pico-Farads when the passive device is in contact. At the resonant frequency (e.g., 1 MHz), a shift from 3 pico-Farads to 2 pico-Farads can be detected.

FIG. 18Bdepicts a graph of frequency versus mutual capacitances Cm_1and Cm_2from the example ofFIGS. 17A-17Bwhere the impedance circuit is a series tank circuit. In this example, the touch screen12does a frequency sweep. At all frequencies besides the resonant frequency of the series tank circuit, Cm_1and Cm_2will be 2 pico-Farads when the passive device is in contact. At the resonant frequency (e.g., 1 MHz), a shift from 2 pico-Farads to 3 pico-Farads can be detected.

FIGS. 19A-19Bare examples of detecting capacitance change.FIG. 19Adepicts a graph of frequency versus capacitance with a channel spacing of 100 KHz. In this example, the passive device is in contact with the touch screen and is also being touched by a user. Using a frequency sweep, the self-capacitance change from the user touching the conductive shell is detectable at 100 Khz in this example. In accordance with the tank circuit impedance circuit examples discussed previously, the mutual capacitance change from the impedance circuit and conductive plates is detectable at a resonant frequency of the tank circuit (e.g., 1 MHz). Therefore, when the frequency of detectable impedance changes is known, the touch screen is able to sweep those frequencies to determine the presence and various functions of the passive device.

FIG. 19Bdepicts a graph of frequency versus capacitance with a channel spacing of 100 KHz. In this example, the passive device is in contact with the touch screen and is also being touched by a user. Further, the passive device includes a switching mechanism which affects the impedance of the impedance circuit. For example, the resonant frequency of the impedance circuit when the switch mechanism is closed increases. Using a frequency sweep, the self-capacitance change from the user touching the conductive shell is detectable at 100 Khz.

In accordance with the tank circuit impedance circuit examples discussed previously, the mutual capacitance change from the impedance circuit and conductive plates when the switch is open is detectable at a first resonant frequency (e.g., 1 MHz). The mutual-capacitance change from the impedance circuit and conductive plates when the switch is closed is detectable at a second resonant frequency (e.g., 2 MHz). As such, detecting the self-capacitance change from the user touching the device as well as detecting the second frequency (2 MHz) indicates a particular user function (e.g., select, zoom, highlight, erase, scroll, etc.).

A drive sense circuit of the touch screen is operable to transmit a self and a mutual frequency per channel for sensing but also has the ability to transmit multiple other frequencies per channel. As an additional example of performing a frequency sweep, one or more frequencies in addition to the standard self and mutual frequency can be transmitted per channel. The one or more additional frequencies change every refresh cycle and can aid in detecting devices/objects and/or user functions. For example, a set of known frequencies can be transmitted every refresh cycle and detected frequency responses can indicate various functions. For example, an object responds to a particular frequency and the touch screen interprets the object as an eraser for interaction with the touch screen.

FIG. 20is a schematic block diagram of an embodiment of a touch screen system86that includes a user input passive device88in contact with a touch screen12.FIG. 20is similar to the example ofFIG. 6Abut only the conductive plates (P1-P6) and impedance circuits (Z1-Z3) of the user input passive device88are shown.FIG. 20shows a simplified depiction of the touch screen12as a touch screen electrode pattern that includes rows of electrodes85-rand columns of electrodes85-c. Here, the conductive cells for the rows (light gray squares) and columns (dark gray squares) are on different layers (e.g., the rows are layered above the columns). Alternatively, the rows and columns may be on the same layer. A mutual capacitance is created between a row electrode and a column electrode. An electrode cell may be 1 millimeter by 1 millimeter to 5 millimeters by 5 millimeters depending on resolution.

The conductive plates P1-P6are shown as approximately four times the area of an electrode cell in this example (e.g., an electrode cell is 5 millimeters by 5 millimeters and a conductive plate is 10 millimeters by 10 millimeters) to affect multiple electrodes per plate. The size of the conductive plates can vary depending on the size of the electrode cells and the desired impedance change to be detected. For example, the conductive plate may be substantially the same size as an electrode cell.

One or more of the plurality of impedance circuits and plurality of conductive plates cause an impedance and/or frequency effect when in close proximity to an interactive surface of the touch screen12(e.g., the passive device88is resting on the touch screen12) that is detectable by the touch screen12. As shown here, the conductive plates of user input passive device88are aligned over the conductive cells of the touch screen12such that the mutual capacitances of four row and column electrodes are fully affected per conductive plate.

FIG. 21is a schematic block diagram of an example of a mutual capacitance change gradient110caused by the user input passive device88on the touch screen12in accordance with the example described with reference toFIG. 20(e.g., the conductive plates align with conductive cells of the touch screen12). For simplicity, only the conductive cells for the row electrodes (light gray squares) are shown. The mutual capacitance effect is created between a row electrode and a column electrode.

When the conductive plates of the user input passive device88align with conductive cells of the touch screen12in the most ideal situation, the mutual capacitance of four row and column electrodes are affected per conductive plate. Each mutual capacitance change108in the area of the user input passive device creates a mutual capacitance change gradient110that is detectable by the touch screen12.

Capacitance change detection, whether mutual, self, or both, is dependent on the channel width of the touch screen sensor, the thickness of the cover glass, and other touch screen sensor properties. For example, a higher resolution channel width spacing allows for more sensitive capacitive change detection.

FIG. 22is a schematic block diagram of another example of a mutual capacitance change gradient110caused by the user input passive device88on touch screen12in accordance with the example described with reference toFIG. 20(e.g., the conductive plates align with conductive cells of the touch screen12). For simplicity, only the conductive cells for the row electrodes (light gray squares) are shown. The mutual capacitance effect is created between a row electrode and a column electrode.

When the conductive plates of the user input passive device88align with conductive cells of the touch screen12in the most ideal situation, the mutual capacitance between four row column electrodes are affected per conductive plate. Each mutual capacitance change108in the area of the user input passive device creates a mutual capacitance change gradient110that is detectable across the touch screen12.

In this example, the two lower plates of the user input passive device create a different mutual capacitance change than the other four conductive plates. For example, impedance circuits Z1and Z2(seeFIG. 20for reference) are series tank circuit causing the mutual capacitance of the electrodes to raise during a resonant frequency sweep. The impedance circuit Z3may be a parallel tank circuit with the same resonant frequency as the series tank circuit such that the mutual capacitance of the electrodes lowers during the resonant frequency sweep. The difference in mutual capacitance changes108across the mutual capacitance change gradient110can indicate orientation of the user input passive device.

FIG. 23is a schematic block diagram of an embodiment of a touch screen system86that includes a user input passive device88in contact with a touch screen12.FIG. 23is similar toFIG. 20except here the conductive plates of the user input passive device88are not aligned over the electrode cells of the touch screen12. For example, one conductive plate of the passive device88fully covers one electrode cell and only portions of the eight surrounding electrode cells.

FIG. 24is a schematic block diagram of another example of a mutual capacitance change gradient110caused by the user input passive device88on touch screen12in accordance with the example described with reference toFIG. 23(e.g., the conductive plates do not align with electrode cells of the touch screen12).

With one conductive plate of the user input passive device88fully covering only one conductive cell, the greatest mutual capacitance change112is detected from the fully covered electrodes (e.g., shown by the dark gray squares and the largest white arrows). Each conductive plate also covers portions of eight surrounding electrode cells creating areas of lesser mutual capacitance changes (e.g., shown by the lighter shades of grays and the smaller white arrows).

Thus, the touch screen12is operable to detect the user input passive device88from a range of mutual capacitance change gradients110(i.e., mutual capacitance change patterns) from a fully aligned gradient (as illustrated inFIGS. 21 and 22) to a partially aligned gradient.

The touch screen12is operable to recognize mutual capacitance change patterns as well as detect an aggregate mutual capacitance change within the mutual capacitance change gradients110. For example, the touch screen12can recognize a range of aggregate mutual capacitance changes within a certain area that identify the user input passive device (e.g., aggregate mutual capacitance changes of 12 pF-24 pF in a 30 millimeter by 30 millimeter area are representative of the user input passive device).

FIG. 25is a schematic block diagram of an example of determining relative impedance that includes user input passive device88in contact with touch screen12. For simplicity, the touch screen12is shown as touch screen electrode pattern that includes rows of electrodes85-rand columns of electrodes85-c. Here, the conductive cells for the rows (white squares) and columns (dark gray squares) are on same layer but may be on different layers as discussed previously.

As the user input passive device88contacts the touch screen12surface, impedance circuits Z1-Z3and corresponding conductive plates P1-P6cause mutual capacitance changes to the touch screen12. Detecting exact mutual capacitance changes in order to identify the user input passive device88and user input passive device88functions can be challenging due to small capacitance changes and other capacitances of the touch screen potentially altering the measurements. Therefore, in this example, a relative impedance effect is detected so that exact impedance measurements are not needed.

For example, the relationship between the impedance effects of Z1, Z2, and Z3(and corresponding conductive plates) are known and constant. The impedance effects of Z1, Z2, and Z3are individually determined, and based on the relationship between those effects, the user input passive device88can be identified (e.g., as being present and/or to identify user functions). For example, Z1/Z2, Z2/Z3, and Z1/Z3are calculated to determine a first constant value, a second constant value, and a third constant value respectively. The combination of the first constant value, the second constant value, and the third constant value is recognized as an impedance pattern associated with the user input passive device88. The methods for detecting the user input passive device and interpreting user input passive device functions described above can be used singularly or in combination.

FIG. 26is a schematic block diagram of an example of capacitance of a touch screen12in contact with a user input passive device95. In this example, the user input passive device95includes a conductive material. The user input passive device95may include a conductive shell with a hollow center, a solid conductive material, a combination of conductive and non-conductive materials, etc. The user input passive device95may include a spherical, half-spherical, and/or other rounded shape for user interaction with the touch screen12. Examples of the user input passive device95will be discussed further with reference toFIGS. 27-31.

The user input passive device95is capacitively coupled to one or more rows and/or column electrodes proximal to the contact (e.g., Cd1and Cd2). A zoomed in view is shown here to illustrate contact between the user input passive device95and two electrodes of the touch screen12, however, many more electrodes are affected when the user input passive device95is in contact (or within a close proximity) with the touch screen12because the user input passive device95is much larger in comparison to an electrode. In this example, there is a human touch (e.g., via a palm and/or finger97) on the conductive material of the user input passive device95.

When a person touches the conductive material of the passive device95, the person provides a path to ground such that the conductive material affects both the mutual capacitance (Cm_0) and the self-capacitance. Here, parasitic capacitances Cp1and Cp2are shown as affected by CHB (the self-capacitance change caused by the human body).

Drive-sense circuits (DSC) are operable to detect the changes in self capacitance and/or other changes to the electrodes and interpret their meaning. For example, as a person moves the user input passive device95, the DSCs of the touch screen12interpret changes in electrical characteristics of the affected electrodes as a direction of movement. The direction of movement can then be interpreted as a specific user input function (e.g., select, scroll, gaming movements/functions, etc.).

FIG. 27is a schematic block diagram of an embodiment of the user input passive device95interacting with the touch screen12. In this example, the user input passive device95in a half spherical shape with a flat top surface. The user input passive device95is made of a rigid conductive material such that the user input passive device95retains its shape when applied pressure. A user may rest a palm and/or a finger on the flat top surface to maneuver the spherical shape in various directions in one location and/or across the touch screen12surface.

As shown on the left, the user input passive device95is used in an upright position and is affecting a plurality of electrodes on the touch screen12surface. On the right, the user input passive device95is tilted, thus, shifting the location of the plurality of affected electrodes. The amount of electrodes affected, the location of affected electrodes, the rate of the change in the location of affected electrodes, etc., can be interpreted as various user functions by the touch screen12. For example, the user input passive device95can be utilized as a joystick in a gaming application.

FIG. 27Ais a schematic block diagram of another embodiment of the user input passive device95interacting with the touch screen12. In this example, the user input passive device95in a half spherical shape with a flat top surface. In comparison toFIG. 27, the half spherical shape shown here is shorter and smaller such that the flat top surface (e.g., the touch plate) is extends beyond the half spherical shape. The user input passive device95is made of a rigid conductive material such that the user input passive device95retains its shape when applied pressure. A user may rest a palm and/or a finger on the flat top surface to maneuver the spherical shape in various directions in one location and/or across the touch screen12surface.

As shown on the top ofFIG. 27A, the user input passive device95is used in an upright position and is affecting a plurality of electrodes on the touch screen12surface. On the bottom, the user input passive device95is tilted, thus, shifting the location of the plurality of affected electrodes and affecting additional electrodes with the flat top surface.

The flat top surface of the user input passive device95is a conductive material. As the user input passive device95is tilted, the flat top surface affects electrodes of the touch screen12with an increasing affect (e.g., a change in capacitance increases as the flat top surface gets closer) as it approaches the surface of the touch screen12. As such, an angle/tilt of the device can be interpreted by this information. Further, the flat top surface in close proximity to the touch screen12(e.g., a touch) can indicate any one of a variety of user functions by the touch screen (e.g., a selection, etc.).

FIG. 28is a schematic block diagram of another embodiment of the user input passive device95interacting with the touch screen12. In this example, the user has a palm and/or a finger on the user input passive device95but also has two fingers directly on the touch screen12surface. For example, the user has a palm and three fingers resting on the top surface of the user input passive device95and a thumb and pinky on either side of the user input passive device95directly on the touch screen12. When interaction with the user input passive device95is detected (e.g., by detection of a region of affected electrodes, by the type of affected electrodes (e.g., a certain self-capacitance change is detected over a certain area, etc.) etc.), the detection of a finger touch nearby can indicate further user functions.

For example, the user input passive device95is directly over a list of files and a finger can be used on the touch screen to initiate a scrolling function. As another example, the user input passive device95is directly over an image and placing one or two fingers on the screen initiates a zooming function.

FIG. 29is a schematic block diagram of another embodiment of the user input passive device95interacting with the touch screen12. In this example, the user input passive device95includes a flexible conductive material such that when a touch and/or pressure is applied, the user input passive device95changes shape. For example, when pressure is applied in the center of the top of the user input passive device95the area in contact with the touch screen12increases thus affecting more electrodes. As such, applying pressure can indicate any number of user input functions (e.g., select, zoom, etc.).

FIG. 30is a schematic block diagram of another embodiment of the user input passive device95interacting with the touch screen12.FIG. 30is similar to the example ofFIG. 29where the user input passive device95includes a flexible conductive material such that when a touch and/or pressure is applied, the user input passive device95changes shape.

In this example, pressure is applied off center on the top of the user input passive device95. The pressure increases and shifts the area in contact with the touch screen12thus affecting more electrodes in a different location. Therefore, the shift in location as well as an increased number of affected electrodes can indicate any number of user input functions. For example, the user input passive device95can be tilted forward to indicate a movement and pressure can be applied to indicate a selection.

FIGS. 31A-31Gare schematic block diagrams of examples of the user input passive device95. InFIG. 31A, the user input passive device95is a half-spherical shape with a flat top surface that includes a plurality of protruding bumps or dimples for interaction with the touch screen. The entire surface may be conductive, the dimples may be conductive, and/or some combination thereof may be conductive. The pattern and size of the dimples can aid the touch screen12in detecting the user input passive device95and interpreting user input functions.

InFIG. 31B, the user input passive device95is a smooth, half-spherical shape with a flat top surface that includes a top handle for ease of use by the user. The top shape of the user input passive device95can correspond to a game piece (e.g., an air hockey striker) or resemble a gaming joy stick to allow for intuitive and easy use for a variety of applications and functions.

InFIG. 31C, the user input passive device95is a spherical shape that includes a plurality of protruding bumps or dimples for interaction with the touch screen. The entire surface may be conductive, the dimples may be conductive, and/or some combination thereof may be conductive. The pattern and size of the dimples can add the touch screen12in detecting the user input passive device95and interpreting user input functions. With a full sphere, the user can roll the user input passive device95across the touch screen with a palm.

InFIG. 31D, the user input passive device95is a smooth spherical shape. InFIG. 31E, the user input passive device95a smooth, half-spherical shape with a flat top surface that has a conductive outer shell and a hollow center.

InFIG. 31F, the user input passive device95is a smooth, half-spherical shape with a flat top surface that includes non-conductive material and conductive wires in a radial pattern. InFIG. 31G, the user input passive device95is a smooth, half-spherical shape with a flat top surface that includes non-conductive material and conductive wires in a circular pattern. The examples, ofFIGS. 31F and 31Gare similar toFIGS. 31A and 31Cin that the conductive wires interact with the touch screen12in a unique way and/or pattern. The unique pattern enhances user input passive device95detection and user function recognition.

Any of the examples described inFIGS. 31A-31Gmay include rigid or flexible conductive material as discussed previously.

FIG. 32is a logic diagram of an example of a method for interpreting user input from the user input passive device. The user input passive device may include a conductive shell with a hollow center, a solid conductive material, a combination of conductive and non-conductive materials, etc. The user input passive device may include a spherical, half-spherical, and/or other rounded shape for user interaction with the touch screen. Examples of the user input passive device95will be discussed further with reference toFIGS. 27-31.

The method begins with step117where a plurality of drive sense circuits (DSCs) of an interactive display device transmit a plurality of signals on a plurality of electrodes of the interactive display device. The interactive display device includes the touch screen, which may further include a personalized display area to form an interactive touch screen.

The method continues with step119where a set of DSCs of the plurality of DSCs detect a change in an electrical characteristic of a set of electrodes of the plurality of electrodes. For example, the self and mutual capacitance of an electrode is affected when a user input passive device is capacitively coupled to the interactive display device.

The method continues with step121where a processing module of the interactive display device interprets the change in electrical characteristic to be a direction of movement caused by a user input passive device in close proximity to an interactive surface of the interactive display device. For example, the change in electrical characteristic is an increase or decrease in self and/or mutual capacitance by a certain amount to a certain number of electrodes that is indicative of movement by the user input passive device.

The method continues with step123where the processing module of the interactive display device interprets the direction of movement as a specific user input function. For example, a direction of movement may indicate a movement (e.g., in a game, with a cursor, etc.), a selection, a scroll, etc.

FIG. 33is a schematic block diagram of another embodiment of the interactive display device10(e.g., shown here as an interactive table top) that includes the touch screen12, which may further include a personalized display area18to form an interactive touch screen display (also referred to herein as interactive surface115). The personalized display area18may extend to all of the touch screen12or a portion as shown. When the user input passive device88is in contact with the interactive surface, a digital pad114is generated for use with the user input passive device88.

The interactive display device10is operable to interpret user inputs received from the user input passive device88within the digital pad114area as functions to manipulate data on the personalized display area18of the interactive display device10. For example, moving the user input passive device88within the digital pad114maps to movements on the personalized display area18so that the user can execute various functions within the personalized display area18without having to move the user input passive device88onto the personalized display area18. This is particularly useful when the personalized display area18is large, and the user cannot easily access the entire personalized display area.

The digital pad114is operable to move with the user input passive device88and is of a predetermined size and shape, a user defined size and shape, and/or a size and shape based on the size and shape of the user input passive device88. Further, the size of the digital pad114may be determined and dynamically adjusted based on available space of the interactive display device10(e.g., where available space is determined based on one or more personalized display areas, detected objects, etc.). Moving the digital pad114onto the personalized display area18can cause the personalized display area18to adjust so that the digital pad114is not obstructing the personalized display area18. Alternatively, moving the digital pad114onto the personalized display area18may disable the digital pad114when the user intends to use the user input passive device88directly on the personalized display area18. A more detailed discussion of adjusting a personalized display area based on an obstructing object is discussed with reference to one or more ofFIGS. 36-44.

When the user input passive device88is in contact with the interactive surface, a virtual keyboard116may also be generated for use by the user. The virtual keyboard116is displayed in an area of the touchscreen in accordance with the user input passive device88's position. For example, the virtual keyboard116is displayed within a few inches of where the user input passive device88is located. User information (e.g., location at the table, right handed or left, etc.) available from the user input passive device and/or user input aids in the display of the virtual keyboard116. For example, a user identifier (ID) (e.g., based on a particular impedance pattern) associated with the user input passive device88indicates that the user is right-handed. Therefore, the virtual keyboard116is displayed to the left of the user input passive device88.

As such, use of the user input passive device88triggers the generation of one or more of the digital pad114and the virtual keyboard116. Alternatively, a user input triggers the generation of one or more of the digital pad114and the virtual keyboard116. For example, the user hand draws an area (e.g., or inputs a command or selection to indicate generation of the digital pad114and/or the virtual keyboard116is desired) on the touchscreen to be used as one or more of the digital pad114and the virtual keyboard116. When the digital pad114area is triggered without the user input passive device, the user can optionally use a finger and/or other capacitive device for inputting commands within the digital pad114. As with the user input passive device88, the interactive display device10is operable to interpret user inputs received within the digital pad114area as functions to manipulate data on the personalized display area18of the interactive display device10.

As another example, a keyboard has a physical structure (e.g., a molded silicon membrane, a transparent board, etc.). The interactive display device can recognize the physical structure as a keyboard using a variety of techniques (e.g., a frequency sweep, capacitance changes, a tag, etc.) and also know its orientation (e.g., via passive device recognition techniques discussed previously). When the physical keyboard is recognized, the touch screen may display the virtual keyboard underneath the transparent structure for use by the user.

The physical keyboard includes conductive elements (e.g., conductive paint, a full conductive mechanical key structure, etc.) such that interaction with the conductive element by the user is interpreted as a keyboard function. For example, the keyboard is a molded silicon membrane with conductive paint on each key. The user physically presses down on a key such that the conductive paint contacts the touch screen. Each key may have a different conductive paint pattern such that the touch screen interprets each pattern as a different function (i.e., key selection, device ID, etc.).

The touch screen of the interactive display device10may further include a high resolution section for biometric input (e.g., a finger print) from a user. The biometric input can unlock one or more functions of the interactive display device10. For example, inputting a finger print to the high resolution section may automatically display one or more of a digital pad114, virtual keyboard116, and the personalized display area in accordance with that user's preferences.

FIGS. 34A-34Bare schematic block diagrams of examples of digital pad114generation on an interactive surface115of the interactive display device. Interactive surface115includes touch screen12and personalized display area18.FIG. 34Adepicts an example where using the user input passive device88on the interactive surface115triggers generation of a digital pad114for use with the user input passive device88on the interactive surface115. For example, setting the user input passive device88on the interactive surface115generates the digital pad114. Alternatively, a user requests generation of the digital pad114via an input interpreted via the user input passive device88or other user input.

The interactive display device10is operable to interpret user inputs received from the user input passive device88within the digital pad114area as functions to manipulate data on the personalized display area18of the interactive display device10. For example, moving the user input passive device88around the digital pad114maps to movements around the personalized display area18so that the user can execute various functions within the personalized display area18without having to move the user input passive device88onto the personalized display area18. The digital pad114is operable to move with the user input passive device88and is of a predetermined shape and size, a user defined size and shape, and/or a size and shape based on the size and shape of the user input passive device88.

FIG. 34Bdepicts an example where a user input triggers the generation of the digital pad114for use with or without the user input passive device88. For example, the user hand draws an area and/or inputs a command or selection to indicate generation of the digital pad114is desired on the interactive surface115. When the digital pad114area is triggered without the user input passive device, the user can optionally use a finger or other capacitive device for inputting commands within the digital pad114. As with the user input passive device88, the interactive display device10is operable to interpret user inputs received within the digital pad114area as functions to manipulate data on the personalized display area18of the interactive display device10.

FIG. 35is a logic diagram of an example of a method for generating a digital pad on an interactive surface of an interactive display device for interaction with a user input passive device. The method begins with step118where a plurality of drive sense circuits (DSCs) of the interactive display device transmit a plurality of signals on a plurality of electrodes of the interactive display device.

The method continues with step120where the plurality of DSCs detect a change in electrical characteristics of a set of electrodes of the plurality of electrodes. For example, the plurality of DSCs detect a change to mutual capacitance of the set of electrodes. The method continues with step122where a processing module of the interactive display device interprets the change in the electrical characteristics of the set of electrodes to be caused by a user input passive device in close proximity to an interactive surface of the interactive display device. For example, the mutual capacitance change detected on the set of electrodes is an impedance pattern corresponding to a particular user input passive device. User input passive device detection is discussed in more detail with reference to one or more ofFIGS. 5-32.

The method continues with step124where the processing module generates a digital pad on the interactive surface for interaction with the user input passive device. The digital pad may or may not be visually displayed to the user (e.g., a visual display may include an illuminated area designating the digital pad's area, an outline of the digital pad, a full rendering of the digital pad, etc.). The digital pad moves with the user input passive device as the user input passive device moves on the interactive surface of the interactive display device. The digital pad may be of a predetermined size and shape, a size and shape based on the size and shape of the user input passive device, a size and shape based on a user selection, and/or a size and shape based on an available area of the interactive display device.

For example, available area of the interactive display device may be limited due to the size of the interactive display device, the number and size of personalized display areas, and various objects that may be resting on and/or interacting with the interactive display device. The interactive display device detects an amount of available space and scales the digital pad to fit while maintaining a size that is functional for the user input passive device. The size of the digital pad is dynamically adjustable based on the availability of usable display area on the interactive display device.

Moving the digital pad onto a personalized display area can cause the personalized display area to adjust so that the digital pad is not obstructing the view of the personalized display area. A more detailed discussion of adjusting display areas based on obstructing objects is disclosed with reference to one or more ofFIGS. 36-44. Alternatively, moving the digital pad onto the personalized display area disables the digital pad so that the user input passive device can be used directly on the personalized display area.

The method continues with step126where the processing module interprets user inputs received from the user input passive device within the digital pad as functions to manipulate data on a display area of the interactive display device. For example, moving the user input passive device around the digital pad maps to movements around a personalized display area of the interactive display device so that the user can execute various functions within the personalized display area without having to move the user input passive device directly onto the personalized display area.

The digital pad may also have additional functionality for user interaction. For example, the digital pad may consist of different zones where use of the user input passive device in one zone achieves one function (e.g., scrolling) and use of the user input passive device in another zone achieves another function (e.g., selecting). The digital pad is also operable to accept multiple inputs. For instance, the user input passive device as well as the user's finger can be used directly onto the digital pad for additional functionality.

In an alternative example, instead of use of the user input passive device triggering generation of the digital pad, a user input can trigger the generation of the digital pad. For example, a user can hand draw an area and/or input a command or selection to indicate generation of the digital pad on the interactive surface of the interactive display device. When the digital pad is triggered without the user input passive device, the user can optionally use a finger or other capacitive device for inputting commands within the digital pad. As with the user input passive device, the interactive display device is operable to interpret user inputs received within the digital pad area as functions to manipulate data on the personalized display area of the interactive display device.

Generation of the digital pad can additionally trigger the generation of a virtual keyboard. When the user input passive device triggers the digital pad, the virtual keyboard is displayed in an area of the interactive surface in accordance with the user input passive device's position. For example, the virtual keyboard is displayed within a few inches of where the user input passive device is located. User information (e.g., user location at a table, right handed or left handed, etc.) available from the user input passive device or other user input aids in the display of the virtual keyboard. For example, a user identifier (ID) (e.g., based on a particular impedance pattern) associated with the user input passive device indicates that the user is right handed. Therefore, the virtual keyboard is displayed to the left of the user input passive device.

Alternatively, a user input triggers the generation of the virtual keyboard. For example, the user hand draws the digital pad and the digital pad triggers generation of the virtual keyboard or the user hand draws and/or inputs a command or selection to indicate generation of the virtual keyboard on the interactive surface.

FIG. 36is a schematic block diagram of another embodiment of the interactive display device10that includes the touch screen12, which may further include a personalized display area18to form interactive surface115. The personalized display area18may extend to all of the touch screen12or a portion as shown. The interactive display device10is shown here as an interactive table top that has interactive functionality (i.e., a user is able to interact with the table top via the interactive surface115) and non-interactive functionality (i.e., the interactive table top serves as a standard table top surface for supporting various objects).

In this example, the interactive display device10has three objects on its surface: a non-interactive and obstructing object128(e.g., a coffee mug), a non-interactive and non-obstructing object130(e.g., a water bottle), and a user input passive device88. In contrast to the user input passive device88which the interactive display device10recognizes as an interactive object (e.g., via a detected impedance pattern, etc.) as discussed previously, the non-interactive objects128and130are not recognized as items that the interactive display device10should interact with. The non-interactive and obstructing object128is an obstructing object because it is obstructing at least a portion of the personalized display area18. The non-interactive and non-obstructing object130is a non-obstructing obstructing object because it is not obstructing at least a portion of the personalized display area18.

The interactive display device10detects non-interactive objects via a variety of methods. For example, the interactive display device10detects a two-dimensional (2D) shape of an object based on capacitive imaging (e.g., the object causes changes to mutual capacitance of the electrodes in the interactive surface115with no change to self-capacitance as there is no path to ground). For example, a processing module of the interactive display device10recognizes mutual capacitance change to a set of electrodes in the interactive surface115and a positioning of the set of electrodes (e.g., a cluster of electrodes are affected in a circular area) that indicates an object is present.

As another example, the interactive display device10implements a frequency scanning technique to recognize a specific frequency of an object and/or a material of an object and further sense a three-dimensional (3D) shape of an object. The interactive display device10may implement deep learning and classification techniques to identify objects based on known shapes, frequencies, and/or capacitive imaging properties.

As another example, the interactive display device10detects a tagged object. For example, a radio frequency identification (RFID) tag can be used to transmit information about an object to the interactive display device10. For example, the object is a product for sale and the interactive display device10is a product display table at a retail store. A retailer tags the product such that placing the product on the table causes the table to recognize the object and further display information pertaining to the product. One or more sensors may be incorporated into an RFID tag to convey various information to the interactive display device10(e.g., temperature, weight, moisture, etc.). For example, the interactive display device10is a dining table at a restaurant and temperature and/or weight sensor RFID tags are used on plates, coffee mugs, etc. to alert staff to cold and/or finished food and drink, etc.

As another example, an impedance pattern tag can be used to identify an object and/or convey information about an object to the interactive display device10. For example, an impedance pattern tag has a pattern of conductive pads that when placed on the bottom of objects is detectable by the interactive display device10(e.g., the conductive pads affect mutual capacitance of electrodes of the interactive display device10in a recognizable pattern). The impedance pattern can alert the interactive display device10that an object is present and/or convey other information pertaining to the object (e.g., physical characteristics of the object, an object identification (ID), etc.). As such, tagging (e.g., via RFID, impedance pattern, etc.) can change a non-interactive object into an interactive object.

As another example of an interactive object, a light pipe is a passive device that implements optical and capacitive coupling in order to extend the touch and display properties of the interactive display device beyond its surface. For example, a light pipe is a cylindrical glass that is recognizable to the interactive display device (e.g., via a tag, capacitive imaging, dielectric sensing, etc.) and may further include conductive and/or dielectric properties such that a user can touch the surface of the light pipe and convey functions to the touch screen. When placed on the interactive display device over an image intended for display, the light pipe is operable to display the image with a projected image/3-dimensional effect. The user can then interact with the projected image using the touch sense properties of touch screen via the light pipe.

When a non-interactive object and obstructing object128is detected by the interactive display device10, the interactive display device10is operable to adjust the personalized display area18based on a position of a user such that the object is no longer obstructing the personalized display area18. Examples of adjusting the personalized display area18such that an obstructing object is no longer obstructing the personalized display area18are discussed with reference toFIGS. 37A-37D.

FIGS. 37A-37Dare schematic block diagrams of examples of adjusting a personalized display area18such that an obstructing object128is no longer obstructing the personalized display area18. The interactive surface115of the interactive display device10(e.g., ofFIG. 36) detects a two-dimensional shape of an object via one of the methods discussed with reference toFIG. 36. For example, an object changes mutual capacitance in electrodes of the interactive surface115such that the interactive surface115develops a capacitive image of the object. Because the personalized display area18is oriented toward a particular user, this known orientation is used to adjust the personalized display area with respect to the user's view. In the examples ofFIGS. 37A-37D, the adjusting is done assuming a user is looking straight across from or straight down at the personalized display area18. Generating personalized display areas according to user orientations are discussed with more detail in reference toFIGS. 45-48.

InFIG. 37A, an obstructing object128(e.g., the coffee mug ofFIG. 36) is detected and the personalized display area18is shifted over to create an adjusted display132such that the obstructing object128is no longer obstructing the personalized display area18. Adjusting the personalized display area18also includes determining available display space of the interactive display device10. For example, when there is limited available space (e.g., other objects and personalized display areas are detected) the personalized display area18may be adjusted such that the adjusted personalized display area18takes up less space.

For example, inFIG. 37B, the obstructing object128is detected and the personalized display area18wraps around the obstructing object128to create the adjusted display132. The type of adjustment may also depend on the type of data that is displayed in the personalized display area18. For example, if the personalized display area18displays a word document consisting of text, the best adjustment may be the example ofFIG. 37Aso that the text displays correctly.

InFIG. 37C, the obstructing object128is detected and the personalized display area18is broken into three display windows where display window2is shifted over such that the obstructing object128is no longer obstructing the personalized display area18. InFIG. 37D, the obstructing object128is detected and the personalized display area18is broken into three display windows to create adjusted display132where display windows2and3are shifted over such that the obstructing object128is no longer obstructing the personalized display area18.

FIG. 38is a logic diagram of an example of a method of adjusting a personalized display area based on detected obstructing objects. The method begins with step134where a plurality of drive sense circuits (DSCs) of an interactive display device (e.g., an interactive table top such as a dining table, coffee table, end table, etc.) transmit a plurality of signals on a plurality of electrodes of the interactive display device (e.g., where the electrodes include one or more of wire trace, diamond pattern, capacitive sense plates, etc.).

The method continues with step136where a set of DSCs of the plurality of DSCs detect a change in an electrical characteristic of a set of electrodes of the plurality of electrodes. The method continues with step138where a processing module of the interactive display device determines that the change in the electrical characteristic of the set of electrodes is a change in mutual capacitance. The method continues with step140where the processing module determines a two-dimensional shape of an object based on the change in mutual capacitance of the set of electrodes and based on positioning of the set of electrodes (e.g., a cluster of electrodes are affected in a circular area).

The method continues with step142where the processing module determines whether the two dimensional shape of the object is obstructing at least a portion of a personalized display area of the interactive display device. When the object is obstructing the at least the portion of the personalized display area of the interactive display device, the method continues with step144where the processing module determines a position of a user of the personalized display area. For example, the personalized display area is oriented toward a particular user. Therefore, the processing module assumes a user is looking straight across from or straight down at the personalized display area from that known orientation.

The method continues with step146where the processing module adjusts positioning of at least a portion of the personalized display area based on the position of the user and the two-dimensional shape, such that the object is no longer obstructing the at least the portion of the personalized display area. For example, the personalized display area is adjusted to create an adjusted display as in one or more of the examples described inFIGS. 37A-37D.

As another example, if the detected obstructing object is larger than or smaller than a certain size, the processing module can choose to ignore the item (e.g., for a certain period) and not adjust the personalized display area. For example, a briefcase is placed on the interactive display device entirely obstructing the personalized display area18. Instead of adjusting the personalized display area18when the object is detected, the user is given a certain amount of time to move the item.

FIG. 39is a schematic block diagram of another embodiment of the interactive display device10that includes the touch screen12, which may further include a personalized display area18to form an interactive surface115. The personalized display area18may extend to all of the touch screen12or a portion as shown. The interactive display device10is shown here as an interactive table top that has interactive functionality (i.e., a user is able to interact with the table top via the interactive surface115) and non-interactive functionality (i.e., the interactive table top serves as a standard table top surface for supporting various objects). The interactive display device10further includes an array of embedded cameras154facing outward from a border of the interactive display device10separate from the interactive surface115(e.g., not incorporated into a top or bottom surface of the interactive display device10).

In this example, a user is seated at the interactive display device10such that the user has line(s) of sight148to a personalized display area18on the interactive surface115. The interactive display device10detects a non-interactive and obstructing object128(e.g., a coffee mug) in any method described with reference toFIG. 36(e.g., capacitive imaging). The detection provides the obstructing object's two-dimensional (2D) obstructing area150. The methods discussed with reference toFIG. 36can determine three-dimensional (3D) characteristics of an object (e.g., via frequency scanning, classification, deep learning, and/or tagging, etc.). However, the obstructing object's 3D obstructing area152changes based on the user's lines of sight148to the personalized display area18. The user's line of sight148changes based on the height of the user, whether the user is sitting or standing, a position of the user (e.g., whether the user is leaning onto the table top or sitting back in a chair), etc.

Here, the user is shown sitting straight up in a chair and looking directly down at the personalized display area18such that the obstructing object128is between the lines of sight148and the personalized display area18. Thus, the obstructing object's 3D obstructing area152is a small shadow behind the obstructing object128. In order to gain information regarding a user's line(s) of sight, the interactive display device10includes an array of embedded cameras154. Image data from the embedded cameras154is analyzed to determine a position of the user with respect to the personalized display area18, an estimated height of the user, whether the user is sitting or standing, etc. The image data is then used to determine the obstructing object's 3D obstructing area152in order to adjust the personalized display area18accordingly.

FIG. 40is a schematic block diagram of another embodiment of the interactive display device10that includes a core control module40, one or more processing modules42, one or more main memories44, cache memory46, a video graphics processing module48, a display50, an Input-Output (I/O) peripheral control module52, one or more input interface modules, one or more output interface modules, one or more network interface modules60, one or more memory interface modules62, an image processing module158, and a camera array156.

The interactive display device10operates similarly to the example ofFIG. 2except the interactive display device10ofFIG. 40includes the image processing module158and the camera array156. The camera array156includes a plurality of embedded cameras. The cameras are embedded in a portion of the interactive display device10to capture images surrounding the interactive display device10. For example, the interactive display device10is an interactive table top (e.g., a coffee table, a dining table, etc.) and the cameras are embedded into a structural side perimeter/border of the table (e.g., not embedded into the interactive surface of the interactive display device10).

The cameras of the camera array156are small and may be motion activated such that when a user approaches the interactive display device10, the cameras activated by the motion capture a series of images of the user. Alternatively, the cameras of the camera array156may capture images at predetermined intervals and/or in response to a command. The camera array156is coupled to the image processing module158and communicates captured images to the image processing module158. The image processing module158processes the captured images to determine user characteristics (e.g., height, etc.) and positional information (e.g., seated, standing, distance, etc.) at the interactive display device10and sends the information to the core module40for further processing.

The image processing module158is coupled to the core module40where the core module40processes data communications between the image processing module158, processing modules42, and video graphics processing module48. For example, the processing modules42detects a two dimensional object is obstructing a personalized display area18of the interactive display device10. The user characteristics and/or positional information from image processing module158are used to further determine a three-dimensional obstructed area of the personalized display area18where the processing modules42and video graphics processing module48can produce an adjusted personalized display area based on the three-dimensional obstructed area for display to the user accordingly.

FIG. 41is a schematic block diagram of another embodiment of the interactive display device10that includes the touch screen12, which may further include a personalized display area18to form an interactive surface115.FIG. 41is similar to the example ofFIG. 39except that a taller non-interactive and obstructing object160is depicted (e.g., a water bottle) on the interactive surface115. In comparison toFIG. 39, the obstructing object's two dimensional (2D) obstructing area162is approximately the same however the obstructing object's three dimensional (3D) obstructing area164is much larger due to the height of the obstructing object160.

The object detection methods discussed with reference toFIG. 36can determine 3D characteristics of an object160(e.g., via frequency scanning, classification, deep learning, and/or tagging, etc.). Once 3D characteristics are determined, an estimation of the obstructing object's 3D obstructing area164can be determined based on a predicted user orientation to the personalized display area18. However, a more accurate obstructing object 3D obstructing area164can be determined by determining the user's line of sight148to the personalized display area18based on image data captured by the embedded cameras154. For example, the image data can show that the user is sitting off to the side of the personalized display area18looking down such that the obstructing object160is directly between the user's line of sight148and the personalized display area18.

FIG. 42is a schematic block diagram of another embodiment of the interactive display device10that includes the touch screen12, which may further include a personalized display area18to form an interactive surface115.FIG. 42is similar toFIG. 41except that the user is now standing at the interactive display device10instead of sitting. In comparison toFIG. 41, the obstructing object's two dimensional (2D) obstructing area162is approximately the same however the obstructing object's three dimensional (3D) obstructing area164is now much smaller due to the user's improved line of sight148to the personalized display area18.

Therefore,FIG. 42illustrates that to determine an accurate obstructing object 3D obstructing area164, a user's line of sight148to the personalized display area18needs to be determined (e.g., by capturing image data by the embedded cameras154for analysis).

FIGS. 43A-43Eare schematic block diagrams of examples of adjusting a personalized display area18such that an obstructing object's two-dimensional (2D) obstructing area and three-dimensional (3D) obstructing area (e.g., obstructing object's 2D obstructing area162and obstructing object's 3D obstructing area164ofFIG. 42) are no longer obstructing the personalized display area18.

InFIG. 43A, the interactive surface115detects a 2D and/or 3D shape of an object via one of the methods discussed previously. For example, an object changes mutual capacitance in electrodes of the interactive surface115such that the interactive surface115develops a 2D capacitive image of the object. The interactive surface115also processes image data captured by a camera array to determine an accurate 3D obstructing area based on a user's line of sight, user characteristics, and/or other user positional information. The personalized display area18is then adjusted accordingly.

InFIG. 43B, the obstructing object's 2D obstructing area162and the obstructing object's 3D obstructing area164are detected and the personalized display area18is shifted over to create an adjusted display132such that the obstructing object's 2D obstructing area162and the obstructing object's 3D obstructing area164are no longer obstructing the personalized display area18. Adjusting the personalized display area18also includes determining available display space of the interactive display device10. For example, when there is limited available space (e.g., other objects and personalized display areas are detected) the personalized display area18may be adjusted in a way that takes up less space on the interactive surface115.

For example, inFIG. 43C, the obstructing object's 2D obstructing area162and the obstructing object's 3D obstructing area164are detected and the personalized display area18wraps around the obstructing object's 2D obstructing area162and the obstructing object's 3D obstructing area164to create an adjusted display132. The type of adjustment may also depend on the type of data that is displayed in the personalized display area18. For example, if the personalized display area18displays a word document consisting of text, the best adjustment may be the example ofFIG. 43Bso that the text displays correctly.

InFIG. 43D, the obstructing object's 2D obstructing area162and the obstructing object's 3D obstructing area164are detected and the personalized display area18is broken into three display windows where display window2is shifted over such that the obstructing object's 2D obstructing area162and the obstructing object's 3D obstructing area164are no longer obstructing the personalized display area18.

InFIG. 43E, the obstructing object's 2D obstructing area162and the obstructing object's 3D obstructing area164are detected and the personalized display area18is broken into three display windows to create an adjusted display132where display windows2and3are shifted over such that the obstructing object's 2D obstructing area162and the obstructing object's 3D obstructing area164are no longer obstructing the personalized display area18.

FIG. 44is a logic diagram of an example of a method of adjusting a personalized display area based on a three-dimensional shape of an object. The method begins with step166where a plurality of drive sense circuits (DSCs) of an interactive display device (e.g., an interactive table top such as a dining table, coffee table, end table, etc.) transmit a plurality of signals on a plurality of electrodes of the interactive display device (e.g., where the electrodes may be wire trace, diamond pattern, capacitive sense plates, etc.).

The method continues with step168where a set of DSCs of the plurality of DSCs detect a change in an electrical characteristic of a set of electrodes of the plurality of electrodes. The method continues with step170where a processing module of the interactive display device determines that the change in the electrical characteristic of the set of electrodes is a change in mutual capacitance.

The method continues with step172where the processing module determines a three-dimensional shape of an object based on the change in mutual capacitance of the set of electrodes (e.g., 2D capacitive imaging), based on positioning of the set of electrodes (e.g., a cluster of electrodes are affected in a circular area), and one or more three-dimensional shape identification techniques.

The one or more three-dimensional shape identification techniques include one or more of: frequency scanning, classification and deep learning, image data collected from a camera array of the interactive display device indicating line of sight of a user to the personalized display area (e.g., based on position, distance, height of user, etc.), and an identifying tag (e.g., an RFID tag, an impedance pattern tag, etc.).

The method continues with step174where the processing module determines whether the three-dimensional shape of the object is obstructing at least a portion of a personalized display area of the interactive display device. When the three-dimensional shape of the object is obstructing the at least the portion of the personalized display area of the interactive display device, the method continues with step176where the processing module determines a position of a user of the personalized display area. For example, the personalized display area is oriented toward a particular user with a known orientation. Therefore, the processing module assumes a user is looking straight across from or straight down at the personalized display area. As another example, image data collected from a camera array of the interactive display device indicates a more accurate position of a user including a line of sight of a user to the personalized display area (e.g., based on user position, distance, height, etc.).

The method continues with step178where the processing module adjusts positioning of at least a portion of the personalized display area based on the position of the user and the three-dimensional shape, such that the object is no longer obstructing the at least the portion of the personalized display area. For example, the personalized display area is adjusted to create an adjusted display as in one or more of the examples described inFIGS. 43A-43E.

As another example, if the detected obstructing three-dimensional object is larger than or smaller than a certain size, the processing module can choose to ignore the item (e.g., for a certain period) and not adjust the personalized display area. For example, a briefcase is placed on the interactive display device entirely obstructing the personalized display area18. Instead of adjusting the personalized display area18when the object is detected, the user is given a certain amount of time to move the item.

FIG. 45is a schematic block diagram of another embodiment of the interactive display device10that includes the touch screen12, which further includes multiple personalized display areas18(e.g., displays1-4) corresponding to multiple users (e.g., users1-4) to form as interactive surface115. In this example, interactive display device10is an interactive table top (e.g., a dining table, coffee table, large gaming table, etc.).

Users1-4are each associated with a particular frequency (e.g., f1-f4). For example, users1-4are sitting in chairs around the interactive display device10where each chair includes a pressure sensor to sense when the chair is occupied. When occupancy is detected, a sinusoidal signal with a frequency (e.g., f1-f4) is sent to the interactive display device10. The chair may be in a fixed position (e.g., a booth seat at a restaurant) such that the signal corresponds to a particular position on the interactive display device10having a particular orientation with respect to the user. When f1-f4are detected, the interactive display device10is operable to automatically generate personalized display areas (e.g., displays1-4) of an appropriate size and in accordance with user1-4's detected positions and orientations. Alternatively, when f1-f4are detected, the interactive display device10is operable to provide users1-4various personalized display area options (e.g., each user is able to select his or her own desired orientation, size, etc., of the display).

As another example, one or more of users1-4may be associated with a user device (e.g., a user input passive device, an active device, a game piece, a wristband, a card, a device that can be attached to an article of clothing/accessory, etc.) that transmits a frequency or is otherwise associated with a frequency (e.g., a resonant frequency of a user input passive device is detectable) when used on and/or near the interactive display device10. When the particular frequency is detected, the interactive display device10is operable to automatically generate a personalized display area in accordance with a corresponding user's detected position and orientation. For example, a user's position and orientation are assumed from a detected location of the user device.

As another example, interactive display device10includes one or more cameras, antennas, and/or other sensors (e.g., infrared, ultrasound, etc.) for sensing a user's presence at the interactive display device. Based on user image data and/or assumptions from sensed data (e.g., via one or more antennas), the interactive display device10assigns a frequency to a user and automatically generates personalized display areas of an appropriate size, positions, and orientation for each user.

As another example, the interactive display device10generates personalized display areas of an appropriate size, positions, and orientation based on a user input (e.g., a particular gesture, command, a hand drawn area, etc.) that indicates generation of a personalized display area is desired. Alternatively, or in addition to, the interactive display device10is operable to track the range of a user's touches to estimate and display an appropriate personalized display area and/or make other assumptions about the user (e.g., size, position, location, dominant hand usage, etc.). The personalized display area can be automatically adjusted based on continual user touch tracking.

In all of the examples above, the interactive display device10is operable to determine the overall available display area of the interactive display device10and generate and/or adjust personalized display areas accordingly. As a specific example, if another user (e.g., user5) were to join the interactive display device10in a chair to the right of user1, user2and4's personalized display areas may reduce in height due to display1moving towards display2and the addition of display5moving toward display4. Alternatively, user2and4's personalized display areas may shift over to accommodate the additional display without reducing in height.

FIG. 46is a schematic block diagram of another embodiment of the interactive display device10that includes the touch screen12, which further includes multiple personalized display areas18(e.g., displays1and2) corresponding to multiple users (e.g., users1and2) to form an interactive surface115. In this example, interactive display device10is an interactive table top (e.g., a dining table, coffee table, large gaming table, etc.).

In this example, user1is associated with an identifying user device (e.g., identifying game piece1) that transmits a frequency f1or is otherwise associated with a frequency f1(e.g., a resonant frequency of a user input passive device is detectable) that is detectable by the interactive display device10when used on and/or near the interactive display device10. User2is associated with an identifying user device (e.g., identifying game piece2) that transmits a frequency f2or is otherwise associated with a frequency f2(e.g., a resonant frequency of a user input passive device is detectable) that is detectable by interactive display device10when used on and/or near the interactive display device10.

FIG. 47is a schematic block diagram of another embodiment of the interactive display device10that includes the touch screen12, which further includes multiple personalized display areas18(e.g., displays1,1-1,2and3) corresponding to multiple users (e.g., users1-3) to form interactive surface115. In this example, interactive display device10is an interactive table top (e.g., a dining table, coffee table, large gaming table, etc.).

Users1and3are located on the same side of the interactive display device10. Personalized display areas display1and display3are generated based on detecting a particular frequency associated with users1and3(e.g., generated by sitting in a chair, associated with a particular user device, etc.) and/or sensing user1and/or user2's presence at the table via cameras, antennas, and/or sensors in the interactive display device10. The interactive display device10scales and positions display1and display2in accordance with available space detected on the interactive display device10.

User2hand draws a hand drawn display area180(display2) on a portion of available space of the interactive display device and user1hand draws a hand drawn display area182(display1-1) on a portion of the interactive display device near display1. User1has one personalized display area (display1) that was automatically generated and one personalized display area (display1-1) that was user input generated. User2's hand drawn display area180depicts an example where the display is a unique shape created by the user. Based on how the display area is hand drawn, an orientation is determined. For example, a right handed user may initiate drawing from a lower left corner. Alternatively, the user selects a correct orientation for the hand drawn display area. As another example, a user orientation is determined based on imaging or sensed data from one or more cameras, antenna, and/or sensors of the interactive display device10.

If a user generated display area overlaps with unavailable space of the interactive display device, the display area can be rejected, auto-scaled to an available area, and/or display areas on the unavailable space can scale to accommodate the new display area.

FIG. 48is a logic diagram of an example of a method of generating a personalized display area on an interactive display device. The method begins with step184where a plurality of drive sense circuits (DSCs) of an interactive display device (e.g., an interactive table top such as a dining table, coffee table, end table, gaming table, etc.) transmit a plurality of signals on a plurality of electrodes (e.g., wire trace, diamond pattern, capacitive sense plates, etc.) of the interactive display device.

The method continues with step186where a set of DSCs of the plurality of DSCs detect a change in an electrical characteristic of a set of electrodes of the plurality of electrodes. The method continues with step188where a processing module of the interactive display device determines that the change in the electrical characteristic of the set of electrodes to be caused by a user of the interactive display device in close proximity (i.e., in contact with or near contact) to an interactive surface of the interactive display device.

For example, a user is sitting in a chair at the interactive display device where the chair includes a pressure sensor to sense when the chair is occupied. When occupied, the chair to conveys a sinusoidal signal including a frequency to the interactive display device alerting the interactive display device to a user's presence, location, and likely orientation. The chair may be in a fixed position (e.g., a booth seat at a restaurant) such that the signal corresponds to a particular position on the interactive display device having a particular orientation with respect to the user.

As another example, a user may be associated with a user device (e.g., user input passive device, an active device, a game piece, a wristband, etc.) that transmits a frequency or is otherwise associated with a frequency (e.g., a resonant frequency of a user input passive device is detectable) that is detectable by the interactive display device when used on and/or near the interactive display device.

As another example, the interactive display device includes one or more cameras and/or antennas for sensing a user's presence at the interactive display device. As yet another example, a user inputs a command to the interactive display device to alert the interactive display device to the user's presence, position, etc.

The method continues with step190where the processing module determines a position of the user based on the change in the electrical characteristics of the set of electrodes. For example, the chair sending the frequency is in a fixed position (e.g., a booth seat at a restaurant) that corresponds to a particular position on the interactive display device having a particular orientation with respect to the user. As another example, the user's position and orientation are assumed from a detected location of a user device. As another example, the user's position and orientation are detected from imaging and/or sensed data from the one or more cameras, antennas and/or sensors of the interactive display device. As a further example, a user input indicates a position and/or orientation of a personalized display area (e.g., a direct command, information obtained from the way a display area is hand drawn, location of the user input, etc.).

The method continues with step192where the processing module determines an available display area of the interactive display device. For example, the processing module detects whether there are objects and/or personalized display areas taking up space on the interactive surface of the interactive display device.

The method continues with step194where the processing module generates a personalized display area within the available display area based on the position of the user. For example, the interactive display device automatically generates a personalized display area of an appropriate size, position, and orientation based on the position of the user (e.g., determined by a particular frequency, device, user input, sensed data, image data, etc.) and the available space. Alternatively, when a user is detected, the processing module is operable to provide the user with various personalized display area options (e.g., a user is able to select his or her own desired orientation, size, etc., of the personalized display area).