Patent Publication Number: US-11644921-B2

Title: Monitoring the status of a touchscreen

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application is a continuation of U.S. application Ser. No. 17/464,569 filed Sep. 1, 2021, which is a continuation-in-part of U.S. application Ser. No. 16/799,377 filed Feb. 24, 2020, which claims the benefit of U.S. Provisional Patent Application Ser. No. 62/810,155 filed Feb. 25, 2019, the disclosures of each of which are hereby incorporated by reference as if fully restated herein. 
    
    
     TECHNICAL FIELD 
     Exemplary embodiments relate generally to systems and methods for monitoring the status of a touchscreen. 
     BACKGROUND AND SUMMARY OF THE INVENTION 
     Increasing supply and decreasing costs have resulted in a greater utilization of touchscreen technology. Such touchscreens are increasingly being used in a number of applications. One such exemplary application is the digital out of home (“DOOH”) advertising industry. DOOH applications include indoor, semi-outdoor, and outdoor digital signage. 
     Some touchscreens require physical depression (e.g., resistive type), while others require contact with a conductive item (e.g., capacitive type). Some touchscreens suffer from inaccurate touch capture, some have slow response speeds, and many touchscreens are only capable of receiving a limited number of touch inputs at a given time. The use of sigma-delta modulation technology and/or in-cell concurrent capacitive imaging (herein interchangeably referred to as “ICCI”) may overcome some or all of these, and other, deficiencies. Such ICCI technology is available from, for example without limitation, SigmaSense, LLC of Austin Tex. (see sigmasense.com). Such ICCI technology may comprise, for example without limitation, systems and method which energize an entire touchscreen array while the touchscreen is operated and monitor for signal reflections. The touchscreen&#39;s steady-state behavior may be monitored and used as a baseline to determine anomalous behavior, such as user input. 
     Touchscreens, particularly those in DOOH applications, may be exposed to harsh conditions. Such harsh conditions may include, for example without limitation, vandalism, smudging, wildlife, other environmental hazards, rain, sun, sleet, snow, high winds, other weather conditions, accidents, some combination thereof, or the like. Such harsh conditions may cause damage affecting the touchscreen&#39;s functionality and may result in operational downtime. Even if such damage does not affect functionality of the touchscreen, it may affect visibility—which can impact revenue, especially in the DOOH market. 
     Even if a touchscreen is remotely monitored, not all damage will be sufficient to trigger an alert. For example, such alerts may be generated only upon certain failure type events. Systems may not exist to monitor for other levels of damage such as, but not limited to, surface scratches, smudges, obstructions, screen cracks, or similar. Furthermore, even if an alert is generated, there is insufficient information provided to determine the type of damage to the touchscreen. For example, a remote user may be unable to determine if the whole unit needs replaced, and therefore a skilled team with replacement parts needs to be dispatched to make the repair, or if the unit is only lightly damaged in an easily repaired fashion (e.g., excess smudging or bird excrement) which might require only an unskilled technician to wipe down the screen. 
     What is needed is a system and method for monitoring the status of a touchscreen. Systems and methods for monitoring the status of a touchscreen are provided. One or more display assemblies may be in communication with a network operations center. Each display assembly may comprise one or more touchscreens, one or more touchscreen controllers, and a network communications device. The touchscreens may utilize ICCI technology. The touchscreens may be monitored periodically, or continuously, for damage. Such monitoring may be reported in substantially real time. The presence of damage may be determined by detecting interruptions in the touchscreen signals distributed across the touchscreen by the touchscreen controller. For example, without limitation, persisting touch input at a given area may indicate a first type of damage, persistent signal interruption across a portion of the touchscreen may indicate a second type of damage, and persistent signal interruption to the entire touchscreen may indicate a third type of damage. The location of damage may be determined based on the area where the signal is interrupted. A visual representation of the touchscreen, along with a visual representation of damage thereto, may be generated. The visual representation may indicate the type and/or location of the damage. 
     It may be desirable to incorporate the touchscreen into a ruggedized subassembly. The touchscreen may comprise multiple components and/or layers which provide protection, improve optical quality, facilitate thermal management, combinations thereof, or the like. 
     Further features and advantages of the systems and methods disclosed herein, as well as the structure and operation of various aspects of the present disclosure, are described in detail below with reference to the accompanying figures. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       In addition to the features mentioned above, other aspects of the present invention will be readily apparent from the following descriptions of the drawings and exemplary embodiments, wherein like reference numerals across the several views refer to identical or equivalent features, and wherein: 
         FIG.  1    is a simplified plan view of an exemplary system in accordance with the present invention; 
         FIG.  2    is a detailed top perspective view of an exemplary display assembly for use with the system of  FIG.  1   ; 
         FIG.  3 A  is a flow chart illustrating exemplary logic for use with the system of  FIG.  1   ; 
         FIG.  3 B  is a flow chart illustrating other exemplary logic for use with the system of  FIG.  1   ; 
         FIG.  4 A  is an exemplary visual representation of an exemplary touchscreen with a first type of damage; 
         FIG.  4 B  is another exemplary visual representation of the exemplary touchscreen with a second type of damage; 
         FIG.  4 C  is another exemplary visual representation of the exemplary touchscreen with a third type of damage; and 
         FIG.  5    is a top sectional view of the system of  FIG.  1    taken along section line A-A. 
     
    
    
     DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENT(S) 
     Various embodiments of the present invention will now be described in detail with reference to the accompanying drawings. In the following description, specific details such as detailed configuration and components are merely provided to assist the overall understanding of these embodiments of the present invention. Therefore, it should be apparent to those skilled in the art that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present invention. In addition, descriptions of well-known functions and constructions are omitted for clarity and conciseness. 
     Embodiments of the invention are described herein with reference to illustrations of idealized embodiments (and intermediate structures) of the invention. As such, variations from the shapes of the illustrations as a result, for example, of manufacturing techniques and/or tolerances, are to be expected. Thus, embodiments of the invention should not be construed as limited to the particular shapes of regions illustrated herein but are to include deviations in shapes that result, for example, from manufacturing. 
       FIG.  1    is a simplified plan view of an exemplary system in accordance with the present invention. One or more display assemblies  100   a - 100   n  may be placed in electronic communication with a network operations center  120 . Any number, size, shape, type, orientation, and the like of display assemblies  100   a - 100   n  may be utilized. A network operations center  120  may comprise one or more electronic devices such as, but not limited to, personal computers, smartphones, tablets, electronic displays, some combination thereof, or the like. Alternatively, or additionally, one or more of the display assemblies  100   a - 100   n  may be placed in electronic communication with one or more remote electronic devices  121  operated outside of the network operations center  120 . 
     Electronic communication with the display assemblies  100   a - 100   n  may be accomplished by way of network communication devices  106  located at each respective display assembly  100   a - 100   n . Such electronic communication may be realized through wired connections, wireless connections, some combination thereof, or the like. Such networks may comprise internets, intranets, the world wide web, cellular networks, some combination thereof, or the like. Each display assembly  100   a - 100   n  may comprise one or more touchscreen electronic displays  102  (hereinafter also, touchscreens  102 ). The touchscreens  102  may comprise liquid crystal displays, organic light emitting diode displays, plasma displays, or the like. Furthermore, the touchscreens  102  may be directly backlit or edge lit. The touchscreens  102  may further comprise various anti-glare layers, anti-smudge layers, protective layers, and the like. In exemplary embodiments, the touchscreens  102  utilize ICCI technology, though any type of touchscreen technology may be utilized. 
     If a respective display assembly  100   a - 100   n  comprises more than one touchscreen  102 . Such touchscreens  102  may be arranged back to back, side by side, above and below one another, some combination thereof, or the like. Each of the touchscreens  102  may be operable by way of touch input. A touchscreen controller  104  may be placed in electronic communication with one or more of the touchscreens  102  in a respective display assembly  100   a - 100   n . Upon receipt of touch input at the touchscreen  102 , the touchscreen controller  104  may command the touchscreen  102  to alter the image displayed thereon based upon the touch input as received and processed at the touchscreen controller  104 . The touchscreen controller  104  may, alternatively or additionally, monitor for touch input at the one or more of the touchscreens  102 . 
     Where a given display assembly  100   a - 100   n  comprises more than one touchscreen  102 , more than one touchscreen controller  104  may be utilized, though such is not required. Each of the touchscreen controllers  104  may comprise software utilizing ICCI technology. Such ICCI technology and software may be available from, for example without limitation, SigmaSense, LLC of Austin Tex. (see sigmasense.com/). The touchscreen controllers  104  may be configured to monitor the touchscreens  102  for touch input and may change the image displayed on the respective touchscreen  102  in response to the received touch input. Activity recorded by the touchscreen controllers  104 , and/or images displayed on the touchscreen  102 , may be remotely monitored at the network operations center  120  and/or at other remotely located electronic devices  121  by way of the network communications device  106 . 
     Each display assembly  100   a - 100   n  may comprise a housing  108 . The housing  108  may partially or wholly enclose the electronic display(s)  102 , the touchscreen controller(s)  104 , and the network communication device  106  for the respective display assembly  100   a - 100   n . The housing  108  may be ruggedized such that the display assembly  100   a - 100   n  may withstand various environmental conditions such as, but not limited to, rain, snow, wind, high temperatures, low temperatures, sunlight, hail, sleet, physical impact, vandalism, and the like. 
       FIG.  2    is a detailed top perspective view of an exemplary display assembly  100   a . In exemplary embodiments, without limitation, such ruggedization may be accomplished by providing an open loop pathway for the ingestion and exhaustion of ambient air  112 . Intake  110  and exhaust apertures may be provided on the display assemblies  100   a - 100   n  to facilitate the intake and exhaust of ambient air  112 . One or more fans may be provided along the open loop pathway to facilitate the ingestion, exhaustion, and other movement of the ambient air  112 . In exemplary embodiments, the open loop pathway travels along the rear surface of the backlight for the touchscreen  102 , though such is not required. 
     A closed loop pathway for circulating air  114  may also be provided within the respective display assembly  100   a - 100   n  in exemplary embodiments, without limitation. The closed loop pathway may be in thermal communication with some or all of the components of the respective display assembly  100   a - 100   n  so as to remove heat generated by such components. The closed loop pathway may be placed in thermal communication with the open loop pathway to transfer heat from the circulating gas  114  to the ambient air  112 . The ambient air  112  may also provide cooling for the display assembly  100   a  and various components thereof. Such thermal communication may be accomplished by way of an air to air heat exchanger. In this way, the circulating gas  114  may remain substantially free from contaminants that may otherwise be present in the ambient air  112 . One or more fans may be provided along the closed loop pathway to facilitate the movement of the circulating gas  114 . In exemplary embodiments, the closed loop pathway may pass between a transparent cover for, and a display surface of, the touchscreen  102 . The various display assemblies  100   a - 100   n  may be the same or different from one another. 
       FIG.  3 A  and  FIG.  3 B  are flow charts illustrating exemplary logic for use with the system of  FIG.  1   . The touchscreen controllers  104  may periodically or continuously transmit signals to the touchscreens  102  and monitor for interruptions. To so monitor the touchscreens  102 , the touchscreen controllers  104  may continuously or periodically disburse signals across the touchscreens  102  and monitor the received signals for weakness, interruption, or other irregularities. 
     Referring particularly to  FIG.  3 A , if such signals are wholly or partially interrupted for less than a predetermined period of time, then the touchscreen controllers  104  may determine that touch input has been received and adjust the image displayed on the touchscreen based on the received touch input. However, if such signals are wholly or partially interrupted for more than the predetermined period of time, the touchscreen controllers  104  may determine that damage has been detected. 
     If damage is detected, the type and location of the damage may be identified. Identification of the type of damage may be achieved by determining the level of interference with touchscreen signals being generated and transmitted across the touchscreen  102  by the touchscreen controllers  104 . For example, without limitation, if a signal is only partially interrupted or weakened, it may be determined that a first type of damage has occurred. Such first types of damage may include, for example without limitation, surface obstructions. If a signal is completely interrupted, it may be determined that a second type of damage has occurred. Such second types of damage may include, for example without limitation, scratches, cracks, or other physical damage. If no signals are returned whatsoever, it may be determined that a third type of damage has occurred. Such third types of damage may include, for example without limitation, failure or downtime events. 
     Alternatively, or additionally, and referring particularly to  FIG.  3 B , if such signal interruptions represent touch input and such interruption has not persisted for at least the predetermined period of time, the touchscreen controllers  104  may determine that touch input has been received and adjust the image displayed on the touchscreen based on the received touch input. The touchscreen controllers  104  may, for example without limitation, determined that the interruption constitutes touch input where the interruption is consistent with other touch inputs received. However, if the interruption is consistent with touch input and has persisted for more than a predetermined period of time, the touchscreen controllers  104  may determine that the first type of damage has occurred. 
     If such signal interruptions do not represent touch input, the touchscreen controllers  104  may determine if the interruption is over part of all of the touchscreen  102 . The touchscreen controllers  104  may, for example without limitation, determined that the interruption does not constitute touch input where the interruption indicates missing signals, scrambled signals, some combination thereof, or the like. If the interruption is over part of the touchscreen  102 , the touchscreen controllers  104  may determine that the second type of damage has occurred. If the interruption is over the enrage touchscreen  102 , the touchscreen controllers  104  may determine that the third type of damage has occurred. 
     Identification of the location of the damage may be achieved by determining the locations on the touchscreen  102  where signals from the touchscreen controllers  104  are interrupted. The entire touchscreen  102  may be scanned to outline the area(s) where the signals are interrupted. 
       FIG.  4 A ,  FIG.  4 B  and  FIG.  4 C  illustrate exemplary visual representations of the touchscreen  102  having a first, second, and third type of damage, respectively. At the network operations center  120 , a visual representation  130  comprising one or more graphical representations of the type and location of damage  132  to the touchscreen  102  may be generated. The visual representation  130  may further comprise one or more graphical representations of the touchscreen  102  and/or the display assemblies  100 . For example, without limitation, the visual representation of damage  132  may be color coded or otherwise patterned to indicate the severity of such damage. The visual representation of damage  132  may likewise be sized and scaled to represent the size, location, or other extent of the damaged area of the touchscreen  102 . Further information may be provided in conjunction with the visual representation  130  such a, but not limited to, a textual description of the type and/or location of the damage  132 , the location or other identifying information for the touchscreen  102 , the location or other identifying information for the display assembly  100 , part numbers for the touchscreen  102  or components thereof, some combination thereof, or the like. Repair efforts may be assigned and undertaken based on the visual representation  130 . This may permit the proper personnel to be assigned to the repair effort, for the proper equipment and other resources to be allocated, and for the repair team to prepare for the repair efforts. Requests for repair may be automatically generated comprising the type and/or location of damage. The request for repair may comprise identifying information for the touchscreen  102 , identifying information for the display assembly  100 , part numbers for the touch screen  102 , billing information, instructions, technical details, some combination thereof, or the like. 
     Referring to the example provided in  FIG.  4 A , the first type of damage may comprise surface obstructions, for example without limitation. The first type of damage may be illustrated on the visual representation  130 A in the size and shape as they have occurred on the touchscreen  102 . Such surface level obstructions may include, for example without limitation, excessive smudging, spilled liquids or other materials, bird or other wildlife droppings, paper, tape, some combination thereof, or the like. The first type of damage may be determined, for example without limitation, where signals corresponding to persistent touch input are received. In such cases, a request for repair may be transmitted to an unskilled or less skilled team. For example, without limitation, the repair request may comprise instructions to spray and wipe down the touchscreen  102  or otherwise check for and remove surface obstructions. The repair request may comprise, for example without limitation, the type of spray materials compatible with the touchscreen. 
     Referring to the example provided in  FIG.  4 B , the second type of damage may comprise scratches or other physical damage to the touchscreen  102 , for example without limitation. The second type of damage may be illustrated on the visual representation  130 B in the size and shape as they have occurred on the touchscreen  102 . Such physical damage may be determined, for example without limitation, where persistent signal interruption is determined for part, but not all, of the touchscreen  102 . In such cases, a request for repair may be transmitted to a skilled team or technician. The requests for repair in such cases may include, for example without limitation, one or more identifiers for replacement parts believed to be required based on the type of damage and the touchscreens  102  installed at the display assembly  100 . Instructions for installation of the replacement parts may be provided with the repair request, for example without limitation. 
     Referring to the example provided in  FIG.  4 C , the third type of damage may comprise a complete failure or downtime of the touchscreen  102 . The third type of damage may be illustrated on the visual representation  130 C with an error message  132 C, for example without limitation. Such failure may be determined where persistent signal interruption is determined for the entire touchscreen  102 . The requests for repair in such cases may include, for example without limitation, one or more identifiers for replacement touchscreens  102  believed to be required based on the type of damage and the touchscreens  102  installed at the display assembly  100 . 
     The damage  132  illustrated in the visual representation  130  may be color coded or patterned to indicate that the damage  132  is a first, second, or third type of damage. The visual representations  132  may be updated in substantially real time and such updates may be performed continuously or at any interval. The repair requests may request that photographic evidence be provided of completed jobs. 
     In exemplary embodiments, such as but not limited to, where the touchscreens  102  utilize ICCI technology, signal interruptions may be determined where received signals vary from an established steady-state behavior. Steady-state behavior may be determined by energizing an entire array for the touchscreen and monitoring the touchscreen for an initial period of time to establish steady-state behavior. Exemplary touch input may be provided to determine exemplary touch input. In such embodiments, comparisons may be made against exemplary touch input to determine signal interruptions which do or do not represent touch input. Regardless, where signal behavior varies from the established steady-state behavior, a determination may be made that the signal is interrupted. Alternatively, or additionally, signal interruption may be determined where no signals are received following transmission. 
     Any of the steps described herein may be performed in any order. Any of the steps may be repeated any number of times. Certain operations described herein may be performed by one or more electronic devices. Each electronic device may comprise one or more processors, electronic storage devices, executable software instructions, and the like configured to perform the operations described herein. The electronic devices may be general purpose computers or specialized computing device. The electronic devices may include the touchscreen controller  104  and/or may be personal computers, smartphone, tablets, databases, servers, or the like. The electronic connections and transmissions described herein may be accomplished by wired or wireless means. 
       FIG.  5    illustrates details of the touchscreen  102  of the display assembly  100 . The touchscreen  102  may comprise multiple layers, some or all of which may be positioned adjacent to one another and/or some or all of which may be spaced apart from one another. In exemplary embodiments, without limitation, the touchscreen  102  comprises a front cover layer  140 , a first layer of adhesive  142 , a touch sensor layer  144 , a second layer of adhesive  146 , a rear cover layer  148 , an electronic display layer  150 , and a backlight  152 . However, not all of the forementioned components or layers need be present, additional layers may be provided, and/or similar layers may be used in substitution. In exemplary embodiments, without limitation, the front cover layer  140  may form at least part of a front portion of the housing  108 . The first and second layers of adhesive  142 ,  146  may comprise the same material, though such is not required. The first layer of adhesive  142  may be sandwiched between the front cover layer  140  and the touch sensor layer  144  to adhere the front cover layer  140  and the touch sensor layer  144 . The second layer of adhesive  142  may be sandwiched between the touch sensor layer  144  and the rear cover  148  to adhere the touch sensor layer  144  to the rear cover  148 . 
     The front cover layer  140 , the first layer of adhesive  142 , the touch sensor layer  144 , the second layer of adhesive  146 , and the rear cover layer  148  may form a first subassembly  162  in exemplary embodiments without limitation. The electronic display layer  150  and the backlight  152  may form a second subassembly  164  in exemplary embodiments without limitation. The rear cover layer  148  may be spaced apart from the electronic display layer  150  to define a front chamber  160  configured to receive, and permit passage of, the circulating air  114 . 
     The circulating air  114  may extend within the housing  108  about at least the second subassembly  164  in exemplary embodiments. A rear chamber  158  may be located rearward of the touchscreen  102  to receive the circulating air  114  from the front chamber  160 . The front chamber  160  may be fluidly connected to the rear chamber  158  to permit the circulating air  114  to circulate about the second subassembly  164 . One or more fans may be provided, such as within or fluidly adjacent to, the rear chamber  158  to cause circulating of the circulating air  114  when activated. 
     An open loop chamber  154  for the ambient air  112  may be provided rearward of the backlight  152 . The open loop chamber  152  may be provided directly adjacent to at least a portion of the backlight  152 , though such is not required. The open loop chamber  152  may form part of the second subassembly  164  or may be separate. In exemplary embodiments, the circulating air  114  may extend rearward of the open loop chamber  152 . 
     A heat exchanger  156  may be located in the rear chamber  158 , though use of the heat exchanger  156  is not necessarily required. The heat exchanger  156  may comprise multiple layers, some of which are configured to receive the circulating air  114 , and others of which are configured to receive ambient air  112 . 
     The touchscreen controller  104 , the network communication device  106 , and/or other components for operating the display assembly  100  (e.g., processors, electronic storage devices, power supplies, video players, combinations thereof, or the like), may be located in the rear chamber  158 , though such is not required. 
     The circulating air  114  need not necessarily be 100% sealed from the ambient air  112 . It may be sufficient that the circulating air  114  is substantially separated from the ambient air  112 . For example, without limitation, separation may be sufficient to meet various industry criteria including, but not limited to, IP65 and/or any one or more of the ingress protection ratings between IP50 and IP69K, such as defined by the International Electrotechnical Commission (available at: https://www.iec.ch/homepage), by way of nonlimiting example. 
     The front cover layer  140  may comprise glass. The front cover layer  140  may be between 5-7 mm thick in exemplary embodiments. The front cover layer  140  may comprise a titanium oxide and/or dioxide easy clean/anti-microbic topcoat over an anti-reflective coating on the front surface. A sintered frit mask (e.g., black) may be provided on a rear surface of the front cover layer  140 . The mask may be provided in a picture frame form at the front cover layer  140 . 
     The first layer of adhesive  142  may comprise one or more optically clear adhesives, which may be curable by UV light. The first layer of adhesive  142  may be between 1-2 mm thick, in exemplary embodiments. 
     The touch sensor layer  144  may comprise a glass substrate with double index matched ITO transmit rows on a first side and receiver columns on a second, opposing side in exemplary embodiments. The touch sensor layer may be between 1-2 mm thick in exemplary embodiments. 
     The second layer of adhesive  146  may comprise the same or different adhesive as the first layer of adhesive  142 . The second layer of adhesive  146  may be between 1-2 mm thick, in exemplary embodiments. 
     The rear cover layer  148  may comprise glass. The rear cover layer  148  may be between 1-3 mm thick in exemplary embodiments. The rear cover layer  148  may comprise a titanium oxide and/or dioxide easy clean/anti-microbic topcoat over an anti-reflective coating on a rear surface. 
     The first subassembly  162  may form a projected capacitive sensor configured to receive touch input. The first subassembly  162  may have a final thickness of between 12-14 mm in exemplary embodiments. 
     The second subassembly  164  may form an electronic display configured to display images. The images may be displayed to correspond with touch input received from first subassembly  162 . The first and second subassemblies  162 ,  164  may together form the touchscreen  102  and may provide an interactive touch experience in a ruggedized fashion which permits thermal management. The touchscreen controller  104  may be in electronic communication with some or all components of one or both of the first and second subassemblies  162 ,  164 . 
     Certain operations described herein may be performed by one or more electronic devices. Each electronic device may comprise one or more processors, electronic storage devices, executable software instructions, combinations thereof, and the like configured to perform the operations described herein. The electronic devices may be general purpose computers or specialized computing devices. The electronic devices may comprise personal computers, smartphones, tablets, databases, servers, or the like. The electronic connections and transmissions described herein may be accomplished by wired or wireless means. The computerized hardware, software, components, systems, steps, methods, and/or processes described herein may serve to improve the speed of the computerized hardware, software, systems, steps, methods, and/or processes described herein. 
     Any embodiment of the present invention may include any of the features of the other embodiments of the present invention. The exemplary embodiments herein disclosed are not intended to be exhaustive or to unnecessarily limit the scope of the invention. The exemplary embodiments were chosen and described in order to explain the principles of the present invention so that others skilled in the art may practice the invention. Having shown and described exemplary embodiments of the present invention, those skilled in the art will realize that many variations and modifications may be made to the described invention. Many of those variations and modifications will provide the same result and fall within the spirit of the claimed invention. It is the intention, therefore, to limit the invention only as indicated by the scope of the claims.