Patent Publication Number: US-11390164-B2

Title: Vehicle interface having electromagnetic communications

Description:
TECHNICAL FIELD 
     This disclosure relates to vehicle interfaces having electromagnetic communications. 
     BACKGROUND 
     Vehicle interfaces enable occupant interaction with vehicle systems and components. Expanding vehicle capabilities increase interface complexity and abundance, leaving occupants uncertain of vehicle interface functions. Mobile devices may provide occupants with interface definitions and functionality. Such methods, however, may require ambient light and proper orientation to provide interface definitions and functionality. 
     SUMMARY 
     A vehicle includes an interface having an electromagnetic transmitter and is operable to receive tactile user input for control of a system of the vehicle. The vehicle includes a controller configured to modulate output having ultraviolet or infrared spectrum wavelength of the electromagnetic transmitter at a receiver capture rate to encode data into the output describing the control interface. The modulated output is responsive to a request. 
     A method by a controller includes displaying, on a graphical user interface, information related to a control interface based on an identification of the control interface such that operation of the control interface is disclosed. The display is responsive to receiving an electromagnetic output being modulated according to a capture rate of a sensor having a wavelength within an ultraviolet or an infrared spectrum defining a data stream with the identification associated with a transmitter of the electromagnetic output. 
     A method by a controller includes displaying, on a graphical user interface, information based on an identification of a control interface related to a one of a plurality of electromagnetic outputs having a greatest intensity and such that operation of the control interface is disclosed. The display being responsive to receiving a plurality of electromagnetic outputs modulated according to a capture rate of a sensor having a wavelength within an ultraviolet or an infrared spectrum defining a data stream with the identification of control interface associated with respective transmitters of the electromagnetic outputs. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a plan view of a vehicle and associated interfaces; 
         FIG. 2  is a schematic view of an interface and a mobile device; 
         FIG. 3  is a graph of a rolling shutter and a received electromagnetic signal; 
         FIG. 4  is an algorithm for displaying interface information from an electromagnetic transmitter; 
         FIG. 5  is an algorithm for sending an electromagnetic signal from a transmitter associated with a control interface; and 
         FIG. 6  is an algorithm for displaying interface information on a mobile device. 
     
    
    
     DETAILED DESCRIPTION 
     Embodiments of the present disclosure are described herein. It is to be understood, however, that the disclosed embodiments are merely examples and other embodiments may take various and alternative forms. The figures are not necessarily to scale; some features could be exaggerated or minimized to show details of particular components. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to variously employ the present invention. As those of ordinary skill in the art will understand, various features illustrated and described with reference to any one of the figures may be combined with features illustrated in one or more other figures to produce embodiments that are not explicitly illustrated or described. The combinations of features illustrated provide representative embodiments for typical applications. Various combinations and modifications of the features consistent with the teachings of this disclosure, however, could be desired for particular applications or implementations. 
     Occupants interact with vehicle systems through vehicle interfaces. Any vehicle type is contemplated through this disclosure. For example, vehicles may enable travel on land, sea, air, and space. Vehicles may include automobiles, cars, trucks, boats, ships, planes, helicopters, drones, or remote controllers. An interface may enable any type of human-machine interaction. For example, interfaces may be push buttons, pull buttons, knobs, dials, indicators, touchscreens, switches. Such interfaces may include electromagnetic transmitters. The electromagnetic transmitters may output modulated signals to be read by a sensor having a rolling shutter. For example, a CMOS sensor of a mobile phone may have a rolling shutter. A rolling shutter may be a modular sensing apparatus of various types. That is, the rolling shutter may be arranged to sense by columns, rows, or other arbitrary segment to systematically or sequentially read electromagnetic signals received by the sensor to form a full frame of the camera. Indeed, electromagnetic communications from each interface may be read by a mobile phone of a vehicle occupant to provide information relating to the functions and operation of the interface. 
     Referring to  FIG. 1 , a vehicle  100 , or portion thereof, is shown. Interfaces  102  may be disposed on the portion of the vehicle  100 . For example, a steering wheel of a vehicle is shown having a plurality of interfaces  102 . The interfaces  102  may provide any type of human-machine interaction and are not limited to push buttons that provide tactile input. For example, the interfaces  102  may be a touchscreen display or graphical user interfaces therein. The interfaces  102  may be located on any portion of the vehicle including external or internal locations. The interfaces  102  may be located in the trunk or engine compartment. 
     The interfaces  102  may include electromagnetic transmitters  104 . The electromagnetic transmitter  104  may emit any type of electromagnetic output. The electromagnetic output may be within the ultraviolet or infrared spectrum. For example, the electromagnetic output may be in the visible light spectrum. Meaning, the electromagnetic transmitter may be a light-emitting diode (LED) as shown in  FIG. 2 . The electromagnetic output may also be within the ultraviolet and infrared spectrum but unperceivable to the human eye. For example, the electromagnetic output may be outside of the visible light range but fall within the ultraviolet and infrared spectrum. Such an implementation would allow communication between the transmitter  104  and the CMOS sensor without distracting vehicle occupants. 
     Referring to  FIG. 2 , a schematic diagram of the interface  102  and a mobile or nomadic device  120  is shown. The interface  102  includes an electromagnetic transmitter  104 . The electromagnetic transmitter  104  is shown as an LED but may be another type of transmitter. The transmitter  104  is driven by a DC source  108  and a resistor  110 . The transmitter  104  is controlled by a controller  106 . The controller is connected to a resistor  112  and a transistor  114  to control the flow of current through the transmitter  104 . The output of the transmitter  104  may be modulated to indicate digital information and the digital information may be encoded according to known methods. The output of the transmitter  104  may be modulated according to the rolling shutter rate or speed of the CMOS sensor  126  of the mobile device. For example, the modulation timing may be synchronized with the rolling shutter speed—e.g., receiving capture rate—of the CMOS sensor  126 . 
     As shown in a simplified diagram, the rolling shutter of the CMOS sensor  126  is operated by a controller  124  to read each sensing segment or transistor of the CMOS sensor  126  systematically. For example, the controller  124  may sequentially take readings from the sensors  126  at 1 μs intervals  302  as shown in  FIG. 3 . 
     The sequential readings may be subsequently stitched together forming a full picture frame. The modulation rate of the modulated output  304  of the transmitter  104  is matched to the rolling shutter rate or receiver capture rate of the collective CMOS sensor  126 . As shown in  FIG. 4  the information received by the controller  124  of the mobile device  120  is used to display reference materials related to the interface  102  to occupants of the vehicle. The mobile device  120  may request further information from servers or database infrastructure outside of the mobile device  120 . The mobile device  120  may have a datastore or other database stored in local memory for retrieval. The reference materials on the display  122  may include diagrams, text, pictures, videos, or other information to assist occupants with the use of each interface  102  or vehicle systems. Additionally, the entire received image from the CMOS sensor  126  may be stitched together and displayed on the mobile device display  122 . Indeed, a captured image of the interface  102  may also include identification or reference materials related to the interface  102  without requiring object recognition of the image. 
     Referring to  FIGS. 3 and 4 .  FIG. 3  depicts a graph  300  having a modulated output  304  and a rolling shutter  302  reading such output at predetermined intervals. The modulated output may be a digital output having a Manchester encoding and an ON-OFF keyed modulation scheme. 
       FIG. 4  shows a communications algorithm  400  for generating the modulated output  304  and reading information therein. Controller  106  operations  402  are shown alongside controller  124  operations  420 . Data  404  associated with the interface may be embedded in any type of memory ready by or included with the controller  106 . The controller  106  reads data and converts it to binary in step  406 . The binary data  406  is encoded by the controller  106 . The encoding may be any known in the art (e.g., Manchester). 
     In some circumstances the encoded data may be multiplexed  410  to increase communication rates or improve reliability. The multiplexing  410  may include associating portions of the encoding  408  with colors or wavelengths of a cluster of transmitters  104 . For example, multiple transmitters  104  may be configured to transmit within predetermined wavelengths. The combination of received colors along a column, row, or arbitrary segment of CMOS sensor  126  may provide increase bandwidth or data validation. That is, a column of CMOS sensors falling within an individual frame of the rolling shutter may detect a plurality of distinct wavelengths or colors related to distinct data streams. After multiplexing, the controller  106  modulates output of the transmitter  104  or plurality of transmitters  104  according to the receive capture rate or rolling shutter of the CMOS sensor  126 . The modulation type may be ON-OFF keyed (OOK). 
     The output  304  is received and processed by the controller  124  and CMOS sensor  126  according to a rolling shutter of the CMOS sensor  126  such that the CMOS camera sensor  422  collects the modulated signal. A rolling shutter  302  is defined as systematically reading any arbitrary segment of CMOS sensor  126 . For example, a column-based rolling shutter may read CMOS pixels aligned in columns of the frame from left to right. A row-based rolling shutter may read CMOS pixels aligned in rows of the frame from top to bottom. Any other systematic selection of pixels may be used. 
     The modulated output  304  is demodulated by virtue of the rolling shutter  424 . That is, the synchronization of the rolling shutter to the modulation frequency enables accurate transmission of data by detecting digital electromagnetic or light values as each segment of the CMOS sensor  126  is read. As such, each segment of the CMOS sensor  126  may provide data in spite of the entire image frame being unread. 
     After receipt of the demodulated data via the rolling shutter  302 , the data may be demultiplexed by recognition of particular light or electromagnetic values received. That is, if coloration of the pixels is used to provide multiplexing, data streams may be identified by colorations or wavelengths. Additionally, where the CMOS sensor could also detect multiple pixels in the segment having the same wavelength or color, the CMOS may use values from pixels having the highest intensity or magnitude. That is, with all other factors being equal, the controller  124  may associate the desired data stream with a particular section of the CMOS segment being read such that the most intense or section having the highest magnitude is used. For example, if a columnar CMOS segment of the rolling shutter is read, a top section may have a higher magnitude (e.g., average value value) of light than a bottom section of the segment. As such, the controller  124  would read the data stream from the top section of the segment for the remainder of the frame of picture taking period to ensure that demultiplexed data was from the same source transmitter  104  if multiple transmitters  104  are detected. 
     The received data is decoded  428  according to the encoding specified above. The controller  124  may then obtain information according to the data or display the data itself on a display  122  of the mobile device  120 . The controller  124  may send the data offboard the mobile device to obtain the information. Further, the controller  124  may send the data to the vehicle  100  for a response. Meaning, the vehicle itself contains information related to the interfaces  102  and can provide information to the mobile device  120  upon presentation of an identifier in the data or other information. Further, the vehicle  100  may convey the information on a visual or audio interface at the request of the mobile device. The mobile device  120  may convey the information visually or aurally in step  432 . A vehicle interface  102  may communicate with a CMOS camera sensor  126  of a mobile device  120  through a rolling shutter of the CMOS camera sensor  126  to provide directionally-based information requests from occupants of the vehicle. 
     Referring to  FIG. 5 , an algorithm  500  for modulating an output is shown. The algorithm starts in step  502 . It should be appreciated that any algorithms disclosed herein may be implemented with steps omitted or occurring in a different sequence than shown. The algorithms may continuously repeat or sporadically stop. In step  504 , the controller  106  receives a communications request. A vehicle operation may initiate a communications request. For example, an engine start or system startup sequence may initiate the request so that all interfaces having such capabilities attempt to communicate. Meaning, the powering of additional devices may initiate the request. The request may be initiated based on a mobile device request. For example, a digital communications connection between the mobile device  120  and the vehicle  100  may enable the mobile device operator to request all of the interfaces to start transmitting. An application on the mobile device  120  may allow the operator to initiate the request. The request may be cryptographically signed for authenticity. 
     In step  506 , the controller  106  modulates the transmitter  104  output at the receiver capture rate. The receiver capture rate may be communicated from the mobile device  120  prior to transmission or be predefined. The modulation may ensure that the data send rate is the same as the rolling shutter rate, as shown in  FIG. 3 . In step  508 , the controller  106  may receive indication from the mobile device  120  that information was properly received or an indication that it was not. If information was not received, the mobile device  120  may update and send a receiver capture rate to the controller  106  or the controller  106  may increment or decrement the modulation frequency to try communications again in step  510 . In step  512 , the controller  106  may stop output of the transmitter  104 . In step  514 , the algorithm  500  may end or repeat. 
     Referring to  FIG. 6 , an algorithm  600  is shown. The algorithm  600  starts in step  602 . In step  604 , the controller  124  determines whether multiple transmitter  104  outputs were received based on the intensity of signals and clustering of the signal magnitude according to the segment of the CMOS sensor  126  being read during the rolling shutter of the frame. The section of the segment of the CMOS sensor  126  is selected in step  606  and used for further communications. In a similar way sections of the segment may be selected to identify multiplexing or other technology. In step  608 , the output is demodulated by the controller  124  according to the rolling shutter of the CMOS sensor such that data is received by the controller  124 . The data may be stored in local memory to be sent offboard. The output signal may be decoded in step  610 . In step  612 , an information request may be sent to a server or data repository on the mobile device  120 , vehicle  100 , or at another location to provide information to the operator. If information is obtained and available in step  614 , the information is displayed for the operator in step  618 . The information may be displayed on a display  122  of the mobile device  120 , another display associated with the vehicle  100 , or a separate display. If information is not available in step  614 , the controller  124  may send an additional request for output  616  with updated data rates or information to properly identify the interface or provide information about the interface. It should be appreciated that any type of data may be communicated. The data communicated may merely identify the interface. The data communicated may otherwise provide any or all of the information related to that interface including maintenance instructions, operation instructions, definitional information, instructional information, or any other information related to the interface. 
     The processes, methods, logic, or strategies disclosed may be deliverable to and/or implemented by a processing device, controller, or computer, which may include any existing programmable electronic control unit or dedicated electronic control unit. Similarly, the processes, methods, logic, or strategies may be stored as data and instructions executable by a controller or computer in many forms including, but not limited to, information permanently stored on various types of articles of manufacture that may include persistent non-writable storage media such as ROM devices, as well as information alterably stored on writeable storage media such as floppy disks, magnetic tapes, CDs, RAM devices, and other magnetic and optical media. The processes, methods, logic, or strategies may also be implemented in a software executable object. Alternatively, they may be embodied in whole or in part using suitable hardware components, such as Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), state machines, controllers or other hardware components or devices, or a combination of hardware, software and firmware components. 
     The words used in the specification are words of description rather than limitation, and it is understood that various changes may be made without departing from the spirit and scope of the disclosure. As previously described, the features of various embodiments may be combined to form further embodiments of the invention that may not be explicitly described or illustrated. While various embodiments could have been described as providing advantages or being preferred over other embodiments or prior art implementations with respect to one or more desired characteristics, those of ordinary skill in the art recognize that one or more features or characteristics may be compromised to achieve desired overall system attributes, which depend on the specific application and implementation. These attributes may include, but are not limited to cost, strength, durability, life cycle cost, marketability, appearance, packaging, size, serviceability, weight, manufacturability, ease of assembly, etc. As such, embodiments described as less desirable than other embodiments or prior art implementations with respect to one or more characteristics are not outside the scope of the disclosure and may be desirable for particular applications.