Patent Description:
Recent decades have brought tremendous advances in communication systems. These advances have enabled the global proliferation of smartphones and many other mobile communication systems. Smartphones have placed WiFi, cellular, Bluetooth, and NFC communication systems, as well as GPS capability, in the pocket of almost every individual, almost all of the time. Improvements in verifying the delivery of messages transmitted by mobile communication systems and the processing of those messages will further enhance the effectiveness of modern communication systems. It is known from <CIT> to provide systems and methods for determining user exposures to content, such as content presented by physical objects (e.g., advertisements on billboards). The systems and methods determine a probability (an "exposure probability") that a user has viewed or consumed content, information, or other visual media presented by a physical object, such as a billboard, vehicle, sign, or other structure, and provide the determined probability to various attribution systems, such as systems that attribute user engagements (e.g., store visits, app downloads, website visits, product purchases, and so on) to earlier content exposures. According to the invention there is provided message verification systems and a method for message verification according to the claims.

Important technical problems pose a challenge to verifying message delivery and processing when mobile entities are involved. As examples, the technical problems include determining whether a mobile receiver is in a position which allows receiving a message transmitted by a mobile entity. An additional technical problem is accounting for challenging environmental conditions. For instance, variations in lighting, topography, and obscuring objects all may impact whether a message can be received. It is also often difficult to determine whether or not the mobile receiver processed the message, even if it is known that the message was delivered to the mobile receiver. The verification system and techniques described below solve these and other technical problems.

<FIG> shows an example environment <NUM> in which a verification system <NUM> operates to verify message delivery and processing. In this particular example, the verification system <NUM> verifies message delivery and processing with respect to messages transmitted by the mobile transmitter <NUM> to mobile receivers, such as the mobile receivers <NUM> and <NUM>. Several examples below are provided for discussion purposes, but note that the particular environment that the verification system <NUM> analyzes may vary extensively, as may the number and types of messages, mobile transmitters, and mobile receivers.

In the example of <FIG>, the environment <NUM> includes the mobile transmitter <NUM> and the mobile receivers <NUM> and <NUM>, which move along a road system <NUM>. There may be any number of mobile transmitters and mobile receivers. In addition, the mobile transmitters and mobile receivers may take a wide variety of forms, as examples: buses, taxis, trains, boats, automobiles, cars, trucks, motorcycles, airplanes, human beings, animals, helicopters, hot air balloons, dirigibles, or other mobile entities.

The mobile transmitters and receivers move along arbitrary routes. In <FIG> for instance, the mobile transmitter <NUM> moves along a predefined repeating route <NUM>, while the mobile receiver <NUM> moves along the route <NUM> from the starting point <NUM> to the destination point <NUM>, and the mobile receiver moves along the route <NUM> from the starting point <NUM> to the destination point <NUM>. The routes may be predefined paths that follow a known timing schedule defined by, as examples, a bus schedule or train schedule. Transmitter and receiver location and timing along the routes may be enhanced, corrected, or interpolated by systems that receive feedback on location, speed, and timing, e.g., via global positioning system (GPS) data, WiFi location data, cellular basestation data, manual location and reporting, or in other manners. For instance, interpolating the route of the mobile receiver <NUM> may add position and time sample points between the starting point <NUM> and the destination point <NUM> to ensure a sample resolution that meets a predefined sampling threshold, e.g., at least one sample per second or one sample per <NUM> feet.

In one implementation, the verification system <NUM> includes communication interfaces <NUM>, verification processing circuitry <NUM>, and verification databases <NUM>. The verification databases <NUM> may include, as examples, map data, road data, image data (e.g., street view images), object data for objects in the environment, weather data, viewshed constructs, location and time data, route schedules, and other types of data. In some implementations, as described below, the verification system <NUM> may build and maintain models <NUM> that include physical models of mobile transmitters and mobile receivers, and probabilistic models used for determining likelihood of message delivery and processing.

The verification processing circuitry <NUM> connects through the communication interfaces <NUM> and networks <NUM> to any desired data sources to obtain data, or to supplement data that the verification system <NUM> already maintains. For instance, the verification processing circuitry <NUM> may receive location and time data defining the routes <NUM>, <NUM>, and <NUM> from mobile data service providers <NUM>. As one example, the routes <NUM> and <NUM> may include location and timestamp information determined by cell towers serving the mobile entities <NUM> and <NUM> or location and timestamp information for individual cell phones carried within the mobile entities <NUM> and <NUM>. As another example, the verification processing circuitry <NUM> may receive bus, train, and water taxi route location and time information from municipal scheduling data sources or other third party data sources <NUM>. The verification system <NUM> may also receive mobile transmitter and mobile receiver characteristics from the data sources. Examples of mobile transmitter and mobile receiver characteristics include width, height, length, speed, acceleration, deceleration, roll, pitch, yaw, steering angle, and brightness of message displays (e.g., LCDs or illuminated signage) mounted on the mobile entities. The verification system <NUM> may include these characteristics (and others) in the models <NUM> of the mobile transmitters and mobile receivers.

The verification system <NUM> may also communicate with messaging systems <NUM>. The messaging systems <NUM> may represent, for instance, systems or organizations which are responsible for providing the messages which the mobile entities transmit. The verification system <NUM> may communicate verification messages <NUM> to the messaging systems <NUM>. The verification messages <NUM> may convey message transmission reports, message delivery statistics, message processing reports, and other message information reports to the messaging systems <NUM>. As an example, the verification messages <NUM> may report whether any particular message was delivered and to which entities it was delivered, as well as whether the message was processed, including how and when it was processed.

As noted above, the verification system <NUM> verifies delivery of messages transmitted by the mobile transmitter <NUM> to the mobile receivers <NUM>, <NUM>. The verification system <NUM> also verifies whether the mobile receivers <NUM>, <NUM> have processed the message. These two aspects of the operation of the verification system <NUM> are described in further detail below.

<FIG> shows an example of the operational logic <NUM> of the verification system <NUM>, including taking into consideration viewshed for determining whether messages were received. The verification system <NUM> receives transmitter route data <NUM> for the mobile transmitter <NUM> (<NUM>) and receives model data <NUM> for any selected mobile transmitters and receivers (<NUM>). This data may be obtained from the verification databases <NUM>, external data sources, or both. The transmitter route data <NUM> may be provided at any desired resolution, e.g., location and timestamp samples at one-second intervals.

The verification system <NUM> constructs a viewshed <NUM> of the environment around the mobile transmitter <NUM> (<NUM>). The viewshed construction may take into consideration the model data <NUM>. The viewshed <NUM> may, for instance, comprise one or more regions of physical locations around the mobile transmitter <NUM> which permit viewing of the message transmitted from the mobile transmitter <NUM>. For instance, when the mobile transmitter carries a two-sided message transmitter, then the viewshed may be an hourglass shape emanating from the mobile transmitter. A fully-wrapped message transmitter may have a sphere--shaped viewshed, as another example. In the example of <FIG>, the mobile transmitter <NUM> includes two LCD displays, <NUM> and <NUM>, one on each side of the mobile transmitter <NUM>. The viewshed <NUM> may include the position and the timestamp associated with the viewshed size and physical extent at that timestamp.

The size and extent of the viewshed <NUM> may vary over time according to the characteristics of the mobile transmitter as captured, for instance, in the model data <NUM>. Example characteristics include size and shape of the mobile transmitter <NUM> and size, shape, brightness, and position of a message display mounted on the mobile transmitter <NUM>. The verification system <NUM> may also determine the viewshed according to other factors such as acceleration, speed, and deceleration of the mobile transmitter <NUM>, as well as direction of travel, weather (e.g., amount of daylight and meteorological conditions), and other inputs. In addition, artificial lighting characteristics may be included in the models <NUM> and may influence the viewshed. For example, the amount and intensity of external lights mounted on the mobile transmitter <NUM> (as well as artificial lights in the erivironment) may impact the spatial and temporal characteristics of the viewshed <NUM> for messages transmitted from the mobile transmitter <NUM>.

The verification system <NUM> also receives receiver route data <NUM> for the mobile receivers (<NUM>). When the receiver route data <NUM> is incomplete or too sparse, then the verification system <NUM> may interpolate to obtain route data, or may request supplemental receiver route data from the external data sources (<NUM>). For instance, the verification system <NUM> may determine whether the route time and location sample interval is less than a pre--defined sampling threshold (such as one location sample per second or one location sample per <NUM> feet), and if so, obtain additional route data.

The verification system <NUM> compares the viewshed <NUM> with the receiver route data <NUM> to determine whether the message was delivered to any particular mobile receiver (<NUM>). In the example of <FIG>, the verification system <NUM> determines that the message has been delivered to the mobile receiver <NUM> because the mobile receiver <NUM> is within the viewshed <NUM> at the time that the mobile transmitter <NUM> is transmitting its message. In some implementations, the verification system <NUM> may also apply a probabilistic viewing model to measure the likelihood that any selected mobile receiver has observed the transmitted message, e.g., an advertisement transmitted from the LCD screens <NUM>, <NUM> on the side of the mobile transmitter <NUM>.

With regard to the viewing model, the verification system <NUM> considers several factors, such as the trajectory and speed of the mobile receiver as it travels, the relative viewing angle to the mobile transmitter, as well as the time that the message was in clear view of the receiver. The viewing model may take the form of a Gaussian process model in which the probability that the message was delivered (e.g., received or viewed by an individual) is governed by the displacement from the center of the field of view of the receiver, the viewing distance to the mobile transmitter, and the relative viewing angle of the mobile receiver to the mobile transmitter. For verification of message delivery to individuals in particular, the viewing model may be fitted to human eye tracking studies over a variety of real-world environments and the process is iterated for each half second block of time (for example) that the mobile transmitter and mobile receiver are coincident. The output of this process is the full posterior distribution for the mobile receiver to have received the message.

The verification system <NUM> may also determine delivery statistics. The delivery statistics may include details such as a count of mobile receivers judged to have received the message or having been at least exposed to the message, a demographic breakdown of the mobile receivers, and other statistics. For instance, the demographic breakdown be based in part on characteristic data provided by the service providers <NUM>. When the mobile receivers are individual viewers, the demographic data may include features such as age, education, gender, occupation, income, marital status, and the like. The verification system <NUM> transmits the verification messages <NUM> including the message delivery determinations and delivery statistics to the messaging systems <NUM> (<NUM>) for reporting purposes.

<FIG> shows another example of operational logic <NUM> of the verification system <NUM>, including taking into consideration obstructions in the viewshed. In particular, the verification system <NUM> may receive object data <NUM> for the environment in and around the mobile transmitter. The object data <NUM> may include, as examples, map and image data (e.g., street view images) that captures obscuring objects, or explicit records of size, location, and facing of: signage, bulletin boards, trees, hedges, fences, hills, telephone poles, and other potentially obstructing objects near the mobile transmitter that affect the viewshed <NUM>.

<FIG> shows several obstructing objects in the environment: the signage <NUM> and <NUM>, and the tree line <NUM>. The verification system <NUM> determines when obstructions in the environment occlude the view from the mobile receiver to the mobile transmitter <NUM>. Those obstructions are removed from the viewshed (<NUM>). In other words, the viewshed no longer includes the areas in which the objects occlude the view of the message.

As shown in <FIG>, in this example, the verification system <NUM> determines that the mobile receiver <NUM> cannot receive the message. The viewshed <NUM> has been reduced by the occluded section <NUM> and is effectively much smaller in the direction of the mobile receiver <NUM> due to the obstructions. As a result, the mobile receiver <NUM> is no longer in the viewshed for the message. On the other hand, the verification system <NUM> determines that the mobile receiver <NUM> only has a partially obstructed viewpoint due to the signage <NUM>, and can receive the message. In this example, the signage <NUM> reduces the viewshed by the occluded section <NUM>, which leaves enough viewshed <NUM> for the mobile receiver <NUM> to receive the message.

<FIG> shows another example of the operational logic <NUM> of the verification system, including determining message processing. The verification system <NUM> may implement a wide variety of approaches to determining whether a mobile receiver has processed the message it has received. In one implementation, the verification system <NUM> obtains or determines expected message effect characteristics (<NUM>), e.g., by retrieving them from the verification database <NUM>, by obtaining them from the external data sources, or both. These characteristics may indicate as examples, assuming the message receiver processed the message, how the message receiver would be expected to move, actions that the message receiver would be expected to take, communications the message receiver would be expected to send or receive, or other indicia of message processing. The verification system <NUM> may determine message processing according to whether or not there is agreement between the effect characteristics and actions taken by the mobile receiver (<NUM>). In addition, the verification system <NUM> may prepare and transmit message processing reports (<NUM>) to the messaging systems <NUM>.

When the message effect characteristics are route characteristics, then the verification system <NUM> may receive subsequent route data <NUM> for the mobile receiver <NUM> for a predefined period of time, e.g., for up to one week after receipt of the message (<NUM>). The verification system <NUM> may then determine whether the subsequent route data <NUM> evidences agreement between the mobile receiver and the effect characteristics (<NUM>). As a specific example, assume that the message concerns free admission to the Kennedy Space Center for the Apollo <NUM>50th Anniversary program. The expected message effect characteristics may then include travel of the mobile receiver <NUM> to or within a predefined region (e.g., to Merritt Island, Florida) one of the days of free admission. The agreement of the mobile receiver route and the expected message effect characteristics, combined with the verification that the message was received indicate that the message was actually processed and acted upon by the mobile receiver <NUM>.

As another example, the verification system <NUM> may receive an explicit confirmation of message processing from the mobile receiver <NUM> (<NUM>). For instance, the mobile receiver <NUM> may transmit a confirmation message to the verification system <NUM>, service providers <NUM>, or other system the mobile receiver <NUM> has claimed free-admission passes to the Kerinedy Space Center in response to the message it received. Confirmation messages received by systems other than the verification system <NUM> may be transmitted to the verification system <NUM> for processing.

Another technique for determining message processing is for the verification system to receive communication indicators (<NUM>) that evidence message processing. As examples, the verification system <NUM> may receive confirmation (e.g., from the mobile device <NUM>, service providers <NUM>, or other data sources <NUM>) that the mobile device <NUM> accessed a web page or IP address for Kennedy Space Center after receiving the message. As another example, the verification system <NUM> may receive confirmation that the mobile receiver <NUM> placed a phone call to a phone number associated with Kennedy Space Center after receiving the message.

In some cases, the verification system <NUM> may receive action confirmations (<NUM>) that indicate message processing. For instance, the verification system <NUM> may receive a confirmation that the mobile device <NUM> redeemed a code (or took some other action required) for free admission, after the mobile device <NUM> received the message. Yet another source of message processing insight includes social media inputs received by the verification processing system <NUM> (<NUM>). For instance, the verification processing system <NUM> may monitor public social media platform posts to determine that the mobile receiver <NUM> acted on the message it received. As examples, the verification processing system <NUM> may detect photos of the Saturn V launch vehicle posted by the mobile receiver <NUM> to social media, or may detect newly posted messages including content tags relating to Kennedy Space Center after the mobile receiver <NUM> received the message.

<FIG> shows an example implementation <NUM> of the verification system <NUM>. The implementation <NUM> includes communication interfaces <NUM>, system circuitry <NUM>, input/output (I/O) interfaces <NUM>, and display circuitry <NUM>. The system circuitry <NUM> may include any combination of hardware, software, firmware, or other circuitry. The system circuitry <NUM> may be implemented, for example, with one or more systems on a chip (SoC), application specific integrated circuits (ASIC), microprocessors, microcontrollers, discrete analog and digital circuits, and other circuitry. The system circuitry <NUM> is part of the implementation of any desired functionality in the verification system <NUM>. Accordingly, the system circuitry <NUM> may implement the operational logic <NUM>, <NUM>, <NUM> described above in <FIG>, as examples. The verification system <NUM> may store and retrieve data from data memories <NUM>. For instance, the data memories <NUM> may store the verification databases <NUM> and the models <NUM>.

The display circuitry <NUM> and the I/O interfaces <NUM> may include a graphical user interface, touch sensitive display, voice or facial recognition inputs, buttons, switches, speakers and other user interface elements. Additional examples of the I/O interfaces <NUM> include Industrial Ethernet, Controller Area Network (CAN) bus interfaces, Universal Serial Bus (USB), Serial Advanced Technology Attachment (SATA), and Peripheral Component Interconnect express (PCle) interfaces and connectors, memory card slots, and other types of inputs. The I/O interfaces <NUM> may further include Universal Serial Bus (USB) interfaces, audio outputs, magnetic or optical media interfaces (e.g., a CDROM or DVD drive), network (e.g., Ethernet or cable (e.g., DOCSIS) interfaces), or other types of serial, parallel, or network data interfaces.

The communication interfaces <NUM> may include transceivers for wired or wireless communication. The transceivers may include modulation / demodulation circuitry, digital to analog converters (DACs), shaping tables, analog to digital converters (ADCs), filters, waveform shapers, filters, pre-amplifiers, power amplifiers and/or other circuitry for transmitting and receiving through a physical (e.g., wireline) medium such as coaxial cable, Ethernet cable, or a telephone line, or through one or more antennas. Accordingly, Radio Frequency (RF) transmit (Tx) and receive (Rx) circuitry <NUM> handles transmission and reception of signals through one or more antennas <NUM>, e.g., to support Bluetooth (BT), Wireless LAN (WLAN), Near Field Communications (NFC), and <NUM>, <NUM>, and <NUM> / Long Term Evolution (LTE) communications.

Similarly, the non--wireless transceivers <NUM> may include electrical and optical networking transceivers. Examples of electrical networking transceivers include Profinet, Ethercat, OPC-UA, TSN, HART, and WirelessHART transceivers, although the transceivers may take other forms, such as coaxial cable network transceivers, e.g., a DOCSIS compliant transceiver, Ethernet, and Asynchronous Transfer Mode (ATM) transceivers. Examples of optical networking transceivers include Synchronous Optical Networking (SONET) and Synchronous Digital Hierarchy (SDH) transceivers, Passive Optical Network (PON) and Ethernet Passive Optical Network (EPON) transceivers, and EPON Protocol over Coax (EPoC) transceivers.

Note that the system circuitry <NUM> may include one or more controllers <NUM>, e.g., microprocessors, microcontrollers, FGPAs, GPUs, and memories <NUM>. The memory <NUM> stores, for example, an operating system <NUM> and control instructions <NUM> that the controller <NUM> executes to carry out desired functionality for the verification system <NUM>. Accordingly, the control instructions <NUM> may implement the logic <NUM>, <NUM>, <NUM> described above and with regard to <FIG> for message delivery verification, message processing verification, and verification message preparation and delivery. The control parameters <NUM> provide and specify configuration and operating options for the control instructions <NUM>.

The methods, devices, processing, circuitry, and logic described above may be implemented in many different ways and in many different combinations of hardware and software. For example, all or parts of the implementations may be circuitry that includes an instruction processor, such as a Central Processing Unit (CPU), microcontroller, or a microprocessor; or as an Application Specific Integrated Circuit (ASIC), Programmable Logic Device (PLD), or Field Programmable Gate Array (FPGA); or as circuitry that includes discrete logic or other circuit components, including analog circuit components, digital circuit components or both; or any combination thereof. The circuitry may include discrete interconnected hardware components or may be combined on a single integrated circuit die, distributed among multiple integrated circuit dies, or implemented in a Multiple Chip Module (MCM) of multiple integrated circuit dies in a common package, as examples.

Accordingly, the circuitry may store or access instructions for execution, or may implement its functionality in hardware alone. The instructions may be stored in a tangible storage medium that is other than a transitory signal, such as a flash memory, a Random Access Memory (RAM), a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM); or on a magnetic or optical disc, such as a Compact Disc Read Only Memory (CDROM), Hard Disk Drive (HDD), or other magnetic or optical disk; or in or on another machine-readable medium. A product, such as a computer program product, may include a storage medium and instructions stored in or on the medium, and the instructions when executed by the circuitry in a device may cause the device to implement any of the processing described above or illustrated in the drawings.

The implementations may be distributed. For instance, the circuitry may include multiple distinct system components, such as multiple processors and memories, and may span multiple distributed processing systems. Parameters, databases, and other data structures may be separately stored and managed, may be incorporated into a single memory or database, may be logically and physically organized in many different ways, and may be implemented in many different ways. Example implementations include linked lists, program variables, hash tables, arrays, records (e.g., database records), objects, and implicit storage mechanisms. Instructions may form parts (e.g., subroutines or other code sections) of a single program, may form multiple separate programs, may be distributed across multiple memories and processors, and may be implemented in many different ways. Example implementations include stand-alone programs, and as part of a library, such as a shared library like a Dynamic Link Library (DLL). The library, for example, may contain shared data and one or more shared programs that include instructions that perform any of the processing described above or illustrated in the drawings, when executed by the circuitry.

Claim 1:
A message verification system (<NUM>) comprising:
a model database (<NUM>); and
verification processing circuitry (<NUM>) configured to:
receive transmitter route data (<NUM>) for a mobile transmitter (<NUM>);
receive receiver route data (<NUM>) for a mobile receiver (<NUM>);
determine a viewshed (<NUM>) for a message transmitted by the mobile transmitter (<NUM>); and
compare the viewshed (<NUM>) to the receiver route data (<NUM>) to determine whether the message was delivered to the mobile receiver (<NUM>);
characterized by:
the model database (<NUM>) configured to store a mobile transmitter physical model; and
the verification processing circuitry (<NUM>) configured to:
determine, based at least in part on object data including street-level image data, the viewshed (<NUM>) for the message transmitted by the mobile transmitter (<NUM>) as modeled by the mobile transmitter physical model, the viewshed (<NUM>) including a region of physical locations around the mobile transmitter (<NUM>) that permit viewing of the message, the viewshed (<NUM>) excluding at least an obstructed location (<NUM>) for which a view of the message is occluded by an obstruction within the street-level image data.