Patent ID: 12223486

The figures are not necessarily to scale. Like numbers used in the figures refer to like components. However, it will be understood that the use of a number to refer to a component in a given figure is not intended to limit the component in another figure labeled with the same number.

DETAILED DESCRIPTION

Various embodiments will be described in detail with reference to the drawings, wherein like reference numerals represent like parts and assemblies through the several views. Reference to various embodiments does not limit the scope of the claims attached hereto. Additionally, any examples set forth in this specification are not intended to be limiting and merely set forth the many possible embodiments for the appended claims.

Whenever appropriate, terms used in the singular also will include the plural and vice versa. The use of “a” herein means “one or more” unless stated otherwise or where the use of “one or more” is clearly inappropriate. The use of “or” means “and/or” unless stated otherwise. The use of “comprise,” “comprises,” “comprising,” “include,” “includes,” and “including” are interchangeable and not intended to be limiting. The term “such as” also is not intended to be limiting. For example, the term “including” shall mean “including, but not limited to.”

A system and method for ascertaining the validity of physical and/or electronic mail items containing monetary solicitations is described herein. The system and method enable a user to obtain an image of the mail item and submit it for analysis. Analysis of the image of the mail item provides the user, in return, with an image overlay that is displayed over, or atop, the image of the mail item and includes one or more validity indicators indicating the valid or fraudulent nature of one or more portions of the mail item. In certain examples, the described system and method include an interactive user-interface wherein a user may select one of the validity indicators for: display of additional information; posing a verbal question; or typing a question related to the validity indicator. In the instance of a posed question, the described system and method can provide a visual or audio response. In certain examples, the system and method work in cooperation with a user's existing credentials that have been established with the user's banking services provider, e.g., bank, credit union, credit card company, etc., to enable payment requested by the mail item via the user's bank account.

Referring toFIG.1, a schematic diagram of an exemplary system10for ascertaining the validity of physical and/or electronic mail items containing monetary solicitations. The system10, in general of terms, includes one or more user devices11such as a smart device12(e.g., mobile phone12a, tablet12b, smart watch12c, augmented reality glasses12d, etc.) or a computer14(e.g., desktop computer14aor laptop14b) that can communicate via a wired or wireless network16to one or more host servers18. In certain examples, the user devices11and/or the host servers18are communicatively coupled to banking services20with access to a user's bank account(s).

FIG.2illustrates an exemplary architecture of a computing device200that can be used to implement aspects of the present disclosure including the smart devices12, computers14and host servers18. The computing device200can be in any suitable form including a microcontroller, a microprocessor, a desktop computer, a laptop computer, a mobile computer device (e.g., smart phone, tablet, smart watch, or other mobile device), or other devices configured to process digital instructions. Accordingly to avoid undue repetition, this description of the computing device200will not be separately repeated herein for each smart device12, computer14and host server18. Rather, it is understood that the exemplary computing device200may be configured specific to its intended use incorporating various peripherals and programming instructions, as described herein, to achieve desired operations. Further, it is understood that the computing device200is an example of programmable electronics, which may include one or more such computing devices, and when multiple computing devices are included, such computing devices can be coupled together with a suitable data communication network so as to collectively perform the various functions, methods, and operations disclosed herein.

In general terms, the computing device200includes at least one processing device and at least one computer readable storage device. The processing device operates to execute data instructions stored in the computer readable storage device to perform various operations, methods or functions described herein.

In more particular terms, and with reference toFIG.2, the computing device200includes at least one processing device202such as a central processing unit (BPU), as a well as a system memory204and a system bus206. The system bus206couples various system components including the system memory204to the processing device202. The system bus is one of any number of types of bus structures including a memory bus, a peripheral bus, and a local bus using any variety of bus architectures.

The system memory204includes program memory208and random access memory (RAM)210. A basic input/output system (BIOS)212containing the basic routines that act to transfer information within the computing device200, such as during start up, is typically stored in the program memory208. In some embodiments, the computing device200also includes a secondary storage device214, such as a hard disk drive or file server for storing digital data. The secondary storage device214is connected to the system bus206by a secondary storage interface (NTF)216. The secondary storage device214, and its associated computer readable media, provides nonvolatile storage of computer readable instructions (including application programs and program modules), data structures, and other data for the computing device200.

Although the exemplary computing device200described herein employs a secondary storage device214, in some embodiments the secondary storage device is eliminated or its hard disk drive/file server configuration is replaced with an alternative form of computer readable storage media. Alternative forms of computer readable storage media include, but are not limited to, magnetic cassettes, flash memory cards, digital video disks, Bernoulli cartridges, compact disk memories, digital versatile disk memories and random access memories. Some embodiments of the second storage devices214include non-transitory media. Further, the computer readable storage media can include local storage or cloud-based storage.

A number of program modules can be stored in the memory, or the secondary storage device214. These program modules include an operating system218, one or more application programs220, other program modules222as described herein, and program data224. The computing device200can utilize any suitable operating system such as Microsoft Windows, Google Chrome, Apple OS, and any other operating system suitable for a computing device.

The computing device200typically includes at least some form of computer readable media, e.g., computer readable media within the memory204or secondary storage device214. Computer readable media includes any available media that can be accessed by the computing device200. By way of example, computer readable media includes computer readable storage media.

Computer readable storage media includes volatile and nonvolatile, removable and non-removable media implemented in any device configured to store information such as computer readable storage instructions, data structures, program modules or other data. Computer readable storage media includes, but is not limited to, random access memory, read only memory, electrically erasable programmable read only memory, flash memory, or other memory technology, compact disk read only memory, digital versatile disks or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by the computing device200.

In some embodiments, a user provides inputs to the computing device200through one or more input devices226. Examples of input devices include a keyboard228, a touch sensor230(such as a touchpad or touch sensitive display), a scanner232and a camera234. Other embodiments include other input devices226necessary for fulfilling the operations of the system10, seeFIG.1. In some embodiments, the input devices are incorporated into the computing device200itself. In some embodiments, the input devices are external to the computing device200and are connected to the processing device202through an input interface236that is coupled to the system bus206. The input devices can be connected by any number of input/output interfaces such as a parallel port, a serial port, a game port, a universal serial bus, or a custom interface. Wireless communication between input device and the input interface236is possible as well, and includes infrared, Bluetooth wireless technology, 802.11/a/b/g/n, cellular or other radio frequency communication systems in some possible embodiments.

In the example embodiment ofFIG.2, the computing device200incorporates within or is operably coupled to a display device238. Examples of the display device238include a monitor, a liquid crystal display device, a projector, or a touch sensitive display device. The display device238is also connected to the system bus via an output interface240, such as a display controller. In addition to the display device238, the computing device200can control via output interface240various other peripherals such as a printing device242or speaker (not shown). As with the input interface236, the output interface240can comprise any number of input/output interfaces such as those described in the paragraph above.

The computing device200further includes a network interface244that includes a network communication device to communicate digital data across a data communication network246. An example of the network interface244includes a wireless transceiver for transmitting digital data over a wireless network. The wireless transceiver is configured to work with one or more wireless communication technologies such as cellular communication, Wi-Fi communication (such as that conforming to one of the IEEE802.11 family of communication protocols), Bluetooth communication, and the like. In other embodiments, the network interface244is an Ethernet network interface device having an Ethernet port for receiving an Ethernet cable to transmit and receive digital data across the Ethernet cable to a network246such as a local area network of the Internet.

When configured for operation as the user device11, the computing device200includes an application program220and/or various program modules222such as a “validity app”300to enable the operations illustrated in the flowchart ofFIG.3. As shown, upon launching the validity app300on the user device11, the user is prompted to enter their credentials to establish themselves a valid user of the app, S302. Such user-entered credentials may include a user name and password, biometric credentials or other suitable forms of identifying the user. In certain examples, the validity app300is independently hosted. In other examples, the validity app300is hosted by the user's banking services provider. Accordingly, the credentials used to establish the user as a valid user of the banking services can also be used to establish them as a valid user of the validity app300. In still other examples, the validity app300is hosted by a different type of service provider to the user and the user may use the credentials established with that service provider to log-in or establish themselves as a valid user of the validity app.

Upon activating the validity app300and establishing themselves as a valid user [YES; S303], the user utilizes the user device11to obtain an image of a physical and/or electronic mail item containing a monetary solicitation that the user wishes to be validated, S304. The image of the mail item can be obtained through any convenient means. For example, the image of the mail item can be obtained: by using the camera234of the user device11; by scanning the mail item with a peripheral scanner; by retrieving an image from memory in the user device11; by taking a screen shot of the display of the user device11; etc. In certain examples, the image of the mail item is in a common image format, e.g., PDF, TIFF, GIF, JPEG, FPX, PCD, PNG, etc. while in other examples the image of the mail item is in a proprietary image format that can be exclusive to the host of the validity app300.

An example of an image of a mail item400containing a monetary solicitation is illustrated inFIG.4. As shown, the exemplary mail item400includes various fields including: (a) user/payor name402; (b) user/payor address404; (c) user/payor account number406; (d) payee name and/or payee logo408; (d) charges to the account410; (e) amount owed by the user/payor412; and (f) physical or electronic address for remittance of payment414. Other fields not shown can also be included in the mail item, for example, a web address for the user/payor to review their bill, one or more fields to enter additional personal information such as a social security number or telephone number, “fine print” language fields that include terms, limitations and/or commitments that may be binding on the user, etc.

Referring once again to the system ofFIG.1and the flowchart ofFIG.3, upon obtaining an image of the mail item, the image is sent by the user from the user device11over the wired or wireless network16to an analysis module600at the host server18, S306, wherein the image will be analyzed and an image overlay will be produced as described in further detail below. Upon analysis, the image overlay is returned from the host server18to the user device11, and displayed atop the image of the mail item, S308. An example of an image overlay500is illustrated inFIG.5Aand described in further detail below. The image overlay500provides an interactive user interface on the user device11from which the user may request the display of additional information from the application host, S310, pose a typed or verbal question to the application host, S312, and/or initiate a desired action, S314.

When configured for operation as the host server18, the computing device200includes an application program220and/or various program modules222such as the analysis module600to enable the operations illustrated in the flowchart ofFIG.6and an advisory module700to enable the operations illustrated in the flowchart ofFIG.7.

Referring toFIG.6, the analysis module600of the host server18operates in conjunction with an analysis database602to receive the image of the mail item (e.g., mail item400), S604and to diagram the image of the mail item to identify the data fields of the image, S606. The identified data fields of the image of the mail item generally correspond to the fields with respect to the exemplary image of the mail item400. For example, the identified data fields of the image of the mail item can include: (a) user/payor name402; (b) user/payor address404; (c) user/payor account number406; (d) payee name and/or payee logo408; (d) charges to the account410; (e) amount owed by the user/payor412; and (f) physical and/or electronic address for remittance of payment414. Other elements of the image can, of course, be identified relative to the specific configuration of the mail item, e.g., a web address for the user/payor to review their bill, one or more fields to enter additional personal information such as a social security number or telephone number, “fine print” language fields that include terms, limitations and/or commitments that may be binding on the user, etc. The identified data fields can remain in image format or can be extracted into a text format or an audio playback format. In certain examples, the analysis module600may be unable to identify all data fields of the mail item and can message the user at the user device11to enter additional information, S607, about the unidentified data field or ignore the unidentified data field, S608.

With all possible data fields of the image of the mail item identified, the analysis module600enables comparison of the identified data fields against similar reference elements, S610that have previously been identified and stored within the analysis database602. The reference elements, in text, image or audio format, can be elements that were obtained from: previous mail item images obtained from the user, previous mail item images that were obtained from users different from the current user, legitimate billing formats obtained directly from the payee or third party organizations, lists of known fraudulent payees or addresses associated with fraudulent payees, lists or data patterns of common fraudulent characteristics of text or image, etc. In certain examples, the comparison can also include a comparison of redundant identified elements from the image of the mail item for consistency. In addition to the comparison, or in lieu of the comparison, the analysis module600can perform a contextual analysis of the mail item to discern the various fields noted above as well as, for example, the use of unusual grammar, misspellings, odd fonts, odd formats, etc. via image analysis, text analysis or audio analysis of the mail item. In certain examples, the contextual analysis is performed using, for example, artificial intelligence, deep learning, cognitive computing and/or other advanced computer-based analysis techniques.

Based on the comparison, the analysis module600enables the generation of the image overlay, e.g., image overlay500, which includes indicators to indicate the validity of the data fields and/or of the overall mail item, S612and the generated image overlay500is sent to the user device11, S614, for display thereon.

Referring once again toFIG.5A, an exemplary image overlay500is illustrated. As noted above, the image overlay500provides an interactive user interface on the user device11from which the user may request the display of additional information and/or pose a typed or verbal question; the image overlay includes at least one validity indicator502indicating the validity or lack of validity of the mail item400. In the present example the image overlay500includes a plurality of validity indicators502including, but not limited to: a) user/payor name502a; (b) user/payor address502b; (c) user/payor account number502c; (d) payee name and/or payee logo502d; (e) charges to the account502e; (f) amount owed by the user/payor502f; and (g) address for remittance of payment502g. Each of the validity indicators502may be in the form of one or more of: a text indicator (e.g., “VALID,” “INVALID”, “WARNING,” etc.), a color indicator (e.g., green=valid, red=invalid, yellow=warning), an outline indicator (e.g., a first shape (rectangle)=valid, a second shape (oval)=invalid, a third shape (triangle)=warning; or line thickness/type-thin line=valid, bold line=invalid, dashed line=warning), or an audio indicator. In certain examples, there are at least two validity indicators statuses, e.g. VALID and INVALID. In certain examples, there are at least three validity indicator statuses, e.g., VALID, INVALID, WARNING. The WARNING indicator can be used as an initial indicator for various purposes. For example, the WARNING indicator can be the initial indicator notifying/advising the user of the presence terms or conditions that may adversely affect the user such as charges to the user account, excess fees, consenting to automatic withdrawals, etc. The WARNING indicator can also be the initial indicator for account health status warnings, e.g. the user account does not contain sufficient funds to pay the requested amount, etc. Selection of the WARNING indicator can provide the user with additional information related to the warning, see description below related to the interactive features of the image overlay500. Additional indicator statuses may be used as appropriate to the mail item image under analysis. In the example ofFIG.5A, both text and shape are used to indicate VALID status (e.g. rectangle and word “VALID”) and to indicate an INVALID status (e.g., oval and word “INVALID”).FIG.5Billustrates the image overlay500in place atop the image of the mail item400.

In certain examples, the image overlay500provides additional information beyond that described above with reference toFIGS.5A and5B. InFIG.5C, an example of an image overlay500is provided with an information bar510. In this example the information bar510is located at the bottom of the image overlay500be can included anywhere within the confines of the image overlay500. The information bar510can include additional information such as a legend of field descriptors512(e.g., “name”, “address”, “name of payee”, etc.), an action recommendation514(e.g., “pay” or “do not pay”), a listing of prior/historical user payments or actions made relative to the payee516by the payor/user, and/or an action button518providing the options of “PAY” or “DO NOT PAY”. In the instance that the validity app300is hosted by the user's banking services provider and the user has established themselves as a valid user of the validity app300, the action button518can be configured with access to the user's bank account. Such that a selection of “PAY” from the action button518results in remittance of the amount owed, e.g., amount412, to the payee, e.g. payee408, identified in the image of the mail item from the user's associated bank account. A selection of “DO NOT PAY” at the action button518results in non-payment. In certain examples, the action button518is customized by the analysis module600in a manner that is specific to the actions that the image of the mail item has requested as determined by the analysis module600. For example, the action button518can include an option to pay a different amount, to pay an amount to a third party, to reject or accept a binding condition, etc.

As noted above with respect to the flowchart ofFIG.3, the image overlay500provides an interactive user interface on the user device11from which the user may request the display of additional information from the application host, S310, S312, S314. More specifically, each of the validity indicators502and/or items within the information bar510may be selected by the user. In certain examples, upon selection of one of the validity indicators502or item within the information bar510, the user is presented with a pop-up menu519(see example inFIG.5A) to select from at least one, and more preferably, two or more informational options520, e.g., to display additional information520a, to enter a typed question520b, and/or to state a verbal question520c; other and/or additional informational options may be provided.

Selection of one of the informational options520from the image overlay500invokes utilization of the advisory module700by the host server18. The advisory module700operates in conjunction with an advisory database702to provide the information requested, see the flowchart ofFIG.7. In the instance of responding to the selection for display of additional information S704, the advisory module700enables the host server18to search the advisory database702for information related to the corresponding selected item (e.g. the validity indicator502or the item from the information bar510), S706, and sends the information to user device11for display and/or read-back, S708. Such additional information found in the advisory database702may include, but is not limited to: the corresponding field name of the selection, historical information about the selection, fraud alerts related to the selection, and/or user warnings related to the selection. User warnings can include, for example, notifying/advising the user of the presence terms or conditions that may adversely affect the user such as charges to the user account, excess fees, consenting to automatic withdrawals, etc. User warnings can also include account health status warnings, e.g. the user account does not contain sufficient funds to pay the requested amount.

In the instance of responding to the selection for entering a typed question, S710, the advisory module700initiates the generation of a question box, see example question box522inFIG.5A, for presentation to the user at the user device11and receives the user's typed question, S712. Upon the typed entry of the user's question, the advisory module700operates to search for an identical or similar question previously stored in the advisory database702and its corresponding response, S714; the corresponding response can then be sent, S716, to the user device11for display and/or verbal read-back by the user device11. In certain examples, alternatively or in addition to searching based on the whole question, S714, the typed question is parsed by the advisory module700for keywords that may be searched within the advisory database702to obtain corresponding response(s), S716, for display and/or verbal read-back by the user device11. In certain examples, alternatively or in addition to searching the advisory database702, the advisory module700can operate to establish a connection with an individual (e.g. a human advisory support representative), S718, who can provide a typed and/or verbal response to the question for display and/or read-back at the user device11. Further, alternatively or in addition to searching the advisory database702, the advisory module700can use advanced computing techniques, for example, artificial intelligence, deep learning, cognitive computing, etc., to determine an appropriate response to any user questions.

In the instance of response to the selection for posing a verbal question, S720, the advisory module700initiates the opening of a microphone of the user device11and directs the host server18to receive the user's verbal question, S722. As with the typed question, the advisory module700operates to search for an identical or similar question previously stored in the advisory database702and its corresponding response. S714; the corresponding response can then be sent, S716, to the user device11for display and/or verbal read-back by the user device11. In certain examples, alternatively or in addition to searching based on the whole question, S714, the typed question is parsed by the advisory module700for keywords that may be searched within the advisory database702to obtain corresponding response(s), S716, for display and/or verbal read-back by the user device11. In certain examples, alternatively or in addition to searching the advisory database702, the advisory module700can use advanced computing techniques, for example, artificial intelligence, deep learning, cognitive computing, etc., to determine an appropriate response to any user questions. In certain examples, alternatively or in addition to searching the advisory database702, the advisory module can operates establish a connection with an individual (e.g. a human advisory support representative or a designated trusted party), S718, who can provide a typed and/or verbal response to the question for display and/or read-back at the user device11. In certain examples, the user is notified at the user device11of the connection with the representative. Any combination of text and speech may be used to pose and/or respond to a user question. A user may continue to select various informational options520for the various validity indicators502and/or items within the information bar510as desired.

In certain examples, additional functionality is provided to the various systems and methods described above. For example, in addition to selecting various informational options520for the various validity indicators502and/or items within the information bar510, the validity app300can allow the user to highlight and/or zoom in on an area on the image of the mail item for specific consideration by the analysis module600and/or advisory module700. Further, the validity app300can provide the user with the functionality to move the image of the mail item and overlay onto another display device such as a connected television, smartphone, or augmented reality headset whereby the user may then utilize the user device11as the control mechanism for the display.

It should be noted that while the above-described system and method reference a mail item containing a monetary solicitation, the system and method can additionally or alternatively be applied to other types of documents such as legal documents, (e.g. purchase agreement, rental agreement, will, business contract) or financial statements (e.g., income statement, expense statement) in single page or multi-page form. The system and method of the present disclosure can essentially be applied to any document about which a user may wish to pose a question. As such the action button, e.g. action button518, can be configured to initiate different types of actions based on the type of document. For example, the action button can be configured to enable an electronic signature.

The various embodiments described above are provided by way of illustration only and should not be construed to limit the claims attached hereto. Those skilled in the art will readily recognize various modifications and changes that may be made without following the example embodiments and applications illustrated and described herein, and without departing from the true spirit and scope of the following claims.