Patent ID: 12243068

DETAILED DESCRIPTION

The present disclosure is directed to systems and methods for using augmented reality (AR) enabled gamification to engage customers of a financial institution or a retail store to learn more about products and services offered by the financial institution or retail store. Documents, advertisements and signage in the financial institution or retail store may be scanned by an AR device to launch interactive content that, once reviewed, may give the customer a reward, such as points or coins. The interactive content may be specifically tailored to a customer to provide information on products and services for which the customer may be interested. In this disclosure, the systems and methods are described for a bank. However, the systems and methods may also be applied to other financial institutions or retail stores. In addition, the systems and method may be used to view interactive content during a tour of a bank museum or other museum.

Gamification as used in this disclosure refers to an interactive session between a user of an AR device and a server computer in which rewards are offered to the user for completing specific activities related to products and services offered by the financial institution or retail store. Both the rewards and the specific activities are tailored to a profile of the user. The profile is based on an identity of the user and includes such items as a history of previous purchases and uses of the products and services. The specific activities may include activities such as watching a video, filling out a form, meeting with a bank employee such as a teller or a loan officer, choosing to use a specific bank service such as bill pay or paperless statements and taking a quiz at the end of a video. Other aspects of the profile of the user and other activities are possible. Gamification may also include game like features such as responding to clues and getting reward points if a correct response is made or an activity is performed as a result of a clue.

The systems and methods are generally described for an AR device that may be worn or carried by a user. An example of and AR device that may be worn by the user is an AR headset. An example of an AR device that may be carried by the user is a smartphone or tablet computer that includes AR components such as a processor, display and camera and an AR software application.

The AR headset includes a wearable computer, a camera and an optical display. The wearable computer includes a wireless telecommunication capability, permitting a wireless connection between the wearable computer and one or more server computers. The wearable computer also includes voice recognition capability, permitting the user to direct the wearable computer via voice commands. In addition, in some implementations, the wearable computer also includes biometric capability such as facial recognition, retinal scan capability, finger print and voice print capability.

One type of AR headset described in this disclosure is a smart glass type of headset, similar to eyeglasses, which may be worn by the user. The user may view AR images in the glass portion of the headset. An example of a smart glass headset is Google Glass, from Google Inc. of Mountain View, CA.

In an example scenario involving AR-enabled gamification, when a customer or potential customer walks into the bank, the customer is identified, as explained later herein. The customer may be wearing an AR headset or the customer may possess another AR device such as an AR-enabled smartphone or tablet computer. While in the bank, the customer may go up to an object in the bank from which an interactive game may be activated via the AR device. For example, the customer may view an advertisement or signage in the bank or the customer may view a document that may include information about the interactive game.

When the customer views the object with the AR headset, the customer looks at the object using the AR headset. When the customer views the object with the AR-enabled smartphone or tablet computer, the customer uses the camera in the AR-enabled smartphone or tablet computer to view the object.

Viewing the object with the AR-enabled device may initiate the interactive game. In some implementations, the object may include a quick response (QR) code. In other implementations the AR-enabled device may be configured to recognize certain language or imagery in the object and initiate the interactive game based on viewing this language or imagery. Initiating the interactive game is discussed in more detail later herein.

When the customer has an account with the bank, the customer may be directed via the interactive game to perform an activity pertinent to the customer. For example, because the identity of the customer is known, the customer may be directed to watch a video describing available home mortgage options or other banking products to which the customer may be interested. However, when it is known from the customer's banking history that the customer would not likely be interested in a mortgage, because for example, the customer has recently obtained a mortgage, the video may be based on something else, specifically tailored to the customer.

When participating in the interactive game, customers may be permitted to opt-in to additional services, for example to sign up for bill pay, to schedule appointments, for example to schedule an appointment with a loan officer and to perform other activities such as filling out an application for a loan or a mortgage.

In order to induce the customer to perform a specific activity, the interactive game may indicate to the customer that a reward may be obtained for performing the specific activity. Rewards may include such items as banking merchandise, virtual coins or reward points. Other rewards are possible.

When the customer does not have an account with the bank and the customer views the object with the AR-enabled device, the customer may be directed to a website from which the customer can download a bank software application. The customer may apply for a bank account using the software application. As an inducement for downloading the bank software application and for opening a new account, the customer may be offered a reward, such as banking merchandise, virtual coins, reward points, or something else.

Rewards may be cashed in for merchandise or for entertainment such as movies and baseball games. In addition, rewards may be applied to other reward programs, for example to obtain double rewards. Rewards may also be tailored to a specific customer. For example, because the identity of the customer is known, the customer may be offered services and products in which, based on a customer profile and history, the customer may be interested.

The AR-enabled device includes a financial software application that supports gamification. The software application may synchronize with bank gift stores and reward programs in real-time to register and track rewards awarded to the customer during the interactive game.

In some implementations the AR-enabled gamification may be implemented via a simulated reality facility, similar to a holodeck. The simulated reality facility is configured to generate an augmented reality scene. Using the simulated reality facility the customer may perform specific activities such as taking a virtual tour of a banking facility, watching videos describing banking products, trying out new banking services, etc. The simulated reality facility may be implemented in various locations, such as in a bank or in the customer's home. For example, the bank may have a special room with equipment that may support the simulated reality facility. In addition, the customer may own or have access to equipment that may support the simulated reality facility.

FIG.1shows an example system100that supports AR-enabled gamification. The example system100includes an AR device102, a server computer104, a database106, an activating object108and a radio frequency identification (RFID) device.

AR device102may be an AR headset device, such as Google Glass or a smartphone or tablet computer with an AR software application. The AR device102may be used to initiate the interactive game and also to display information relating to the interactive game. For example messages, videos and other information related to the interactive game may be viewed on AR device102.

Server computer104is a server computer associated with the bank. A wireless connection may be established between AR device102and server computer104. Server computer104includes or has access to a history and profile for the customer. When the interactive game is initiated, a software application on server computer104determines a content of information to be displayed on AR device102. The content is tailored to an identity of the customer. The content may include such things as a particular video to be displayed on AR device102, a particular direction, such as a suggestion to see a particular bank employee, and/or particular rewards that may be tailored to the customer. Other content is possible. The server computer104may actually be one or a plurality of server computers, located individually or on one or more server farms.

In some implementations, one or more of the history and profile for the customer and the content of one or more interactive games may be stored on database106. In other implementations, the history, profile and content may be stored on server computer104. When stored on database106, the history, profile and content are accessible over a network to server computer104. The network may be an internal network or it may be the Internet.

Activating object108is an object in the bank from which the interactive game may be initiated via AR device102. As discussed, in some implementations the activating object may be signage, such as a wall poster, advertisements or documents. Other activating objects are possible. In some implementations the interactive game may be initiated by viewing a QR code on activating object108with AR device102. In other implementations, AR device may be configured to initiate the interactive game when viewing specific words, phrases, pictures or symbols on activating object108.

As discussed, when the interactive game is initiated, a directive may be displayed on AR device102for the customer to see a bank employee, for example a loan officer. In typical implementations, many bank employees wear a security badge having an RFID device110. RFID device110is used to locate the bank employee, if the bank employee is in the bank. RFID device110may include a unique identifier that corresponds to an identification number of the bank employee.

In an example implementation, when a bank employee is to be located, server computer104initiates a scan of the bank. As a result of the scan, RFID devices in the bank respond with their unique identifiers. In addition, based on a direction and a time for a response to the scan, server computer104determines where the bank employee is located. In some implementations, the position of the bank employee is displayed on a display screen of AR device102.

FIG.2shows example modules200of AR device102. The example modules200include a camera202, a display204, electronics206and GPS software208. The camera202is a camera that is built-in to AR device102. Display204is a display that is also built-in to AR device102. Electronics206includes an electronic computing device with wireless capability. The wireless capability may include Bluetooth, radio frequency identification (RFID), mobile telecommunications, Wi-Fi, or similar types of wireless capability. GPS software208is typically a software application that may be installed on the mobile electronic device. When AR device102is an AR headset, camera202, display,204, electronics206and GPS software208are part of the AR headset.

FIG.3shows an example activating object108. In this example, activating object108is signage, for example a wall poster, in the bank. The poster includes text302inquiring of a person whether the person wants to learn more about the bank's products and services and offering a reward if the person agrees. The text302also instructs the person to scan QR code304.

When the person scans QR code304and the person is an existing bank customer, information tailored to the person is displayed on AR device102. The information is based on an identity and customer profile of the customer. The information may be in the form of a video, text, photos or a directive for the person to see a bank employee. Other types of information may be displayed to the customer.

When the person scans QR code304and the person is not an existing bank customer, different type of information may be displayed to the customer. For example, the person may be asked to view a video providing an overview of products and services offered by the bank. The information may also include a link for the person to download a financial software application offered by the bank. The financial software application may be downloaded and installed on the person's smartphone or tablet computer.

As stated earlier herein, the gamification discussed is specifically tailored to a person, either a customer or a potential customer. When the person is a customer, the gamification is based on a profile and history of the customer. When the person is not a customer, the gamification is oriented toward introducing the bank's products and services to the person to encourage the person to become a customer.

In order to tailor the gamification to the person, either the customer or potential customer, an identity of the person needs to be determined. Using the systems and methods of this disclosure, the identity of the person may be determined by one of several methods.

In one method, when the person is wearing an AR headset, the AR headset may synchronize with a computer system in the bank, for example with server computer104. One way in which the AR headset may synchronize with the computer system in the bank is via a Wi-Fi connection.

In some implementations, when the person enters the bank and the bank has a Wi-Fi system, a notification may be displayed on the AR headset indicating that a Wi-Fi connection is available. When the customer accepts the connection, either via a voice command or via touching a selection area on the AR headset, a wireless connection is made between the AR headset and a server computer in the bank, for example server computer104. Once the wireless connection is made, the person may be prompted for identification information. Example identification information may include a bank account number, a unique identifier for the person, a login and password or other types of information.

In other implementations, the person may initiate a wireless connection to server computer104when viewing signage in the bank. For example, the signage may include a URL for the bank that the person can access to establish the wireless connection. Once the wireless connection is established, the person may send identification information to server computer104over the wireless connection.

Another way in which the AR headset may synchronize with the computer system in the bank is via a near-field communication (NFC) or similar type of connection. The AR headset typically includes support for NFC. When two NFC compatible devices are close to each other, a communication session may be established between the two devices. When one of the two devices is a device that may be connected to or in communication with server computer104, when the communication session is established identification information from AR device102may be transmitted to server computer104. For example, the person with AR device102may walk up to an automated teller machine (ATM). The ATM may include an NFC compatible device, such that when AR device102comes into close contact with the ATM, the communication session is established. Once the communication session is established, identification information from AR device102may be sent to server computer104.

In a second method, facial recognition may be used to identify the person wearing the AR headset. In this second method, a camera in the bank may take a photo of the person as the person enters the bank. For example, the camera may be one of several security cameras in the bank. The photo may be sent to server computer104or another server computer that includes facial recognition software. The facial recognition software may compare the photo with images of bank customers, determine whether with the person with AR device102is a bank customer, and if so, identify the customer.

In a third method, the person may be instructed to log in to a server computer using a terminal or other log-in device in the bank. For example, the person may be prompted to log in via a message on AR device102when the person establishes the communication session with server computer104. The identity of the person is obtained from the login.

In addition to the gamification being uniquely tailored to the customer, the gamification may also be uniquely tailored to a geographical location of the customer. AR device102and other similar AR devices typically include global positioning system (GPS) software by which the geographical location of the AR device can be determined. For example, if AR device102is determined to be in a southern location, for example Florida, a video showing homes in Florida may be shown to the customer as part of the game. Similarly, if AR device102is determined to be in a northern location, for example Minnesota, a video showing homes having a Minnesota style may be shown to the customer as part of the game. The geographical location of the customer may also be determined based on contextual images received from the camera on the AR device. For example, the geographical location of the customer may be determined based on buildings, landmarks, signage, logos or other contextual information received by the AR device camera.

Using the GPS system, rewards offered as part of the game may also be tailored by geographic location. For example, one reward offered to a customer in Florida may be tickets to a Tampa Bay Rays baseball game, whereas one reward offered to a customer in Minnesota may be tickets to a Minnesota Twins baseball game.

FIG.4shows a flowchart of a method400implemented on a server computer for activating an interactive game on an AR device, for example AR device102. The method400is typically implemented in a financial institution such as a bank or in a retail store. For method400, the AR device is an AR headset, such as Google Glass. In other methods, the AR device may be a smartphone or tablet computer having an AR software application.

At operation402, a user of the AR device is identified. The user may be identified when the user enters the bank or retail store. One way in which the user may be identified, is by synchronizing the user's AR device with a software application of a computer system, for example server computer104, at the bank or retail store. In some implementations, messages may be broadcast from server computer104that may be intercepted by AR device102and may cause AR device102to establish a wireless connection with server computer104. When the wireless connection is established, identification information for the user may be sent from AR device102to server computer104.

At operation404, when a determination is made that the user is a customer of the bank or retail store, a profile for the customer is obtained based on the identity of the customer. The profile may include such things as the customer's purchasing history, products or services that the customer has purchased or used, and videos related to products and services offered by the bank or retail store that were previously viewed by the customer.

At operation406, an interactive game is activated on AR device102. The interactive game is sent from server computer104to AR device102via the wireless connection. The interactive game is designed to introduce the customer to products and services that the customer of which the customer may not be aware or may not be using. For example, for a customer of a bank, the interactive game may promote such banking services as bill pay and paperless billing. The interactive game is designed to obtain input from the customer and to provide videos and other information to AR device102based on the input. For example, if the customer indicates that he/she is interested in bill pay, a video on bill pay may be played on AR device102. As another example, if the customer indicates he/she is interested in a loan, server computer104may send information to AR device102regarding a name and location of a loan officer in the bank.

At operation408, the customer receives rewards for playing the interactive game. For example, the customer may receive a reward for watching a video, for talking to a loan officer and for filling out a loan application. The rewards may include such items as merchandise, cash, virtual coins and reward points. The virtual coins and reward points may be accumulated and redeemed for merchandise or cash. Other types of rewards are possible.

FIG.5shows a flowchart of a method500implemented on an AR device, for example on AR device102, for activating an interactive game on AR device102.

At operation502, a customer of a bank or retail store may use AR device102to view signage in the bank or retail store. The signage may be an item such as a wall poster or an advertisement. In some implementations, the signage may be displayed on a display screen of an ATM. The signage may include words, phrases or imagery that are recognized by a software application on AR device102. The signage may also include a QR code.

At operation504, a wireless connection is established between AR device102and a server computer, for example server computer104. The wireless connection is established when the words, phrases, imagery or QR code are recognized by the software application.

At operation506, an interactive game is received at AR device102from server computer104. The interactive game is oriented to the customer and is based on a profile of the customer.

At operation508, the interactive game is activated on AR device102.

As illustrated in the example ofFIG.6, the server computer104includes at least one central processing unit (“CPU”)602, a system memory608, and a system bus622that couples the system memory608to the CPU602. The system memory908includes a random access memory (“RAM”)610and a read-only memory (“ROM”)612. A basic input/output system that contains the basic routines that help to transfer information between elements within the server computer104, such as during startup, is stored in the ROM612. The server computer104further includes a mass storage device614. The mass storage device614is able to store software instructions and data. A central processing unit, system memory and mass storage device similar to that inFIG.6are also included in AR device102.

The mass storage device614is connected to the CPU602through a mass storage controller (not shown) connected to the system bus622. The mass storage device614and its associated computer-readable data storage media provide non-volatile, non-transitory storage for the server computer104. Although the description of computer-readable data storage media contained herein refers to a mass storage device, such as a hard disk or solid state disk, it should be appreciated by those skilled in the art that computer-readable data storage media can be any available non-transitory, physical device or article of manufacture from which the central display station can read data and/or instructions.

Computer-readable data storage media include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable software instructions, data structures, program modules or other data. Example types of computer-readable data storage media include, but are not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory technology, CD-ROMs, digital versatile discs (“DVDs”), other optical storage media, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the server computer104.

According to various embodiments of the invention, the server computer104may operate in a networked environment using logical connections to remote network devices through the network620, such as a wireless network, the Internet, or another type of network. The server computer104may connect to the network620through a network interface unit604connected to the system bus622. It should be appreciated that the network interface unit604may also be utilized to connect to other types of networks and remote computing systems. The server computer104also includes an input/output controller606for receiving and processing input from a number of other devices, including a touch user interface display screen, or another type of input device. Similarly, the input/output controller606may provide output to a touch user interface display screen or other type of output device.

As mentioned briefly above, the mass storage device614and the RAM610of the server computer104can store software instructions and data. The software instructions include an operating system618suitable for controlling the operation of the server computer104. The mass storage device614and/or the RAM610also store software instructions, that when executed by the CPU602, cause the server computer104to provide the functionality of the server computer104discussed in this document. For example, the mass storage device614and/or the RAM610can store software instructions that, when executed by the CPU602, cause the server computer104to display received financial data on the display screen of the server computer104.

Although various embodiments are described herein, those of ordinary skill in the art will understand that many modifications may be made thereto within the scope of the present disclosure. Accordingly, it is not intended that the scope of the disclosure in any way be limited by the examples provided.