Augmented reality security applications

A method implemented on an augmented reality electronic device includes an augmented reality electronic device. A wireless connection is established between the augmented reality device and an automated teller machine (ATM). A user of the augmented reality electronic device is authenticated. After the wireless connection is established and after the user is authenticated, a financial transaction is initiated. Financial information is received from the ATM related to the financial transaction. The financial information is displayed on a display screen of the augmented reality electronic device.

BACKGROUND

Augmented reality is a technology in which a person's conception of reality can be enhanced, typically through augmented sound, video or graphics displays. The augmentation is typically implemented via various technologies, such as a headset that may be worn by the person. One or more augmented views may be presented to the person through the headset.

The augmented reality headset typically includes a wearable computer and an optical display mounted to the headset. The wearable computer may include a wireless telecommunication capability, permitting a wireless connection from the wearable computer to a server computer. Because of the wireless telecommunication capability, the augmented reality headset may be used to interact with the server computer to accomplish various tasks.

SUMMARY

Embodiments of the disclosure are directed to a method implemented on an augmented reality electronic device comprising: establishing a wireless connection between the augmented reality device and an automated teller machine (ATM); authenticating a user of the augmented reality electronic device; after the wireless connection is established and after the user is authenticated, initiating a financial transaction; receiving financial information from the ATM related to the financial transaction; and displaying the financial information on a display screen of the augmented reality electronic device.

In another aspect, a method implemented on an augmented reality electronic device comprises: synchronizing the augmented reality electronic device with a security system, the security system including a server computer; receiving an image of a person at the augmented reality electronic device, the image being obtained using a camera in the augmented reality electronic device; sending the image to the server computer; receiving information from the server computer that the person is one of interest; and displaying a message that the person is one of interest on the augmented reality electronic device.

In yet another aspect, an augmented reality electronic computing device comprises: a processing unit; and system memory, the system memory including instructions which, when executed by the processing unit, cause the augmented reality electronic computing device to: establish a wireless connection between the augmented reality device and an ATM; authenticate a user of the augmented reality electronic device; after the wireless connection is established and after the user is authenticated, initiate a financial transaction; receive financial information from the ATM related to the financial transaction; and display the financial information on a display screen of the augmented reality electronic device.

DETAILED DESCRIPTION

The present disclosure is directed to systems and methods for using augmented reality (AR) to help conduct secure financial transactions, particularly at an automated teller machine (ATM) and for using AR in a banking or retail environment to alert security personnel of sightings of “wanted” individuals. The “wanted” individuals may be known bank or ATM robbers or may be individuals of high risk to a banking or retail organization.

In an example implementation, a user may conduct a financial transaction at an ATM machine without having financial information displayed on an ATM display screen. Instead, financial information regarding the financial transaction is displayed on a display screen of the AR device. By displaying financial information on the display screen of the AR device and not on the ATM display screen, the financial transaction may be more secure. For example, other individuals who may be near the ATM display screen, for example people waiting in line to use the ATM, are not able to see the financial information of the financial transaction. Only the user of the AR device is able to see the financial information.

In another implementation relating to identifying wanted or undesirable individuals, when a person wearing an AR device enters a bank or retail store, the AR device may synchronize with a bank security system or a retail store security system. As the person walks around in the bank or retail store, images of individuals viewed via a camera in the AR device may be sent to the bank security system or retail store security system. The images may be compared at the bank security system or retail store security system with images of wanted or undesirable individuals. When a match is found, security personnel in the bank or retail store may be alerted.

In yet another implementation, a greeter or other employee of a bank or retail store may use an AR device to identify wanted or undesirable individuals. For example, if the greeter is wearing an AR device, as the greeter greets people entering the bank or retail store, an image of the person being greeted is sent to the bank security system or retail store security system. The bank security system or retail store security system may determine, for example, that the individual is wanted for a crime such as bank robbery or that the individual has a record of stealing items from the retail store. The bank security system or retail store security system may issue an alert that may be displayed on a display screen of the AR device. The alert may inform the greeter that there is a wanted individual in the bank or retail store and may display instructions as how to proceed, for example by activating an alarm or calling911.

The systems and methods are generally described for an AR headset that may be worn by a user. The AR headset includes a wearable computer, a camera and an optical display. The wearable computer includes a wireless telecommunication capability, permitting a wireless connection between the wearable computer and one or more server computers. The wearable computer also includes voice recognition capability, permitting the user to direct the wearable computer via voice commands. In addition, in some implementations, the wearable computer also includes biometric capability such as facial recognition, retinal scan capability, finger print and voice print capability. The biometric capability permits biometric authentication of the user, as described in more detail later herein.

One type of AR headset described in this disclosure is a smart glass type of headset, similar to eyeglasses, which may be worn by the user. The user may view AR images in the glass portion of the headset. An example of a smart glass headset is Google Glass, from Google Inc. of Mountain View, Calif.

The systems and methods are also described for a mobile electronic device, such as a smart telephone or laptop computer, having an AR software application. The smart telephone and laptop computer each includes a camera that may be used to view individuals in a bank or retail organization. The AR software application may permit an initiation of a financial transaction at an ATM machine using voice commands. The AR software application may also provide one or more methods of biometric authentication of the user of the mobile electronic device. In addition, financial information related to the financial transaction may be overlaid on a display screen of the mobile electronic device.

FIG. 1shows an example system100for which AR may be used to enhance security applications. The example system100includes a server computer102, an AR headset104, an ATM112and a server computer120. The example AR headset104includes a headset camera106, headset electronics108and a headset display110. The ATM112is a typically located inside a bank or near a bank. However, the ATM may also be located elsewhere, for example in a shopping mall, an airport, etc.

The example server computer102is typically a server computer at a bank or banking organization. A wireless connection122may be established between the AR headset104and the server computer102. Information may be exchanged via a wireless connection122between the AR headset104and the server computer102. One type of information that may be exchanged between the AR headset104and the server computer102is biometric authentication information that may be used to authenticate a user of the AR headset104, as explained in more detail later herein.

The server computer120is also typically a server computer at a bank or banking organization. In some implementations, server computer102and server computer120are the same server computer. In other implementations, server computer102and server computer120are different server computers. The server computer120is accessible from the ATM112and processes transactions at the ATM112. Information relating to a financial transaction from the ATM112may be transmitted to server computer120. Financial information generated by server computer120may be transmitted to the ATM112and then from the ATM112to AR headset104.

The example AR headset104includes a headset camera106, headset electronics108and a headset display110. The example headset camera106is a camera that is typically mounted to the headset such that a lens of the camera has a same orientation as the user. When the user looks at an object or a scene, the camera is positioned to record or transmit what the user sees. The headset electronics108includes an electronic computing device with wireless capability. The wireless capability may include Bluetooth or a similar type of wireless capability. The headset electronics108may also include optical character recognition (OCR).

The ATM112includes an ATM display screen114, an ATM keyboard116and ATM electronics118. The ATM electronics118also includes an electronic computing device with wireless capability. The wireless capability may include Bluetooth or a similar type of wireless capability.

In an example implementation, a user wearing the AR headset104approaches the ATM112. When the user comes within a predetermined distance of the ATM112, for example several feet, a wireless connection124is initiated between the AR headset104and the ATM112. The wireless connection may be initiated via a Bluetooth communication protocol or via a similar type of wireless communication protocol.

When the ATM112is not being used by another individual, the user initiates a financial transaction at the ATM112. In some implementations, the user may initiate the financial transaction by inserting an ATM card into a card reader of the ATM112. In this implementation, the user typically enters a personal identification number (PIN) via ATM keyboard116. The PIN authenticates the user at the ATM112. After the user is authenticated, the user may select a transaction, for example withdrawing money from the user's checking account, via the ATM keyboard116. However, in this implementation, some or all messages regarding the financial transaction that may usually be displayed on the ATM display screen114are instead displayed on the headset display110.

For example, financial information such as the user's account balance is displayed on headset display110. In some implementations, the ATM display screen114is blank when the financial information is displayed on the headset display110. In other implementations, the ATM display screen114displays other information, such as marketing information, instead of the user's financial information. In this way, only the user can see the financial information (via the headset display110). Other people near the ATM, such as people waiting in line, are not able to see the financial information.

In another example implementation, when the ATM112is not being used by another individual, the user may be authenticated by a biometric authentication method such as a facial scan, retinal scan, finger print scan or voice print. When the user is authenticated by the biometric authentication method, the user no longer needs to enter a PIN via ATM keyboard116in order to be authenticated. However, in order to give the user a sense of security regarding use of the ATM112, the user is typically asked to enter a security code into the ATM112. The security code is sent from server computer120and displayed on headset display110after the user is authenticated. In this implementation, after the user enters the displayed security code into ATM112and the security code is validated at the server computer120, the user is enabled to initiate a financial transaction. For example, a list of available financial transactions may be displayed on the headset display110. In addition, because this implementation uses biometric authentication and a security code, an actual ATM card may not need to be used.

In some situations, a user may approach the ATM112and find that the ATM is currently being used by another person. In some situations, there may also be a line of people waiting ahead of the user. In these situations, when the user is wearing the AR headset104and the user is within the predetermined distance to the ATM112, the wireless connection124may still be established between the AR headset104and server computer120. The user may be still be authenticated using a biometric authentication method and a security code may still be displayed on the headset display110of the user, but the user is not able to complete an execution of a financial transaction at the ATM112. In some implementations, the user may be able to utter the security code to fully authenticate the user. Once the user is fully authenticated, the user may be able to initiate, but not fully execute financial transactions. Instead, the user may be able to initiate one or more transactions and have the transactions queued at the server computer120until such time as the ATM112becomes available. In this way, the user can complete the transactions faster at the ATM112when the ATM112becomes available.

FIG. 2shows another example system200for which AR may be used to enhance security applications. For the example system200, a mobile electronic computing device with an AR software application is used instead of an AR device. The mobile electronic device is typically a smart telephone or a tablet computer.

The example system200includes server computer102, mobile electronic device202, ATM112and server computer120. The mobile electronic device202is typically a smart telephone or a tablet computer. The mobile electronic device202includes camera204, electronics206and display208. The electronics206includes an electronic computing device with wireless capability. The wireless capability may include Bluetooth or a similar type of wireless capability.

In system200, a wireless connection210may be established between the mobile electronic device202and server computer102. Another wireless connection212may be established between the mobile electronic device202and the ATM112. The wireless connection210may be used to authenticate the user of the AR headset via a biometric authentication method. The wireless connection212may be used to transmit financial transaction information between the mobile electronic device202and the ATM112.

In an example implementation using system200, when a user with mobile electronic device202comes within a predetermined distance from the ATM112and the user has initiated a mobile banking application on the mobile electronic device202, the wireless connection212is established between the mobile electronic device202and the ATM112.

At some point either before the wireless connection212is fully established or after the wireless connection212is established, the user is authenticated. In some implementations the user is authenticated via a biometric authentication method. In these implementations biometric authentication typically is achieved via sending biometric information to server computer102and obtaining an indication of biometric authentication from server computer102. The biometric information may be obtained, for example, by using the camera204to focus on the face of the user. A facial scan or retinal scan may then be performed on the image of the face using an AR software application on the mobile electronic device202. Server computer102compares information regarding the facial scan or retinal scan with facial and retinal information previous stored on server computer102or accessible from server computer102. When a match occurs, server computer102generates an indication of biometric authentication.

The indication of biometric authentication is then sent to server computer120via wireless connection212and ATM112. In some implementations, biometric authentication may be performed via server computer120instead of via server computer102. In other implementations, authentication is implemented by the user entering a PIN into ATM112.

When server computer120receives the financial transaction information from the ATM112, server computer120processes the transaction. Server computer120may then send additional financial information related to the financial transaction to ATM112. The additional financial information may include such items as a confirmation number for the transaction, an account balance, etc. In the example implementation using system200, when ATM112receives the additional financial information, ATM112sends the additional financial information via wireless connection212to mobile electronic device202. The additional financial information is displayed on display208instead of on ATM display screen114.

FIG. 3shows an example system300for which AR may be used to identify wanted or undesirable individuals in a bank. The system300may also be used in a retail store in addition to a bank.

The example system300includes a bank security system302, an AR headset304and banking environment312. The bank security system302comprises one or more electronic devices in the bank or accessible from the bank. The electronic devices may include one or more computing devices including server computers. The AR headset304includes a headset camera306, headset electronics308and a headset display310. The banking environment312includes individuals that may be in a particular bank. The individuals may include banking customers, bank employees and any other people that may be in the bank.

When a customer wearing the AR headset304enters the bank, the AR headset304is synchronized with the bank security system302. When the customer moves around the bank, the headset camera306sends images of what the customer sees to the bank security system302. The bank security system302compares the images to stored images of known wanted or undesirable people. For example, the wanted or undesirable people may include known criminals, including bank robbers, people otherwise wanted by law enforcement organizations, people with a history of causing problems at the bank or other locations, etc. The images may be stored at the bank security system302or on a data store accessible from the bank security system302.

When a determination is made at the bank security system302that an undesirable person is in the bank, the bank security system may send an alert to the AR headset304. The alert is displayed on headset display310.

FIG. 4shows an example alert400that may be displayed on AR headset304. The alert400indicates that there is a suspect in the XYZ Bank and that the suspect committed robberies on Oct. 23, 1996. The alert also displays example instructions for the user of the AR headset304. As shown inFIG. 4, the user is instructed to stay calm, activate a hold-up alarm, call911and call a security response center. Other example alerts are possible.

FIG. 5shows a system500, similar to system300that may be used in a retail environment instead of in a bank. The system500includes a retail store security system502instead of bank security system302and a retail environment512instead of the banking environment312. For example, a customer wearing AR headset504in a retail store may send images of people in the retail store to the retail store security system502. The retail store security system502compares the images to images of known wanted persons stored at the security system. When a match occurs, the retail store security system sends an alert to the customer's AR headset504. The retail store security system may take other action, such as notifying retail store security personnel or notifying law enforcement.

The system500may also be used by a greeter in a retail store to issue security alerts. For example, the greeter may wear AR headset504. When the greeter greets an individual entering the retail store, camera506of the AR headset504may scan the face of the individual. The scanned image of the individual's face may then be sent to the retail store security system502. The retail store security system502compares the image with stored images of known wanted persons or persons that the retail store considers as undesirable. For example, an undesirable person may be a person having a history of disruptive behavior in the retail store, including such behaviors as theft of merchandise and rowdiness.

When the retail store security system502determines that a wanted or undesirable person is in the retail store, the retail store security system may issue an alert. The alert may be displayed on the headset display510of the AR headset504to alert the greeter that there is a wanted or undesirable person in the retail store. The alert may also be sent to retail store security personnel and law enforcement.

FIG. 6shows a flowchart of a method600for displaying ATM transaction information on an AR headset. The AR headset104is used for this method.

At operation602, the AR headset104is moved to a close proximity of an ATM, in this method ATM112. Moving the AR headset104to close proximity of ATM112comprises moving the AR headset104to within a predetermined distance of ATM112. The predetermined distance is a distance close enough so that a Bluetooth communication protocol is activated on the AR headset104and on the ATM112. In method600, the predetermined distance is 10 feet, although in other implementations a range of up to about 10 meters (about 32 feet) may be used.

At operation604, a wireless connection is established between the AR headset104and the ATM112. For method600, a Bluetooth connection is established. Because the ATM112has a connection to a server computer, for example server computer120, once the wireless connection is established between AR headset104and ATM112, information can be transmitted between AR headset104and server computer120.

At operation606, the user of AR headset104is authenticated. In the example method600, the authentication is implemented by a biometric authentication method such as facial recognition, retinal scan, finger print scan or voice print. In other methods, a manual authentication method such as a PIN may be used. For the method600, the authentication is implemented via a wireless connection between AR headset104and server computer102. In this implementation, server computer102includes AR software that permits biometric authentication. In this implementation, server computer120is dedicated to serving one or more ATM machines and does not include support biometric authentication.

At operation608, a menu of ATM choices is displayed on the AR headset. The menu of ATM choices is similar to a menu that may be displayed on the ATM display screen114. For example, the menu may display choices such as receive cash or make a deposit, etc.

At operation610, a financial transaction is initiated at the AR headset. The financial transaction is typically a common ATM transaction such as receiving cash or depositing a check. The transaction may be initiated by selecting from the menu of choices displayed on the AR headset or by uttering the name of a transaction item—such as “receive cash” or “deposit check.” In other implementations, the user may use a keypad on the ATM112to initiate the financial transaction.

At operation612, the transaction information is sent to server computer120. For example, if the user decides to receive cash in an amount of $100, a message containing information related to a cash withdrawal of $100 from the user's checking account is sent to server computer120. When the user initiates the financial transaction from the AR headset104, the information is sent from the AR headset104via the Bluetooth connection to the ATM112and then from ATM112to server computer120. When the financial transaction is initiated from the ATM112, the information is sent from ATM112to server computer120.

At operation614, financial information related to the financial transaction is received at the AR headset104. The financial information is sent by server computer120, received at ATM112and sent via the Bluetooth connection to ATM112. The financial information may include such items as a confirmation that the financial transaction has been completed, an account balance, a receipt, etc. Other types of financial information are possible.

At operation616, the financial information is displayed on headset display110of the AR headset104.

FIG. 7shows a flowchart of a method for the operation606for authenticating the user. At operation702, biometric information is obtained from the user. Typically, the biometric information is obtained from one of facial recognition, retinal scan, voice print or finger print scan. Facial recognition comprises scanning the user's face with the headset camera106and extracting specific facial features from the scanned image of the user's face. Retinal scan uses unique patterns on the user's iris in order to identify the user. Voice print obtains a spectrogram of the user's voice, for example when the user utters a command. Finger print scan uses finger print recognition to identify the user. In some implementations, finger print recognition hardware and software are built into the AR device. For example, when the user touches the headset display110, the user's finger print may be scanned.

At operation704, the biometric information for the user obtained at operation702is compared with previously obtained biometric information for the user. Typically, when the user is assigned an AR device, a biometric profile is compiled for the user and stored on the AR device. The profile may include one or more of a facial profile, a retinal profile, a voice print and a finger print.

At operation706, a determination is made as to whether the biometric information obtained at operation702matches the previously obtained biometric information for the user. When a determination is made that there is a match, at operation1008, the user is designated as being authenticated.

When a determination is made that there is not a match, at operation710, a message is displayed on the headset display110indicating that there is an authentication failure.

FIG. 8shows a flowchart of a method800for displaying a security alert on an AR device. For this method, the AR device is the AR headset104. The method800may apply to a variety of environments, including a banking environment and a retail environment.

In method800, a user who is wearing the AR headset104is in an environment such as a bank or retail store with customers of the bank or retail store. The bank or retail store has a security system, such as bank security system302or retail store security system502. At operation802, the AR device is synchronized with the bank security system302, the retail store security system502or another security system, as the case may be.

The user may be a customer or employee at a bank, a greeter in a retail store, an employee at the retail store, etc. As the user walks around the bank or retail store, headset camera106scans images of people in the bank or retail store with whom the user comes in contact. At operation804, the user receives a scanned image of a person at the AR headset104.

At operation806, the image is sent to a server computer, for example server computer102. Server computer102is a server computer, typically at a financial institution or retail business. The AR headset has access to server computer102via a wireless connection.

At operation808, information is received from the server computer102that the person whose image was sent to server computer102is a person of interest. The person of interest is typically a person that is wanted by the bank or retail institution or one who is considered undesirable by the bank or retail institution. For example, the person may be a known criminal, such as a bank robber or a person that has previously caused a problem at the retail institution. The server computer102determines that the person is one of interest by comparing the scanned image of the person with images of people who are wanted or undesirable. The images being compared are typically compiled by the bank, the retail store or law enforcement agencies.

At operation810, an alert is displayed on the AR headset104. The alert informs the user that the person is a person of interest and typically provides a message as to why the person may be wanted by the bank or retail institution. The alert may also provide instructions to the user, such as remain calm, call911, etc.

As illustrated in the example ofFIG. 9, the AR headset104includes at least one central processing unit (“CPU”)902, a system memory908, and a system bus922that couples the system memory908to the CPU902. The system memory908includes a random access memory (“RAM”)910and a read-only memory (“ROM”)912. A basic input/output system that contains the basic routines that help to transfer information between elements within the AR headset104, such as during startup, is stored in the ROM912. The AR headset104further includes a mass storage device914. The mass storage device914is able to store software instructions and data. A central processing unit, system memory and mass storage device similar to that inFIG. 9are also included in server computer102.

The mass storage device914is connected to the CPU902through a mass storage controller (not shown) connected to the system bus922. The mass storage device914and its associated computer-readable data storage media provide non-volatile, non-transitory storage for the AR headset104. Although the description of computer-readable data storage media contained herein refers to a mass storage device, such as a hard disk or solid state disk, it should be appreciated by those skilled in the art that computer-readable data storage media can be any available non-transitory, physical device or article of manufacture from which the central display station can read data and/or instructions.

According to various embodiments of the invention, the AR headset104may operate in a networked environment using logical connections to remote network devices through the network920, such as a wireless network, the Internet, or another type of network. The AR headset104may connect to the network920through a network interface unit904connected to the system bus922. It should be appreciated that the network interface unit1404may also be utilized to connect to other types of networks and remote computing systems. The AR headset104also includes an input/output controller906for receiving and processing input from a number of other devices, including a touch user interface display screen, or another type of input device. Similarly, the input/output controller906may provide output to a touch user interface display screen or other type of output device.

As mentioned briefly above, the mass storage device914and the RAM910of the AR headset104can store software instructions and data. The software instructions include an operating system918suitable for controlling the operation of the AR headset104. The mass storage device914and/or the RAM910also store software instructions, that when executed by the CPU902, cause the AR headset104to provide the functionality of the AR headset104discussed in this document. For example, the mass storage device914and/or the RAM910can store software instructions that, when executed by the CPU902, cause the AR headset104to display received financial data on the display screen of the AR headset104.