Buying products within video content by voice command

A user registers for an account with an account management system, configures account settings to permit the account management system to receive user computing device data from a user computing device associated with the user, and logs into the account via the user computing device. The account management system receives a user voice purchase command and determines a purchase command context based on the received user computing device data. The account management system identifies a product that the user desires to purchase based on the purchase command context and directs the user computing device web browser to a merchant website to set up a transaction for the identified product.

TECHNICAL FIELD

The present disclosure relates to improving user convenience in online shopping by enabling a user to purchase via voice command a product displayed or broadcasted to the user on a user computing device or on an electronic device in proximity to a user computing device.

BACKGROUND

Users often desire to purchase a product while watching or listening to an advertisement about the product or a review of the product. For example, a user may watch an advertisement for the product on television and desire to purchase the product. However, it may be inconvenient and time consuming for a user to search for the desired product and purchase the product online. For example, the user may have to input a product name into a search engine, select a merchant website offering the product for sale, and add the product to a virtual shopping cart on the merchant website. A user may use a voice command as a convenient way to input a product name into a search engine. Current technology does not, however, provide for determining a context of a voice purchase command submitted by a user, determining a product the user desires to purchase based on the context, and setting up a purchase transaction for the product with a merchant system offering the product for sale.

SUMMARY

Techniques herein provide computer-implemented methods to set up a purchase transaction for a product identified based on a context of a voice purchase command submitted by a user. In an example embodiment, a user registers for an account with an account management system, configures account settings to permit the account management system to continuously receive user computing device data from a user computing device associated with the user, and logs into the account via the user computing device. The account management system receives a user voice purchase command and determines a purchase command context based on the received user computing device data. The account management system identifies a product that the user desires to purchase based on the purchase command context and directs the user computing device web browser to a merchant website to set up a transaction for the identified product.

In certain other example aspects described herein, systems and computer program products to set up a purchase transaction for a product identified based on a context of a voice purchase command submitted by a user are provided.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

Overview

The example embodiments described herein provide computer-implemented techniques for setting up a purchase transaction for a product identified based on a context of a voice purchase command submitted by a user. In an example embodiment, a user registers for an account with an account management system, configures account settings to permit the account management system to continuously receive user computing device data from a user computing device associated with the user, and logs into the account via the user computing device. The account management system receives a user voice purchase command and determines a purchase command context based on the received user computing device data. The account management system identifies a product that the user desires to purchase based on the purchase command context and directs the user computing device web browser to a merchant website to set up a transaction for the identified product.

In an example embodiment, a user accesses an account management system website, establishes an account with the account management system, and downloads an application onto the user computing device. In an example embodiment, the user establishes a voice signature to associate with voice purchase commands that the account management system can use to verify the authenticity of voice commands by the user to purchase a product. For example, the voice signature may be a password, a frequency of the user's voice, an accent of the user's voice, or other relevant signature characteristic of the user's voice or method of making voice purchase commands. For example, the user may configure the account such that the account management system will only regard purchase commands that begin with “Hello! I would like to purchase . . . ”

The user logs into the account via the application and configures the account settings of the account to permit the account management system to continuously receive user computing device data. In an example embodiment, the user may permit the user computing device to transmit data to the account management system associated with an environment external to the user computing device. For example, the user may permit the user computing device to transmit location data, an audio feed, a video feed, and/or other data describing an environment of the user computing device. In another example embodiment, the user may permit the user computing device to transmit data to the account management comprising an internal status of the user computing device. For example, the user may permit the user computing device to transmit data comprising what video file the user is currently watching, what audio file the user is currently hearing, what website the user is currently viewing, what text the user is currently viewing, and/or what image the user is currently viewing on the user computing device. In an example embodiment, a user logs into the account and the user computing device continuously and/or periodically transmits applicable data to the account management system via a network according to the user's configurations.

In certain example embodiments, the user opens a video on the user computing device. In another example embodiment, the user opens an image, a website, a text file, an audio file, or an application on the user computing device. The user watches the video or otherwise interacts with the user computing device to access an image, a website, a text file, an audio file, or an application stored on the user computing device or accessible to the user computing device via a network. In an example embodiment, as the user is watching the video or other media, the user views a product that the user desires to purchase. The user inputs a voice purchase command to the user computing device. In an example embodiment, the voice purchase command comprises an ambiguous term or part of speech. For example, the user views a video on the user computing device, sees a product, and says “I want to buy that,” wherein the word “that” is an ambiguous term. The account management system receives the user's voice command from the user computing device and continuous and/or period user computing device data.

In certain other example embodiments, the user activates an electronic device external to the user computing device, such as a television or a radio, and watches/listens to video, audio, and/or text on the electronic device. For example, the user turns on a television and watches a television program. The user views, hears, or otherwise becomes aware of a product that the user desires to purchase via observation of the output of the electronic device. For example, the user views an advertisement on the television program for pizza from merchant A and desires to purchase merchant A pizza. The user inputs a voice command to the user computing device. In an example embodiment, the voice purchase command comprises an ambiguous term or part of speech. For example, the user views a video on the user computing device, sees a product, and says “I want to buy that,” wherein the word “that” is an ambiguous term. The account management system receives the user's voice command from the user computing device and continuous and/or period user computing device data.

In an example embodiment, the account management system verifies the voice signature of the user's voice purchase command according to the user's account settings. For example, the account management system verifies the quality of the user's tone, accent, or other quality of speech or voice and compares it against one or more true voice recordings provided previously by the user. In another example embodiment, the account management system determines whether a necessary password or passphrase is present in the voice command. In an example embodiment, the account management determines that the voice purchase command is invalid because it does not match the voice signature provided by the user and ignores the command. In another example embodiment, the account management system determines that the voice purchase command is valid because it matches the voice signature provided by the user. In this example embodiment, the account management system determines a purchase command context associated with the voice purchase command.

In an example embodiment, the account management system determines the grammar of the received voice command and identifies any ambiguous terms. For example, the user's voice command comprises “I want to purchase this,” wherein “this” is an ambiguous term. The account management system retrieves a user computing device data feed for a predefined time before the voice purchase command was submitted. For example, the account management system may retrieve the data feed for a time range of between one minute before the voice command was submitted and the time at which the voice command was submitted. In example embodiment, the account management system determines whether a file was being displayed to a user on the user computing device at the time the voice command was submitted. For example, the account management system can determine, based on the data feed, whether the user computing device was displaying a video, image, text, audio, or if a program, application, or website was open on the user computing device at the time the user submitted the voice purchase command. In certain example embodiments, the account management system can determine to what point of the video or audio the user the user has progressed. For example, the account management system determines that at the time of the submission of the voice purchase command the user was three minutes and fifty-four seconds through a five minute video. In an example embodiment, the account management system comprises annotated video and/or audio clips that describe which products are displayed or broadcasted at what time during the video and/or audio file.

If the account management system determines that a file was being displayed to a user at the time the voice command was submitted, the account management system determines if a product was being shown to a user on the displayed file that can provide a purchase command context to one or more ambiguous terms of the received voice purchase command. If the account management system determines that a product was being shown to the user on the displayed file at the time the voice purchase command was submitted that provides a satisfactory purchase command context to the one or more ambiguous terms of the command, the account management determines the displayed file provides a purchase command context for the voice purchase command.

If the account management system determines that a file was not being displayed to a user at the time the voice command was submitted, or is unable to find a satisfactory purchase command context to one or more ambiguous terms of the voice purchase command based on a displayed file, the account management system determines the location of the user computing device at the time the voice command was submitted. The account management system determines programming of the external electronic device based on the user computing device location and external data associated with the data feed. For example, external data comprises an audio feed, a video feed, or other applicable input to the user computing device that describes an environment of the user computing device. In an example embodiment, the account management system determines that the programming of the external electronic device provides a product purchase content for the voice purchase command.

In certain example embodiments, the account management system analyzes applicable data internal to the user computing device to attempt to identify a desired product associated with a user's voice purchase command. For example, the account management system knows that, for a particular five minute video displayed on the user computing device, at 2:54, the video displays a brand A baseball bat, at 3:15, the video displays a brand B baseball bat, and at 3:40, the video displays a brand A baseball glove to the user via the user computing device. If a voice purchase command comprising, “buy this now,” is received when the video is at 3:41, the account management system may determine that the purchase command context comprises a user's desire to purchase the displayed brand A baseball glove. In another example embodiment, the account management system is unable to identify a product based on internal user computing device data. In certain example

In certain example embodiments, the account management system analyzes applicable data external to the user computing device to attempt to identify a desired product associated with a user's voice purchase command. For example, the account management system identifies a programming schedule for the electronic device in the user's area and receives an audio input of the electronic device output to determine exactly what the user was watching or listening to at the time of the voice command. In an example embodiment, the account management system comprises annotated video and/or audio clips that describe which products are displayed or broadcasted at what time during the electronic device program. For example, the account management system knows that, for a particular hour long television program, at 12:55-14:55, the television program displays a pizza advertisement, at 35:05-40:00, the television program displays vacuum cleaner advertisement, and at 55:55-57:30, the television program displays an advertisement for potato chips. If a voice purchase command comprising, “buy this now,” is received when the television program is at 13:05, the account management system may determine that the purchase command context comprises a user's desire to purchase the pizza displayed in the pizza advertisement. In another example embodiment, the account management system is unable to identify a product based on one or of internal user computing device data and external user computing device data.

In certain example embodiments, the account management system requests and receives user input to attempt to identify a desired product associated with a user's voice purchase command. In these example embodiments, the user input may be used in addition to one or more of internal user computing device data and external user computing device data to identify the desired product. In an example embodiment, the account management system may request user input to further determine product specifications. For example, the account management system determines that the user wants to purchase a particular brand and model of hat, then asks the user, via the user computing device, what size hat the user wants to purchase. In this example, the user transmits a response to the account management system via the user computing device comprising a selection of a medium sized hat. After receiving user input, if the account management system is still unable to identify a product associated with the user's voice purchase command, the account management system may request further user input or may request that the user manually enter a description of the product that the user desires to purchase.

In an example embodiment, the account management system finds an offer associated with the identified product that the user desires to purchase. The account management system redirects the user computing device web browser to a merchant system website offering the product for sale. The user submits payment information and shipping information to the merchant system website and selects an option to initiate a payment transaction with the merchant system website to purchase the product. In another example embodiment, the user has an established digital wallet account with the account management system and the account management system provides user selected payment information to the merchant system website. In an example embodiment, the merchant system processes the payment transaction. For example, the merchant system communicates a payment authorization request to a credit card issuer system associated with credit card payment information entered by the user as payment information. In this example, the merchant system receives an approval of the payment authorization request from the issuer system and communicates a receipt to the user computing device indicating a successful payment transaction.

By using and relying on the methods and systems described herein, the account management system and the user computing device enable the user to purchase a product displayed or broadcasted to the user on a user computing device or on an electronic device in proximity to a user computing device via voice command. Additionally, because the account management system determines a purchase command context associated with the user voice purchase command, the user does not have to specifically provide the name of the desired product in the voice purchase command. As such, the systems and methods described herein may provide a convenience to the user.

Example System Architecture

Turning now to the drawings, in which like numerals indicate like (but not necessarily identical) elements throughout the figures, example embodiments are described in detail.

FIG. 1is a block diagram depicting a system100for setting up a purchase transaction for a product identified based on a context of a voice purchase command submitted by a user, in accordance with certain example embodiments. As depicted inFIG. 1, the system100includes network computing devices110,120,130, and140that are configured to communicate with one another via one or more networks150. In some embodiments, a user associated with a device must install an application and/or make a feature selection to obtain the benefits of the techniques described herein.

In example embodiments, the network150can include a local area network (“LAN”), a wide area network (“WAN”), an intranet, an Internet, storage area network (“SAN”), personal area network (“PAN”), a metropolitan area network (“MAN”), a wireless local area network (“WLAN”), a virtual private network (“VPN”), a cellular or other mobile communication network, Bluetooth, Bluetooth low energy, NFC, or any combination thereof or any other appropriate architecture or system that facilitates the communication of signals, data, and/or messages, and allows for the measurement of a received signal strength indicator (“RSSI”) or other similar measurement such as the free space path loss, the received channel power indicator (“RCPI”), the time of arrival (“TOA”), and/or the round trip time (“RTT”). Throughout the discussion of example embodiments, it should be understood that the terms “data” and “information” are used interchangeably herein to refer to text, images, audio, video, or any other form of information that can exist in a computer-based environment.

Each network computing device110,120,130, and140includes a device having a communication module capable of transmitting and receiving data over the network160. For example, each network computing device110,120,130, and140can include a server, desktop computer, laptop computer, tablet computer, a television with one or more processors embedded therein and/or coupled thereto, smart phone, handheld computer, personal digital assistant (“PDA”), or any other wired or wireless, processor-driven device. In the example embodiment depicted inFIG. 1, the network computing devices110,120,130, and140are operated by users101, merchant system120operators, account management system130operators, and users101, respectively.

An example user computing device110comprises a communication application111, a web browser112, a data storage unit115, an audio/video recorder module117, an application118, and a user interface119.

In an example embodiment, the user101can use a communication application111, such as a web browser112application or a stand-alone application118, to view, download, upload, or otherwise access documents or web pages via a distributed network160. In an example embodiment, the communication application111can interact with web servers or other computing devices connected to the network150, including the user computing device110, a web server121of a merchant system120, and a web server131of the account management system131.

In an example embodiment, the web browser112can enable the user101to interact with web pages using the user computing device110. In an example embodiment, the user101may access a website133of the account management system130to establish an account with the account management system130via the web browser112. In an example embodiment, the user101may access the user's101account maintained by the account management system130via the web browser112. In certain example embodiments described herein, one or more functions performed by the application118may also be performed by a web browser112application associated with the account management system130.

In an example embodiment, the data storage unit115comprises a local or remote data storage structure accessible to the user computing device110suitable for storing information. In an example embodiment, the data storage unit115stores encrypted information, such as HTML5 local storage.

In an example embodiment, the audio/video recorder module117records and/or receives as input an audio data feed or video data feed of the environment of the user computing device110. For example the audio/video recorder module117comprises a camera module or an audio recorder module. In an example embodiment, the application118and/or the account management system130instructs the audio/video recorder module117to record input. In an example embodiment, the audio/video recorder module117continuously and or periodically records an input of the user computing device110module. For example, the audio/video recorder module117records an audio output of a television that a user101is watching in the vicinity of the user computing device110.

In an example embodiment, the application118is a program, function, routine, applet, or similar entity that exists on and performs its operations on the user computing device110. In certain embodiments, the user101must install the application118and/or make a feature selection on the user computing device110to obtain the benefits of the techniques described herein. In an example embodiment, the application118communicates with the account management system130. In an example embodiment, a user101downloads the application118from an account management system website133onto a user computing device110. In an example embodiment, a user101may access the user's101account via the application118and configure account permissions to allow the user computing device110to transmit internal data describing an internal operating state of the user computing device110, external data describing an environment of the user computing device110, and/or user101input submitted by the user101to the account management system130. In an example embodiment, the user101may configure account settings via the application118to allow the account management system130to receive voice purchase commands submitted by the user101via the user computing device110. In certain example embodiments, the application118is a digital wallet account that communicates with the account management system130, which manages the user's101digital wallet account. In this this example embodiment, the user101may add, edit, or delete payment account information in the user's101digital wallet account via the application118. In an example embodiment, when the user101is logged into the user's101account via the application118and the user101has configured account settings to allow the user computing device110to transmit data to the account management system130, the application instructs the user computing device110to transmit one or more of internal data, external data, or user101input to the account management system130. In this example embodiment, the application118may also instruct the audio/video recorder module117to record external data describing an environment of the user computing device110. In this example embodiment, the application118continuously or periodically logs internal data describing an internal operating state of the user computing device110.

In certain example embodiments described herein, one or more functions performed by the application118resident on the user computing device110may also be performed by a web browser112application associated with the account management system130. In certain example embodiments described herein, one or more functions performed by the account management system130may also be performed by the application118. In certain example embodiments described herein, one or more functions performed by the web browser112application associated with the account management system130may also be performed by the application118.

In certain example embodiments described herein, the application118maintains periodic or constant communication with the account management system130via the network150. In certain example embodiments, the application118and is able to send and receive data associated with the user's101account to and from the account management system130when appropriate. For example, the application118may communicate user101interactions with the application118via the user interface119to the account management system130.

In an example embodiment, the user interface119may be a touch screen, a voice-based interface or any other interface that allows the user101to provide input and receive output from an application or module on the user computing device110. In an example embodiment, the user interface119enables the user101to interact with the application118or a web browser112application associated with the account management system130.

An example merchant system120comprises a server121, a website123, and a data storage unit125.

In an example embodiment, the server121provides the content accessible by the user101through the web browser112and/or application118resident on the user computing device110, including but not limited to html documents, images, style sheets, and scripts. In an example embodiment, the server121supports the merchant system website123.

In an example embodiment, the website123is a means by which the user101purchases a product from the merchant system120. In an example embodiment, the user computing device110web browser112is redirected by the account management system130to the merchant system website123to purchase a product that a user101request via a voice purchase command. In an example embodiment, the website123receives user101payment information from the user computing device110or from the account management system130. In example embodiment, the website123processes a payment transaction using the user's101payment account information. For example, the website123may communicate with an issuer system or financial institution system associated with a user101payment account to process the payment transaction. In an example embodiment, the website123may transmit a receipt to the user computing device110in response to successfully processing a payment transaction.

In an example embodiment, the data storage unit125comprises a local or remote data storage structure accessible to the merchant system120suitable for storing information. In an example embodiment, the data storage unit125stores encrypted information, such as HTML5 local storage.

An example account management system130comprises a server131, a website133, a data storage unit135, a context determination module137, and an offer module139.

In an example embodiment, the server131provides the content accessible by the user101through the web browser112and/or application118resident on the user computing device110, including but not limited to html documents, images, style sheets, and scripts. In an example embodiment, the server131supports the account management system website133.

In an example embodiment, the website133is a means by which the user101establishes an account with the account management system130. In an example embodiment, the user101interacts with the account managed by the account management system130via the website133. In an example embodiment, the user101accesses the website133via the web browser112. In another example embodiment, the user101accesses the website133via the application118resident on the user computing device110.

In an example embodiment, the data storage unit135comprises a local or remote data storage structure accessible to the account management system130suitable for storing information. In an example embodiment, the data storage unit135stores encrypted information, such as HTML5 local storage. In an example embodiment, the data storage unit135stores information associated with a user's101account. In an example embodiment, the data storage unit135may store a media database comprising annotated video files, audio files, websites, and/or other media. In this example embodiment, each file is annotated with product names describing products featured in the respective media. For example, an annotated video file in the database comprises annotations describing products displayed during certain time intervals of the corresponding video.

In an example embodiment, the context determination module137determines a purchase command context for a received voice purchase command. In an example embodiment, the context determination module137receives a voice purchase command. In an example embodiment, the context determination module137receives one or more of internal data, external data, and/or user101from the user computing device110. Example internal data comprises a description of any websites, videos, audio, applications, or other matter operating on the user computing device110displayed to the user101on the user interface119of the user computing device110. Example external data comprises a live video or audio data feed recorded by the user computing device110and location data logged by the user computing device110describing a location of the user computing device110. In an example embodiment, the context determination module137determines a purchase command context internal to the user computing device110. For example, the context determination module137, based on the internal data, identifies a video, audio file, website, image, or other applicable media that was executing on the user computing device110at the time the user submitted the voice purchase command. In this example embodiment, the context determination module137accesses the media database and finds an annotated media file corresponding to the media file that was executing on the user computing device110at the time the voice purchase command was submitted. In this example embodiment, the context determination module137identifies a purchase command context from the annotated media file. In another example embodiment, the context determination determines a purchase command context external to the user computing device110. For example, the context determination module137, based on the external data, identifies an output of an electronic device140which the user101was watching or listening at the time the user101submitted the voice purchase command as the purchase command context. In another example, the context determination module137, based on the external data, identifies a live demonstration of a product at a merchant location120located at or within a certain distance to the user's101location as the purchase command context. In an example embodiment, the context determination module137determines a product that the user101desires to purchase based on the determined purchase command context.

In an example embodiment, the offer module139identifies a merchant system120offering the user's101desired product for sale. In an example embodiment, the offer module138identifies a merchant system120website123that directs the user computing device110to a virtual shopping cart of the merchant system120comprising the user101desired product. In an example embodiment, the offer module138provides a link for the merchant system120website123for purchasing the desired product to the user computing device110.

In an example embodiment, the electronic device140comprises a television, radio, personal computer, tablet device, mobile device, or other electronic device140other than and external to the user computing device110. In an example embodiment, the electronic device140outputs a purchase command context in which a user101submits a voice purchase command to the user computing device110. For example, the user101is watching an advertisement on the electronic device140and submits a voice purchase command to the user computing device110to purchase the product displayed on the electronic device140.

It will be appreciated that the network connections shown are example and other means of establishing a communications link between the computers and devices can be used. Moreover, those having ordinary skill in the art having the benefit of the present disclosure will appreciate that the user computing device110, the merchant system120, the account management system130, and the electronic device140illustrated inFIG. 1can have any of several other suitable computer system configurations. For example, a user computing device110embodied as a mobile phone or handheld computer may or may not include all the components described above.

In example embodiments, the network computing devices and any other computing machines associated with the technology presented herein may be any type of computing machine such as, but not limited to, those discussed in more detail with respect toFIG. 9. Furthermore, any modules associated with any of these computing machines, such as modules described herein or any other modules (scripts, web content, software, firmware, or hardware) associated with the technology presented herein may by any of the modules discussed in more detail with respect toFIG. 9. The computing machines discussed herein may communicate with one another as well as other computer machines or communication systems over one or more networks, such as network150. The network150may include any type of data or communications network, including any of the network technology discussed with respect toFIG. 9.

Example Processes

The example methods illustrated inFIGS. 2-8are described hereinafter with respect to the components of the example operating environment100. The example methods ofFIGS. 2-8may also be performed with other systems and in other environments.

FIG. 2is a block diagram depicting a method200for setting up a purchase transaction for a product identified based on a context of a voice purchase command submitted by a user101, in accordance with certain example embodiments. The method200is described with reference to the components illustrated inFIG. 1.

In block210, a user101registers an account with an account management system130.

FIG. 3is a block diagram depicting a method210for registering an account with an account management system130, in accordance with certain example embodiments. The method210is described with reference to the components illustrated inFIG. 1.

In block310, the user101accesses an account management system website133. In an example embodiment, the user101accesses the account management system website133via the web browser112of the user computing device110. For example, the user101enters the website133address in the address bar of the web browser112to access the website133. In another example embodiment, the user101accesses the account management system website133using an application resident on the user computing device110. For example, the user101selects an application on the user computing device110that connects the user101to the account management system website133.

In block320, the user101establishes an account with the account management system130. In an example embodiment, the user account comprises a digital wallet account, an email account, a file storage and sharing account, or other account managed by the account management system130. In an example embodiment, the user101registers a username and a password associated with the account to use to sign in to the user101account. In an example embodiment, the user101may register a username and a password that is used for multiple user101accounts associated with the account management system130. In an example embodiment, the user101establishes a digital wallet account with the account management system130. In this example embodiment, the user101enters payment account information corresponding to one or more payment accounts of the user101. For example, the user101may enter payment account information corresponding to one or more credit accounts, bank accounts, or merchant system120credit accounts. In an example embodiment, the user101selects an option on the user101account to enable the account management system130to respond to voice command purchase requests. For example, the user101selects an option on the user101account that reads, “enable voice-based purchasing.”

In block330, the user101establishes a voice signature for voice purchase requests. In an example embodiment, a voice signature comprises characteristics of a user's101voice that are unique from characteristics of voices of other users101. For example, a voice signature may comprise one or more voice biometrics of the user's101voice. In an example embodiment, to establish a voice signature for voice purchase requests, the user101transmits one or more voice recordings of the user's101voice to the account management system130via the user computing device110. For example, the account management system130, via the user computing device110, may request that the user101to submit a recording of his or her voice as the user101is reading a requested passage of text to establish a voice signature of the user's101voice. In this example, the account management system130instructs the user computing device110to record, the user101reads the requested passage, and the user computing device submits the voice recording to the account management system130via the network150.

In certain example embodiments, the voice signature may also comprise a password or passphrase for use in voice purchase commands. For example, the user101may configure account settings to only permit the account management system130to execute a voice purchase command if the user101speaks a certain combination of words. For example, the user101configures the voice signature such that the account management system130only regards voice purchase commands ending in “for me.” In this example, the account management system130executes a voice purchase command comprising “Buy this for me!” but disregards a voice purchase command comprising, “Buy this!”

In block340, an application118is downloaded onto the user computing device110. In another example embodiment, the user101may download various applications associated with the user account from the account management system130in addition to the application118. For example, the user101downloads one or more of an email application, a mapping application, a social network application, or a search application associated with the account management system130. In this example, the user101may maintain a universal username and/or password to access all the applications associated with the account management system130. In another example embodiment, the application118is downloaded onto the user computing device110before the user101establishes the user101account with the account management system130. In certain example embodiments, the user101does not download the application118onto the user computing device110. For example, the user101may access the account using a web browser112application associated with the account management system130or may access the account directly on the account management system website133using the web browser112. In an example embodiment, the application118communicates with the account management system130over the network150. In this example, the application118communicates with the account management system130, which administers the user101account.

In block350, the user101logs into the account via the application118. For example, the user101selects the application118on the user computing device110, enters a user name and/or password via the user interface119, and selects an option on the application118reading “log in to my account.” In an example embodiment, the application118communicates with the account management system130when the user101is logged in to the application118. In an example embodiment, the account management system130is unable to process voice purchase commands when the user101is not logged into his account via the application118and/or a web browser112application operating on the user computing device110and communicating with the account management system130.

In block360, the user101configures account settings to permit the account management system130to continuously receive user computing device110data. In an example embodiment, the user101accesses account settings via the application118. In an example, the user's101account comprises information notifying the user101that the account management system130is only able to process voice purchase command when the user101permits the user computing device110to transmit data to the account management system130. In an example embodiment, the application118displays an option for the user101to permit the user computing device110to transmit data the account management system130continuously, periodically, or in response to receiving a voice purchase command from the user101.

In this example embodiment, the user101selects the option to permit the user computing device110to transmit data to the account management system130by actuating one or more objects of the user interface119of the user computing device. In an example embodiment, the application118communicates the user's101configuration of the user's101account settings to the account management system130via the network150. In an example embodiment, if the user101selects the option to allow the user computing device110to transmit data to the account management system130, the user computing device110continuously or periodically transmits data to the account management system130while the user101is logged into the application118. In this example embodiment, if the user101signs out of the application118, then the user computing device110ceases to transmit data to the account management system130until the user101signs back into the application118.

In an example embodiment, data transmitted by the user computing device110to the account management system130comprises internal data describing an internal operating state of the user computing device110. For example, the user computing device110may transmit internal data describing what is currently being displayed to the user101, for example, if the user101is watching a particular video or listening to a particular song on the user computing device110. In another example, the user computing device110transmits internal data comprising the internet address of a website that the user101is currently viewing. In another example embodiment, data transmitted by the user computing device110to the account management system130comprises external data describing an external environment of the user computing device110. For example, the user computing device110may transmit external data comprising a live audio or video feed recorded by the user computing device110to the account management system130.

In an example embodiment, the user computing device110transmits internal data and/or external data to the account management system130continuously or periodically. For example, the user computing device110continuously or periodically transmits external data comprising a live video or audio data feed recorded by the user computing device110. In another example, the user computing device110continuously or periodically transmits internal data comprising a description of any websites, videos, audio, applications, or other matter operating on the user computing device110that is currently being displayed to the user101on the user interface119of the user computing device110. In another example embodiment, the user computing device110transmits internal and/or external data to the account management system130after the user101submits a voice purchase command. In this example embodiment, the user computing device110continuously and/or periodically logs internal and/or external data and stores the data in a data storage unit115of the user computing device110to retrieve and transmit to the account management system130when appropriate. In another example embodiment, the user computing device110logs internal and/or external data in response to receiving a voice purchase command from a user. In this example embodiment, the user computing device110logs internal and/or external data for a certain period of time after receiving the voice purchase command and then transmits the logged data to the account management system130. In certain example embodiments, the user computing device110may only transmit the data to the account management system while the user101is logged in to the application118and if the user101account settings are configured by the user101to allow the user computing device110to transmit such data.

From block360, the method210proceeds to block220ofFIG. 2.

Returning toFIG. 2, in block220, the account management system130receives a continuous data feed from the user computing device110according to user101configured settings.

In block230, the account management system130receives a user101voice purchase command.

In an example embodiment, the user101desires to purchase a product in a purchase command context internal to the user computing device110. In this example embodiment, the user101becomes aware of through interaction with the user computing device110. In this example embodiment, the user101may be watching a video, listening to audio, looking at a website, or interacting with some other applicable program, module, application, or file that is executing on the user computing device110. In this example embodiment, the user101becomes aware of a product that the user101desires to purchase via the user's101interaction with the user computing device110. For example, the user101is watching a video executing on the user computing device110comprising a review of a product. In this example, at some point during the video the product is displayed in the video, the user101notices the product, desires to purchase the product, and submits a voice purchase command. For example, upon seeing the product displayed on the video executing on the user computing device110, the user101shouts a voice purchase command comprising “I want to buy that!” In this example embodiment, the user computing device110records the user101voice purchase command.

In another example embodiment, the user101desires to purchase a product in a purchase command context external to the user computing device110. In this example embodiment, the user101desires to purchase a product that the user101becomes aware of through interaction with an electronic device140external to the user computing device110. In this example embodiment, the user101may be watching a video, or interacting with some other applicable program, module, application, or file that is executing on the electronic device140in proximity to the user computing device110. In this example embodiment, the user101is logged into the application118and the user computing device110is able to record an output of the electronic device140as the user is interacting with the electronic device140. For example, the electronic device140is a television and the user computing device110records an audio output of the television as the user101is watching the television. In this example embodiment, the user101becomes aware of a product that the user101desires to purchase via the user's101interaction with the electronic device140. For example, the user101is watching a video executing on the electronic device140comprising an advertisement of a product. In this example, at some point during the video the product is displayed in the video, the user101notices the product, desires to purchase the product, and submits a voice purchase command. For example, upon seeing the product displayed on the video executing on the electronic device140, the user101shouts a voice purchase command comprising “I want to buy that!” In this example embodiment, the user computing device110records the user101voice purchase command.

FIG. 4is a block diagram depicting a method230for receiving a voice purchase command from a user in a purchase command context internal to a user computing device110, in accordance with certain example embodiments. The method230is described with reference to the components illustrated inFIG. 1.

In block410, the user101opens a video on the user computing device110. In an example embodiment, the user101actuates one or more objects on the user interface119of the user computing device110to open a video. In an example embodiment, the user101accesses the video via a web browser112over the network150or via an application resident on the user computing device110. In another example embodiment, the user101opens an audio file, accesses a website, opens an application, or otherwise interacts with the user computing device110.

In block420, the user101watches the video on the user computing device110. In another example embodiment, the user101listens to a song or an audio file executing on the user computing device110. In another example embodiment, the user101reads text or looks at images displayed on the user computing device110.

In block430, the user101views a product that the user101desires to purchase. For example, the user views the product displayed in the video. For example, the user101views a video advertisement for a pizza offered by a delivery service playing on the user computing device110and desires to purchase the advertised pizza. In another example embodiment, the user101views a product that the user101desires to purchase on a web site displayed in the web browser112. In another example embodiment, the user101listens to the name of a product or to a description of a product on an audio file executing on the user computing device110.

In block440, the user101inputs a voice purchase command to the user computing device110. For example, the user101says “buy it!” and the user computing device records the user's101command. In an example embodiment, when the user computing device110receives the voice purchase command input by the user101, the user computing device110logs a time stamp to associate with the voice purchase command. For example, the user computing device110records a month, day, year, hour, minute, second, time zone, and/or any other applicable measures of time to record the time at which the user computing device110receives the voice purchase command. For example, the user computing device110receives a user101voice purchase command, logs a timestamp comprising “Feb. 10, 2015-13:25 EST,” and associates the time stamp with the voice purchase command. In an example embodiment, the user computing device110stores the voice purchase command. In another example embodiment, the user computing device110does not store the voice purchase command and immediately transmits the voice purchase command to the account management system130along with the logged timestamp. In an example embodiment, the user computing device110transmits the user101voice purchase command to the account management system130. In an example embodiment, the user computing device110transmits the logged timestamp along with the user101voice purchase command.

In block450, the account management system130receives the user's101voice purchase command from the user computing device110. In an example embodiment, the account management system130also receives a timestamp generated by the user computing device110at the time that the user computing device110received the user101voice purchase command.

In certain example embodiments, as previously discussed, the user computing device110transmits internal data and/or external data to the account management system130continuously or periodically. In an example, the user computing device110continuously or periodically transmits external data comprising a live video or audio data feed recorded by the user computing device110. In another example, the user computing device110continuously or periodically transmits internal data comprising a description of any websites, videos, audio, applications, or other matter operating on the user computing device110that is currently being displayed to the user101on the user interface119of the user computing device110.

In certain other example embodiments, the user computing device110transmits internal and/or external data to the account management system130after the user101submits a voice purchase command. In an example embodiment, the user computing device110transmits to the account management system130internal and/or external data logged over a time period of a certain length before the time at which the voice purchase command was received by the user computing device110. In this example embodiment, the user computing device110continuously and/or periodically logs internal and/or external data and stores the data in a data storage unit115of the user computing device110to retrieve and transmit to the account management system130at the appropriate time. In certain example embodiments, the user computing device110may only transmit the data to the account management system while the user101is logged in to the application118and if the user101account settings are configured by the user101to allow the user computing device110to transmit such data.

From block450, the method230proceeds to block240inFIG. 2.

In certain example embodiments, instead of receiving a voice purchase command from a user in a purchase command context internal to a user computing device110, the user computing device110receives a voice purchase command from a user in a purchase command context external to a user computing device110.

FIG. 5is a block diagram depicting a method230for receiving a voice purchase command from a user in a purchase command context external to a user computing device, in accordance with certain example embodiments. The method230is described with reference to the components illustrated inFIG. 1.

In block510, the user101activates an electronic device140. For example, the user101activates a television or a radio device. In an example embodiment, the electronic device140is in physical proximity to the user computing device110to enable the user computing device110to receive or record an output of the electronic device140. For example, the user computing device110is able to record or receive a sound output of a television, personal computer, tablet, radio, or other electronic device140that generates at least an audio output via a sound recording capability of the user computing device110. In another example, the user computing device110is able to record a video output of a television via a video recording capability of the user computing device110.

In block520, the user101watches video, watches text, or listens to audio on the electronic device140. In an example embodiment, while the video is interacting with the electronic device140, the user computing device110records or receives the output of the electronic device140

In block530, the user views or hears of a product that the user101desires to purchase. For example, the user views the product displayed in the video. For example, the user101views a video advertisement for a pizza offered by a delivery service playing on the electronic device140and desires to purchase the advertised pizza. In another example, the user101listens to the name of a product or to a description of a product on a radio program executing on a radio electronic device140and desires to purchase the product.

In block540, the user101inputs a voice purchase command to the user computing device110. For example, the user101says “buy that!” and the user computing device records the user's101command. In an example embodiment, when the user computing device110receives the voice purchase command input by the user101, the user computing device110logs a time stamp to associate with the voice purchase command. For example, the user computing device110records a month, day, year, hour, minute, second, time zone, and/or any other applicable measures of time to record the time at which the user computing device110receives the voice purchase command. For example, the user computing device110receives a user101voice purchase command, logs a timestamp comprising “Feb. 10, 2015-13:25 EST,” and associates the time stamp with the voice purchase command. In an example embodiment, the user computing device110stores the voice purchase command. In another example embodiment, the user computing device110does not store the voice purchase command and immediately transmits the voice purchase command to the account management system130along with the logged timestamp. In an example embodiment, the user computing device110transmits the user101voice purchase command to the account management system130. In an example embodiment, the user computing device110transmits the logged timestamp along with the user101voice purchase command. In certain example embodiments, the user computing device110does not continuously log external and/or internal data and, in response to receiving the user101voice purchase command, the user computing device110logs external and/or internal data for a certain period of time and then transmits the logged data to the account management system130in addition to the logged timestamp and the user101voice purchase command.

In block550, the account management system130receives the user's101voice purchase command from the user computing device110. In an example embodiment, the account management system130also receives a timestamp generated by the user computing device110at the time that the user computing device110received the user101voice purchase command.

In certain example embodiments, as previously discussed, the user computing device110transmits internal data and/or external data to the account management system130continuously or periodically. In an example, the user computing device110continuously or periodically transmits external data comprising a live video or audio data feed recorded by the user computing device110. In another example, the user computing device110continuously or periodically transmits internal data comprising a description of any websites, videos, audio, applications, or other matter operating on the user computing device110that is currently being displayed to the user101on the user interface119of the user computing device110.

In certain other example embodiments, as previously discussed, the user computing device110transmits internal and/or external data to the account management system130after the user101submits a voice purchase command. In an example embodiment, the user computing device110transmits to the account management system130internal and/or external data logged over a time period of a certain length before the time at which the voice purchase command was received by the user computing device110. In this example embodiment, the user computing device110continuously and/or periodically logs internal and/or external data and stores the data in a data storage unit115of the user computing device110to retrieve and transmit to the account management system130at the appropriate time. In certain example embodiments, the user computing device110may only transmit the data to the account management system while the user101is logged in to the application118and if the user101account settings are configured by the user101to allow the user computing device110to transmit such data.

From block550, the method230proceeds to block240inFIG. 2.

Returning toFIG. 2, in block240, the account management system130verifies a voice signature of the received voice purchase command. In an example embodiment, the account management system130accesses a first saved user101voice signature associated with the user's101account, determines a second voice signature of the received user101voice purchase command, and compares the first saved voice signature against the second voice signature. In this example embodiment, the account management system130may determine a similarity between the first voice signature and the second voice signature. For example, the account management system130may compare one or more biometric qualities or properties of the first voice signature against the second voice signature to determine a similarity between the voice signatures. In another example, the account management system130may retrieve a password or passphrase from the first voice signature. In this example, the account management system130determines whether the passphrase or password from the first voice signature is present in the second voice signature.

If the account management system130determines that the second voice signature associated with the received voice purchase command is invalid, the method200proceeds to block250. In an example embodiment, the account management system130determines that the first voice signature and the second voice signature are not similar at least to a certain threshold value and the account management system130determines that the voice signatures do not match and that the second voice signature is invalid. In another example embodiment, the account management system130determines that the second voice signature does not comprise a password or passphrase established by the user in the first voice signature and the account management system130determines that the voice signatures do not match and that the second voice signature is invalid. For example, the user101established, in the first voice signature and/or via one or more configurations of user101account settings, that all user101voice purchase requests must comprise the phrases “please” at the beginning and “for me” at the end. In this example, the voice command submitted by the user101comprises “please purchase this” and does not end in “for me,” therefore the account management system130determines that the second voice signature does not comprise the required passphrases from the first voice signature.

In block250, the account management system130disregards the user101voice purchase command. In another example embodiment, the account management system130transmits, via the user computing device110, a request for the user101to provide a password associated with the user101account. In this example embodiment, the account management system130may lock the user101out of the user101account as a security measure until the user101provides the requested password.

Returning to block240, if the account management system130determines that the voice signature is valid, the method200proceeds to block260. In this example embodiment, the account management system130determines that the first voice signature and the second voice signature are similar at least to a certain threshold value and the account management system130determines that the voice signatures match and that the second voice signature is valid. In another example embodiment, the account management system130determines that the second voice signature comprises a required password or passphrase established by the user in the first voice signature and the account management system130determines that the voice signatures match and that the second voice signature is valid. For example, the user101established, in the first voice signature and/or via one or more configurations of user101account settings, that all user101voice purchase requests must comprise the phrases “please” at the beginning and “for me” at the end. In this example, the voice command submitted by the user101comprises “please purchase this for me,” therefore the account management system130determines that the second voice signature comprises the required passphrases from the first voice signature and that the second voice signature is valid.

In block260, the account management system130determines a purchase command context for the voice purchase command.

FIG. 6is a block diagram depicting a method260for determining a purchase command context and identifying a product associated with a voice purchase command, in accordance with certain example embodiments. The method260is described with reference to the components illustrated inFIG. 1.

In block610, the account management system130determines the meaning of the voice purchase command and identifies ambiguous terms. In an example embodiment, the account management system130uses one or more rules of grammar and/or a dictionary to determine the meaning of the voice command and identify ambiguous terms. For example, the voice purchase command comprises, “buy it.” In this example, the account management system130determines that the user101desires to execute a voice purchase command based on the verb “buy.” In this example, the account management system130identifies the ambiguous term “it” in the voice purchase command. In another example, the voice purchase command comprises, “buy this pizza.” In this example, the account management system130determines that the user101desires to execute a voice purchase command to purchase a pizza based on the verb “buy” and the direct object “pizza.” In this example, the account management system130identifies the ambiguous term “this” in the voice purchase command.

In block620, the account management system130retrieves a user computing device110data feed for a predefined time before the time at which the voice purchase command was submitted by the user101. For example, the data feed comprises internal and/or external data logged by the user computing device110. For example, the account management system130, based on a timestamp received from the user computing device110identifying the time at which the voice purchase command was received by the user computing device110, extracts all data received from the user computing device110from a predefined time period prior to the time of the timestamp all the way up until the time of the timestamp. In another example embodiment, the account management system130, based on a timestamp received from the user computing device110identifying the time at which the voice purchase command was received by the user computing device110, extracts data received from the user computing device110from the time of the timestamp up to a predefined time period after the time of the timestamp. In yet another example embodiment, the account management system130receives, from the user computing device110a data feed logged by the user computing device110during the predefined time along with the voice purchase command and/or timestamp. For example, the account management system130receives a data feed from the user computing device110comprising data logged during a predefined time before the voice purchase command and/or the time associated with the timestamp. In another example, when the user101submits the voice purchase command to the user computing device110, the user computing device110is not currently logging internal or external data. In this example, the user computing device110, in response to receiving the user101voice purchase command, begins to log internal and/or external data for a certain period of time. In this example, the user computing device110, after logging the data for the certain period of time, transmits the logged data to the account management system130along with the user101voice purchase command. In this example, the account management system130receives the logged data associated with the certain period of time subsequent to the user computing device110receiving the voice purchase command, a timestamp associated with the time at which the user computing device110received the voice purchase command, and the user101voice purchase command. In yet another example embodiment, in response to receiving the voice purchase command and determining that the voice signature is valid, the account management system130transmits to the user computing device110a request for a data feed for a predefined time before the time at which the voice purchase command was submitted by the user101. In this example embodiment, the user computing device110receives the request for a data feed and transmits the requested data feed to the user computing device110. For example, the user computing device110retrieves the generated timestamp associated with the voice purchase command and transmits all applicable or relevant data logged, captured, or recorded by the user computing device110for a predefined period of time prior to the time specified in the time stamp. In an example embodiment, the data feed transmitted by the user computing device110and received by the account management system130comprises internal and/or external data. For example, external data may comprise a live video or audio data feed recorded by the user computing device110. For example, internal data may comprise a description of any websites, videos, audio, applications, or other matter operating on the user computing device110displayed to the user101on the user interface119of the user computing device110.

In block630, the account management system130determines, based on the user computing device110data feed, whether a file was displayed or was playing on the user computing device110at the time of the voice purchase command. In an example embodiment, the account management system130analyzes received internal data from the user computing device110to determine whether a file was displayed or was playing on the user computing device110at the time of the voice purchase command. In another example embodiment, the account management system130may determine whether a file was displayed or was playing on the user computing device110at the time of the voice purchase command or during a predetermined time period before the time of the voice purchase command.

If a file was not being displayed or playing on the user computing device110at the time of the voice purchase command, the method260proceeds to block660. For example, if a file was not being displayed or playing on the user computing device110, the purchase command context of the user's101voice purchase command may be a purchase command context that is external to the user computing device110and the account management system130may determine the purchase command context based on an external environment of the user computing device110instead of an internal operating state of the user computing device110.

In block660, the account management system130determines the location of the user computing device110at the time the voice purchase command was submitted by the user101. In an example embodiment, external data comprises one or more locations of the user computing device110logged by the user computing device110.

Returning to block630, if a file was being displayed or was playing on the user computing device110at the time of the voice purchase command, the method260proceeds to block640. In another example embodiment, if a file was being displayed or was paying on the user computing device110at the time of the voice purchase command, or within a predetermined time period prior to the time of the voice purchase command, the method260proceeds to block640. For example, if a file was being displayed or playing on the user computing device110, the purchase command context of the user's101voice purchase command may be a purchase command context that is internal to the user computing device110and the account management system130may determine the purchase command context based on an internal operating state of the user computing device110instead of an external environment of the user computing device110. In another example, if a file was being displayed or playing on the user computing device110two seconds before a voice purchase command was received by the user computing device110, the purchase command context of the user's101voice purchase command may be a purchase command context that is internal to the user computing device110and the account management system130may determine the purchase command context based on an internal operating state of the user computing device110instead of an external environment of the user computing device110.

In block640, the account management system130accesses the file. In an example embodiment, the account management system130comprises a media database comprising annotated video files, audio files, websites, and/or other media. In this example embodiment, each file is annotated with product names describing products featured in the respective media. In an example embodiment, an annotated video file in the database comprises annotations describing products displayed during certain time intervals of the corresponding video. For example, an annotated video file corresponding to a video of five minutes duration might be annotated showing that a brand A window cleaning product is shown between 1 minute and 22 seconds and 1 minute and 55 seconds of the video and a brand B vacuum cleaner is shown between 2 minutes and 3 minutes and 25 seconds of the video. In an example embodiment, the account management system130annotates one or more video files, audio files, website files, and/or other media files corresponding to one or more videos, audio clips, websites, and/or other media. For example, the account management system130comprises a web crawler that gathers videos, audio, images, and/or website information over the network140. In this example, the account management system130analyzes the content of the media and generates annotated files corresponding to each media that describe the products displayed or broadcasted in a media file and specific timing information for when the products are displayed in the media.

In block650, the account management system130determines whether the purchase command context is an internal function of the user computing device110or an external environment to the user computing device110. For example, an internal purchase command context results when the user101intends to purchase a product that was displayed or otherwise an output of the user computing device110. In this example, the purchase command context is internal because it is explained by an internal operation of the user computing device110. In another example, an external purchase command context results when the user101intends to purchase a product that was displayed or otherwise output on an electronic device140external to the user computing device110or otherwise displayed or output in an environment external to the user computing device110.

If the account management system130determines that the purchase command context is an internal function of the user computing device110, the method260proceeds to block680. In an example embodiment, if the file executing on the user computing device110was displaying or broadcasting a product to the user101at the time the user101submitted a voice purchase command, then the account management system130determines that the purchase command context is internal to the user computing device.

In block680, the account management system130determines the desired product associated with the voice purchase command based on the purchase command context. For example, the account management system130determines the desired product associated with the voice purchase command based on an internal purchase command context. For example, the file executing on the user computing device110was displaying or broadcasting a product to the user101at the time the user101submitted a voice purchase command. In this example, the account management system130identifies the file based on internal data received from the user computing device110. For example, internal data corresponding to a video may comprise a video file identifier or other media file identifier, a total length of the video and the user's current progress in viewing the video. For example, for a five minute video, the user101was at three minutes and twenty three seconds through the video when the user101submitted the voice purchase command. In an example embodiment, the account management system130may extract an annotated media file corresponding to the media file identifier from the account management system's130media database. In this example, the account management system130may determine what product or products were being displayed or broadcasted to the user101via the user computing device110at the three minute and twenty three second mark of the user101and/or during a predefined period beforehand. In an example embodiment, the account management system130determines that the product identified in the media file as being displayed to the user101at the time the user101voice purchase command was received by the user computing device110is the user's101desired product in the voice purchase command.

Returning to block650, if the account management system determines that the purchase command context is an external environment of the user computing device110, the method260proceeds to block660. In an example embodiment, the voice purchase command does not make sense in light of the file executing on the user computing device110at the time of the user101submission of the voice purchase command. In this example embodiment, the file may not have been displaying a product at the time of the user101submission of the voice purchase command or during a threshold time before the user101submitted the voice purchase command. In another example embodiment, a different product than the product specified in the user's101voice purchase command was being displayed to the user101at the time the voice purchase command was submitted. For example, a pizza offer was being displayed but the voice purchase command comprises “buy that vacuum cleaner please!” In these example embodiments, the account management system130determines that there is no purchase command context internal to the user computing device110, therefore the purchase command context may be external to the user computing device110. For example, the user101submitted the voice purchase command with an intent to purchase a product displayed on an electronic device140external to the user computing device110instead of a product displayed on the user computing device110. In another example, the user101submitted the voice purchase command with an intent to purchase a product displayed in a live display, such as product shown at a showroom exposition at a merchant system120location.

In block660, the account management system130determines the location of the user computing device110at the time the voice purchase command was submitted by the user101. For example, determining the location of the user computing device110at the time the voice purchase command was submitted by the user101may provide information to aid in identifying an external purchase command context. In an example embodiment, the account management system130transmits a request to the user computing device110for the location information logged by the user computing device110at a time corresponding to the timestamp associated with the time the voice purchase command was received by the user computing device110. In this example embodiment, the user computing device110receives the request for the location information and transmits the location information to the account management system130. In another example embodiment, the account management system130received, from the user computing device110along with the voice purchase command and associated timestamp, the location information of the user computing device110corresponding to the time the voice purchase command was received. Example location information comprises global positioning system (“GPS”) coordinates logged by the user computing device110, a physical street address, an indication of proximity to a Wi-Fi beacon, network device, or other device along with the hardware identifier corresponding to the device, which the account management system130may use to identify an approximate location of the user computing device110if the device is known to the account management system130. In an example embodiment, the user101has configured the user's101account settings to allow transmission of location information from the user computing device to the account management system130. In this example embodiment, the user101may be required to be signed in to the application118resident on the user computing device110or be signed in to the user's101account via a web browser112application to allow the user computing device110to transmit user computing device110location information.

In block670, the account management system130determines the programming of an electronic device140based on the user computing device110location and external data feed. In an example embodiment, the account management system130first determines that the user101was listening to or watching an output of the electronic device140at the time of the user101submission of the voice purchase command. For example, the user computing device110external data feed received by the account management system130may comprise an audio feed of the output of a television electronic device140. In this example, the user computing device110may identify the electronic device as a television and may identify a particular television channel based on an output sound frequency or output sound characteristics of the electronic device140as found in the audio data feed. Likewise, the account management system130may determine that the electronic device140is a radio or a personal computing device other than the user computing device110. In an example embodiment, if the account management system130determines a particular television channel or radio station that was outputted by the electronic device140at the time of the voice purchase command, the account management system130may correlate the user computing device110location to a programming schedule received from the television station or radio station to identify exactly which programming the user101was watching or listening to at the time the user101submitted the voice purchase command.

In another example embodiment, the external purchase command context is not programming displayed by an electronic device140, but may be a live demonstration of a product at a merchant system120location. For example, the account management system130receives a schedule of live demonstrations of products associated with a merchant system120location from the merchant system120. In this example, the account management system120correlates the location of the user computing device110at the time of the voice purchase command to within a threshold distance of the site of a merchant system120location within a certain length of time before the user101submission of the voice purchase command or at the same time as the user101submission of the voice purchase command.

From block670, the method260proceeds to block270inFIG. 2.

Returning toFIG. 2, in block270, the account management system130determines the user's101desired product based on the purchase command context.

FIG. 7is a block diagram depicting a method680for determining a desired product associated with a voice purchase command based on a purchase command context, in accordance with certain example embodiments. The method680is described with reference to the components illustrated inFIG. 1.

In block710, the account management system130analyzes applicable data from an internal data feed received from the user computing device110. For example internal data transmitted by the user computing device110to the account management system130describes what is currently being displayed to the user101or a history describing was displayed to the user101over a certain time period. For example, the user computing device110transmits internal data indicating that the user computing device110was playing a particular video or a particular song on the user computing device110at the time the user101submitted the voice purchase command. In another example, the user computing device110transmits internal data comprising the internet address of a website that the user101is currently viewing or has viewed at a certain time in the past. In an example embodiment, the user computing device110continuously logs internal data describing videos, audio, or websites running on or being accessed by the user computing device110and transmits the internal data periodically in batches to the account management system130.

In block720, the account management system130is able or unable to identify a desired product based on the internal data feed received from the user computing device110.

If the account management system130is able to identify a desired product based on the internal data feed received from the user computing device110, the method680proceeds to block270inFIG. 2. In an example embodiment, the account management system130accesses an annotated media file from the database comprising annotated media files corresponding to the video, audio, or text that the user101was watching, listening to, or reading at the time of the user101submission of the voice purchase command. In an example embodiment, the account management system130may determine, from the annotated media file, what product or products were being displayed or broadcasted to the user101via the user computing device110at the time of and/or during a predefined period before the user101submission of the voice purchase command. In this example embodiment, the account management system130determines that the product identified in a media file as being displayed to the user101at the time the user101voice purchase command is the user's101desired product in the voice purchase command.

Returning toFIG. 720, if the account management system130is unable to identify a desired product based on the internal data feed received from the user computing device110, the method680proceeds to block730. For example, the user computing device110was not displaying any video, music, images, or websites to the user101at the time the user submitted the voice purchase command. In another example, the account management system130identifies a desired product based on the internal data feed, but does not have enough information to set up a purchase transaction for the desired product. For example, if the desired product is an item of clothing, the account management system130may still need user101input comprising a user's101clothing size or external data comprising a user101location to be able to identify specifically which product the user101desires to purchase.

In block730, the account management system130analyzes applicable data received from an external data feed of the user computing device110. For example, external data comprises an audio feed, a video feed, or other applicable input to the user computing device that describes an environment of the user computing device110. In an example, the external data feed may be an audio feed, a video feed, or other applicable input recorded by or received by the user computing device110that corresponds to an output of an electronic device140in the vicinity of the user computing device110, such as a television, a personal computer, or a radio. In this example, the external data feed may further comprise one or more locations of the user computing device110logged by the user computing device110.

In block740, the account management system130is able or unable to identify a desired product based on the external data feed received from the user computing device110.

If the account management system130is able to identify a desired product based on the external data feed received from the user computing device110, the method680proceeds to block270inFIG. 2. In an example embodiment, the account management system130accesses an annotated media file from the database comprising annotated media files corresponding to the programming the user101was watching on an electronic device140. In this example embodiment, the account management system130may identify a product that was being displayed to the user101via the electronic device140at the time of the voice purchase command or within a certain length of time before the submission of the voice purchase command. In yet another example embodiment, based on a determination that the user computing device110was present at the merchant system120location associated with the live display of the product at the time of the submission voice purchase command, the account management system120determines that the user101intended to purchase the product demonstrated in the live display.

Returning toFIG. 740, if the account management system130is unable to identify a desired product based on the external data feed received from the user computing device110, the method680proceeds to block750. For example, the account management system130is unable to identify a product from the electronic device140programming being outputted to the user101at the time the user101submitted the voice purchase command based on the external data feed. In this example, the account management system130may determine what electronic device140programming was being outputted to the user101at certain time intervals before the time the user101submitted the voice purchase command based on the external feed. In this example, the user101may have watched an advertisement for pizza on the electronic device140between 12:05-12:07 p.m. but not issue the voice command “Buy that pizza” until 12:08 p.m., at which time the electronic device140was outputting programming other than the pizza advertisement. In this example, the account management system130does not determine an external voice command context based on the programming being output by the electronic device at 12:08, but determines that the output between 12:05-12:07 corresponds to the pizza advertisement and determines that the user101intended to purchase the pizza displayed in the pizza advertisement. In another example embodiment, the account management system130identifies a desired product based on the external data feed, but does not have enough information to set up a purchase transaction for the desired product. For example, if the desired product is an item of clothing, the account management system130may still need user101input comprising a user's101clothing size In another example, if the account management system130did not understand the user's101voice purchase command, the account management system130may request that the user101type in or otherwise input a purchase command manually into the user computing device110.

In block750, the account management system130transmits a request to the user computing device110for user101input to display on the user computing device110.

In block760, the user computing device110receives and displays the request for user101input. For example, the account management system130identified the product the user101desires to purchase as being a brand A hat. In this example, the input request displayed on the user computing device110may read, “what size brand A hat would you like to purchase?” or “what is your hat size.” In an example, the user computing device110may display a list of options from which the user101may select an option to transmit as input. In another example. In another example, the user computing device110may display a request for a text input from the user101. For example the user computing device110may display a request comprising, “please input your hat size.” In another example, in which the account management system130did not understand the user's101voice purchase command, the user computing device110may display a request reading, “we're you trying to purchase something? If so, what would you like to purchase?”

In block770, the user101enters input onto the user computing device110in response to seeing the request displayed on the user computing device110. For example, for input regarding a hat the user101desires to purchase, the user101may select one of multiple user interface119objects reading “small,” “medium,” “large,” or “extra large.” In another example, the user101may input text reading “medium” when asked which hat size the user101desires. In another example, the user101may manually enter text comprising a purchase command if the account management system130did not understand a previous voice purchase command submitted by the user101.

In block780, the account management system130receives the user input from the user computing device. For example, the account management system130receives an indication of a user101selection of an option from a list of input options. In another example, the account management system130receives a text input submitted by the user101.

In block790, the account management system130is able or unable to identify the desired product associated with the voice purchase command based on the user101input received from the user computing device110.

If the account management system130is unable to identify the desired product associated with the voice purchase command based on the user101input, the method680returns to block750.

In block750, the account management system130transmits a request to the user computing device110for user101input to display on the user computing device110. For example, the account management system130requests second input from the user101via the user computing device110and receives second input101from the user computing device110submitted by the user101. In this example, the account management system130determines the user's101desired product based on the second input, any previous user101input, internal data received from the user computing device110, and/or external data received from the user computing device110. If the account management system130is still unable to identify the product the user101desires to purchase after receiving the second user101input, the account management system130may disregard the user's101voice purchase command.

Returning to block790, if the account management system130is able to identify the desired product associated with the voice purchase command based on the user101input, the method270proceeds to block280inFIG. 2.

Returning toFIG. 2, in block280, the account management system130directs user computing device to a merchant system website123to set up a transaction for identified product.

FIG. 8is a block diagram depicting a method270for directing a user computing device110to a merchant system website123to set up a transaction for an identified product associated with a voice purchase command, in accordance with certain example embodiments. The method270is described with reference to the components illustrated inFIG. 1.

In block810, the account management system130finds an offer associated with the identified product that the user101desires to purchase. In an example embodiment, in addition to identifying a product the user101desires to purchase from the internal data feed, external data feed, and/or user101input data received from the user computing device110, the account management system130identifies a merchant system120that sells the product. For example, the user101was watching a video on the user computing device produced by the merchant system120and the account management system130identifies the merchant based on the received internal user computing device110data describing the video. In this example embodiment, the account management system130accesses a website123of the merchant system130to identify an offer associated with the product. In another example embodiment, after identifying the product that the user101desires to purchase, the account management system130searches for offers from online merchant systems120and finds one or more offers offering the product for sale online. In this example embodiment, the account management system130selects an offer from the one or more offers and identifies a website123corresponding to the merchant system120offering the product for sale.

In block820, the account management system130redirects the user computing device web browser112to a merchant system website123offering the product for sale. In an example embodiment, the website123to which the user computing device110web browser112is redirected displays a virtual shopping cart with an icon representing the user's101desired product in the virtual shopping cart.

In block830, the user101submits payment information and shipping information to the merchant system website123. For example, the user101submits payment account information corresponding to a payment accounts of the user101, for example, a credit account, a bank account, or a merchant system120store credit account. In another example embodiment, the user101has a digital wallet account with the account management system130and the account management system130automatically fills in payment information for the user101to the website123and all the user101has to do is to confirm the transaction to initiate the transaction. In another example embodiment, in which the user101has a digital wallet account with the account management system130, a digital wallet application on the user computing device110or a digital wallet web browser112application may execute on the user computing device110requesting the user101to select a payment option from multiple payment options for which the user101has provided payment account information to the digital wallet account. In this example embodiment, the user101selects a payment account option on the appropriate digital wallet application and the account management system130fills in the payment information into the virtual shopping cart on the merchant system120website. In these example embodiments, the user101may also be required to provide a security code, a card verification code, or a password associated with selected or inputted payment account information.

In block840, the user101selects an option to initiate a payment transaction with the merchant system website123to purchase the product. For example, the user101activates one or more objects on the user interface119directing the merchant website to initiate the payment transaction. For example, the user101selects an option on the virtual shopping cart that reads “confirm payment.”

In block850, the merchant system120processes the payment transaction. In an example embodiment, the merchant system120website123receives, from the user computing device110via the network150, an indication of a user101selection of an option to confirm the payment transaction after the user101reviews the payment information and shipping information. In an example embodiment, the merchant system120communicates with an issuer system or financial institution associated with payment information received from the user101to process the payment transaction. For example, the user101submitted credit card payment information and the merchant system120transmits a payment authorization request to the credit card issuer via the credit card acquirer system comprising the payment information, the amount of the transaction, and payment account information associated with an account of the merchant system120. In this example embodiment, the issuer system approves the payment authorization request, schedules a payment in the transaction amount to an account of the merchant system120, adds a charge to the user101credit card account, and notifies the merchant system120that the payment authorization request was approved. In an example embodiment, the merchant system120transmits a receipt to the user computing device110comprising a summary of the approved transaction and an estimated shipping date.

Other Example Embodiments

FIG. 9depicts a computing machine2000and a module2050in accordance with certain example embodiments. The computing machine2000may correspond to any of the various computers, servers, mobile devices, embedded systems, or computing systems presented herein. The module2050may comprise one or more hardware or software elements configured to facilitate the computing machine2000in performing the various methods and processing functions presented herein. The computing machine2000may include various internal or attached components such as a processor2010, system bus2020, system memory2030, storage media2040, input/output interface2060, and a network interface2070for communicating with a network2080.

The computing machine2000may be implemented as a conventional computer system, an embedded controller, a laptop, a server, a mobile device, a smartphone, a set-top box, a kiosk, a vehicular information system, one more processors associated with a television, a customized machine, any other hardware platform, or any combination or multiplicity thereof. The computing machine2000may be a distributed system configured to function using multiple computing machines interconnected via a data network or bus system.

The processor2010may be configured to execute code or instructions to perform the operations and functionality described herein, manage request flow and address mappings, and to perform calculations and generate commands. The processor2010may be configured to monitor and control the operation of the components in the computing machine2000. The processor2010may be a general purpose processor, a processor core, a multiprocessor, a reconfigurable processor, a microcontroller, a digital signal processor (“DSP”), an application specific integrated circuit (“ASIC”), a graphics processing unit (“GPU”), a field programmable gate array (“FPGA”), a programmable logic device (“PLD”), a controller, a state machine, gated logic, discrete hardware components, any other processing unit, or any combination or multiplicity thereof. The processor2010may be a single processing unit, multiple processing units, a single processing core, multiple processing cores, special purpose processing cores, co-processors, or any combination thereof. According to certain embodiments, the processor2010along with other components of the computing machine2000may be a virtualized computing machine executing within one or more other computing machines.

The system memory2030may include non-volatile memories such as read-only memory (“ROM”), programmable read-only memory (“PROM”), erasable programmable read-only memory (“EPROM”), flash memory, or any other device capable of storing program instructions or data with or without applied power. The system memory2030may also include volatile memories such as random access memory (“RAM”), static random access memory (“SRAM”), dynamic random access memory (“DRAM”), and synchronous dynamic random access memory (“SDRAM”). Other types of RAM also may be used to implement the system memory2030. The system memory2030may be implemented using a single memory module or multiple memory modules. While the system memory2030is depicted as being part of the computing machine2000, one skilled in the art will recognize that the system memory2030may be separate from the computing machine2000without departing from the scope of the subject technology. It should also be appreciated that the system memory2030may include, or operate in conjunction with, a non-volatile storage device such as the storage media2040.

The storage media2040may include a hard disk, a floppy disk, a compact disc read only memory (“CD-ROM”), a digital versatile disc (“DVD”), a Blu-ray disc, a magnetic tape, a flash memory, other non-volatile memory device, a solid state drive (“SSD”), any magnetic storage device, any optical storage device, any electrical storage device, any semiconductor storage device, any physical-based storage device, any other data storage device, or any combination or multiplicity thereof. The storage media2040may store one or more operating systems, application programs and program modules such as module2050, data, or any other information. The storage media2040may be part of, or connected to, the computing machine2000. The storage media2040may also be part of one or more other computing machines that are in communication with the computing machine2000such as servers, database servers, cloud storage, network attached storage, and so forth.

The computing machine2000may operate in a networked environment using logical connections through the network interface2070to one or more other systems or computing machines across the network2080. The network2080may include wide area networks (WAN), local area networks (LAN), intranets, the Internet, wireless access networks, wired networks, mobile networks, telephone networks, optical networks, or combinations thereof. The network2080may be packet switched, circuit switched, of any topology, and may use any communication protocol. Communication links within the network2080may involve various digital or an analog communication media such as fiber optic cables, free-space optics, waveguides, electrical conductors, wireless links, antennas, radio-frequency communications, and so forth.

The processor2010may be connected to the other elements of the computing machine2000or the various peripherals discussed herein through the system bus2020. It should be appreciated that the system bus2020may be within the processor2010, outside the processor2010, or both. According to some embodiments, any of the processor2010, the other elements of the computing machine2000, or the various peripherals discussed herein may be integrated into a single device such as a system on chip (“SOC”), system on package (“SOP”), or ASIC device.

The example systems, methods, and acts described in the embodiments presented previously are illustrative, and, in alternative embodiments, certain acts can be performed in a different order, in parallel with one another, omitted entirely, and/or combined between different example embodiments, and/or certain additional acts can be performed, without departing from the scope and spirit of various embodiments. Accordingly, such alternative embodiments are included in the scope of the following claims, which are to be accorded the broadest interpretation so as to encompass such alternate embodiments.