Personal digital assistant

Disclosed herein are system, method, and computer program product embodiments for managing completion of a task. An embodiment operates by receiving a selection of a collection configured to complete the task. The embodiment determines a context indicator for analyzing an application object in the selected collection. The embodiment processes the application object in based on the context indicator to generate information for completing the task. Another embodiment operates by receiving a selection of a collection configured to complete a task. The embodiment then determines that completion of the task requires solving a situation defined by a situation definition. The embodiment then identifies an anchor application object in the selected collection based on the situation definition. The embodiment then generates related information and a set of actions for solving the situation based on the anchor application object and the situation definition.

BACKGROUND

Background

Computer users typically work on tasks that involve multiple complex steps. For example, a purchasing agent may want to create a purchase order. To create a purchase order, the purchasing agent may need to identify a list of suppliers, determine whether a specific supplier sells the needed type and quantity of products, and process a bill from the supplier for the products.

To create the purchase order, the purchasing agent may use several different software applications. For example, the purchasing agent may search a list of suppliers using a contact management software application. Similarly, the purchasing agent may process a supplier invoice using an accounting software application.

But a user may not think in terms of different software applications. Rather, the user may think in terms of completing the task. While different software applications provide predefined functionality for completing parts of a task, they may not enable the user to complete the task according to their personal way of working. Specifically, a user may have to remember relevant information when switching between different applications. In addition, a user may be forced to complete part of the task in one application before switching to another application. Finally, the user may not receive relevant predictions, notifications, and actions for one application while working in another application.

DETAILED DESCRIPTION

Provided herein are system, apparatus, device, method and/or computer program product embodiments, and/or combinations and sub-combinations thereof, for managing completion of a task. An embodiment operates by receiving a selection of a collection in collection list from a user. The embodiment determines a context indicator in response to receiving the selection. The embodiment then processes an application object in the selected collection based on the context indicator. The embodiment then provides information based on the processed application object to a user.

Another embodiment operates by receiving a selection of a collection in collection list from a user. The other embodiment determines a condition in a situation definition is valid. The other embodiment then identifies an anchor application object in the selected collection based the situation definition. The other embodiment then generates related information and a set of actions from the anchor application object. The other embodiment then provides the related information and the set of actions for output to a user.

FIG. 1is a block diagram of a personal assistant system100for managing completion of a task, according to some embodiments. Personal assistant system100may bridge the gap between predefined application functionality in separate applications and the personal way in which a user works on a task.

InFIG. 1, personal assistant system100includes server102and client device104. Server102may communicate with client device104operated by a user over network106. Network106may be any network or combination of networks including the Internet, a local area network (LAN), a wide area network (WAN), a wireless network, a cellular network, or various over types of networks as would be appreciated by a person of ordinary skill in the art.

In some embodiments, client device104may be any computing device that can communicate with server102including a desktop computer, laptop computer, tablet computer, smartphone, personal digital assistant, wearable electronic device, or various other electronic devices as would be appreciated by a person of ordinary skill in the art. Client device104may include personal assistant connector108. Personal assistant connector108may output a user interface on client device104for managing a task in personal assistant engine110. Specifically, personal assistant connector108may output a collection data structure corresponding to the task on client device104. In some embodiments, personal assistant connector108may be a software application that presents a user interface. For example, personal assistant connector108may be a web application.

In some embodiments, personal assistant connector108may receive user input to create, modify, or delete a collection data structure corresponding to a task in personal assistant engine110. Personal assistant connector108may receive user input to perform an action related to a task. Personal assistant connector108may output a prediction or notification related to a task.

FIG. 2is a block diagram of personal assistant engine110, according to an example embodiment. In some embodiments, personal assistant engine110may include a collection list202, users204, situations206, input module208, output module210, object analyzer212, logic engine214, notification engine216, natural language engine218, application connector220, and applications222.

In some embodiments, personal assistant engine110manages one or more collections in collection list202. Collection list202may be an array, linked list, or various other data structure as would be appreciated by a person of ordinary skill in the art.

In some embodiments, a collection is a data structure that may represent a user task in personal assistant engine110. For example, a collection may represent a purchase order issued by a business to a supplier. A collection may bridge the gap between predefined software application functionality and the personal way in which a user works by grouping together one or more application objects according to a user task. An application object may represent a logical entity in a software application.

FIG. 3is a block diagram of collection list202, according to an example embodiment. Collection list202includes one or more collections302-1to302-N. A collection302may be associated with a user in users204.

In some embodiments, a collection302may represent a user task. A collection302may include a collection title. A collection title may be a text description of a user task. A collection302may include a collection identifier that identifies the collection302to personal assistant engine110. In some embodiments, a collection identifier may be a universally unique identifier (UUID). A collection302may include a user identifier that identifies a user in users204that created the collection302.

In some embodiments, a collection302may include one or more application objects304related to the task of the collection302. An application object304may be a logical entity in a software application in applications222. For example, an application object304may be chat message in an instant messaging application, a note in a note taking application, a contact in a contact management application note, an image in a photo management application, an alert in an event planning application, an activity in a calendar application, a screenshot, or an aggregator of one or more other application objects304.

In some embodiments, a collection302may include one or more application objects304from the same software application. In some embodiments, a collection302may include one or more application objects304from different software applications.

In some embodiments, personal assistant engine110may store one or more collections302and one or more application objects304in a database. The database may be a relational database, object-oriented database, hierarchical database, network database, or various other type of database as would be appreciated by a person of ordinary skill in the art.

In some embodiments, an application object304may include a set of fields and a set of actions. A field in an application object304may be a data item in the associated logical entity. For example, an application object304for an email message may include an address field, from field, subject field, and message body field.

An action in an application object304may include application logic corresponding to the application in applications222. For example, an application object304may represent an email message and the set of actions may include send the email, reply to the email, forward the email, and delete the email. In some embodiments, executing an action may launch a component of the application or a new application in applications222. In some other embodiments, executing an action may open a web page, dial a telephone number, or various other actions as would be appreciated by a person of ordinary skill in the art.

In some embodiments, an application object304may include an application object identifier. The application object identifier may be an identifier that identifies the application object304to personal assistant engine110. In some embodiments, the application object identifier may be a universally unique identifier (UUID).

In some embodiments, an application object304may include an intent attribute. The intent attribute may control how output module210outputs the application object304. The intent attribute may also specify to input module208how a user navigates the application object304.

In some embodiments, an application object304may include an application object type. The application object type may identify a type of logical entity in an application in applications222. The application object type may identify the set of fields and set of actions accessible in the application object304.

In some embodiments, application connector220is an application programming interface (API) to applications202. Specifically, application connector202may enable communication between an application object304and a logical entity in an application in applications222. Application connector202may make the logical entity accessible from the application object304. Specifically, application connector220exposes data items of the logical entity as a set of fields in the application object304and application logic in the associated application as a set of actions in application object304.

In some embodiments, application connector220exposes data items and application logic to an application object304based on a manifest file. Application connector220may select a manifest file from a set of manifest files based on an application object type of the application object304. The manifest file may define a set of mappings between a set of data items of the logical entity and the set of fields in the application object304. The manifest file may also define a set of mappings between application logic in the associated application and the set of actions accessible from the application object304.

In some embodiments, the manifest file may be an Extensible Markup Language (XML) file. In some embodiments, the set of mappings may specified as a set of annotations of the data items of the logical entity and the application logic of the application.

In some embodiments, input module208may receive user input from personal assistant connector108to create, modify, or delete a collection302or application object304in collection list202. In some embodiments, the user input may textual data. In other some embodiments, the user input may be voice, gestures, touch data, visual data, or other types of user input as would be appreciated by a personal of ordinary skill in the art.

In some embodiments, the user input may be content data or a user command. For example, content data may include a first and last name for a contact, a message body of an email, and a date and time for an event. A user command may include a create collection command, delete application object command, and delete situation command. As would be appreciated by a personal of ordinary skill in the art, input module208may receive various other types of user command.

In some embodiments, input module208may convert user input from one Balm into another. For example, in some embodiments, input module208may convert a voice user input into textual content data using a voice recognition engine. In some other embodiments, input module208may convert a gesture user input into a user command.

In some embodiments, output module210may generate a user interface for personal assistant connector108. In some embodiments, the user interface may be a visual user interface. In some other embodiments, the user interface may be an auditory user interface. For example, output module210may output textual content data as audio content data using a text-to-speech recognition engine.

In some embodiments, output module210generates a user interface based on a template associated with an application object type of an application object304. The template may specify how to layout out user interface elements for the application object304. The template may also specify what data to display from the application object304and when to display the data.

In some embodiments, the template may specify the layout out of user interface elements for the application object304based on one or more device characteristics of client device104. A device characteristic may include the display screen size of client device104, the display screen orientation of client device104, or various other device characteristics of client device104as would be appreciated by a person of ordinary skill in the art. Thus, the template may enable output module210to provide a scalable user interface on different types of client device104.

In some embodiments, object analyzer212may save user time by auto filling one or more fields of an application object304in a collection302. Object analyzer212may also save user time by proposing one or more actions associated with an application object304. Object analyzer212may also save user time by generating notifications for application object304. Object analyzer may also save user time by generating a new application object304automatically or based on user demand. Thus, object analyzer212enables a collection302to be a living, gradually growing artifact that accompanies a user on their way from issue to solution.

In some embodiments, object analyzer212may determine a context indicator in order to analyze an application object304. Object analyzer212may determine the context indicator based on an application object type of the application object304, the associated collection302of application object304, an application object304currently being output to client device104by output module210, activity information of the user interacting with the application object304, a user role of the user interacting with the application object304, preferences of the user interacting with the application object304, and various other criteria as would be appreciated by a person of ordinary skill in the art. In some embodiments, object analyzer212may analyze the context indicator criteria using natural language engine216, machine learning techniques, or both.

In some embodiments, object analyzer212auto fills one or more fields of an application object304with content from the associated collection302. In some other embodiments, object analyzer212may auto fill the one or more fields of the application object304based on the determined context indicator.

In some embodiments, object analyzer212may provide one or more proposed actions for an application object304. Object analyzer212may generate the one or more proposed actions from the set of actions associated with the application object304. Object analyzer212may also generate the one or more proposed actions from a set of actions associated with a different application object304that is related to the application object304. Object analyzer212may provide the proposed actions to a user for execution based on the determined context indicator. Object analyzer212may rank the one or more proposed actions based on the determined context indicator.

In some embodiments, object analyzer212may generate a notification for an application object304. Object analyzer212may generate the notification from one or fields in the application object304and various criteria associated with application object304. Object analyzer212may provide the notification to notification engine216based on the determined context indicator. Notification engine214may deliver the notification to output module210for output on client device104.

In some embodiments, object analyzer212may proactively generate a new application object304by analyzing an existing application object304and highlighting one or more matching application objects304in collection list202. Specifically, object analyzer212may parse one or more fields of an application object304in a collection302. In some embodiments, object analyzer212may parse the one or more fields of the application object304using natural language engine216. In some embodiments, object analyzer212may parse the one or more fields of the application object304based on the determined context indicator.

In some embodiments, object analyzer212may compare one or more parsed fields in the application object304to fields in other application objects304. The other application objects304may be in the same or a different collection than the parsed application object304.

In some embodiments, object analyzer212may add the other application objects304to the same collection302as the parsed application object304based on the parsed application object304and the one or more other application objects304sharing identical field values. In some other embodiments, object analyzer212may provide a user an option to add the other application objects304to the same collection302as the parsed application object304based on the parsed application object304and the one or more other application objects304sharing identical field values.

In some embodiments, object analyzer212may add other application objects304to the same collection302as the parsed application object304based on the parsed application object304and the other application objects304sharing similar field values. In some other embodiments, object analyzer212may provide a user a selectable option to add the other application objects304to the same collection302as the parsed application object304based on the parsed application object304and the other application objects304sharing similar field values.

In some embodiments, object analyzer212may provide a selectable option to a user to generate a new application object304based on the parsed application object304having a field value similar to an existing application object304. For example, object analyzer212may provide a selectable option to a user to generate a new contact application object based on a person name being in an email application object.

In some embodiments, object analyzer212may parse an application object304upon creation. In some other embodiments, object analyzer212may parse an application object304in a collection302at regular time intervals.

In some embodiments, logic engine214performs situational analysis across a collection302. Situation analysis may enable improved user workflow across one or more different application objects304. Specifically, logic engine214may identify a situation in which a user in users204may need additional information, or may need to take an action to complete a task. In response, logic engine214may provide additional related information and a set of actions to the user to solve the situation.

For example, logic engine214may identify an overdue purchase order and provide additional related information to a purchasing agent user regarding the overdue purchase order. The related information may include supplier information. The logic engine214may also identify one or more actions the purchasing agent may need to take to solve the overdue purchase order situation. The actions may include adjusting the purchase order quantity, canceling the purchase order, splitting the purchase order, and ignoring the overdue purchase order. Thus, logic engine214may provide related information and actions for solving a task to a user at the right time.

In some embodiments, logic engine214performs situational analysis according to one or more situation definitions in situations206. In some embodiments, a user in users204may create a situation definition in situations206. In some other embodiments, logic engine214may generate a situation definition in situations206based on a machine learning analysis of collections list202.

In some embodiments, a situation definition includes one or more anchor application objects, a condition, a situation trigger, related information, and one or more actions capable of being performing by a user in users204.

In some embodiments, an anchor application object is an application object304in a collection302. An anchor application object may act as an entry point for retrieving related information and a set of actions. In some embodiments, the related information may include one or more related application objects304. In some other embodiments, the related information may include information stored outside collection list202.

In some embodiments, the set of actions may include one or more actions defined for the one or more anchor application objects. In some other embodiments, the set of actions may include the one or more actions for application objects304related to the one or more anchor application objects.

In some embodiments, an anchor application object may include a path expression that defines one or more application object304that are related to the anchor application object. In some embodiments, a path expression may relate one or more application objects to the anchor application object based on an application object type. Logic engine214may evaluate the path expression to retrieve one or more related application objects304having the application object type.

In some embodiments, an anchor application object may include an external link to information stored outside collection list202. For example, an anchor application object may include an external link to a related web page.

In some embodiments, a condition in a situation definition is an application rule, query, or an application relevant event. For example, a condition may be an overdue purchase order. As would be appreciated by a person of ordinary skill in the art, a condition may be specified using various notational forms.

In some embodiments, a trigger controls when a condition is evaluated. A trigger may specify to logic engine214that the associated condition needs to be evaluated periodically such as once a week, daily, or hourly. In some other embodiments, a trigger may specify a system update event such as the creation of a new application object304, e.g., a new purchase order application object. In some other embodiments, a trigger may be an active request by a user in users204to have the latest information from personal assistant engine110.

In some embodiments, a situation definition may include a textual description of the situation to solve. The textual description may include corporate rules, department rules, or links to relevant process information. In some embodiments, a situation definition may include a situation type.

In some embodiments, a situation definition includes one or more tags to form a group of situations having similar attributes. This enables assigning a group of situations to groups of users or individual users. A tag may be an application domain, application area, department, group of users, list of individual users, or various other tag types as would be appreciate by a person of ordinary skill in art.

In some embodiments, a situation definition may include a calculation method for determining a business relevance score. A business relevance score may control which users in users204receive the related information and set of actions associated with the situation.

In some embodiments, logic engine214may evaluate the calculation method for determining the business relevance score using one or more fields in the anchor application object. In some other embodiments, logic engine214may evaluate the calculation method for determining the business relevance score using application logic and rules of an organization.

In some embodiments, a situation definition may include a calculation method for determining a user relevance score. A user relevance score may control which users in users204receive the related information and set of actions associated with the situation. In some embodiments, logic engine214may evaluate the calculation method for determining a user relevance score based on a user identifier associated with the anchor application object. Specifically, logic engine214may use the user identifier to retrieve user information from users204. Logic engine214may then use the received user information in evaluating the calculation method for determining the user relevance score.

In some embodiments, the user information may include a history of user interactions with the anchor application object, tags assigned to the anchor application object, user activity associated with application objects304related to the anchor application object, the user assigned to the anchor application object, the role of the user assigned to the anchor application object, and various other criteria as would be appreciated by a person of ordinary skill in the art.

In some embodiments, the situation definition may include notification criteria. The notification criteria may control when logic engine214presents related information and a set of actions of a situation to a user via notification engine216. In some embodiments, notification criteria may include a relevance value and threshold value. The relevance value may be calculated from a business relevance score, user relevance score, or a combination of both. In some embodiments, logic engine214may present the related information and the set of actions of a situation to a user when the relevance value exceeds the threshold value.

In some embodiments, logic engine214evaluates a situation condition based on one or more application objects304. Logic engine212may generate a situation instance with related information and a set of actions when the associated condition is valid.

In some embodiments, logic engine214may present a situation instance to a user based on a business relevance score, a user relevance score, or both. In some embodiments, logic engine214presents all related information and actions required for handling the situation instance to a user.

In some embodiments, output module210generates a user interface for outputting a situation instance based on a template associated with the definition of the situation. The template may specify to output module210how to layout out user interface elements for the situation. The template may also specify to output module210what data to display from the situation and when to display the data.

In some embodiments, the template may specify the layout out of user interface elements for the situation instance based on one or more device characteristics of client device104. A device characteristic may include the display screen size of client device104, the display screen orientation of client device104, or various other device characteristics of client device104as would be appreciate by a person of ordinary skill in the art.

FIG. 4is a flowchart for a method400for managing a task using personal assistant engine110, according to an embodiment. Method400can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown inFIG. 4, as will be understood by a person of ordinary skill in the art.

Method400shall be described with reference toFIGS. 2 and 3. However, method400is not limited to that example embodiment.

In402, input module208receives a selection of a collection302in collection list202from a user in users204. The selection may comprise a collection identifier that identifies the collection302in collection list202.

In404, object analyzer212determines a context indicator in order to analyze an application object304in the selected collection302. In some embodiments, the object analyzer212may determine the context indicator based an application object type of the application object304, the selected collection302, activity information of the user interacting with the application object304, a user role of the user interacting with the application object304, preferences of the user interacting with the application object304, and various other criteria as would be appreciated by a person of ordinary skill in the art.

In404, object analyzer212processes the application object304in the selected collection302based on the context indicator. Specifically, object analyzer212may auto fill one or more fields of the application object304. Object analyzer212may also generate one or more proposed actions for the application object304. Object analyzer212may also generate a new application object based on the application object304. Object analyzer212may also generate a notification based on the application object304.

In some embodiments, object analyzer212may process the application object304upon creation of the application object304in the collection302. In some other embodiments, object analyzer212may process the application object302in the collection302at periodic intervals.

In406, object analyzer212provides information based on the processed application object304to output module210for output to a user.

FIG. 5is a flowchart for a method500for performing situational analysis using personal assistant engine110, according to an embodiment. Method500can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown inFIG. 5, as will be understood by a person of ordinary skill in the art.

Method500shall be described with reference toFIGS. 2 and 3However, method500is not limited to that example embodiment.

In502, input module208receives a selection of a collection302in collection list202from a user in users204. The selection may comprise a collection identifier that identifies the collection302in collection list202.

In504, logic engine214executes a trigger in a situation definition associated with the selected collection302.

In506, logic engine214determines a condition in a situation definition in situations206is valid in response to the executed trigger.

In508, logic engine214identifies an anchor application object in the selected collection302based the situation definition.

In510, logic engine214generates related information and a set of actions from the anchor application object according to the situation definition. The related information may include one or more related application objects304. The related information may also include information stored outside collection list202. The set of actions may include one or more actions defined for the one or more anchor application object. The set of actions may include the one or more actions for one or more application objects304related to the anchor application object.

In512, logic engine214optionally calculates a relevancy score for the related information and the set of actions based on the anchor application object and the situation definition. In some embodiments, the relevancy score may be a business relevance score, user relevancy score, or a combination of both.

In514, logic engine214provides the related information and the set of actions to output module210for output to a user based on the relevancy score.

Various embodiments can be implemented, for example, using one or more computer systems, such as computer system600shown inFIG. 6. Computer system600can be used, for example, to implement method400ofFIG. 4. For example, computer system600can process an application object in a selected collection based on a context indicator. Computer system600can further generate related information and a set of actions from an application object based on a situation definition, according to some embodiments. Computer system600can be any computer capable of performing the functions described herein.

Computer system600can be any well-known computer capable of performing the functions described herein.

Computer system600includes one or more processors (also called central processing units, or CPUs), such as a processor604. Processor604is connected to a communication infrastructure or bus606.

Computer system600also includes a main or primary memory608, such as random access memory (RAM). Main memory608may include one or more levels of cache. Main memory608has stored therein control logic (i.e., computer software) and/or data.

Computer system600may also include one or more secondary storage devices or memory610. Secondary memory610may include, for example, a hard disk drive612and/or a removable storage device or drive614. Removable storage drive614may be a floppy disk drive, a magnetic tape drive, a compact disk drive, an optical storage device, tape backup device, and/or any other storage device/drive.

Removable storage drive614may interact with a removable storage unit618. Removable storage unit618includes a computer usable or readable storage device having stored thereon computer software (control logic) and/or data. Removable storage unit618may be a floppy disk, magnetic tape, compact disk, DVD, optical storage disk, and/any other computer data storage device. Removable storage drive614reads from and/or writes to removable storage unit618in a well-known manner.

According to an exemplary embodiment, secondary memory610may include other means, instrumentalities or other approaches for allowing computer programs and/or other instructions and/or data to be accessed by computer system600. Such means, instrumentalities or other approaches may include, for example, a removable storage unit622and an interface620. Examples of the removable storage unit622and the interface620may include a program cartridge and cartridge interface (such as that found in video game devices), a removable memory chip (such as an EPROM or PROM) and associated socket, a memory stick and USB port, a memory card and associated memory card slot, and/or any other removable storage unit and associated interface.

Computer system600may further include a communication or network interface624. Communication interface624enables computer system600to communicate and interact with any combination of remote devices, remote networks, remote entities, etc. (individually and collectively referenced by reference number628). For example, communication interface624may allow computer system600to communicate with remote devices628over communications path626, which may be wired and/or wireless, and which may include any combination of LANs, WANs, the Internet, etc. Control logic and/or data may be transmitted to and from computer system600via communication path626.

In an embodiment, a tangible apparatus or article of manufacture comprising a tangible computer useable or readable medium having control logic (software) stored thereon is also referred to herein as a computer program product or program storage device. This includes, but is not limited to, computer system600, main memory608, secondary memory610, and removable storage units618and622, as well as tangible articles of manufacture embodying any combination of the foregoing. Such control logic, when executed by one or more data processing devices (such as computer system600), causes such data processing devices to operate as described herein.