Source: https://patents.google.com/patent/US10007897B2/en
Timestamp: 2019-04-21 13:19:41+00:00

Document:
2013-07-30 Assigned to MICROSOFT CORPORATION reassignment MICROSOFT CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: KIM, JANE T., AUER, NATHANIEL E.B., DANHOPE, STEFANIE J., REYES, LORRISSA J., GIACOBBE, GAIL, POSSING, DENIS L.
The description relates to auto-calendaring. One example can obtain content relating to a calendarable event. The example can also communicate the content to an auto-calendaring service. The example can also receive a response from the auto-calendaring service. The response can indicate that the calendarable event has been added to a calendar or can request further information about the calendarable event.
Electronic calendars have been in use for quite some time. However, many events never make it to the calendar because of the perceived inconvenience to the user to manually add the event to his/her calendar. For instance, the user may think it takes too long to add the event to his/her calendar. As a result, the user may procrastinate and not add the event to the calendar or may just write it down on a piece of paper. Further, the user may be utilizing a type of device, such as a smart phone that does not lend itself to traditional manual calendar entry. As such, the user's electronic calendar tends to be an incomplete representation of their schedule which may cause them to inadvertently double-book events or miss events because they forget about them.
The described concepts relate to auto-calendaring. One example can obtain content relating to a calendarable event. The example can also communicate the content to an auto-calendaring service. The example can also receive a response from the auto-calendaring service. The response can indicate that the calendarable event has been added to a calendar or can request further information about the calendarable event.
Another example can obtain a data item from a user and extract text from the data item. The example can analyze the text to identify at a first confidence level whether the text includes a potential event. This example can also determine at a second confidence level whether the potential event relates to a known event. The example can further identify at a third confidence level whether the potential event matches an event profile of the user. The example can take an action on behalf of the user based upon the first, second, and third confidence levels.
The above listed examples are intended to provide a quick reference to aid the reader and are not intended to define the scope of the concepts described herein.
The accompanying drawings illustrate implementations of the concepts conveyed in the present document. Features of the illustrated implementations can be more readily understood by reference to the following description taken in conjunction with the accompanying drawings. Like reference numbers in the various drawings are used wherever feasible to indicate like elements. Further, the left-most numeral of each reference number conveys the figure and associated discussion where the reference number is first introduced.
FIGS. 1A-1C, 2, and 3 illustrate a computing device upon which auto-calendaring functionalities can be accomplished in accordance with some implementations of the present concepts.
FIG. 4 illustrates an example of a system upon which auto-calendaring functionalities can be accomplished in accordance with some implementations of the present concepts.
FIGS. 5-6 show examples of auto-calendaring methods in accordance with some implementations of the present concepts.
This patent relates to auto-calendaring. The user can have any type of data item, such as an email, a picture, a text communication, etc. and indicate that he/she wants to utilize auto-calendaring. The auto-calendaring can detect one or more events from the data item and auto-calendar some or all of the events to the user's calendar. Thus, with as little as a single action, the user can have events automatically added to his/her electronic calendar.
For purposes of explanation consider FIGS. 1A-1C which collectively show a computing device 102 in several consecutive instances (e.g., FIG. 1A shows instances 1-3, FIG. 1B shows instances 4-6, FIG. 1C shows instances 7-8). In this example, computing device 102 is manifest as a smart phone type mobile computing device that has a display 104. The computing device 102 can be configured to present a graphical user interface (GUI) 106 on the display 104.
While a smart phone type mobile computing device is illustrated, the concepts apply to other types of computing devices, such as notebook computers, desktop computers, and/or tablet computers, among others. For purposes of explanation, assume that computing device 102 belongs to a user named “Denis”.
In this example, the computing device 102 can provide multiple functionalities or features to the user. As illustrated in Instance 1, the features can include an email functionality 108, a phone functionality 110, a text functionality 112, a camera functionality 114, and a web browsing functionality 116. Of course, the computing device can include alternative or additional functionality, such as a media player functionality and/or a social media functionality, among others. Assume for purposes of explanation that in Instance 1, the user has selected the email functionality 108.
Instance 2 shows the computing device 102 responsive to the user selection related to Instance 1. In this case, the computing device is now displaying features relating to the email functionality 108. Specifically, the GUI relates to Denis' inbox as indicated at 118. In this example, the inbox shows two emails 120. The first email 120(1) is from Gail as indicated at 122. The second email 120(2) is from Jane as indicated at 124.
In this case, assume that Denis reads Gail's email 120(1) and wants to go to dinner. Toward this end, he can tap the display 104 (or otherwise control the device).
Instance 3 shows subsequent GUI 106 that allows Denis to “Reply”, “Forward”, or “Delete” the email from Gail at 126. Assume for purposes of explanation that Denis selects “Reply”.
Instance 4 shows a subsequent GUI 106 that shows the reply email to Gail at 128. Further, Denis has added text at 130 that says “Sounds good. See you then. -Denis”. Assume for purposes of explanation that Denis would like to have the dinner event on his calendar, but does not want to take the time to add it manually.
Instance 5 shows the GUI 106 with a feature for automatically calendaring the dinner event for Denis. In this case, Denis simply adds “Auto-calendar” as a CC address to the reply email as indicated at 132. In this case “Auto-calendar” can be thought of as an alias for an auto-calendar feature or service rather than the actual email address. At this point, assume that Denis sends the reply email to Gail.
Instance 6 shows a subsequent GUI 106. This example shows a new email 134 in Denis' inbox 118. The new email 134 is from the auto-calendar service. The new email indicates at 136 that the dinner event has been added to Denis' calendar. The email also offers the option for Denis to view details at 138 or view his calendar at 140. Instance 7 shows an example where Denis selects to view details 138. Instance 8 shows an example where Denis selects to view his calendar. Of course, the illustrated email response is only one possible email response that could be generated by the auto-calendaring service. Another example may indicate to Denis that the event occurs at a time that is already booked on Denis' calendar and ask for clarification. Other examples are described below.
Instance 7 shows an auto-calendar view 142 of the scheduled dinner event. Details of the scheduled dinner event include “When: Tomorrow at 6:00 P.M.” as indicated at 144, “Who: Gail and You” as indicated at 146, and “Where: Undetermined” at 148. Notice also that at 150, the auto-calendar view offers the user the option of receiving suggestions for the undecided detail (e.g., “(Click here for suggestions)”). Thus, Denis could utilize this feature to find restaurants for dinner. For example, the restaurants could be selected based upon various parameters, such as proximity to Gail and Denis, ratings of restaurants from the web, known preferences of Gail and/or Denis, past dinner calendar events that Gail and/or Denis have attended, targeted advertisements, and/or coupons, among others.
Instance 8 shows a version of GUI 106 generated responsive to Denis selecting to see his calendar at Instance 6. In this case, a calendar view 152 is presented for Denis. The Calendar view shows the day the dinner event was auto-calendared for (e.g., “Tomorrow, Jan. 19, 2013”) as indicated at 154. The dinner event can be seen at 6:00 P.M. as indicated at 156. In summary, the present auto-calendaring concepts enable a user to have an event populated to their calendar with very little effort on their part. In this example, the user simply added an email address of the auto-calendaring service as a recipient of an email that included details about the event. Of course, this is just one example of how the auto-calendaring service can be invoked. For example, the user could alternatively forward an existing email to the auto-calendaring service.
FIG. 2 shows another auto-calendaring implementation on a device 202 at Instances 9-12. In this case, as seen in Instance 9, assume that a user of the device sees a flyer 204 for a concert. The user looks at the flyer and decides that he/she wants to go to the concert. In this implementation, the user can take a picture of the flyer 204 with device 202 as indicated at 206.
Instance 10 shows a picture of the flyer on device 202 at 208. The device can offer the user various options for the picture. The options can be accessed in various ways. In this case, assume that the user taps the picture (or otherwise requests to take an action relative to the picture, such as through a voice command or gesture).
Instance 11 shows a set of options for the picture at 210. In this case, the options include a “Save” option at 212, a “Delete” option at 214, and an “Auto-calendar” option 216. In this case, assume that the user wants to have the picture auto-calendared and selects the “Auto-calendar” option 216. The picture can then be auto-calendared for the user. The auto-calendaring can be performed by a user-invokable auto-calendaring service that is resident on the device 202. Alternatively, the auto-calendaring service can be remote from the device. For instance, user selection of the auto-calendaring option can cause the picture 210 to be communicated to the auto-calendaring service. For instance, the picture can be emailed to the auto-calendaring service. The auto-calendaring service can then add the event from the picture to the user's calendar.
Instance 12 shows a confirmation 218 that the event of the picture has been added to the user's calendar. Note that in this case, greater detail is included in the auto-calendar event indicated in the confirmation than was contained in the picture itself. The auto-calendaring service can utilize various resources to obtain these details. For example, the auto-calendaring service can use optical character recognition (OCR) technology to identify the text in the picture. The auto-calendaring service could also submit metadata (such as GPS data) from the picture to a location resource to determine that the picture was taken at Marymoor Park in Redmond, Wash., for example. The auto-calendaring service could then submit the text data and the location data to a search engine resource. Results from the search engine could confirm that in fact there is a concert scheduled for Marymoor Park on that day and at the specified time (rather than the more ambiguous “sundown” provided in the flyer).
FIG. 3 shows another auto-calendaring scenario at instances 13-16. This auto-calendaring scenario relates to auto-calendaring in a texting environment represented on a mobile computing device 302.
At Instance 13, for purposes of explanation, assume that the user Denis sent an initial text to Jane and Lorissa asking “Hey do you want to go to the zoo tomorrow?” as indicated at 304. As indicated at 306, Jane replied “I'm in. Let's go at 2.” Similarly, Lorissa replied “I'm in too, but I can't go until 3.” as indicated at 308. Finally, Denis replies “Ok, see you then.” as indicated at 310. Assume for purposes of explanation that Denis lives near Seattle, Wash. and that Seattle has a ‘zoo’ (i.e., a place to view animals) and a (hypothetical) sports bar named ‘the zoo’. These aspects will be discussed below beginning at Instance 15.
At this point, assume that Denis wants to use the auto-calendar feature to add the event to his calendar. At Instance 14, Denis uses a verbal command (e.g., “Auto-calendar”) as indicated at 312 to control his mobile computing device 302. (Of course, other types of commands can be utilized). The verbal auto-calendar command can cause content (e.g., ‘text’) described relative to Instance 13 to be obtained by the auto-calendaring service. Alternatively, a screen capture can be performed on the mobile computing device's display and OCR used to obtain the text from the screen capture. The text can then be analyzed to generate the event. Two potential outcomes are described below relative to Instances 15-16.
Instance 15 shows one potential response 314 to the user's auto-calendaring command of Instance 14. In this case, the auto-calendaring service detected an event in the text. However, in this case, some of the details of the event are ambiguous to the auto-calendaring service so the potential response 314 requests clarification from the user. The other details are populated into the calendar. In this example, the clarification relates to the ‘where’ detail (e.g., location) of the event as indicated at 316. Assume for purposes of example that the auto-calendaring service accessed search engine resources relative to the term “zoo” relative to the Seattle area and obtained results for “Woodland Park Zoo” and “The Zoo sports bar”. In this case, the auto-calendaring service was not able to determine with a high degree of certainty which location the user intended and thus requested the clarification. The user can make the clarification. The user clarification can then be utilized to update the calendar event. In summary, the response can indicate what action was taken on behalf of the user by the auto-calendaring service and allow the user to provide clarification that can be utilized to augment the action.
Instance 16 shows an alternative potential response 318. In this case, assume that the auto-calendaring service accessed search engine resources relative to the term “zoo” relative to the Seattle area and obtained results for “Woodland Park Zoo” and “The Zoo sports bar” as described relative to Instance 15. However, assume further that the auto-calendaring service also applied a user profile (e.g., Denis' profile) to the search results. Assume in this example that Denis' profile indicated that Denis is an avid sports fan who especially likes watching football. However, the profile does not include any information about animals. Assume further that the auto-calendaring service returns to the search engine resources with the information from the user profile and discovers that the championship football game is tomorrow at 3:00 P.M. and that the game will be shown at The Zoo sports bar. With this information, the auto-calendaring service can automatically calendar the event with a relatively high degree of certainty that the details of the event are correct.
To summarize, in this latter case, the auto-calendaring service determined that the supplied content (e.g., the text history) contained an event, what the event was, and that the event was something that the user was likely to do. In some implementations, where the confidences associated with these determinations are above respective threshold levels, the event may be auto-calendared without sending a reply to the user (e.g., when the confidence is high that the event is ‘correct’ the event can calendared without bothering the user with a notification). Stated another way, the auto-calendaring service can determine similarity scores between the event detected in the supplied content and results returned from the search engine resource to determine a likelihood that the identified (e.g., potential event) corresponds to a known event. Similarly, the auto-calendaring service can determine similarity scores between the event detected (and/or the known event) and the user profile to determine if the detected event and/or the known event match something that the user is likely to do.
In an alternative situation, if the confidence levels are above a first threshold value, the event can be tentatively scheduled and presented to the user for approval (e.g., confirmation). If the confidence levels are below both thresholds, the user can be queried for more information that can then be used in the auto-calendaring process.
FIG. 4 shows system 400 that can enable the auto-calendaring concepts described above. In this example, the system 400 includes several devices 402. In this case, the devices are manifest as a smartphone type computer 402(1), a pad type computer 402(2), a notebook type computer 402(3), and a set of cloud-based server type computers 402(4). Smartphone type computer 402(1) is similar to devices 102, 202, and 302 described above relative to FIGS. 1A-1C, 2, and 3, respectively. (In this discussion, the use of a designator with the suffix, such as “(1)”, is intended to refer to a specific device instance. In contrast, use of the designator without a suffix is intended to be generic). Of course not all device implementations can be illustrated and other device implementations should be apparent to the skilled artisan from the description above and below.
The devices 402 can communicate over one or more networks 404 (represented by ‘lightning bolts’). The devices can also communicate with resources 406. Non-limiting examples of resources can include a global contacts/calendaring service resources 406(1), enterprise directory service resources 406(2), search engine resources 406(3), and monetization engine resources 406(N). Other non-illustrated examples of resources can include optical character recognition resources, machine learning resources, natural language processing/generating resources, and/or user profile database resources, among others. In some cases, the present concepts can be implemented by an individual device 402 acting in isolation. In other cases, the device can implement the present concepts by operating cooperatively with one or more other devices and/or the resources 406. These variations are described in more detail below.
Devices 402 can include several elements which are defined below. For example, these devices can include a processor 410, storage/memory 412, and/or an auto-calendaring component 414. The auto-calendaring component can perform some or all of the auto-calendaring services described above. The devices can alternatively or additionally include other elements, such as input/output devices (e.g., touch, voice, and/or gesture), buses, graphics cards, Wi-Fi circuitry, cellular circuitry, positional circuitry (absolute location (e.g., GPS)) and/or relative location (accelerometers, magnetometers, among others) etc., which are not illustrated or discussed here for sake of brevity.
The term “device”, “computer”, or “computing device” as used herein can mean any type of device that has some amount of processing capability and/or storage capability. Processing capability can be provided by one or more processors (such as processor 410) that can execute data in the form of computer-readable instructions to provide a functionality. Data, such as computer-readable instructions, can be stored on storage, such as storage/memory 412 that can be internal or external to the computer. The storage can include any one or more of volatile or non-volatile memory, hard drives, flash storage devices, and/or optical storage devices (e.g., CDs, DVDs, etc.), among others. As used herein, the term “computer-readable media” can include signals. In contrast, the term “computer-readable storage media” excludes signals. Computer-readable storage medium/media includes “computer-readable storage devices.” Examples of computer-readable storage devices include volatile storage media, such as RAM, and non-volatile storage media, such as hard drives, optical discs, and flash memory, among others.
Examples of devices 402 can include traditional computing devices, such as personal computers, desktop computers, notebook computers, cell phones, smart phones, personal digital assistants, pad type computers, mobile computers, cameras, or any of a myriad of ever-evolving or yet to be developed types of computing devices. A mobile computer can be any type of computing device that is readily transported by a user and may have a self-contained power source (e.g., battery).
In the illustrated implementation, devices 402 are configured with a general purpose processor 410 and storage/memory 412. In some configurations, a device can include a system on a chip (SOC) type design. In such a case, functionality provided by the device can be integrated on a single SOC or multiple coupled SOCs. One or more processors can be configured to coordinate with shared resources, such as memory, storage, etc., and/or one or more dedicated resources, such as hardware blocks configured to perform certain specific functionality. Thus, the term “processor” as used herein can also refer to central processing units (CPU), graphical processing units (CPUs), controllers, microcontrollers, processor cores, or other types of processing devices suitable for implementation both in conventional computing architectures as well as SOC designs.
In some configurations, the auto-calendaring component 414 can be installed as hardware, firmware, or software during manufacture of the device 402 or by an intermediary that prepares the device for sale to the end user. In other instances, the end user may install the auto-calendaring component 414, such as in the form of a downloadable application. Further, in some instances individual devices 402 can include robust auto-calendaring components. In other cases individual devices may have less robust or thin auto-calendaring components where a majority of the functionality is performed by other devices, such as cloud based devices, for presentation on the thin device. In some cases, the local device (e.g., the user's device) can provide a web-view of content generated remotely, such as by the cloud based devices.
Stated another way, in some implementations, an individual device, such as device 402(1) may have a less robust instance of the auto-calendaring component 414(1) such that some or all of the functionality provided by the auto-calendaring component 414(1) is performed remotely, such as at cloud-based device 402(4) and communicated back to device 402(1) for presentation to the user. Further, the auto-calendaring component may include individual resources 406 or access individual resources. For example, the auto-calendaring component may include a natural language processing/generation resource or may access a remote natural language processing/generation resource.
The auto-calendaring component 414 can be a freestanding application or the auto-calendaring component can be an element of a contact management application or a calendaring application, among others. Examples of contact management applications can include Outlook® from Microsoft® Corporation, Apple Contacts™, and/or Google Gmail™.
Auto-calendaring component 414 can be configured to cause a calendar related GUI to be presented to a user on the user's computing device. The auto-calendaring component can derive event data from content submitted by the user. This event data can then be utilized with various resources to determine whether the event data relates to one or more events, what the events are, and whether the user is likely to participate in such events.
The resources 406 can be local or remote relative to the auto-calendaring component 414. The auto-calendaring component can utilize individual resources to obtain data related to user supplied content that may relate to an event. The obtained data can then be leveraged with other resources. For instance, the auto-calendaring component 414 can utilize OCR resources to derive text from user supplied content. The auto-calendaring component 414 can supply the output derived text from the OCR resources to natural language processing resources to analyze the derived text. The auto-calendaring component can then utilize the output of the natural language processing resources to facilitate a meaningful search utilizing the search engine resources. The auto-calendaring component can utilize information from a user profile to refine the search. In the absence of a user profile or in addition to a user profile, the auto-calendaring component can utilize other resources relating to the user, such as data from their personal data (e.g., global contacts/calendaring service resources 406(1) or work data enterprise directory service resources 406(2)) to further determine or refine the event details.
Of course, security of personal confidential information can be maintained while implementing the inventive concepts. The user can be queried for permission prior to any access to, or use of, personal data. Further, any use of the personal data can be strictly limited to the authorized uses and be securely maintained. Also, the user permission may be conditional and/or relate to specific data sources. For example, the user may give permission to use their data to construct the user profile but not for any use other use. Further, the user may authorize access to their calendar data, but not their email data, for instance. Further still, if the user does not give consent to use of their personal information, the present concepts can be employed satisfactorily with the available resources.
As mentioned above, the auto-calendaring component 414 can access the user's profile as a resource. In some cases, the auto-calendaring component 414 can create and/or contribute to (e.g., update) the user profile. The user profile can be created manually (e.g., by asking the user a set of questions about his/her interests and activities). Alternatively or additionally, the user profile can be constructed from analysis of user data. For instance, the user's calendar can be accessed to identify what types of events the user has participated in previously. Similarly, the user's texts and/or emails can be analyzed to construct the user profile.
Note further that the user profile may include multiple user profiles. For example, the search engine resource may generate a user profile related to the user's search history and an email application may generate a profile of the user's email history. These various user profiles can contribute information that can be utilized by the auto-calendaring component 414.
In summary, the user profile can indicate details of what the user tends to do and/or other details, such as whom they do certain types of events with and/or where they do them. Further, the user profile can be searchable in various ways. For example, the user may tend to engage in certain types of events during the week after work with people from work and engage in different activities on the weekend with different people. This information can contribute to the accuracy of identifying the correct events from user supplied content.
The auto-calendaring component 414 can aid the user in creating a user account or otherwise configuring auto-calendaring services. For instance, the user can identify who they are (e.g., supply their account information) and indicate which calendar(s) they want the auto-calendared events to be populated to. For instance, the user may define that certain types of events be calendared to their work calendar and other types of events be calendared to their private calendar. For example, the user can define that emails from certain addresses or containing certain words, go to a particular calendar.
FIG. 5 shows a flowchart of a method 500 relating to auto-calendaring.
The method can obtain content relating to a calendarable event at 502. Examples of content are described above. Content can be thought of as anything that the user desires to generate a calendar event from. For example, the content can be manifest as text and/or media. As such, the content can be thought of as a data item that can be processed by, or on behalf of, an auto-calendaring service.
The method can communicate the content to an auto-calendaring service at 504. In some cases the communicating can occur within a device. For instance, an image, screenshot, or text displayed on a device can be communicated to the auto-calendaring service resident on the device. For example, the auto-calendaring service can be provided by an auto-calendaring component, such as an application stored on the device. In other configurations, the auto-calendaring service can be remote from the device and the communicating can entail communicating the content over a network to the remote auto-calendaring service.
The method can receive a response from the auto-calendaring service at 506. In some cases, the response indicates that the calendarable event has been added to a calendar or requests further information about the calendarable event.
FIG. 6 shows a flowchart of a method 600 relating to auto-calendaring.
The method can obtain a data item from a user at 602. In some cases, the data item can be obtained from the user via text or email, among other technologies. Examples of data items can include a text, an email, a video, a document, a data container, and/or an image, or multiple instances of any one or more of these, among others.
The method can extract text from the data item at 604. In some cases, optical character recognition can be performed on the data item to extract the text. In other cases, voice recognition processing can be performed on the data item to extract the text. In some cases, various types of text may be processed to obtain plain text. For instance, the data item may be manifest as hypertext markup language (HTML) text. The HTML text can be processed to obtain plain text.
The method can analyze the text to identify at a first confidence level whether the text includes a potential event at 606. The analyzing can include identifying at least some details of the potential event, such as what the potential event relates to, where the potential event occurs, and/or whom the potential event relates to. The presence of these details can contribute to determining the confidence level that an event has been identified. Of course, an event can be identified without all of the details, for instance, an event may include ‘when’ details and ‘who’ details, but no ‘where’ details, or some combination thereof.
The method can determine at a second confidence level whether the potential event relates to a known event at 608. For instance, the determining can include comparing the details identified at block 606 to search results produced by a search engine based upon the details. The weight given to the second confidence level can be adjusted based upon the event details of block 606. For instance, an event that appears to be a private event will likely have a lower correspondence to the search results than a more public event. Thus, if the details are “dinner with Jane on Friday at 6” the correlation to the search results tends not to be very determinative. However, if the details are “Apple Cup football game at Husky stadium on November 19th” and the search results show that to be a real event, the determinative value of satisfying the second confidence level can be very high.
The method can identify at a third confidence level whether the potential event matches an event profile of the user at 610. For example, the event profile can include previous events that the user has participated in and interests of the user. Continuing with the above example, if the user's profile indicates that the user is a big college football fan, a Husky fan, and has attended college football games at Husky stadium in the past, the third confidence level can be very high and very determinative.
The method can take an action on behalf of the user based upon the first, second, and third confidence levels at 612. For example, in an instance where the first, second, and third confidence levels are above a predefined value, the action can be automatically scheduling the potential event on a calendar(s) associated with the user.
Alternatively, in an instance where one or more of the first, second, and third confidence levels are above a predefined value, but below a second higher predefined value, then the action can be tentatively scheduling the potential event on a calendar(s) associated with the user and requesting a confirmation from the user. In still another instance where one or more of the first, second, and third confidence levels are below a predefined value, the action can be requesting more information from the user prior to auto-calendaring the potential event.
The order in which the methods are described is not intended to be construed as a limitation, and any number of the described blocks can be combined in any order to implement the method, or an alternate method. Furthermore, the method can be implemented in any suitable hardware, software, firmware, or combination thereof, such that a computing device can implement the method. In one case, the method is stored on computer-readable storage medium/media as a set of instructions such that execution by a computing device causes the computing device to perform the method.
Although techniques, methods, devices, systems, etc. pertaining to auto-calendaring are described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claimed methods, devices, systems, etc.
add the upcoming event to a calendar associated with the user with an indication that the upcoming event will occur at the first public establishment.
2. The system of claim 1, embodied on a single device.
3. The system of claim 2, wherein the single device comprises a mobile device.
4. The system of claim 1, wherein the first public establishment is a sports bar, the second public establishment is a zoo, the ambiguous location term is “zoo,” and the user event profile does not indicate an interest in animals.
use the search results obtained from the search engine to confirm the upcoming event involves the particular game being shown at the sports bar at the particular time.
6. The system of claim 1, wherein the data item is a picture that includes the ambiguous location term.
perform optical character recognition on the picture to extract the ambiguous location term.
submit the location metadata to the search engine together with the ambiguous location term extracted from the picture via the optical character recognition.
determine a confidence that the user will attend the upcoming event; and add the upcoming event to the calendar associated with the user responsive to a determination that the confidence exceeds a threshold.
use the further search results obtained from the search engine to confirm that the upcoming event is scheduled to occur at the first public establishment at the particular time.
send the calendar from the web-based cloud service over a network to a user device associated with the user, the calendar being sent with the upcoming event added to the calendar.
automatically schedule the upcoming event for the recipient of the message at the first public establishment.
automatically schedule the upcoming event by populating a calendar of the recipient with the upcoming event.
include the message sender and the first public establishment as event details of the upcoming event on the calendar of the recipient.
15. The device of claim 12, wherein the message is an email.
obtain the predetermined interests from a user profile of the recipient.
17. The device of claim 12, embodied as a mobile computer of the recipient.
18. The device of claim 12, embodied as a server computer located remotely from the recipient.
automatically scheduling the upcoming event at the first public establishment on an electronic calendar of the recipient of the electronic message.
20. The method of claim 19, the electronic message comprising an email.
21. The method of claim 19, the electronic message comprising a text message other than an email.
22. The method of claim 19, further comprising: obtaining the recipient-preferred activity from a profile of the recipient.
23. The method of claim 22, wherein the first type of activity and the second type of activity are recreational activities.
MX2015016071A MX2015016071A (en) 2013-05-20 2014-05-19 Auto-calendaring.
""Assemble Friends for Spontaneous Fun With GiddyUp"; retrieved athttp://mashable.com/2012/05/14/giddyup-app/; accessed on Dec. 17, 2012; 6 pages."
"DotNetNuke Social Networks", retrieved at «http://www.activemodules.com/ActiveSocial/Features.aspx» on Oct. 25, 2012, 5 pages.
"Groupme for Windows Phone", retrieved at «http://blog.groupme.com/post/18436501807/groupme-for-windows-phone-its-back-and-better», Feb. 28, 2012, 7 pages.
"International Search Report and Written Opinion" From PCT Patent Application No. PCT/US/1438529, dated Dec. 16, 2014.
"Leveraging Social Portlets, Activities Tracking and User Connections", retrieved at «http://www.liferay.com/documentation/liferay-portal/6.1/user-guide/-/ai/lp-6-1-ugen08-leveraging-social-portlets-activities-tracking-and-user-con-0» on Oct. 23, 2012, 16 pages.
"Non-Final Office Action dated Mar. 27, 2015 from U.S. Appl. No. 13/766,559, 27 pages."
"Office Action Issued in European Patent Application No. 14730384.6", dated Nov. 13, 2017, 8 Pages.
"Office Action Issued in Japanese Patent Application No. 2016-514980", dated Feb. 7, 2018, 7 Pages.
"Office Action Issued in Russia Patent Application No. 2015149777", dated Apr. 2, 2018, 7 pages. (W/o English Translation).
"Second Office Action Issued in Chinese Patent Application No. 201480029294.9", dated Mar. 5, 2018, 6 Pages.
"Speech Recognition-Wikipedia", Retrieved From «https://en.wikipedia.org/w/index.php.title=Speech_recognition&oldid=555081415», May 14, 2013, 13 pages.
"Speech Recognition—Wikipedia", Retrieved From «https://en.wikipedia.org/w/index.php.title=Speech_recognition&oldid=555081415», May 14, 2013, 13 pages.
Applicant-Initiated Interview Summary dated Jul. 29, 2016 from U.S. Appl. No. 13/766,559, 6 pages.
Applicant-Initiated Interview Summary dated Nov. 3, 2015 from U.S. Appl. No. 13/766,559, 15 pages.
Communication pursuant to Rules 70(2) and 70a(2) EPC dated Jul. 12, 2016 from European Patent Application No. 14708130.1, 1 page.
Communication pursuant to Rules 70(2) and 70a(2) EPC dated Jun. 28, 2016 from European Patent Application No. 14708357.0, 1 page.
Communication pursuant to Rules 70(2) and 70a(2) EPC dated Oct. 18, 2016 to European Patent Application No. 14730384.6, 1 page.
Counts, Scott, "Group-based Mobile Messaging in Support of the Social Side of Leisure", Proceedings on Computer Supported Cooperative Work, Apr. 1, 2007, 23 pages.
Dale, Steve, "Knowledge Hub-part 4: Social Graph and Activity Stream", Retrieved at «http://steve-dale.net/2011/03/10/knowledge-hub-4-social-graph-and-activity-stream/», Mar. 10, 2011, 4 pages.
Dale, Steve, "Knowledge Hub—part 4: Social Graph and Activity Stream", Retrieved at «http://steve-dale.net/2011/03/10/knowledge-hub-4-social-graph-and-activity-stream/», Mar. 10, 2011, 4 pages.
Das, Debajyoti., "Gmail's Automatic "Add to Calendar" Suggestions for Meetings, Events", Retrieved at «http:// www.snaphow.com/4445/gmails-automatic-add-to-calendar-suggestions-for-meetings-events», Aug. 2010, pp. 5.
Final Office Action dated Aug. 6, 2015 from U.S. Appl. No. 13/766,559, 38 pages.
Final Office Action dated Jan. 17, 2017 from U.S. Appl. No. 13/754,899, 10 pages.
Final Office Action dated Nov. 17, 2016 from U.S. Appl. No. 13/766,559, 58 pages.
Final Office Action dated Sep. 18, 2015 from U.S. Appl. No. 13/754,899, 15 pages.
First Office Action dated Aug. 1, 2017 from Chinese Patent Application No. 201480008726.8, 12 pages.
First Office Action dated May 27, 2017 from Chinese Patent Application No. 201480029294.9, 13 pages.
International Preliminary Report on Patentability dated Aug. 13, 2015 from PCT Patent Application No. PCT/US2014/013093, 7 pages.
International Preliminary Report on Patentability dated Aug. 27, 2015 from PCT Patent Application No. PCT/US2014/015463, 6 pages.
International Preliminary Report on Patentability dated Dec. 3, 2015 from PCT Patent Application No. PCT/US14/38529, 6 pages.
International Search Report and Written Opinion dated Dec. 16, 2014 from PCT Patent Application No. PCT/US14/38529, 10 pages.
International Search Report and Written Opinion dated Sep. 2, 2014 from PCT Patent Application No. PCT/US2014/013093, 10 pages.
International Search Report and Written Opinion dated Sep. 5, 2014 from PCT Patent Application No. PCT/US2014/015463, 9 pages.
Kim, et al., "Multi-layered Structure-based Context Information using Adaptive Service Decision", Proceedings of the Conference on Wireless Information Networks & Business Information System, vol. 3, No. 2; Mar. 2, 2012, pp. 127-136.
Microsoft Press, "Microsoft Computer Dictionary", 5th Edition, 2002, p. 452, 2 pages.
Nazir, et al., "Beyond Friendship: Modeling User Activity Graphs on Social Network-Based Gifting Applications", retrieved at «http://www2.research.att.com/˜bala/papers/imc12-uag.pdf», IMC'12, Boston, Massachusetts, USA, Nov. 14, 2012, 13 pages.
Non-Final Office Action dated Apr. 21, 2016 from U.S. Appl. No. 13/766,559, 49 pages.
Non-Final Office Action dated Dec. 24, 2014 from U.S. Appl. No. 13/754,899, 22 pages.
Non-Final Office Action dated Jul. 14, 2016 from U.S. Appl. No. 13/754,899, 9 pages.
Non-Final Office Action dated May 5, 2017 from U.S. Appl. No. 13/754,899, 9 pages.
Non-Final Office Action dated Oct. 24, 2014 fomr U.S. Appl. No. 13/766,559, 22 pages.
Notice of Allowance dated Sep. 6, 2017 from U.S. Appl. No. 13/754,899, 44 pages.
Office Action dated Jun. 5, 2017 from Mexican Patent Application No. MX/a/2015/016071, 7 pages.
Ogasawara, Todd., "iPhone/iPad Convert Email to Calendar Event Tip", Retrieved at «http://socialtimes.com/iphoneipad-convert-email-to-calendar-event-tip_b9371», Dec. 16, 2010, pp. 4.
Response filed Aug. 3, 2017 to the Non-Final Office Action dated May 5, 2017 from U.S. Appl. No. 3/754,899, 16 pages.
Response filed Aug. 8, 2016 to the Communication pursuant to Rules 70(2) and 70a(2) EPC dated Jun. 28, 2016 from European Patent Application No. 14708357.0, 16 pages.
Response filed Dec. 17, 2015 to the Final Office Action dated Sep. 18, 2015 from U.S. Appl. No. 13/754,899, 15 pages.
Response filed Jan. 26, 2015 to the Non-Final Office Action dated Oct. 24, 2014 from U.S. Appl. No. 13/766,559, 13 pages.
Response filed Jul. 29, 2016 to the Non-Final Office Action dated Apr. 21, 2016 from U.S. Appl. No. 13/766,559, 13 pages.
Response filed Mar. 10, 2017 to the Communication pursuant to Rules 70(2) and 70a(2) EPC dated Oct. 18, 2016 to European Patent Application No. 14730384.6, 18 pages.
Response filed Mar. 27, 2017 to the Final Office Action dated Jan. 17, 2017 from U.S. Appl. No. 13/754,899, 11 pages.
Response filed May 20, 2015 to the Non-Final Office Action dated Dec. 24, 2014 from U.S. Appl. No. 13/754,899, 8 pages.
Response filed May 20, 2015 to the Non-Final Office Action dated Mar. 27, 2015 from U.S. Appl. No. 13/766,559, 9 pages.
Response filed Nov. 6, 2015 to the Non-Final Office Action dated Aug. 6, 2015 from U.S. Appl. No. 13/766,559, 11 pages.
Response filed Oct. 4, 2016 to the Non-Final Office Action dated Jul. 14, 2016 from U.S. Appl. No. 13/754,899, 10 pages.
Smith, Dustin Arthur., "Goal-based Event Recognition and Execution", Retrieved at «http://web.mit.edu/˜dsmit/dfiles/dustin-back/dustin-ms-draft.pdf.mine», Sep. 2007, pp. 70.
Supplemental European Search Report dated Sep. 30, 2016 from European Patent Application No. 14730384.6, 9 pages.
Supplementary European Search Report dated Jun. 9, 2016 from European Patent Application No. 14708357.0, 3 pages.
Supplementary Search Report dated Jun. 24, 2016 from European Patent Application No. 14708130.1, 7 pages.
Tempus Nova Trainer, "Add Events to Google Calendar from Gmail", YouTube, retrieved from «https://www.youtube.com/watch?v=9ggOWddW3eo», May 9, 2013, 3 pages.
Voluntary Amendment filed Feb. 18, 2016 to China Patent Application No. 201480008726.8, 9 pages.
Voluntary Amendment filed Mar. 31, 2016 to China Patent Application No. 201480007010.6, 7 pages.
Wikipedia Contributors, "Natural language processing", Wikipedia, The Free Encyclopedia, Oct. 6, 2016, retrieved from «https://en.wikipedia.org/w/index.php?title=Natural_language_processing&oldid-742902283», 8 pages.
Zhao, et al., "The Design of Activity-Oriented Social Networking: Dig-Event", Proceedings of the Conference on Information Integration and Web-based Applications & Services, Ho Chi Minh City, Vietnam; Dec. 5-7, 2011, pp. 6.

References: Application No. 14730384
 Application No. 2016
 Application No. 2015149777
 Application No. 201480029294
 Application No. 14708130
 Application No. 14708357
 Application No. 14730384
 Application No. 201480008726
 Application No. 201480029294
 Application No. 14708357
 Application No. 14730384
 Application No. 14730384
 Application No. 14708357
 Application No. 14708130
 Application No. 201480008726
 Application No. 201480007010