Patent ID: 12216722

DETAILED DESCRIPTION

The subject matter of aspects of the present disclosure is described with specificity herein to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventors have contemplated that the claimed subject matter might also be embodied in other ways, to include different steps or combinations of steps similar to the ones described in this document, in conjunction with other present or future technologies. Moreover, although the terms “step” and/or “block” may be used herein to connote different elements of methods employed, the terms should not be interpreted as implying any particular order among or between various steps herein disclosed unless and except when the order of individual steps is explicitly described. Each method described herein may comprise a computing process that may be performed using any combination of hardware, firmware, and/or software. For instance, various functions may be carried out by a processor executing instructions stored in memory. The methods may also be embodied as computer-useable instructions stored on computer storage media. The methods may be provided by a stand-alone application, a service or hosted service (stand-alone or in combination with another hosted service), or a plug-in to another product, to name a few.

Aspects of the present disclosure relate to technology for facilitating and improving people highlight information presented on personal computing devices. The coalescence of telecommunications and personal computing technologies in the modern era has enabled, for the first time in human history, information on demand combined with a ubiquity of personal computing resources (including mobile personal computing devices and cloud-computing coupled with communication networks). As a result, it is increasingly common for users to rely on one or more mobile computing devices throughout the day for handling various tasks. It is also now possible to provide information to the user regarding other people relevant to a particular task, location, communication, application use, or other contextual situation, referred to collectively as people highlight information or people highlights. But these people highlights may not always be useful to any particular user. In some instances, providing these people highlights may be seen by a particular user as unhelpful. As such, as described below, the disclosed technology also now offers the opportunity to present only people highlight information most likely desired by a user for display, and to conceal or suppress from display other people highlights that are not likely desired for display. Embodiments, as described herein, address a need that arises from very large scale of operations created by software-based services that cannot be managed by humans. The actions/operations described herein are not a mere use of a computer, but address results of a system that is a direct consequence of software used as a service offered in conjunction with user communication through services hosted across a variety of platforms and devices.

Accordingly, solutions provided herein include technologies for improving, or providing improved control over, the presentation or display of people highlights on computing devices. The solution monitors a user device to determine a user interest (explicit interest and/or implicit interest) in a person of interest. A people highlight determiner identifies a set of possible people highlights, with respect to the person of interest. In some aspects, the identified set of possible people highlights is ranked or ordered. A suppression engine determines whether any of the possible people highlights should be suppressed and not presented to the user on the user device. People highlights that are determined to be people highlights that should be suppressed and not presented are removed to create a remaining set of possible people highlights. At least one of the remaining set of possible people highlights are presented on the user device. By suppressing or removing people highlights that are potentially annoying or unhelpful to the user, the remaining people highlights that are presented are more likely to be seen as helpful and desirable by the user.

Turning now toFIG.1, a block diagram is provided showing an example operating environment100in which some embodiments of the present disclosure may be employed. It should be understood that this and other arrangements described herein are set forth only as examples. Other arrangements and elements (e.g., machines, interfaces, functions, orders, and groupings of functions) can be used in addition to or instead of those shown, and some elements may be omitted altogether for the sake of clarity. Further, many of the elements described herein are functional entities that may be implemented as discrete or distributed components or in conjunction with other components, and in any suitable combination and location. Various functions described herein as being performed by one or more entities may be carried out by hardware, firmware, and/or software. For instance, some functions may be carried out by a processor executing instructions stored in memory.

Among other components not shown, example operating environment100includes a number of user computing devices, such as user devices102aand102bthrough102n; a number of data sources, such as data sources104aand104bthrough104n; server106; sensors103aand107; and network110. It should be understood that environment100shown inFIG.1is an example of one suitable operating environment. Each of the components shown inFIG.1may be implemented via any type of computing device, such as computing device500described in connection toFIG.5, for example. These components may communicate with each other via network110, which may include, without limitation, one or more local area networks (LANs) and/or wide area networks (WANs). In exemplary implementations, network110comprises the Internet and/or a cellular network, amongst any of a variety of possible public and/or private networks.

It should be understood that any number of user devices, servers, and data sources may be employed within operating environment100within the scope of the present disclosure. Each may comprise a single device or multiple devices cooperating in a distributed environment. For instance, server106may be provided via multiple devices arranged in a distributed environment that collectively provide the functionality described herein. Additionally, other components not shown may also be included within the distributed environment.

User devices102aand102bthrough102ncan be client user devices on the client-side of operating environment100, while server106can be on the server-side of operating environment100. Server106can comprise server-side software designed to work in conjunction with client-side software on user devices102aand102bthrough102nso as to implement any combination of the features and functionalities discussed in the present disclosure. This division of operating environment100is provided to illustrate one example of a suitable environment, and there is no requirement for each implementation that any combination of server106and user devices102aand102bthrough102nremain as separate entities.

User devices102aand102bthrough102nmay comprise any type of computing device capable of use by a user. For example, in one embodiment, user devices102athrough102nmay be the type of computing device described in relation toFIG.5herein. By way of example and not limitation, a user device may be embodied as a personal computer (PC), a laptop computer, a mobile or mobile device, a smartphone, a tablet computer, a smart watch, a wearable computer, a personal digital assistant (PDA) device, a handheld communications device, a gaming device or system, an entertainment system, a consumer electronic device, a workstation, or any combination of these delineated devices, a combination of these devices, or any other suitable computer device.

Data sources104aand104bthrough104nmay comprise data sources and/or data systems, which are configured to make data available to any of the various constituents of operating environment100, or system200described in connection toFIG.2. For instance, in one embodiment, one or more data sources104athrough104nprovide (or make available for accessing) people data and person of interest data, which may include user-activity related data, to people-data collection component210ofFIG.2. Data sources104aand104bthrough104nmay be discrete from user devices102aand102bthrough102nand server106or may be incorporated and/or integrated into at least one of those components. In one embodiment, one or more of data sources104athrough104ncomprise one or more sensors, which may be integrated into or associated with one or more of the user device(s)102a,102b, or102nor server106. Examples of sensed people data made available by data sources104athrough104nare described further in connection to people-data collection component210ofFIG.2.

Operating environment100can be utilized to implement one or more of the components of system200, described inFIG.2, including components for collecting people data; monitoring or determining user tasks, user activity and events, user patterns (e.g., usage, behavior, or activity patterns), user preferences, and/or similar categories of data regarding a person of interest, context data, or related information to facilitate sharing context or to otherwise provide an improved user experience; and/or presenting selected people highlights to users. Operating environment100also can be utilized for implementing aspects of method400inFIG.4.

Referring now toFIG.2, withFIG.1, a block diagram is provided showing aspects of an example computing system architecture suitable for implementing an embodiment of this disclosure and designated generally as system200. System200represents only one example of a suitable computing system architecture. Other arrangements and elements can be used in addition to or instead of those shown, and some elements may be omitted altogether for the sake of clarity. Further, as with operating environment100, many of the elements described herein are functional entities that may be implemented as discrete or distributed components or in conjunction with other components, and in any suitable combination and location.

Example system200includes network110, which is described in connection toFIG.1, and which communicatively couples components of system200including people-data collection component210, presentation engine220, context determiner230, storage250, and people highlights engine260. People-data collection component210, presentation engine220, context determiner230, storage250, and people highlights engine260may be embodied as a set of compiled computer instructions or functions, program modules, computer software services, or an arrangement of processes carried out on one or more computer systems, such as computing device500described in connection toFIG.5, for example.

In one embodiment, the functions performed by components of system200are associated with one or more personal assistant or assistance applications, services, or routines to provide only selected people highlights in association with a person of interest and a computing experience of the user. In particular, such applications, services, or routines may operate on one or more user devices (such as user device102a) and/or servers (such as server106). Moreover, in some embodiments, these components of system200may be distributed across a network, including one or more servers (such as server106) and client devices (such as user device102a), in the cloud, or may reside on a user device, such as user device102a. Moreover, these components, functions performed by these components, or services carried out by these components may be implemented at appropriate abstraction layer(s) such as the operating system layer, application layer, or hardware layer of the computing system(s). Alternatively, or in addition, the functionality of these components and/or the embodiments described herein can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), or Complex Programmable Logic Devices (CPLDs). Additionally, although functionality is described herein with regards to specific components shown in example system200, it is contemplated that in some embodiments functionality of these components can be shared or distributed across other components.

Continuing withFIG.2, people-data collection component210is generally configured to access or receive (and in some cases also identify) people data (with respect to both a user and a person of interest) from one or more data sources, such as data sources104aand104bthrough104nofFIG.1. In some embodiments, people-data collection component210may be employed to facilitate the accumulation of people data of a particular user (or in some cases, a plurality of users including crowdsourced data) for activity detector232or more generally context determiner230. The data may be received (or accessed), and optionally accumulated, reformatted, and/or combined, by people-data collection component210and stored in one or more data stores such as storage250, where it may be available to other components of system200. For example, the people data may be stored in or associated with a user profile240, as described herein. In some embodiments, any personally identifying data (i.e., people data that specifically identifies particular users) is either not uploaded or otherwise provided from the one or more data sources with people data, is not permanently stored, is de-identified, and/or is not made available to context determiner230. In some embodiments, a user may opt into or out of services provided by the technologies described herein and/or select which user data and/or which sources of user data are to be captured and utilized by these technologies.

People data, generally, may be any information that is related to a person, or group of people, that informs a user about an aspect of that person or group of people, and may be received from a variety of sources and may be available in a variety of formats. By way of example and without limitation, people data may comprise contact information (e.g., email, instant message, phone, and may also specify a person's communication preferences); location information (e.g., a person's current location or location of a particular office where they work); presence; recent activity, which may comprise activity relevant to the user (e.g., that the person of interest has an upcoming meeting with the user or recently emailed the user); task-related information (e.g., outstanding tasks that the user has with regard to the person of interest, or outstanding tasks that the person of interest has with respect to the user); information about the person of interest that they may choose to share (e.g., birthday, anniversary, etc.); information in common with the user (e.g., common project teams, work groups, backgrounds, education, interests or hobbies). Additional examples of people data are described herein.

In some embodiments, people data received via people-data collection component210may be obtained from a data source (such as data source104(a) inFIG.1, which may be a social networking site, a professional networking site, or other data source containing people data) or determined via one or more sensors (such as sensors103aand107ofFIG.1), which may be on or associated with one or more user devices (such as user device102a), servers (such as server106), and/or other computing devices. As used herein, a sensor may include a function, routine, component, or combination thereof for sensing, detecting, or otherwise obtaining information such as people data from a data source104a, and may be embodied as hardware, software, or both. By way of example and not limitation, people data may include data that is sensed or determined from one or more sensors (referred to herein as sensor data), such as location information of mobile device(s), properties or characteristics of the user device(s), user-activity information (for example: app usage; online activity; searches; voice data such as automatic speech recognition; activity logs; communications data including calls, texts, instant messages, and emails; website posts; other people data associated with communication events) including, user history, session logs, application data, contacts data, calendar and schedule data, notification data, social-network data, ecommerce activity, user-account(s) data (which may include data from user preferences or settings associated with a personalization-related application, a personal assistant application or service), global positioning system (GPS) data, other user device data (which may include device settings, profiles, network-related information, payment or credit card usage data, purchase history data, other sensor data that may be sensed or otherwise detected by a sensor (or other detector) component(s) including data derived from a sensor component associated with the user (including location, motion, orientation, position, user-access, user-activity, network-access, user-device-charging, or other data that is capable of being provided by one or more sensor component), data derived based on other data (for example, location data that can be derived from Wi-Fi, Cellular network, or IP address data), and nearly any other source of data that may be sensed or determined as described herein.

People data, particularly in the form of contextual information, can be received by people-data collection component210from one or more sensors and/or computing devices associated with a user. In some embodiments, people-data collection component210, context determiner230(or one or more of its subcomponents), or other components of system200may determine interpretive data from received people data. Interpretive data corresponds to data utilized by the components or subcomponents of system200that comprises an interpretation from processing raw data, such as venue information interpreted from raw location information. Interpretive data can be used to provide context to people data, which can support determinations or inferences carried out by components of system200. Moreover, it is contemplated that some embodiments of the disclosure use people data alone or in combination with interpretive data for carrying out the objectives of the subcomponents described herein. It is also contemplated that some people data may be processed, by the sensors or other subcomponents of people-data collection component210not shown, such as for interpretability by people-data collection component210. However, embodiments described herein do not limit the people data to processed data and may include raw data or a combination thereof, as described above.

In some respects, people data may be provided in user-data streams or signals. A “user signal” can be a feed or stream of people data from a corresponding data source. For example, a user signal could be from a smartphone, a home-sensor device, a GPS device (e.g., for location coordinates), a vehicle-sensor device, a wearable device, a user device, a gyroscope sensor, an accelerometer sensor, a calendar service, an email account, a credit card account, or other data sources. In some embodiments, people-data collection component210receives or accesses data continuously, periodically, or as needed. In some embodiments, the people information/people data received by people-data collection component210is stored in storage250, such as in people information and highlights258.

Context determiner230is generally responsible for monitoring people data (such as that collected by people-data collection component210) for information that may be used for determining user context, which may include features (sometimes referred to herein as “variables”) or other information regarding specific user actions and related contextual information. Embodiments of context determiner230may determine, from the monitored people data, a user context associated with a particular user or user device. As described previously, the user context information determined by context determiner230may include user information from multiple user devices associated with the user and/or from cloud-based services associated with the user (such as email, calendars, social media, or similar information sources), and which may include contextual information associated with the identified user activity (such as location, time, venue specific information, or other people present). Context determiner230may determine current or near-real-time user information and may also determine historical user information, in some embodiments, which may be determined based on gathering observations of a user over time. Further, in some embodiments, context determiner230may determine user context from other similar users.

As described previously, user context features may be determined by monitoring people data received from people-data collection component210. In some embodiments, the people data and/or information about the user context determined from the people data is stored in a user profile, such as user profile240.

In an embodiment, context determiner230comprises one or more applications or services that analyze information detected via one or more user devices used by the user and/or cloud-based services associated with the user, to determine user-related or user-device-related contextual information. Information about user devices associated with a user may be determined from the people data made available via people-data collection component210, and may be provided to context determiner230or people highlights engine270, or other components of system200.

Some embodiments of context determiner230, or its subcomponents, may determine a device name or identification (device ID) for each device associated with a user. This information about the identified user devices associated with a user may be stored in a user profile associated with the user, such as in user accounts and devices246of user profile240. In an embodiment, the user devices may be polled, interrogated, or otherwise analyzed to determine information about the devices. This information may be used for determining a label or identification of the device (e.g., a device ID) so that the user interaction with the device may be recognized by context determiner230. In some embodiments, users may declare or register a device, such as by logging into an account via the device, installing an application on the device, connecting to an online service that interrogates the device, or otherwise providing information about the device to an application or service. In some embodiments, devices that sign into an account associated with the user, such as a Microsoft® account or Net Passport, email account, social network, or the like, are identified and determined to be associated with the user.

Context determiner230, in general, is responsible for determining (or identifying) contextual information about a user or user-device associated with a user. Embodiments of activity detector232may be used for determining current user activity or one or more historical user actions. Some embodiments of activity detector232may monitor people data for activity-related features or variables corresponding to user activity such as, for example, user location; indications of applications launched or accessed; files accessed, modified, or copied; websites navigated to; online content downloaded and rendered or played; or similar user activities.

Contextual information extractor234, in general, is responsible for determining contextual information related to the user activity (detected by activity detector232or context determiner230), such as context features or variables associated with user activity, related information, and user-related activity, and is further responsible for associating the determined contextual information with the detected user activity. In some embodiments, contextual information extractor234may associate the determined contextual information with the related user activity and may also log the contextual information with the associated user activity. Alternatively, the association or logging may be carried out by another service. For example, some embodiments of contextual information extractor234provide the extracted contextual information to context features determiner238, which determines user context using information from the activity detector232, the contextual information extractor234, and/or the semantic information analyzer236.

Some embodiments of contextual information extractor234(or semantic information analyzer236, described below) determine contextual information related to a user action or activity event such as entities identified in a user activity or related to the activity (e.g., recipients of a group email sent by the user), which may include nicknames used by the user (e.g., “mom” and “dad” referring to specific entities who may be identified in the user's contacts by their actual names); information about the current user of the user device (e.g., whether the user is the owner or another user, the age of the current user, the relationship of the current user to the owner, such as a close friend, co-worker, family member, or unknown entity); or user activity associated with the location or venue of the user's device, which may include information about other users or people present at the location. By way of example and not limitation, this may include context features such as location data; contextual information about the location; user device characteristics or user device identification information regarding the device; duration of the user activity; other information about the activity such as entities associated with the activity (e.g., venues, people, objects); information detected by sensor(s) on user devices associated with the user that is concurrent or substantially concurrent to the user activity; or any other information related to the user activity that is detectable that may be used for determining patterns of user activity.

In some embodiments, a device name or identification (device ID) may be determined for each device associated with a user. This information about the identified user devices associated with a user may be stored in a user profile associated with the user, such as in user account(s) and device(s)246of user profile240. In an embodiment, the user devices may be polled, interrogated, or otherwise analyzed to determine contextual information about the devices. This information may be used for determining information about a current user of a user device, or determining a label or identification of the device (e.g., a device ID) so that user activity on one user device may be recognized and distinguished from user activity on another user device. Further, as described previously, in some embodiments, users may declare or register a user device, such as by logging into an account via the device, installing an application on the device, connecting to an online service that interrogates the device, or otherwise providing information about the device to an application or service. In some embodiments devices that sign into an account associated with the user, such as a Microsoft® account or Net Passport, email account, social network, or the like, are identified and determined to be associated with the user.

In some implementations, contextual information extractor234may receive people data from people-data collection component210, parse the data, in some instances, and identify and extract context features or variables (which may also be carried out by context features determiner238). Context variables may be stored as a related set of contextual information associated with the user activity, and may be stored in a user profile such as in user context information component242. Contextual information also may be determined from the people data of one or more users, in some embodiments, which may be provided by people-data collection component210in lieu of or in addition to user activity information for the particular user.

Semantic information analyzer236is generally responsible for determining semantic information associated with the user-activity related features identified by activity detector232or contextual information extractor234. For example, while a user-activity feature may indicate a specific website visited by the user, semantic analysis may determine the category of website, related websites, themes or topics, or other entities associated with the website or user activity. Semantic information analyzer236may determine additional user-activity related features semantically related to the user activity, which may be used for further identifying user context.

In particular, as described previously, a semantic analysis is performed on the user activity information and/or the contextual information, to characterize aspects of the user action or activity event. For example, in some embodiments, activity features associated with an activity event may be classified or categorized (such as by type, time frame or location, work-related, home-related, themes, related entities, other user(s) (such as communication to or from another user) and/or relation of the other user to the user (e.g., family member, close friend, work acquaintance, boss, or the like), or other categories), or related features may be identified for use in determining a similarity or relational proximity to other user activity events, which may indicate a pattern. In some embodiments, semantic information analyzer236may utilize a semantic knowledge representation, such as a relational knowledge graph. Semantic information analyzer236may also utilize semantic analysis logic, including rules, conditions, or associations to determine semantic information related to the user activity. For example, a user activity event comprising an email sent to someone who works with the user may be characterized as a work-related activity.

Semantic information analyzer236may also be used to characterize contextual information, such as determining that a location associated with the activity corresponds to a hub or venue of the user (such as the user's home, work, gym, or the like) based on frequency of user visits. For example, the user's home hub may be determined (using semantic analysis logic) to be the location where the user spends most of her time between 8 PM and 6 AM. Similarly, the semantic analysis may determine the time of day that corresponds to working hours, lunch time, commute time, or other similar categories. Similarly, the semantic analysis may categorize the activity as being associated with work or home, based on other characteristics of the activity. In this way, the semantic analysis provided by semantic information analyzer236may provide other relevant features that may be used for determining user context.

Context features determiner238is generally responsible for determining user context-related features (or variables) associated with the user, which may include the context of a user device associated with the user. Context features may be determined from information about a user activity, from related contextual information, and/or from semantic information. In some embodiments, context features determiner238receives information from activity detector232, contextual information extractor234, and/or semantic information analyzer236and analyzes the received information to determine a set of one or more features associated with the user's context.

Examples of user context-related features include, without limitation, location-related features, venue-related information associated with the location, or other location-related information; other users present at the venue or location; time-related features; current-user-related features, which may include information about the current or recent user of the user-device; user device-related features, such as device type (e.g., desktop, tablet, mobile phone, fitness tracker, heart rate monitor), hardware properties or profiles, OS or firmware properties, device IDs or model numbers, battery or power-level information, network-related information, position/motion/orientation-related information about the user device, network usage information, user account(s) accessed or otherwise used (such as device account(s), OS level account(s), or online/cloud-services related account(s) activity, such as Microsoft® account or Net Passport, online storage account(s), email, calendar, or social networking accounts); content-related features, such as online activity (e.g., searches, browsed websites, purchases, social networking activity, communications sent or received including social media posts); or any other features that may be detected or sensed and used for determining the user context.

Example system200also includes a people highlights engine260that is generally responsible for crawling and indexing content, determining possible people highlights, ranking the highlights and suppressing selected highlights, and presenting at least a portion of the remaining ranked people highlights to a user (or passing the portion of remaining ranked people highlights to presentation engine220). People highlights engine260includes a person of interest content crawler262and a person of interest indexer264that are generally responsible for scanning or crawling and indexing, for example, content relating to a person of interest, which may include data, programs, applications, or information related to data, programs or applications associated with a user account, user device(s), and/or storage. Content crawler262identifies user content, person of interest content, and related information, as broadly defined above, as an input. Again, this content and information could be present on user device(s) or stored remotely, and could also come from people-data collection component210or people data/information (which may have been already received by component210and) stored on storage250. The information and content identified by person of interest content crawler262is accessed by, or passed to, a person of interest content indexer264. In some embodiments, person of interest content indexer264creates a classified content data index associating metadata with the content.

Person of interest content crawler262may also access user preferences248, which may have settings indicating which user devices and storage may be accessed, and may also specify or include user-defined labels or categories to be used by person of interest content indexer264. People highlights engine260may also access user accounts and devices246to identify online or cloud-based storage accounts, email, calendars, and similar sources of content to be classified. Person of interest content crawler262and person of interest indexer264may identify and classify information about a person of interest similar to user context information242, including, for example, organizational role (job, social, location within an organizational chart, work collaborator); background (school(s) attended, family members, connected friends); demographic information (social status, hometown, birthdays, work anniversary, cities lived-in, cities or companies worked-in); behavioral information (conversations, frequency of contact (close connection, old acquaintance, stranger)); tasks or obligations (a promised draft or communication on a certain date); and other similar types of information.

People highlights engine260also includes a highlights determiner266. In some embodiments, highlights determiner266uses highlights logic252to determine a set of possible people highlights for a person of interest relative to the user. Highlights determiner266may take as inputs information from person of interest content crawler262, person of interest indexer264, people-data collection component210, context determiner230and/or information from user profile240. Highlights logic252may include rules, conditions, associations, classification models, or other criteria to identify and classify the set of possible people highlights. The highlights logic252can take many different forms. For example, the classification logic may comprise fuzzy logic, neural network, finite state machine, support vector machine, logistic regression, clustering, topic modeling, or machine learning techniques, similar statistical classification processes or, combinations of these to identify and classify content items, or to predict relevancy of content items to be considered for highlighting. Highlights determiner266passes a collected set of people highlights to a ranking engine268and in some embodiments the people highlights may be stored in people information and highlights258. In some embodiments the people highlights (or possible/candidate highlights) that are determined by people highlights engine260(and/or highlights determiner266) may be stored in (or received from) people information and highlights258. For example, and without limitation, the set of highlights may include things such as: “Say congratulations to James on his 10 year work anniversary”; “See email from John” (calling attention to an unread email); “You have several unread emails from people attending your upcoming meeting”; “Jessica has recently changed positions at the company”; “Kayla is a new hire and will be on the call today”; “Three of the meeting attendees viewed your profile, would you like to update it?”; “Tracy also graduated from the University of Iowa”; “You and Mike have a shared connection with Randy”.

The ranking engine268receives the collected set of people highlights from highlights determiner266and ranks them using ranking logic254. Ranking logic254, executed by ranking engine268generally determines, from the collected set of people highlights, a narrower set of people highlights ranked to determine the right people highlights to present to a user at the right time. Ranking logic254generally considers, among other things: where the people highlight is going to be presented (screen real estate on a phone, laptop to large screen; within a specific surface (word processing application, calendaring application, communication application or other program or application)); the relationship of the person of interest to the user (close connection, a reconnection of an older acquaintance, or a new contact such as a stranger); user inputs and preferences; and/or preferences of other users that are similar to the user. The ranking logic254may also limit the ranked set to a specific number of people highlights, as presenting more than a specific number of people highlights may not be desirable. Like highlights logic252, ranking logic254may include rules, conditions, associations, classification models, or other criteria apply a ranking to the set of possible people highlights received from highlights determiner266. The ranking logic254can take many different forms. For example, the ranking logic may comprise Boolean logic, fuzzy logic, decision trees or random forest, finite state machine, support vector machine, logistic regression, clustering, topic modeling, or machine learning techniques, similar statistical classification processes or, combinations of these to rank the received set of people highlights. The ranking engine268passes the ranked set of people highlights to the suppression engine270.

Suppression engine270receives the ranked set of people highlights from the ranking engine268and generally determines whether any of the received ranked set of people highlights should be suppressed and not presented to the user. Suppression engine270uses suppression logic256to determine any highlights that should be suppressed. By suppressing certain highlights, the people highlights that are presented are more likely to be valuable and useful to the user. User fatigue caused by too many presented people highlights can be avoided, as well as the potential for user annoyance if the people highlight has already been presented, or if the people highlight is stale or otherwise not timely.

Suppression logic256comprises rules, conditions, associations, classification models, or other criteria to identify and/or classify a set of one or more possible people highlights to be suppressed. In some embodiments, suppression logic256may comprise logic used for making a binary determination (e.g., whether or not to suppress a particular people highlight or type of highlight), or may comprise logic used by suppression engine270for determining a suppression score or suppression weighting, which may be evaluated against a pre-determined threshold (e.g., 0.5 or 50 percent) to determine whether to suppress a particular highlight. In some of these embodiments, the threshold may be determined by the user (e.g., via user preferences) or by a system administrator, may be dependent on the particular highlight type, or may be based on user behavior (e.g., where a user rarely uses highlights or often dismisses highlights, then the threshold may be adjusted lower so that more highlights are likely to satisfy the threshold and thus be suppressed).

Some embodiments of suppression logic256comprise logic for modifying a suppression weighting associated with one or more possible people highlights. As described above, a higher suppression weighting for a particular possible people highlight may result in a greater likelihood of that highlight being suppressed. For example, a suppression weighting may be modified based on whether a highlight has been previously shown, dismissed, or interacted with. For instance, in an embodiment, showing a highlight may result in its suppression weighting being increased (making it more likely not to be shown again) and a user dismissing a highlight may also increase its weighting, in some cases even more so than showing the highlight. In some of these example embodiments, determining that a user interacted with a highlight may increase or decrease the weighting, depending on the particular highlight type. For instance, where a user has a pattern of interacting with a particular highlight type, such as contact information, status, or task-related highlights, then the user's interaction may indicate that the user relies on this information and thus this type of highlight is valuable to the user. Accordingly, the suppression weighting for this highlight (or for highlights of this type) may be decreased. For other types of highlights, such as those that are temporal and only likely to result in user action once, suppression weighting may be increased following a determination of user interaction (or likely user interaction) with the highlight. For example, where the highlight type is a date event or a highlight type that is temporal, such as a birthday or anniversary, then the interaction might indicate that the user has acted on the particular highlight. Therefore, its suppression weighting may be increased, according to suppression logic256, making it less likely to be presented again.

In some embodiments, suppression logic256may utilize user behavior rules, periodic/time based rules, user interest, and/or people highlight context rules. In particular, user behavior rules (or logic) utilizes historic user activity patterns or behaviors for the user or similar users (e.g., the user never views or interacts with people highlights relating to birthdays or social information; the user only views work information; the user has quickly dismissed or closed similar people highlights in the past; or other user activity or patterns of user activity related to people highlights), to determine whether to suppress one or more particular people highlights. The user activity or behavior information may be received from user context information242or may be determined by context determiner230or activity detector232. For example, a highlight of a certain category that is consistently dismissed by a user's colleagues (similar users), such as a person's birthday, may be determined to be suppressed for the user.

In some embodiments, periodic and/or time based rules (or logic) may comprise logic for suppressing a highlight based on whether it has been presented recently or whether the highlight is stale. Some embodiments of periodic and/or time based rules or logic comprise logic for increasing, decreasing or modifying a suppression weighting associated with a people highlight; for instance, a suppression weighting may be increased each time the highlight is shown to the user. Similarly, in some embodiments, this logic may increase the suppression weighting of a people highlight after an interval of time (e.g., 1 day, 2 days, 1 week, or other time interval). In this way, highlights that may grow stale, such as date-related highlights like birthdays, anniversaries, or events, are less likely to be presented as time passes after the date associated with the highlight. The particular time interval may be based on the particular content of the highlight or the type of highlight (e.g., birthday, which may be for 1-2 days, a promotion announcement, which may be for 1 week), for example.

User interest rules (or logic) may be utilized to determine or predict a likely user suppression interest (i.e., whether a user would likely want to see or not want to see a particular highlight or highlight type) associated with a particular people highlight or type of highlight. For example, the user interest logic may utilize implicit information to infer or predict the user interest or explicit information, such as explicit user settings (where the user has indicated they do not value or desire certain people highlights, or categories of highlights, such as social highlights, birthdays, work anniversaries, or employment history). Without limitation, examples of implicit information for determining or predicting user interest may comprise user settings of similar users or activity of the user (which may include user behavior or an activity pattern) or similar users with regard to similar people highlights or similar people highlight types.

People highlight context rules (or logic) utilizes contextual information to determine an indication of whether to suppress a highlight or type of highlight. The contextual information may be determined by context determiner230or received from user context information242. For example, contextual information may indicate that a particular task has been completed, and thus a highlight associated with the task should be suppressed because it is no longer relevant. Similarly, contextual information may indicate that a particular people highlight was already presented within a recent time interval (e.g., within the past few days, or within a session).

In some embodiments, suppression logic256may comprise logic for determining suppression based on information about the behavior of other users similar to the user (e.g., similar users have interacted with a similar people highlight; or similar users never respond to people highlights with work-related information). Suppression logic256also may include rules based on personal preferences or settings of the user or the person of interest. For example, a particular person of interest may desire not to show his birthday information as a highlight, and thus may configure a setting to do so.

The suppression logic256can take many different forms. For example, the suppression logic may comprise Boolean logic, fuzzy logic, neural network, finite state machine, support vector machine, decision tree or random forest, logistic regression, clustering, topic modeling, reinforcement learning model (e.g., wherein user dismissal of a highlight results in a penalty and/or where user interaction with a highlight results in a reward) or other machine learning techniques, similar statistical classification or machine-learning prediction processes or, combinations of these to determine any of the ranked set of people highlights to suppress. In some embodiments, suppression logic256may be based on activity or activity-patterns learned from other users, which may include other similar users. For example, where it is learned that a particular user's coworkers (similar users) often dismiss or ignore birthday-related highlights, it may be inferred that the office culture does not celebrate birthdays. From the observed pattern of dismissing birthday highlights, suppression logic256may learn a rule to suppress people highlights for birthday information for the particular user. In some embodiments, this learned logic may include rules that are explicitly defined by other users (such as user configurations of other users specifying to suppress people highlights of a social nature when the current context indicates that user is at work) or implicit rules (which may be determined based on monitoring the user activity of another user, as described above for the primary user). In this way, as new people highlights are developed or evolved, embodiments of the technologies described herein can adapt and continue to provide the benefits that a user desires. Suppression engine270thus removes or suppresses any identified people highlights from the ranked set of people highlights received from the ranking engine268and passes the revised set of remaining ranked people highlights to presentation engine272or a separate presentation engine220.

One or more of the highlights determiner266, ranking engine268, or presentation engine272may determine how many people highlights, if any, should be presented to the user. This determination might be made, for example, based upon the device's screen size (with potentially more people highlights presentable on a laptop computer as compared to a cell phone) or the surface on which the people highlights will be presented (for example a calendaring application, communication platform, or other application or program).

An exemplary schematic screen display300is shown inFIG.3A, which represents any of the computing device(s)102(n) discussed above with respect toFIG.1. Screen display300is shown having a schematic surface302, which could be any of a number of different programs, applications or other display on the computing device screen display300. A smaller display space304is shown, which could be, for example, a people card or other type of informational display. Within display space304, a number of people highlights are shown as People Highlight 1 (306), People Highlight 2 (308), and People Highlight 3 (310) for an example person (305). In this exemplary screen display300, three people highlights are shown and may be the highest three people highlights ranked by ranking engine268. Another exemplary screen display300is shown inFIG.3B, which also shows three people highlights for the person (305). Here, however, within display space304, the people highlights shown are People Highlight 1 (306), People Highlight 3 (310), and People Highlight 4 (312). The difference betweenFIG.3AandFIG.3Bis that People Highlight 2 (308) is suppressed (with respect toFIG.3B) by, for example, suppression engine270and/or according to suppression logic256. With People Highlight 2 suppressed, and with the screen real estate to show, for example, three people highlights, another people highlight, e.g., People Highlight 4 (312), can be displayed, if available.

Example system200also includes storage250. Storage250generally stores information including data, computer instructions (e.g., software program instructions, routines, or services), logic, profiles, and/or models used in embodiments described herein. In an embodiment, storage250comprises a data store (or computer data memory). Further, although depicted as a single data store component, storage250may be embodied as one or more data stores or may be in the cloud.

As shown in example system200, storage250includes user profile240and highlights logic252, ranking logic254and suppression logic256. One example embodiment of a user profile240is illustratively provided inFIG.2. Example user profile240includes information associated with a particular user such as user context information242(from context determiner230), information about user accounts and devices246, and user preferences248. The information stored in user profile240may be available to the people highlights engine260and other components of example system200.

As described previously, user context information242generally includes information describing the overall state of the user, which may include information regarding the user's device(s), the user's surroundings, activity events, related contextual information, activity features, or other information determined via context determiner230, and may include historical or current user activity information. In some embodiments, user context information242may also include semantic information determined by semantic information analyzer236.

User accounts and devices246generally includes information about user devices accessed, used, or otherwise associated with a user, and/or information related to user accounts associated with the user; for example, online or cloud-based accounts (e.g., email, social media) such as a Microsoft® Net passport, other accounts such as entertainment or gaming-related accounts (e.g., Xbox live, Netflix, online game subscription accounts, or similar account information), people data relating to such accounts such as user emails, texts, instant messages, calls, other communications, and other content; social network accounts and data, such as news feeds; online activity; and calendars, appointments, application data, other user accounts, or the like. Some embodiments of user accounts and devices246may store information across one or more databases, knowledge graphs, or data structures. As described previously, the information stored in user accounts and devices246may be determined from people-data collection component210or context determiner230(including one or more of its subcomponents).

User preferences248generally include user settings or preferences associated with the user, and specifically may include user settings or preferences associated with the people highlights engine260. By way of example and not limitation, such settings may include user preferences about specific people highlights or people highlight categories that the user desires to be suppressed given certain user contexts. The user may specify which data or content should be included for use with the people highlights engine260. In one embodiment, preferences248may include user-defined rules (or group-policy rules, which may be set by an administrator) for presenting people highlights based on a context; for instance, suppressing specifically designated people highlights that are deemed unhelpful or undesirable to a specific user. Further, a graphical user interface may facilitate enabling the user to easily create, configure, or share these user-preferences. For example, in one embodiment, right-clicking on (or touching, or otherwise selecting) a particular people highlight may invoke a menu, a rules wizard, or other user interface that enables the user to specify treatment (i.e., whether to show or suppress a people highlight and according to which context) of that particular people highlight or similar people highlights.

Example system200may also include a separate presentation engine220that is generally responsible for presenting content and related information to a user, such as the people highlights received from people highlights engine260, and may work in conjunction with presentation engine272of people highlights engine260. The content may be presented via one or more presentation components516, described inFIG.5. Presentation engine220may comprise one or more applications or services on a user device, across multiple user devices, or in the cloud. For example, in one embodiment, presentation engine220manages the presentation of content to a user across multiple user devices associated with that user. Presentation engine220may determine on which user device(s) content is presented, and presents information determined by people highlights engine270. Presentation engine220presents this information, including any substitutions, reorganizations, or highlights as directed by the people highlights engine270.

Turning toFIG.4, a method400for intelligently selecting and presenting people highlights on one or more user devices is shown. As shown at block402, the method400includes collecting user information. The collected user information may include, for example, collecting information about the user and/or about people of possible interest to the user. The information may be collected from, for example, people-data collection component210, any of the data source(s)104, or from storage250such as in user profile(s)240. The method400also includes, as shown at block404, determining a user interest in a person. This determining could be from explicit interest from a user (such as hovering over a contact, or initiating a communication with the person) or implicit (such as the person being on a meeting invite, new to a company, or owed a task-result). The method400, as shown at block406, includes determining the context and/or social distance between the user and the determined person of interest, as may be performed for example by context determiner230, person of interest content crawler262, and/or person of interest indexer264.

As shown at block408, the method400includes collecting (or determining) a set of possible (or candidate or potential) people highlights. This collected set of possible people highlights may include, for example, any of the exemplary people highlights discussed above, and may be determined by highlights determiner266. At block410, the method includes ranking and/or ordering the received set of possible people highlights, and may be done by ranking engine268, using ranking logic254, for example. The method400also includes, at block412, determining whether any of the people highlights should be suppressed. The suppression determination may be done, for example, using suppression engine270and/or suppression logic256.

At block412, suppression logic256and/or suppression engine270may consider, for example, past user behaviors at block412A, periodic and/or time based rules at block412B, user interest at block412C, and/or people highlight context at block412D. Past user behaviors412A generally considers whether user (or similar user) behavior indicates that one or more particular people highlights in the set of possible people highlights should be suppressed. For example, user behaviors412A may determine, if a people highlight relates to a person in a meeting request previously declined by the user, and accordingly suppress that people highlight. As another example, user behaviors412A may consider whether the user has already dismissed the same highlight, or a similar highlight, and if so, the people highlight may be suppressed. As yet another example, if the user (or a similar user) has consistently dismissed or ignored a people highlight of a certain category (such as a people highlight indicating a person's birthday), the people highlight may be suppressed.

Periodic and/or time based rules412B generally use an aspect of suppression logic256to determine if the timing of a people highlight indicates it should be suppressed. For example, if the people highlight has already been shown x number of times during a time frame y or for a certain number of sessions z, periodic and/or time based rules412B may suppress the people highlight. As another example of periodic and/or time based rules421B, if a user ignores a presented people highlight for two sessions (a session may be, for example a window of time (such as 12 hours) in which the people highlight has been presented one or more times), then suppression logic256may determine that the highlight is not valuable or no longer relevant to the user and therefore suppress the highlight.

User interest block412C uses an aspect of suppression logic256to consider or determine user interest with respect to a people highlight, and suppresses any people highlights where the logic indicates a likely user interest to suppress the highlight. For example, user interest at block412C may be indicated by explicit user settings (where the user has indicated they do not value or desire certain people highlights, or categories of highlights, such as social highlights, birthdays, work anniversaries, or employment history). User interest at block412C, somewhat similarly to user behavior at block412A, can also determine user interest implicitly, such as by considering similar users' interest, or similar users' preferences and settings.

Suppression logic256may also consider, for example, a people highlight context at block412D. For example, if the people highlight context or history indicates a likely desire to suppress a people highlight (e.g., a people highlight task has already been completed, or the people highlight was already presented to the user and dismissed), people highlight context412D may suppress the people highlight. At block414, the method400includes presenting a selected number of remaining (those not suppressed) people highlights. The method may also include, for example, continued monitoring of the user-activity with respect to the people highlights and/or the user context. As the user-activity with respect to the people highlights or the context changes, the method may include determining whether any further modifications to the suppression of people highlights is warranted.

As another example, assume the last two ranked people highlights relate to a work anniversary this week (“Scott has worked with the firm for 25 years”), and a people highlight related to educational history (“Scott also attended the University of Iowa”). The ranking engine268, using ranking logic254will rank these two highlights. In this example, the work anniversary may be ranked higher, given the fact that it is timely, and temporal. Accordingly, the work anniversary people highlight may be presented to, and viewed by, the user. In this example, the work anniversary people highlight may continue to be presented any other time that people highlights are presented for the person of interest during a particular session (such as a 12 hour period). The next time the user sees the card and if the highlight is still available (e.g., not stale, for example, two days later), the user may again see the work anniversary people highlight. Now, assume the user viewed the people highlight, but did not interact with the people highlight (such as by clicking on it). On the next session, (e.g. a third day), the work anniversary people highlight may be suppressed (such as at block412B), and the next ranked people highlight (here, the educational history highlight) may be presented to the user because the work anniversary people highlight was ignored by the user in the previous two sessions, and/or has become stale.

Another form of suppression may occur by user interaction. For example, if the user clicks an ‘X’ displayed on the highlight to explicitly remove the highlight, the highlight is suppressed immediately. In some embodiments, explicitly suppressed highlights will not be presented to the user again, while implicitly suppressed highlights may be presented again at a later time, such as months later.

Accordingly, we have described various aspects of technology directed to systems and methods for intelligently selecting and presenting people highlights on personal computing devices. It is understood that various features, sub-combinations, and modifications of the embodiments described herein are of utility and may be employed in other embodiments without reference to other features or sub-combinations. Moreover, the order and sequences of steps shown in the example method400are not meant to limit the scope of the present disclosure in any way, and in fact, the steps may occur in a variety of different sequences within embodiments hereof. Such variations and combinations thereof are also contemplated to be within the scope of embodiments of this disclosure.

Having described various implementations, an exemplary computing environment suitable for implementing embodiments of the disclosure is now described. With reference toFIG.5, an exemplary computing device is provided and referred to generally as computing device500. The computing device500is but one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of embodiments of the disclosure. Neither should the computing device500be interpreted as having any dependency or requirement relating to any one or combination of components illustrated.

Embodiments of the disclosure may be described in the general context of computer code or machine-useable instructions, including computer-useable or computer-executable instructions, such as program modules, being executed by a computer or other machine, such as a personal data assistant, a smartphone, a tablet PC, or other handheld device. Generally, program modules, including routines, programs, objects, components, data structures, and the like, refer to code that performs particular tasks or implements particular abstract data types. Embodiments of the disclosure may be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, or more specialty computing devices. Embodiments of the disclosure may also be practiced in distributed computing environments where tasks are performed by remote-processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

With reference toFIG.5, computing device500includes a bus510that directly or indirectly couples the following devices: memory512, one or more processors514, one or more presentation components516, one or more input/output (I/O) ports518, one or more I/O components520, and an illustrative power supply522. Bus510represents what may be one or more busses (such as an address bus, data bus, or combination thereof). Although the various blocks ofFIG.5are shown with lines for the sake of clarity, in reality, these blocks represent logical, not necessarily actual, components. For example, one may consider a presentation component such as a display device to be an I/O component. Also, processors have memory. The inventors hereof recognize that such is the nature of the art and reiterate that the diagram ofFIG.5is merely illustrative of an exemplary computing device that can be used in connection with one or more embodiments of the present disclosure. Distinction is not made between such categories as “workstation,” “server,” “laptop,” or “handheld device,” as all are contemplated within the scope ofFIG.5and with reference to “computing device.”

Computing device500typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by computing device500and includes both volatile and nonvolatile, removable and non-removable media. By way of example, and not limitation, computer-readable media may comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVDs) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computing device500. Computer storage media does not comprise signals per se. Communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above should also be included within the scope of computer-readable media.

Memory512includes computer storage media in the form of volatile and/or nonvolatile memory. The memory may be removable, non-removable, or a combination thereof. Exemplary hardware devices include for example solid-state memory, hard drives, and optical-disc drives. Computing device500includes one or more processors514that read data from various entities such as memory512or I/O components520. Presentation component(s)516presents data indications to a user or other device. Exemplary presentation components include a display device, speaker, printing component, vibrating component, and the like.

The I/O ports518allow computing device500to be logically coupled to other devices, including I/O components520, some of which may be built in. Illustrative components include a microphone, joystick, game pad, satellite dish, scanner, printer, or a wireless device. The I/O components520may provide a natural user interface (NUI) that processes air gestures, voice, or other physiological inputs generated by a user. In some instances, inputs may be transmitted to an appropriate network element for further processing. An NUI may implement any combination of speech recognition, touch and stylus recognition, facial recognition, biometric recognition, gesture recognition both on screen and adjacent to the screen, air gestures, head and eye tracking, and touch recognition associated with displays on the computing device500. The computing device500may be equipped with depth cameras, such as stereoscopic camera systems, infrared camera systems, RGB camera systems, and combinations of these, for gesture detection and recognition. Additionally, the computing device500may be equipped with accelerometers or gyroscopes that enable detection of motion. The output of the accelerometers or gyroscopes may be provided to the display of the computing device500to render immersive augmented reality or virtual reality.

Some embodiments of computing device500may include one or more radio(s) (or similar wireless communication components). The radio transmits and receives radio or wireless communications. The computing device500may be a wireless terminal adapted to receive communications and media over various wireless networks. Computing device500may communicate via wireless protocols, such as code division multiple access (“CDMA”), global system for mobiles (“GSM”), or time division multiple access (“TDMA”), as well as others, to communicate with other devices. The radio communications may be a short-range connection, a long-range connection, or a combination of both a short-range and a long-range wireless telecommunications connection. When we refer to “short” and “long” types of connections, we do not mean to refer to the spatial relation between two devices. Instead, we are generally referring to short range and long range as different categories, or types, of connections (i.e., a primary connection and a secondary connection). A short-range connection may include, by way of example and not limitation, a Wi-Fi® connection to a device (e.g., mobile hotspot) that provides access to a wireless communications network, such as a WLAN connection using the 802.11 protocol; a Bluetooth connection to another computing device is a second example of a short-range connection, or a near-field communication connection. A long-range connection may include a connection using, by way of example and not limitation, one or more of CDMA, GPRS, GSM, TDMA, and 802.16 protocols.

Many different arrangements of the various components depicted, as well as components not shown, are possible without departing from the scope of the claims below. Embodiments of the disclosure have been described with the intent to be illustrative rather than restrictive. Alternative embodiments will become apparent to readers of this disclosure after and because of reading it. Alternative means of implementing the aforementioned can be completed without departing from the scope of the claims below. Certain features and sub-combinations are of utility and may be employed without reference to other features and sub-combinations and are contemplated within the scope of the claims.