Templates associated with content items based on cognitive states

A processor-implemented method, system, and/or computer program product alter a template used to display content on a user interface. One or more processors receive a transmission from the one or more hardware devices that describes a current cognitive state of a user. The processor(s) receive content to be presented to the user on a user interface, and identify template components that are appropriate for presenting the content to the user based on the current cognitive state of the user. The processor(s) alter an initial template used to present the content, such that altering the initial template creates an altered template, and where the altered template uses the template components that are determined to be appropriate for presenting the content to the user based on the current cognitive state of the user. The processor(s) then present the content on the user interface using the altered template.

BACKGROUND

The present disclosure relates to the field of computer systems, and specifically to the field of computer systems that display content on graphical user interfaces. Still more specifically, the present disclosure relates to creating and/or selecting templates to display content based on a cognitive state of a user.

SUMMARY

A processor-implemented method, system, and/or computer program product alter a template used to display content on a user interface. One or more processors receive a transmission from the one or more hardware devices that describes a current cognitive state of a user. The processor(s) receive content to be presented to the user on a user interface, and identify template components that are appropriate for presenting the content to the user based on the current cognitive state of the user. The processor(s) alter an initial template used to present the content, such that altering the initial template creates an altered template, and where the altered template uses the template components that are determined to be appropriate for presenting the content to the user based on the current cognitive state of the user. The processor(s) then present the content on the user interface using the altered template.

DETAILED DESCRIPTION

With reference now to the figures, and in particular toFIG. 1, there is depicted a block diagram of an exemplary system and network that may be utilized by and in the implementation of the present invention. Note that some or all of the exemplary architecture, including both depicted hardware and software, shown for and within computer101may be utilized by software deploying server149and/or client computer151shown inFIG. 1.

Exemplary computer101includes a processor103that is coupled to a system bus105. Processor103may utilize one or more processors, each of which has one or more processor cores. A video adapter107, which drives/supports a display109, is also coupled to system bus105. System bus105is coupled via a bus bridge111to an input/output (I/O) bus113. An I/O interface115is coupled to I/O bus113. I/O interface115affords communication with various I/O devices, including a keyboard117, a mouse118, a media tray121(which may include storage devices such as CD-ROM drives, multi-media interfaces, etc.), a camera123(capable of capturing still and/or video images), and external USB port(s)125. While the format of the ports connected to I/O interface115may be any known to those skilled in the art of computer architecture, in one embodiment some or all of these ports are universal serial bus (USB) ports.

As depicted, computer101is able to communicate with a software deploying server149and/or client computer151using a network interface129. Network interface129is a hardware network interface, such as a network interface card (NIC), etc. Network127may be an external network such as the Internet, or an internal network such as an Ethernet or a virtual private network (VPN).

A hard drive interface131is also coupled to system bus105. Hard drive interface131interfaces with a hard drive133. In one embodiment, hard drive133populates a system memory133, which is also coupled to system bus105. System memory is defined as a lowest level of volatile memory in computer101. This volatile memory includes additional higher levels of volatile memory (not shown), including, but not limited to, cache memory, registers and buffers. Data that populates system memory135includes computer101's operating system (OS)137and application programs143.

OS137includes a shell139, for providing transparent user access to resources such as application programs143. Generally, shell139is a program that provides an interpreter and an interface between the user and the operating system. More specifically, shell139executes commands that are entered into a command line user interface or from a file. Thus, shell139, also called a command processor, is generally the highest level of the operating system software hierarchy and serves as a command interpreter. The shell provides a system prompt, interprets commands entered by keyboard, mouse, or other user input media, and sends the interpreted command(s) to the appropriate lower levels of the operating system (e.g., a kernel141) for processing. Note that while shell139is a text-based, line-oriented user interface, the present invention will equally well support other user interface modes, such as graphical, voice, gestural, etc.

As depicted, OS137also includes kernel141, which includes lower levels of functionality for OS137, including providing essential services required by other parts of OS137and application programs143, including memory management, process and task management, disk management, and mouse and keyboard management.

Application programs143include a renderer, shown in exemplary manner as a browser145. Browser145includes program modules and instructions enabling a world wide web (WWW) client (i.e., computer101) to send and receive network messages to the Internet using hypertext transfer protocol (HTTP) messaging, thus enabling communication with software deploying server149and other computer systems.

Application programs143in computer101's system memory (as well as software deploying server149's system memory) also include an User Interface Template Management Logic (UITML)147. UITML147includes code for implementing the processes described below, including those described inFIGS. 2-6. In one embodiment, computer101is able to download UITML147from software deploying server149, including in an on-demand basis, wherein the code in UITML147is not downloaded until needed for execution. Note further that, in one embodiment of the present invention, software deploying server149performs all of the functions associated with the present invention (including execution of UITML147), thus freeing computer101from having to use its own internal computing resources to execute UITML147.

Computer101(and particularly client computer151) includes one or more biometric sensors153, which are able to detect body heat, pupil size, respiration, skin sweat, etc. used to determine the current physiological and/or psychological state of a user. For example, if a person is sweating and has constricted pupils, then a state of anxiety in the user may be assumed.

Described herein is a method and system of creating or selecting a template for display of content, such that the representation adapts to a real-time assessment of the cognitive state of the user. In one or more embodiments, the system learns what templates, or template alterations, are best for different cohorts of users.

In the prior art, content is associated with templates without any regard to the real-time cognitive state of the viewer. The present invention, however, associates templates with content items based on the current mood and current cognitive state of the person or persons, and optionally along with a consideration of the content and the screen size to display the content item.

The term template can have many meanings. For example, it may concern the control of Web page formatting and eBook formatting, where items are arranged on a page and how they respond to user interface (UI) interactions. Templates can also control the layout of items on other kinds of UIs, such as interactive UIs that allow a user to enter data, request information, etc.

Applications, eBooks, and/or web resources (e.g., websites) that seek to display dynamic content face the challenge of displaying the dynamic content in a visually appealing way in combination with editorially chosen content. The challenge appears when arranging content within templates where the templates may be designed for editorially chosen content, but dynamic content, coming from outside sources, may not fit the rigid templates when it is desirable to improve the template based on a user's cognitive state. Thus, layouts of the content may be arranged in an odd or ineffective manner (or may not be useful for classes of viewers) and not be responsive to a cognitive state of a user (including fatigue level, distraction level, medical conditions, etc.).

As mentioned above, in the prior art content is often arranged into rigid templates that are pre-made and not chosen based on content or a real-time assessment of a user state. Rather, content is made to fit the template. This results in little variability in displays and no consideration of automated assessment of user mood, cognitive state, fatigue level, distraction level, membership in a particular cohort, etc.

In an embodiment of the present invention, template composition utilizes “template” snippets that are composed in real time, based on a user's real-time context. Additionally, a module includes a learning component that refines templates over time.

Applications may request and/or receive content items from a database, which may be any online repository storing content items, and a content cluster may be created based on application requirements and on real-time cognitive considerations. Application requirements may be, for instance, that the content cluster is a boxed content cluster (i.e., content items are presented with boxed borders), unboxed content cluster (i.e., there are no boxes around content items), and the like. The content cluster may also be arranged based on available screen space. Content items may be associated with templates within the content cluster until there are no more content items to display or a maximum column count has been reached. The maximum column count may depend on the available screen space.

Templates may be selected for content items based on automated assessment of user mood, cognitive state, fatigue level, distraction level, cohort membership, etc., along with the presence of an image, size of an image, content image aspect ratio, title length, content length, abstract presence, snippet presence, template class, and the like. Once all content items have been associated with a template, the set of content items is displayed within the content cluster.

At a high level, the present invention presents a method and system of creating or selecting a template for display of content, with the representation changing based on the cognitive state of user. The cognitive state is any of nervousness, elation, fatigue, frustration, anger, distraction level, etc.

The cognitive state may be related to a user type/cohort (e.g., certain medical and/or psychological states). Certain psychological states may be estimated by an analysis of how words are strung together by a user. That is, certain patterns of words and how they are strung together may indicate a certain medical/psychological state of the user.

In one or more embodiments of the present invention, the cognitive state is inferred (with a certain level of confidence) by any of: user biometric, facial expression analysis, noise level in the environment, possible distractions caused by number of open activities (e.g., windows open) on a computer desktop, concurrent use of other devices (e.g., a phone), possible fatigue based on meeting schedules for the day (as determined by entries on an electronic calendar for the user), etc.

The cognitive state may also inferred by any of: user profile, caregiver input, demographic factors, etc.

In one or more embodiments of the present invention, the template change may involve graphics and layouts, scenes meant to soothe the user, cartoons designed to amuse the user, etc. That is, the template change may involve changes to textual attributes, such as words meant to sooth the user. The template change may involve changes to audio attributes, such as a change to or addition of music, sounds of nature, a user's favorite song, etc. The template change may involve the building of an object in a virtual world. The template change may involve the inclusion or specification of a puzzle, joke, or riddle. The template change may involve the size of the graphical elements.

Templates may be higher dimensional constructions in which the higher dimension relates to such considerations as progression of a cognitive disease, progression of an emotional state, etc.

The system may learn what templates are best for different cohorts of users.

The present invention affords the utility of selecting and changing templates based on cognitive considerations, including changing the template based on cognitive state of user. Depending on the class of user a person is in, a person may want a different template that is more useful to me based on my current state or cohort (e.g., medical or psychological cohort) and cognitive style, etc. Thus, User A may need to use a UI that utilizes a standard template, while User B, who has a medical/psychological condition that makes using the standard template difficult if not impossible, is better off with a template/UI that uses a different spacing, arrangement, sizing, etc. than that of the standard template.

The system learns what templates are best for different cohorts (classes of user) so that other users in the same class/cohort can benefit as the system learns. For example, a user may tap on a certain style of template when the user “likes” it. In other embodiments, the system may determine (with a certain level of confidence) that a user is becoming impatient, nervous, etc.

With reference nowFIG. 2, a graph202depicts various clusters that are best suited to particular types of template characteristics. Plotted on graph202are two template characteristics. For example, template characteristic204may be the use of text on a template/UI (with the quantity of text usage increasing along the vertical axis for template characteristic204). Template characteristic206may be the use of complex pie graphs on a template/UI (with the quantity of complex pie graphs used on a single UI increasing from left to right along the horizontal axis for template characteristic206).

For example, assume that cohort208is made up of young persons. As such, history has shown that members of cohort208best interact with a template/UI that is has little text (as indicated by cohort208being in the lower part of graph202) and few pie charts (as indicated by cohort208being in the left area of graph202).

Assume now that cohort210is for persons having a certain neurological condition. Historical data shows that persons in cohort210would rather use little text but is high in pie charts.

Assume further that cohort212is for users who are in one or more particular environmental states, such as being in a loud area that is moderately bright. As such, members of cohort212(i.e., users who are in these environments) prefer a template/UI that is heavy in text (as indicated by being in the upper area of graph202) and a moderate quantity of pie charts (as indicated by being in the middle (from left to right) area of the graph202.

Various approaches are possible for estimating a user's cognitive state. For example, an application (“app”) for a computer or smart phone may use face-tracking technology that allows computers to read facial expressions. As shown inFIG. 3, a system (e.g., client computer151shown inFIG. 1) takes a digital photograph301(e.g., using camera123shown inFIG. 1) of the user. The UITML147(e.g., within the computer101shown inFIG. 1) recognizes certain shapes of the photographed face in the digital photograph301, and converts those shapes into an emoji303and/or a line graph305representing a state of happiness for the user of the system.

With reference now toFIG. 4, an exemplary system400used in one or more embodiments of the present invention is presented. System400dynamically assigns content items to templates based on current real-time cognitive states of the user.

System400includes a computing device451(analogous to client computer151shown inFIG. 1), which communicates via a network427with a database402and a computer401(analogous to computer101shown inFIG. 1), which determines what types of layouts are used on the client computing device451. Database402may be any database that stores content items (i.e., layout items), and is accessible by entities submitting proposed content items to be used on a template/UI.

The template engine404is part of any device (such as computer401) that is capable of implementing the present invention. The receiving component406is configured to receive content items (such as newspaper stories and photos). The validating component408validates a template class, if any, associated with a content item. A template class refers generally to an identifier representing an indication of specific properties of a content item. Template classes may be based on whether an image is present, the size of the image, the orientation of the image, whether the content item is a video or slide show, etc. The template classes may also be used to control the layout origination of content.

The layout editor410is configured to associate content items with templates based in part on the cognitive state of a current viewer. The layout editor410may evaluate a variety of factors including a user's cognitive state, a template class, screen size, etc.

The displaying component407is a driver/interface (e.g., video adapter107shown inFIG. 1) for displaying the dynamically adjusted template on the UI.

With reference now toFIG. 5, a detailed flow chart of one or more steps performed by one or more processors and/or other hardware devices to generate and/or select a UI template in accordance with one or more embodiments of the present invention is presented.

After initiator block501, the system obtains user context and searches for candidate templates, as described in block503. That is, the system identifies the context of the user, including the user's current emotional state (as determined by photo analysis, readings from biometric sensors on the user, etc.), the user's general state (e.g., level of maturity, physical and psychological state, etc.), the environment (e.g., a noisy environment, a bright environment, etc.), etc. The system then determines which initial template is best suited for displaying content (e.g., from a webpage, an eBook, a data collection/survey page, a product ordering page, etc.).

As shown in block505, the system performs several steps for each template or complementary template that matches the user context.

As shown in block507, the system compares template features. For example, as shown in table508, one template (“Template1”) may output content verbally but performing a text-to-speech conversion. This type of output is useful if the content is appropriate for verbal communication, and if the user is in a state (e.g., quiet environment, is not hearing impaired, etc.) in which he/she can hear the content. Another template (“Template2”) may display text and/or other visual content on a heads-up display (e.g., on a windshield of a car). As shown in table508, if the vehicle is in motion and/or the driver is tired, then displaying text information on a heads-up display is inappropriate and/or dangerous, such that the text-to-speech template will override the text projection template, even if the user's other cognitive features favor the heads-up display.

As described in block509, the system then composes certain template features that best match the current cognitive and environment of the user. For example, if a user is a toddler who is known to interact best with cartoons and clickable icons, then these features are retrieved and/or created by the system to be used for that user.

As described in block511, the system then identifies any default settings (e.g., cartoons and clickable buttons for toddlers). However, in block513, any exceptions to these default settings (e.g., a toddler having limited hand/eye coordination) are identified. As shown in block515, an expert system (e.g., a heuristic capability of UITML147shown inFIG. 1) resolve the exception (e.g., use an aural template instead of a cartoon/clickable button template). These new template features are then added to the template (block517), and each new feature/term is validated (block519) as being appropriate for the user (e.g., based on how well the user interacts with this new template). Once the system confirms that this new template is useful, it is published for use by other systems (block521). The flow chart ends at terminator block523.

As described herein, the present invention customizes a template for a particular real-time cognitive state and environment of a particular user. The generated/selected specific templates are targeted to persons with certain cognitive abilities. These templates may be selected based on the determination of cognitive state, and then adjusted over time as certain states progress.

For example, assume that two cognitive states are considered. The first cognitive state is a normal cognitive state, and the second cognitive state is an impaired cognitive state. Thus, when the template asks for the user's date of birth, the original template (to be used with a person having a normal cognitive state) asks the user to “Enter MM-DD-YY”. However, the cognitively impaired template (for the user with a cognitive impairment) will display or say to the user “I see you were born in the month of January. Please enter the day of your birth in that month.”. Or the cognitively impaired template may say “I see you were born on January 1. What year were you born in?” Similarly, for an address entry, the original template may state “Enter your Street Address”, which the cognitively impaired template may state “What's the name of your street?” or “I see you live on Main Street. What is the number on Main Street where you live?”

If the UI is directed to obtaining a food order from a person, an original template may ask the user to “Please choose an entree from the list below” (where a listing of menu items is displayed below). However, a cognitively impaired template may ask “Do you want to eat breakfast, lunch, or supper?”. If the user enters “supper”, then the cognitively impaired template would as follow-up questions “Do you want to eat beef, chicken, or fish?” and then “Do you want to have a red sauce or a white sauce?”.

With reference now toFIG. 6, a high level flow chart of one or more steps performed by one or more processors and/or other hardware devices to generate and/or select a UI template in accordance with one or more embodiments of the present invention is presented.

After initiator block602, one or more processors receive a transmission from the one or more hardware devices that describes a current cognitive state of a user (e.g., camera123shown inFIG. 1, as described in block604.

As described in block606, the processor(s) receive content to be presented on a user interface to the user (e.g., a webpage, an eBook, a questionnaire, etc.).

As described in block608, the processor(s) identify template components that are appropriate for presenting the content to the user based on the current cognitive state of the user, such as certain fonts of text, cartoon characters, sounds (e.g., audible words or music), etc.

As described in block610, the processor(s) then alter an initial template that was being used to present the content on the user interface, As described herein, altering the initial template creates an altered template, and the altered template uses the template components that are determined to be appropriate for presenting the content to the user based on the current cognitive state of the user.

As described in block612, the processor(s) then present the content on the user interface using the altered template.

The flow chart ends at terminator block614.

In an embodiment of the present invention, the current cognitive state of the user is from a group consisting of nervousness, elation, fatigue, frustration, anger, and distraction.

In an embodiment of the present invention, the processor(s) identify a cohort of users who share the current cognitive state of the user. For example, cohort members may all have a same cognitive deficiency, be temporarily suffering a same level of anxiety, etc. The processor(s) then determine a preferred template that was used in the past to present content to members of the cohort of users, and present the content on the user interface using that preferred template.

In an embodiment of the present invention, the processor(s) determine the current cognitive state of the user based on biometric sensor readings from biometric sensors associated with the user (e.g., biometric sensors153shown inFIG. 1).

In an embodiment of the present invention, the processor(s) determine the current cognitive state of the user based on facial expression analysis of the user (e.g., using the camera123shown inFIG. 1).

In an embodiment of the present invention, the processor(s) determine the current cognitive state of the user based on a noise level in an environment of the user (e.g., using readings from environmental sensors155inFIG. 1).

In an embodiment of the present invention, the processor(s) determine the current cognitive state of the user based on distractions to the user caused by a quantity of open windows on a computer desktop being used by the user. That is, computer101can detect how many applications and/or open windows are running on the client compute155. Based on this information, computer101will determine how to format the template used to display content on the client computer151.

In an embodiment of the present invention, the processor(s) determine the current cognitive state of the user based on a concurrent use of other devices by the user. That is, the computer101may determine, based on monitoring of cell phone networks or from photos taken by camera123inFIG. 1, that the user of client computer151is currently on the phone. Based on this information, the template used to present information to the user will be adjusted accordingly (e.g., using only visual information without any verbal/oral information being presented on the client computer151).

In an embodiment of the present invention, the processor(s) determine the current cognitive state of the user based on fatigue of the user based on meeting schedules for the user from an electronic calendar. That is, the computer101will examine the electronic calendar of the user of the client computer151. If the user of the client computer151has been in meetings all day, then the computer101may add soft music to the presentation, reduce the amount of information being presented, etc.

In an embodiment of the present invention, the processor(s) determine the current cognitive state of the user based on a user profile of the user.

In an embodiment of the present invention, altering the initial template to create the altered template includes changing graphics and layouts of the initial template.

In an embodiment of the present invention altering the initial template to create the altered template comprises an addition of visual scenes (e.g., calming ocean waves) that are designed to soothe the user.

In an embodiment of the present invention, altering the initial template to create the altered template includes adding sound (e.g., rippling water, soft music, etc.) that has been predetermined to sooth the user.

In one or more embodiments, the present invention is implemented in a cloud environment. It is understood in advance that although this disclosure includes a detailed description on cloud computing, implementation of the teachings recited herein are not limited to a cloud computing environment. Rather, embodiments of the present invention are capable of being implemented in conjunction with any other type of computing environment now known or later developed.

Characteristics are as follows:

Service Models are as follows:

Deployment Models are as follows:

Workloads layer90provides examples of functionality for which the cloud computing environment may be utilized. Examples of workloads and functions which may be provided from this layer include: mapping and navigation91; software development and lifecycle management92; virtual classroom education delivery93; data analytics processing94; transaction processing95; and template alteration processing96(for altering templates used on UIs as described herein).