INTELLIGENT GAMING ASSISTANT FOR PERSONAL WELLNESS

Systems and methods for providing wellness assistance to a user during a gaming session are disclosed. Telemetry data associated with a user is received, e.g., by an information handling system or hub device, during a gaming session for a gaming application. A wellness level of the user is monitored during the gaming session, based, at least in part, on the telemetry data. Upon determining, based on the monitoring, that the user needs wellness assistance during a portion of the gaming session, a wellness assistance session is initiated to provide the wellness assistance via at least one interface of a device associated with the user.

FIELD OF THE DISCLOSURE

The instant disclosure relates to information handling systems. More specifically, portions of this disclosure relate to an information handling system for assisting users of video game applications (or gamers) with their personal wellness.

BACKGROUND

One increasingly popular use for information handling systems is gaming. Information handling systems can be utilized by users to execute a variety of gaming applications, such as strategy games, adventure games, first person shooter games, racing games, sports games, simulation games, role playing games, platformer games and other games. Gaming applications may range in difficulty from easy to learn by a non-gamer to difficult even for veteran gamers.

Oftentimes, when a user encounters a particularly difficult segment of a game or wishes to improve their skills, the user may consult online videos, written strategy guides, forums, message boards, friends, and other information sources for tips, strategies, and other assistance related to improving their skills with respect to a particular game. Such information sources, however, are seldom used by gamers for their own personal wellness. Moreover, some gamers may participate in extended gaming sessions that last several hours without taking their personal well-being into account, for example, by taking regular breaks to rest their eyes or managing their stress or excitement levels before bedtime.

Shortcomings mentioned here are only representative and are included simply to highlight that a need exists for improved information handling systems. Embodiments described herein address certain shortcomings but not necessarily each and every one described here or known in the art. Furthermore, embodiments described herein may present other benefits than, and be used in other applications than, those of the shortcomings described above.

SUMMARY

Information handling systems may be used to execute applications, such as gaming applications. Embodiments of this disclosure may be used to provide an information handling system, such as a hub device, which uses artificial intelligence (AI) to provide assistance with a user's personal wellness during an application session. The application session may be a gaming session of the user with a gaming application executed by the information handling system or hub device. Gaming applications may include any of various single-player or multi-player games, such as puzzle games, role playing games, adventure games, first person shooter games, sports games, simulation games, strategy games, platformer games, and other games. Such games may be hosted by the information handling system to support different types of gameplay in one or more gaming environments, such as one or more rooms of a household for gameplay in a family environment or a cloud-based gaming environment for online gameplay.

In some embodiments, the information handling system or hub device may include an intelligent personal wellness assistance service (e.g., an automated wellness assistant or wellness coach) that analyzes telemetry data acquired from various sources to monitor the user's personal wellness during the gaming session and provide appropriate assistance or feedback, e.g., to encourage the user to take appropriate action for their personal well-being. In some embodiments, the telemetry data may include or correspond to session data, biometric data, and relationship data. The session data may indicate one or more gaming characteristics of the user that may affect the user's state of health or wellness (such as the user's age, skill or experience level, preferred gaming time, and duration of each gaming session). The biometric data (also referred to herein as “wellness data”) may indicate one or more physiological characteristics of the user, such as the user's level of stress during the gaming session. The biometric data associated with the user may include various measurements (e.g., heart rate, skin temperature, skin moisture or perspiration, etc.) collected by different sensors or measurement devices that are coupled to or integrated with a peripheral device, such as a wireless game controller, of the user during the gaming session. Such sensors may include, for example, one or more motion sensors (e.g., one or more of a gyroscope and an accelerometer), a heart rate sensor, a temperature sensor, a moisture sensor, a microphone, a fingerprint sensor (for user identification), and one or more location sensors or devices. The relationship data may include, for example, information connecting the user with one or more other users associated with one or more gaming environments (e.g., one or more rooms within the user's house). Such other users may be, for example, other family members who are also gamers residing in the same household as the user. In some implementations, the relationship data may be determined based on the session data associated with one or more of the users. For example, the session data associated with a first user may indicate one or more interactions between the first user and a second user, which may then be used to determine a relationship between the first and second users.

In some embodiments, the wellness assistance service of the information handling system may include an AI engine that analyzes the telemetry data along with information relating to a gaming context of the gaming application to monitor the user's level of health or personal wellness during the gaming session. Based on the monitoring, the AI engine may determine whether the user needs wellness assistance during at least a portion of the gaming session, such as while playing a particularly challenging stage of the game. The assistance may be provided in various ways, such as through in-game assistance via an on-screen display, through an interface of a gaming companion application executable at a mobile device of the user, or through controller feedback via a haptic or lighting interface of the user's game controller or other peripheral device, or through any combination of the foregoing interfaces. The type of assistance provided to the user during the session may depend on a gaming context associated with the gaming application. The gaming context may be based on the type of gaming application (e.g., single-player vs. multi-player game) and the type of gaming environment (e.g., a family environment with gamers located in the same household vs. a cloud environment with online gamers in different remote locations). In some embodiments, the AI engine may employ a machine learning model for the data analysis and wellness monitoring operations. The machine learning model may be trained to assess the user's gaming performance and wellness levels based on session history data collected over time during previous gaming sessions of the user or other users with similar gaming profiles (or both) for the gaming application. The AI engine may also incorporate user feedback to evaluate the effectiveness of the assistance and retrain the model if necessary to improve the assistance to be provided during subsequent gaming sessions based on the user's particular gaming characteristics and level of health or wellness.

In some embodiments, the aspects described herein may be used to support the execution of gaming applications in different environments. Gaming sessions may execute on a service, either locally on a device, on another system on the network, or in the cloud. A device may access the gaming session by executing an application that communicates with the service to receive and transmit user input to the service and provide feedback to the user from the service. The device may include its own audio/visual (AV) output for displaying a graphical user interface and/or a rendered display from the gaming session. Different environments at a location may include different AV systems, and the device may be automatically paired with an AV system and may be reconfigured to support interaction with an application session using the paired AV system.

A user's home is one example location that may have multiple environments, such as a living room, a dining room, a study, and/or a bedroom, each with different screen configurations, speaker configurations, and/or network availability. Aspects of embodiments disclosed herein may provide a system that enables game play from a set of candidate game hosts and environments to consumption devices of a user's choice while the user moves about their home between the different environments. The system may employ methods to determine where a user is located within the home, availability and selection of candidate game hosting and target environments, homing and direction of related I/O, and/or AV for consumption. The system then migrates the user and their information to the determined environment by coordinating gameplay by the user. The solution accommodates multiple users simultaneously within the home, whether in single player, multiplayer using the same screen, or multiplayer using separate screen games. The solution may configure AV and input/output (I/O) such that multiple users can consume one or multiple games in the home simultaneously, whether in separate locations or when seated together in front of the same consumption device, e.g., a large television, where multiple games might be hosted simultaneously.

The mobility of a user between services and applications for executing an application session may be supported by an information handling system that uses available telemetry from multiple sources to build a confidence-based knowledge graph of the user's gaming environments and determine a position of the user within that graph. A system with knowledge of devices in a user's gaming environment may build a knowledge graph by aggregating and comparing telemetry. For example, network telemetry may reveal that devices are positioned relatively near each other, a mobile device may reveal an absolute location based on GPS data, and/or an infrared presence sensor may reveal that the user is sitting in front a device. An intelligent system may assemble these individual pieces of telemetry into a broader knowledge graph based on the absolute and/or relative locations of the user's devices, the location of the user in relation, and or characteristics of the devices. This knowledge graph may be updated in real time and/or based on changes in device telemetry.

While embodiments of the disclosed wellness assistance techniques are described in the context of users of gaming applications, it should be appreciated that these techniques may be applied to general computer users and extended to applications beyond gaming. For example, wellness assistance may be provided to a user of a desktop or mobile computing device (such as a laptop) based on telemetry data acquired using various sensors embedded in or near I/O devices of the computing device (such as in a palm rest adjacent to a touchpad or keyboard of the laptop). Such sensors may include, but are not limited to, a heart rate monitor, one or more motion sensors, a camera, and a microphone. The data captured using such sensors (e.g., heart rate data) may be combined with visual and audio input data analysis techniques (such as key stroke analysis, analysis of pupil dilation, and voice modulation analysis) to gauge the user's excitement or stress levels (e.g., based on key stroke frequency, pupil dilation, and breathing patterns). This sensory data may be further combined with other personal data, such as health data accumulated with the user's consent, to determine whether the user may be experiencing high stress levels during a portion of the application session and to provide relevant feedback accordingly. The feedback may include, for example, wellness tips or suggestions (or links to online wellness resources). Such feedback may be provided in real time during the application session, e.g., via a pop-up window displayed within a graphical user interface of the application, or via email or other messaging interface after the session. The application in this example may be an office or work-related application, such as a spreadsheet editor or meeting presentation program.

According to one embodiment, a method for execution by an information handling system, such as a hub device, includes receiving, by the hub device, telemetry data associated with a user during a gaming session for a gaming application executing on the hub device. The method may also include monitoring, by the hub device, a wellness level of the user during the gaming session, based, at least in part, on the telemetry data. The method may further include determining that the user needs wellness assistance during a portion of the gaming session, based on the monitoring and initiating a wellness assistance session to provide the wellness assistance via at least one interface of a device associated with the user, based on the determination.

In certain embodiments, the telemetry data includes session data and biometric data associated with the user, where the session data is indicative of one or more gaming characteristics of the user during the gaming session, and where the biometric data is indicative of a stress level of the user during the gaming session.

In some embodiments, the session data may indicate that an age of the user is below a minimum threshold age, and the telemetry data may further include relationship data identifying one or more designated contacts related to the user. Furthermore, initiating the wellness assistance session may include transmitting a recommendation for the wellness assistance to one or more corresponding devices of the one or more designated contacts.

In some embodiments, determining that the user needs the wellness assistance includes: determining whether the stress level of the user is above a threshold stress level; and determining that the user needs the wellness assistance when the stress level of the user is above the threshold stress level during the portion of the gaming session. In some implementations, the threshold stress level is based on an average stress level of others users associated with the gaming application.

In some embodiments, the at least one interface of the device associated with the user includes a display interface of a display device coupled to the hub device, and initiating the wellness assistance session includes: displaying a prompt via the display interface of the display device, the prompt requesting confirmation that the user needs the wellness assistance during the gaming session; receiving a response to the prompt from the device of the user; and initiating the wellness assistance session as an in-game assistance session via the display interface when the response includes the requested confirmation.

In some embodiments, the at least one interface of the device associated with the user further includes a haptic interface of a peripheral device coupled to the hub device, and initiating the wellness assistance session further includes triggering haptic feedback via the haptic interface of the peripheral device based on the received response. In some implementations, the peripheral device is a wireless game controller coupled to the hub device.

The method may be embedded in a computer-readable medium as computer program code comprising instructions that cause a processor to perform operations corresponding to the steps of the method. In some embodiments, the processor may be part of an information handling system including a first network adaptor configured to transmit data over a first network connection; and a processor coupled to the first network adaptor, and the memory.

As used herein, the term “coupled” means connected, although not necessarily directly, and not necessarily mechanically; two items that are “coupled” may be unitary with each other. The terms “a” and “an” are defined as one or more unless this disclosure explicitly requires otherwise. The term “substantially” is defined as largely but not necessarily wholly what is specified (and includes what is specified; e.g., substantially parallel includes parallel), as understood by a person of ordinary skill in the art.

DETAILED DESCRIPTION

These example embodiments describe and illustrate various aspects of a configurable and dynamic gaming environment that can be supported through the use of a hub device, which may be an information handling system. A hub device may be located in a user's home and used to arrange game play sessions (or more generically application sessions) between host devices and services. The host devices may execute an application for receiving an AV stream for displaying rendered content from a game play session (or other application session), and in some configurations also receive user input for interacting with the session from a peripheral device, such as a gaming controller. The AV stream presented by the host device may be generated by a service. The service may execute on the hub device or another information handling system, such as a cloud computing resource. A home may include one or several host devices (e.g., televisions, mobile computers, tablet computers, and personal computers) and may include one or several information handling systems executing the service (e.g., a hub devices and personal computers).

The user's home may be divided into different gaming environments defined by a space around a host device. For example, a living room with a television may be one environment and a bedroom with a personal computer may be another environment. A user may use a peripheral device in one of the environments and the hub device may configure a host device, a service, and the peripheral device for operation in the environment by determining the corresponding environment using a knowledge graph. The knowledge graph provides a database of historical information about the environments from which the hub device may use current characteristics of the peripheral device to deduce the location, and thus current environment, of the peripheral device. For example, the knowledge graph may include information about location of rooms (e.g., environments) in the house based on wireless signatures of devices within the different rooms. This difference in signatures reflects that a device on a one side of the house may receive beacon signals from different neighboring access points than a device on an opposite side of the house. When a user carries the peripheral device around the house, the hub device may determine a location of the peripheral device based on visible access points to the peripheral device. Other example characteristics beyond wireless signature for determining location are described in further detail below, and the knowledge graph may be used to combine different characteristics to identify the location, and thus environment, of the peripheral device.

Based on the location of the peripheral device determined from the knowledge graph, the hub device may initialize an application session for the peripheral device by determining an appropriate host device and service for the application session. For example, if the peripheral device is in the living room and is requesting a game that is within the capabilities of the service on the hub device to execute, the hub device may initialize an application session for the peripheral device between the television as a consumption device and the hub device as a service. The service on the hub device executes the game and streams rendered content to an application executing on the television consumption device.

The hub device may be used to migrate the peripheral device to a different environment and/or migrate the application session between host devices and/or services. For example, initially the application session may use a communication link between the peripheral device and the television host device for receiving user input, in which the application executing on the television host device relays user input to the service through a backhaul communication link from the television host device to the hub device. During the application session, the hub device may monitor characteristics of the peripheral device, including signal strength of connection to other components, and determine that the communication link from the peripheral device to the hub device is stronger than the peripheral device to the television host device. The hub device may migrate the peripheral device to a communications link with the hub device such that the service executing on the hub device directly receives the user input but the streaming session continues from the service to the application executing on the television host device. Such a change is illustrated in the change in configuration fromFIG.3Ato the configuration ofFIG.3Bdescribed in further detail below.

Other aspects of the application session may also be migrated. For example, if the peripheral device is determined to move to a different environment, then the hub device may migrate the application session to an application executing on a host device within the new environment. As another example, if a connection between the television host device and the service becomes unstable, the hub device may recommend and/or initiate a migration of the application session to a different host device. One scenario for such a migration may be where the television host device is connected through a wireless link to the service in which the wireless link quality is reducing quality of the streaming and a second host device with a wired connection is available in a nearby environment. Each of these example migrations may be determined based on information in the knowledge graph regarding locations of environments and capabilities within those environments. As yet another example, a user may request execution of an application, such as a particular game, during the application session for which a better configuration exists than the current host device and/or current service. The request for a different application, such as a game requiring a certain GPU capability, may cause the hub device to determine that a second device executing a second service is better for hosting the application and migrate the peripheral device to the second service by, for example, reconfiguring network connections.

The hub device may support connecting to multiple peripheral devices. In one example, the hub device may support two peripheral devices using a shared session on one host device to play the same or different games on the host device. In another example, the hub device may support two peripheral devices in different environments using different sessions with different host devices. The hub device may determine the environment of each of the peripheral devices based on characteristics of the device and the knowledge graph and configure application session for each of the peripheral devices accordingly. Different arrangements of peripherals and players may be supported. For example, one hub device executing a service and one host device executing an application can support a configuration with Game A and one player (P1) with peripheral (C1) and Game B and one player (P2) with peripheral (C2); or can support a configuration with Game A and one player (P1) with peripheral (C1) and Game A and one player (P2) with peripheral (C2); or can support a configuration with Game A and two players (P1, P2) with peripherals (C1, C2).

FIG.1is a block diagram illustrating aspects of a configurable system for providing services to users according some embodiments of the disclosure. A system100includes users102who may have access to a shared library of applications106including applications108A-108N. The users102may have separate libraries, with some overlapping applications between the libraries. The users102may access the library106through devices110A-I, such as mobile gaming device110A, tablet computing device110B, phone computing device110C, television110D, personal computing device110E, desktop computing device110F, laptop computing device110G, game controller110H, VR headset110I. The devices110may access services at any of locations112, including cars, busses, homes, hotels, offices, parks, etc. One or more of the devices110may communicate with an application session executing on a computing device114, such as a home application hub114A, a server114B, or a cloud execution environment114C. In some embodiments, environments may only exist for fixed devices, e.g., desktop computers, televisions, etc.

In some implementations, an application session, such as a gaming session of a gaming application, may execute on a service, either locally on a device, on another system on a network, or in the cloud. A device, such as one of devices110A-I, may access the application session by executing an application that communicates with the service to receive and transmit user input to the service and provide feedback to the user from the service. It is noted that a portion (or entirety) of an application session for a multi-player gaming application accessed by a device associated with a user may be referred to as “a user session.” The device may include its own audio/visual (AV) output for displaying a graphical user interface and/or a rendered display from the application session. Different environments at a location may include different AV systems, and the device may be automatically paired with an AV system and may be reconfigured to support interaction with an application session using the paired AV system.

FIG.2is a block diagram illustrating possible game environments according to some embodiments of the disclosure. A user's home200may include rooms202A-F, and each of the rooms may have different information handling systems present, different AV equipment present, and/or different characteristics. For example, a living room202B may include a large-size television, a bedroom202D may include a personal computer, and a dining room202C may include a table computing device. Gaming environments204A-E in the home200may be defined based on spaces where a user is likely to execute an application session. Each gaming environment204A-E may include numerous devices and gaming environments, devices that may or may not be capable of hosting games, and/or devices that may or may not be capable of receiving game output. A system100may allow multiple users in the home200to simultaneously execute an application session (e.g., a user session associated with each user). In some embodiments, multiple games may be hosted on a single device. In some embodiments, multiple games may target a single output device. In some embodiments, solution manages where games should be hosted, where game output should go, and how to best route peripheral I/O for users.

A user may move between gaming environments204A-E within the home200and continue an application session. For example, a user may take a device, such as a gaming controller, from environment204A to environment204C. The gaming controller may migrate and reconfigure for operation in environment204C from a configuration for environment204A. For example, the controller may transition from an application hosted on a TV in living room202B to an application hosted on TV in dining room202C while remaining connected to a host service executing on a PC in bedroom202D.

Example configurations for applications and services in gaming environments are shown inFIGS.3A-3D.FIG.3Ais a block diagram illustrating application and services hosted in different gaming environments according to some embodiments of the disclosure. InFIG.3A, a first gaming environment304A may include a device, such as a TV or PC, hosting an application302, which is an endpoint for an application session, such as a gaming session. The application302communicates with a service306, which may be hosted on a device in a different gaming environment304B. A controller308may communicate with the application302to receive user input for the application session to control, for example, a character in a game. In some embodiments, the controller308is connected to the environment304A hosting the application and the I/O is configured to be relayed to the environment304B hosting the actual game.

Another arrangement for the application and service is shown inFIG.3B.FIG.3Bis a block diagram illustrating application and services hosted in different gaming environments according to some embodiments of the disclosure. InFIG.3B, the controller308communicates with the service306for providing user input to an application session, with the AV rendering target of the application session being application302in a different gaming environment.

Another arrangement for the application and service is shown inFIG.3C.FIG.3Cis a block diagram illustrating application and service hosted in a common gaming environment according to some embodiments of the disclosure. InFIG.3C, the application302and the service306are executed in the same gaming environment304A, which may be a single device, two devices, or a combination of devices in the gaming environment304A. The controller308may communicate with either the service306and/or the application302.

A further arrangement for the application and service is shown inFIG.3D.FIG.3Dis a block diagram illustrating a cloud-based service arrangement for a gaming environment according to some embodiments of the disclosure. InFIG.3D, the controller308may communicate with a service306hosted in a gaming environment304B that is remote from the gaming environment304A in which the application302is executing. The service306may be executing, for example, on a remote device, even though the user's home includes a gaming environment304C in which the service306may be executed, when the user is engaging with the application302at a location on a different network from their home (e.g., at a friend's house). The service306may also or alternatively be executed, for example, on a cloud computing device available as a subscription service to the user.

In some embodiments, the service306may be used to monitor the wellness level of a user (or “gamer”) in one or more of the gaming environments304A-C ofFIGS.3A-3Ddescribed above and provide wellness assistance based on information relating to the user's wellness characteristics during a gaming session (e.g., a user session) of the application302(e.g., a gaming application). Such information may include telemetry data collected from various sources for the user during the session. The telemetry data may include, for example, a combination of session data and biometric data associated with the user. The session data may include gaming statistics based on user input received from the controller308as well as other gaming attributes or characteristics of the user, e.g., based on a gameplay history of the user with the application302. The biometric data may include different measurements relating to the user's health or wellness, e.g., the user's heart rate, temperature, hand moisture levels, motion, etc., as collected by various sensors that are coupled to or integrated with the controller308. In some embodiments, the telemetry data along with a gaming context associated with the application302may be used by the service306to determine whether the user needs wellness assistance during a portion of the gaming session (e.g., a particular stage of the game with which the user is experiencing some difficulty). The service306may then initiate a wellness assistance session to provide intelligent assistance with the user's personal wellness via at least one interface of a device associated with the user. Additional details regarding such a wellness assistance service (or wellness assistant) will be described with reference to the examples inFIGS.4-9.

FIG.4is a block diagram illustrating a system400for providing assistance with a user's gaming performance during an application session (or gaming session) of a gaming application according to some embodiments of the present disclosure. System400may include or correspond to one or more devices in system100ofFIG.1, as described above.

As shown inFIG.4, system400includes a user device410, a user device420, and a server430. Each of the user devices410and420may be communicatively coupled to the server430via a network402. The network402may include a wired network, a wireless network, or a combination thereof. To illustrate, the network402may include a Bluetooth personal area network (PAN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless network (e.g., a cellular network), a wired network, the Internet, one or more other networks, or a combination thereof, as illustrative, non-limiting examples. In some implementations, the network402may include or correspond to a wired or wireless connection between the server430and each of the user devices410and420.

In some embodiments, system400may be a gaming system in which the server430may include or correspond to an information handling system or hub device, such as a gaming console, a desktop computer, a laptop computer, a tablet, a mobile device, a server (e.g., a cloud server), or other type of computing device for executing games or gaming applications, such as a gaming application432. The gaming application432may be any of various single-player or multi-player games. Examples of such games include, but are not limited to, puzzle games, role playing games, adventure games, first person shooter games, sports games, simulation games, strategy games, platformer games, and other games. In some implementations, server430may be used to host an application session (e.g., a gaming session) for the gaming application432. In some implementations, the server430may operate as a gaming cloud server that provides a cloud-based execution environment for gaming applications, such as the gaming application432, of the gaming system. In some embodiments, the server430may include a wellness assistant434that operates alongside the gaming application432to provide assistance with a user's gaming performance during a gaming session. As will be described in further detail below, the wellness assistant434may be an automated wellness assistance service hosted by the server430that monitors a wellness level of the user during the gaming session and provides wellness assistance or feedback (e.g., wellness advice or recommendations) via the user device410and/or the user device420.

In some embodiments, the user devices410and420may be different types of peripheral devices, such as devices110A-I ofFIG.1, associated with a user of the gaming system. Examples of different types of peripheral devices that may be associated with the user include, but are not limited to, a wired or wireless game controller (e.g., a gamepad or a joystick controller), a mouse, a keyboard, a display, a speaker, a microphone, a headset, a virtual reality (VR) device, an augmented reality (AR) device, a vehicle control simulation device (such as a racing wheel or a vehicle pedal assembly), a musical instrument simulation device (such as a guitar controller), a detachable controller, or a brain-computer interface (BCI) device, a streaming device (e.g., for streaming gaming content from the server430), a handheld gaming device, a smartphone, and a tablet computer. Each type of peripheral device may include one or both of an input device that receives user input (such as user operation of a button or other input device) or an output device that generates an output (such as one or more of an audio output, a visual output, a haptic output, or another output).

Although the user devices410and420are illustrated inFIG.4as being external to the server430, it should be appreciated that, in some implementations, one or more of these devices may be included or integrated within the server430, for example, as part of a handheld gaming system. Also, while not shown inFIG.4, it should be appreciated that certain output (e.g., video or audio output) generated in response to the user input may be provided to or experienced by the user via an output device (e.g., a display device or an audio speaker) that is coupled to or integrated with the server430.

In some embodiments, the user device410may correspond to a first type of peripheral device, such as a wireless game controller (e.g., the game controller110H ofFIG.1), which is communicatively coupled or paired with the server430. In some implementations, the wireless game controller may also be coupled to or paired with one or more other information handling systems (or hub devices) corresponding to other gaming environments associated with the user (e.g., other rooms in the user's house). The user device420may correspond to a second type of peripheral device, such as a mobile device (e.g., the tablet computing device110B or the phone computing device110C ofFIG.1). In some implementations, the user device420(e.g., a mobile device) executes an application422(e.g., a mobile application) configured to communicate with the gaming application432, which enables the user device420to operate as another peripheral device (e.g., a secondary controller or mobile gaming interface) for the user of the gaming system, e.g., for receiving and transmitting user input to the server430and/or generating an output for the user based on information received from the server430.

Each of the user devices410and420may include a processor or other computing circuitry (e.g., a microcontroller, one or more application specific integrated circuits (ASICs), or the like) and various interfaces for input, output, and network communications with other devices (including the server430). Each of the user devices410and420may also include a memory. The memory may include a read only memory (ROM) device, a random access memory (RAM) device, one or more hard disk drives (HDDs), a flash memory device, a solid state drives (SSDs), or any other device or combination of different devices configured to store data in a persistent or non-persistent state. In some implementations, the memory may be used to store one or more instructions, including instructions executable by the processor for a software application (such as the application422of the user device420). Additionally or alternatively, the memory may be used to store one or more thresholds, data, user preferences, one or more device configurations, parameters, other device settings, or any combination of the foregoing.

In some embodiments, each of the user devices410and420may be configured to receive user input (such as from user operation of a button or other input device) and to transmit signals to the server430via the network402based on the user input. Each of the user devices410and420may also be configured to receive information (e.g., instructions or data or both) from the server430via the network402(e.g., in response to the transmitted signals) and to generate an output (such as one or more of an audio output, a visual output, a haptic output, or another output) based on the received information. Accordingly, the user devices410and420may include respective network interfaces419and426for communicating with the server430via the network402. Each of the network interfaces419and426may include one or more communication interfaces, such as a long range (LoRa) interface, a Wi-Fi interface (e.g., an Institute of Electrical and Electronics Engineers (IEEE) 802.11 interface), a cellular interface (e.g., a fourth generation (4G) or long term evolution (LTE) interface, a fifth generation (5G) new radio (NR) interface, or the like), a Bluetooth interface, a Bluetooth low energy (BLE) interface, a Zigbee interface, a non-LoRa interface, another type of network interface, or any combination thereof.

In some embodiments, the user device410may further include one or more input controls412, one or more sensors414, a haptic interface416, and a lighting interface418. In some implementations, the input controls412, the sensors414, the haptic interface416, and the lighting interface418may correspond to different input/output (I/O) interfaces for respective input and output devices of the user device410. Such I/O interfaces may include one or more input interfaces for receiving user input based on the user's physical interaction with the one or more input controls412.

The input controls412may include, for example, one or more depressible buttons, triggers, thumbsticks, scroll wheels, directional pads, touchscreens or touchpads, microphones, and other types of input controls that may be coupled to or integrated within the user device410. The user device410may also include additional input interfaces for receiving sensor input (or measurements) from the one or more sensors414. The one or more sensors414may include various sensors or measurement devices for collecting different types of biometric or wellness data associated with the user. Such sensors may be coupled to or integrated with the user device410(e.g., a game controller), as shown inFIGS.5A and5B.

FIG.5Ais a diagram illustrating a top view500A of a game controller including various sensors for collecting wellness data associated with a user during a gaming session according to some embodiments of the disclosure. As shown in the example ofFIG.5A, the sensors of the game controller may include, but are not limited to, a microphone502, a fingerprint sensor504, one or more motion sensors506, a temperature sensor508, and a moisture sensor510. Each sensor may be strategically placed in a location on the game controller that maximizes some performance or measurement criteria (e.g., based on the type of sensor) while satisfying one or more design constraints. For example, the microphone502may be placed in a central location on the game controller that is away from user input devices (e.g., buttons, thumbsticks, touchpads, etc.), and thus, less likely to be obstructed by the user's hands, to maximize a sound reception performance of the microphone502.

The microphone502may be used to capture sounds (e.g., voice data for the user or other users in the user's vicinity) during the gaming session. The fingerprint sensor504may be used to scan the user's fingerprint, and the corresponding fingerprint data may be transmitted to an information handling system or hub device (e.g., the server430ofFIG.4) for purposes of user identification or account registration. In some implementations, the voice data from the microphone502may also be used (e.g., together with the fingerprint data from the fingerprint sensor504) to identify the user before or during the gaming session. The motion sensor(s)506may be used to detect a motion of the user during the gaming session. In some implementations, the motion sensor(s)506may include an accelerometer or a gyroscope (or a combination of both) to detect and measure the frequency and/or direction of the user's motion during the gaming session. The temperature sensor508and the moisture sensor510may be used to measure the user's hand temperature and moisture levels, respectively, during the gaming session.

In some embodiments, the condition of the user's hand, as indicated by the measurements from the temperature sensor508and the moisture sensor510, may serve as a proxy for the user's overall health or wellness level during the gaming session. Additionally or alternatively, the measurements of the user's hand temperature and moisture levels may be correlated with additional biometric or wellness data collected by one or more other sensors (e.g., motion data collected by the motion sensor(s)506) to determine one or more physiological characteristics of the user, such as a stress level and/or an excitement level of the user at the time the data was collected. For example, measurements indicating relatively low hand temperature and moisture levels for the user with little to no motion detected may correspond to a low stress level and/or a low excitement level. By contrast, a relatively high hand temperature and/or moisture level with frequent motion detected may indicate a high excitement level. It is assumed for purposes of this example that the user's motion, temperature, and moisture are detected and/or measured while the user is holding or operating the game controller.

In addition to the sensors shown inFIG.5A, the game controller may include other sensors or measurement devices for collecting additional biometric or wellness data associated with the user, as shown inFIG.5B. InFIG.5B, a bottom view500B of the game controller ofFIG.5Ais shown with a hear rate sensor512and an antenna514. The heart rate sensor512may be used to measure a heart rate of the user during the gaming session. In some embodiments, the heart rate measurement from the heart rate sensor512may be correlated with the user's hand temperature and moisture data from the respective temperature and moisture sensors508and510along with the motion data collected by the motion sensor(s)506to determine the user's stress level and/or excitement level, as described above. For example, the telemetry from these different sensors may be combined to establish user-specific benchmarks for the biometric data used by the information handling system or hub device (e.g., the server430) to gauge the user's stress level and/or excitement level during the current gaming session. In some embodiments, such benchmarks may be established and/or updated for the current gaming session based on biometric data collected for the user during one or more previous gaming sessions.

In addition to or as an alternative to detecting the user's motion using the motion sensor(s)506, the antenna514may be used to measure or track the user's movement within one or more gaming environments (e.g., one or more rooms of the user's home, as described above). In some implementations, the antenna514may be an ultra-wideband (UWB) antenna used to track a location of the user relative to the known locations of other devices within the gaming environment(s). The user's location may correspond to a location of the game controller relative to one or more reference points (e.g., UWB tags or other network devices) corresponding to the one or more gaming environments. For example, the relative location of the user within a gaming environment may be based on differences in wireless signatures or wireless signal strengths measured or detected by the antenna514indicating a relative distance between the user (or controller) and a corresponding reference point.

Additionally, GPS data from the user's mobile device (e.g., GPS data collected by GPS424of the user device420inFIG.4) may reveal an absolute location of the user and/or infrared data from an infrared presence or proximity sensor may reveal that the user is sitting in front of an AV device (e.g., a device within the gaming environment for displaying a graphical user interface and/or a rendered display from the gaming session). In some embodiments, the information handling system or hub device may combine individual pieces of telemetry from different sensors or devices into a confidence-based knowledge graph of the user's gaming environments and determine a position of the user within that graph. As described above, such a knowledge graph may be based on the absolute and/or relative locations of the user's devices, the location of the user in relation to each device, and or characteristics of the respective devices. The knowledge graph and the user's location within the one or more gaming environments may be tracked and updated in real time and/or based on changes in device telemetry. For example, the knowledge graph may include information about location of rooms (e.g., environments) in the house based on wireless signatures of devices within the different rooms. The difference in signatures reflects that a peripheral device (e.g., game controller) on one side of the house may receive beacon signals from different neighboring access points than a device on an opposite side of the house. When a user carries the peripheral device around the house, the hub device may determine a location of the peripheral device (and the user) based on visible access points to the peripheral device.

Referring back toFIG.4, the measurements collected by the sensors414of the user device410(e.g., one or more of the microphone502, the fingerprint sensor504, the motion sensor(s)506, the temperature sensor508, the moisture sensor510, the heart rate sensor512, and the antenna514ofFIGS.5A and5B, as described above) may be transmitted to the server430as telemetry data415via the network402. In some embodiments, the telemetry data415may also include session data collected by the user device410. The session data may include, for example, input data indicating the particular input controls412used or operated by the user during the gaming session for the gaming application432. In some implementations, the input controls412may be sorted by frequency of usage during gameplay by the user to obtain input usage statistics associated with the user for the gaming application432. The input data generated by the input controls412may include, for example, the user's button selections or other input entered by the user using the input controls412during the gaming session.

Like the user device410, the user device420may include various I/O interfaces for input, output, and network communications. In some embodiments, the user device420may further include the application422and a global positioning system (GPS)424. As described above, GPS data collected by the GPS424may be combined with other telemetry data (e.g., a portion of the telemetry data415collected by one or more of the sensors414of the user device410) to track the location of the user within one or more gaming environments.

As described above, the application422executed by the user device420may enable the user device420to operate as a peripheral device for the user of the gaming system. In some implementations, the application422may serve as a mobile gaming companion for the gaming application432executed by the server430. For example, the application422may be used to extend the functionality of the gaming application432to the user device420. The application422may include a graphical user interface (GUI) that enables the user to conveniently access different features of the gaming application432from the user device420. For example, the user may interact with the GUI of the application422to register or update the user's gaming profile or account for the gaming application432(or a gaming cloud service associated therewith) by entering registration information (e.g., the user's age, gender, and other relevant user info) before or during the gaming session. In addition to such account registration features, the application422may include gameplay features (e.g., for enabling the user device420to serve as a game controller or other peripheral device for the gaming session) and/or assistance notification features (e.g., for receiving gaming assistance, such as relevant gaming advice or tips, via the GUI of the application422or other interface, such as via email or other messaging interface, at the user device420).

In some embodiments, the data input by the user (e.g., registration information entered by the user) via the GUI of the application422and/or the GPS data collected by the GPS424may be transmitted as telemetry data425from the user device420to the server430via the network402. As will be described in further detail below, the telemetry data415from the user device410and the telemetry data425from the user device420may be analyzed by the wellness assistant434at the server430to monitor the user's gaming performance and provide assistance with at least a portion of the gaming application432during a gaming session. As part of the assistance provided by the wellness assistant434, the server430may transmit output data435to the user device410and/or the user device420via the network402. The I/O interfaces of the respective user devices410and420may also include one or more output interfaces to provide one or more outputs for the user via corresponding output devices coupled to or integrated with the user device410and/or the user device420, based on the output data435received from the server430. The output(s) may include, for example, an audio output via one or more speakers, a visual output via a display, a light output via one or more light sources (e.g., a light ring), a haptic output via one or more haptic or vibration devices (e.g., haptic feedback via a haptic touchpad or a rumble motor), or any combination of the foregoing. For example, the output interfaces of the user device410may include the haptic interface416for the haptic output and the lighting interface418for the light output.

The server430may include or correspond to a computing device, e.g., computing device114ofFIG.1. Server430may include one or more processors (not shown), such as a CPU or other computing circuitry (e.g., a microcontroller, one or more ASICs, or the like) and may have one or more processing cores. Server430may further include a memory436and a network interface438for communicating with user device410and user device420over network402. The processor(s), memory436, network interface438and other components of server430may be coupled to each other via a data bus.

The memory436of the server430may include a read only memory (ROM) device, a random access memory (RAM) device, one or more hard disk drives (HDDs), a flash memory device, a solid state drives (SSDs), another devices configured to store data in a persistent or non-persistent state, or a combination of different memory devices. The memory436may be a computer-readable medium used to store one or more instructions, such as instructions or code for the gaming application432and the gaming assistant434. To illustrate, the memory436may store instructions that, when executed by the processor(s) of the server430, cause the processor(s) to perform operations relating to the gaming performance assistance techniques disclosed herein. Additionally, the memory436may be used to store one or more thresholds, data, preferences, other settings, or any combination thereof. In some embodiments, the memory436may also be used to store a machine learning (ML) model452and data for a gaming context454associated with the gaming application432and/or gaming session of the user.

In some implementations, such data may include historical session data collected for the user over multiple gaming sessions and stored in a database (DB)460coupled to the server430. Such historical data may include the user's gameplay history, including historical information about different gaming environments (e.g., different rooms of the user's house) from which the server430(e.g., implemented as a hub device) may use current characteristics of at least one of the user's peripheral devices (e.g., user devices410and/or420) to deduce the device's location, and thus current gaming environment of the user. In some implementations, this historical information may be stored within the DB460in association with a user profile or account registered to the user. The user profile or account may include the user's age, gender, and other relevant data about the user. Additionally, the DB460may be used to store device metadata (e.g., in associated with the user's profile), which identifies the user devices410and420and one or more application environments (such as gaming environments204A-E ofFIG.2, gaming environments304A-C ofFIGS.3A-3D, or a combination thereof) in which each user device was detected. The device metadata may include a unique device identifier, a device type, a device manufacturer, a device model, or any combination thereof. In some embodiments, a similar user profile may be stored for each user associated with the gaming environments (e.g., each family member or gamer who resides in the user's house).

As described above, the wellness assistant434executed by the server430may operate alongside the gaming application432to provide assistance with a user's personal wellness during a gaming session hosted by the server430. The assistance may be based, at least in part, on telemetry data received by the server430via the network402and the network interface438. The network interface438includes one or more communication interfaces. For example, the network interface438may include a LoRa interface, a Wi-Fi interface (e.g., an IEEE 802.11 interface), a cellular interface (e.g., a 4G or LTE interface, a 5G NR interface, or the like), a Bluetooth interface, a BLE interface, a Zigbee interface, a non-LoRa interface, another type of network interface, a combination thereof, or the like.

In some embodiments, the wellness assistant434may include a wellness monitor442, an AI engine444, and an output generator446. In some implementations, the gaming application432and/or the wellness assistant434(including the wellness monitor442, the AI engine444, and the output generator446) may be included in the processor or processing circuitry of the server430. The wellness monitor442may use the AI engine444to monitor the user's personal wellness based on the telemetry data415received from the user device410and/or the telemetry data425received from the user device420via the network402during the gaming session. In some embodiments, the telemetry data415and425may be aggregated and correlated by the wellness monitor442. For example, the wellness monitor442may include a telemetry analyzer (not shown) to analyze and parse the telemetry data415and425into session data and biometric data. The session data and biometric data may then be provided to the AI engine444. The AI engine444may apply this data to train the ML model452for further analysis and processing relating to the wellness monitoring operations associated with the wellness monitor442.

The AI engine444may also use the ML model452to determine and select one or more appropriate options for providing wellness assistance to the user during the gaming session. Based on the selected assistance option, the output generator446may initiate a wellness assistance session. In some embodiments, the assistance options may be based on the gaming context454associated with the gaming application432, as will be described in further detail below with respect to the example data flow ofFIG.6.

In some embodiments, the AI engine444may incorporate user feedback to evaluate the effectiveness of the assistance session and, if necessary, retrain the ML model452to improve the assistance provided during subsequent gaming sessions based on the user's wellness characteristics over time. In some embodiments, the feedback may be based on a comparison of the user's wellness level (e.g., based on a wellness score assigned to the user) before and after the wellness assistance session.

FIG.6is a diagram of an example data flow600illustrating inputs and outputs of a wellness assistant module or service (e.g., the wellness assistant434ofFIG.4) for providing assistance with a user's personal wellness during a gaming session of a gaming application (e.g., gaming application432ofFIG.4). As shown inFIG.6, the inputs of a wellness assistant610may include session data602, biometric data604, gaming cloud data606, and relationship data608. The outputs of the wellness assistant610may include various output options620for initiating a wellness assistance session via one or more interfaces of a device associated with the user. The wellness assistant610may be executed by an information handling system or hub device (e.g., the server430ofFIG.4) alongside the gaming application to monitor a wellness level of the user based, at least in part, on the session data602and the biometric data604received for the user during a gaming session. In some embodiments, the session data602and the biometric data604may be received by the wellness assistant610at the information handling system or hub device from various data sources. Such data sources may include one or more peripheral devices associated with the user (e.g., the user device410and/or user device420ofFIG.4, as described above) and one or more data repositories, such as a local database coupled to the hub device (e.g., the DB460ofFIG.4) or a cloud data repository (e.g., an online data repository associated with a cloud gaming service hosted by another information handling system or a cloud computing resource). In some implementations, such a cloud data repository may be used to store gaming cloud data606.

The session data602may include user input data602a, gameplay history602b, and household data602c. The user input data602amay include input data indicating the particular input controls (e.g., input controls412ofFIG.4) of a peripheral device (e.g., game controller) used or operated by the user during the gaming session for the gaming application. In some embodiments, the user input data602amay also include user profile data (e.g., the user's age, gender, etc.) entered by the user before or during the gaming session, e.g., as part of an account registration process via a mobile application executable at the user's mobile device (e.g., via the application422of the user device420, as described above). In addition to the information received from one or more devices of the user, the session data602may include the user's gameplay history602band household data602c, which may be accessed by the information handling system or gaming assistant610from the local database or the cloud data repository described above. Although the gameplay history602band the household data602care shown separately from the gaming cloud data606, it should be appreciated that, in some implementations, the gameplay history602band the household data602c(or different portions thereof) may be included within the gaming cloud data606accessed or received from the cloud data repository, as described above.

The gameplay history602bmay include, for example, the user's gaming preferences, such as the user's preferred gaming time(s) (e.g., time of day, day of week, and/or other time period) and the duration of each gaming session at the corresponding time(s). Additionally, the gaming characteristics may include information that is indicative of the user's level of experience or skill, such as an amount of time the user has spent gaming, an amount of time the user has spent playing games in a particular genre, an amount of time the user has spent playing a particular game, a level of completion the user has attained in a particular game, levels of completion the user has attained in games of a particular genre, and in-game performance statistics, such as kill/death ratios, speed-run time records and other statistical data regarding the user's gaming performance obtained from the gaming application during each gaming session.

The household data602cmay include information about the size of the user's house, the location of rooms (or gaming environments) within the house, the identity of other users/gamers (e.g., family members) who reside in the house, and any other relevant information about the user's location relative to that of other users within the gaming environments. As described above, such information may be based on a knowledge graph indicating the locations of various devices within each gaming environment based on differences between the wireless signatures and other characteristics of devices and/or access points associated with each environment.

The biometric data604may include measurements relating to the user's health or wellness as collected by the one or more sensors414that are coupled to or integrated with the user device410(e.g., a game controller). The biometric data604may include, for example, hand or skin temperature data604a(e.g., as measured by the temperature sensor508ofFIG.5A), hand or skin moisture data604b(e.g., as measured by the moisture sensor510ofFIG.5A), heart rate data604c(e.g., as measured by the heart rate sensor512ofFIG.5B), and motion data604d(e.g., as measured by the one or more motion sensors506ofFIG.5A).

The relationship data608may include any information that may be used to establish a relationship between the user and other users within one or more gaming environments (e.g., different rooms of the user's house). Such information may be obtained from the user profile data described above and/or the profile data associated with other related users (e.g., other members/gamers of the user's family who also reside in the house). The relationship data608may indicate, for example, if the user is playing alone within the gaming environment or if there are any other known users located nearby. The indication may be based on, for example, the recognition of another user's voice captured by a microphone coupled to or integrated with a peripheral device of the user (e.g., the microphone502of the game controller ofFIG.5, as described above) during the gaming session.

In some embodiments, the gaming assistant610(or an AI engine thereof) may use a ML model612to analyze the session data602and the biometric data604along with information relating to a gaming context614of the gaming application to monitor the user's gaming performance during the gaming session and to determine whether the user needs assistance with at least a portion of the gaming application. The session data602may indicate one or more gaming characteristics of the user, such as a skill level of the user, during the gaming session. The biometric data604may indicate an excitement level or mood of the user during the gaming session, such as whether the user in a mood that is receptive to accepting or receiving gaming assistance. For example, the gaming assistant610may determine that the user needs assistance with at least a portion of the gaming application during the gaming session if, for example, the analysis of the session data602indicates that the skill level of the user is below a threshold skill level for that portion of the gaming application and the analysis of the biometric data604indicates that the user is in a mood that is receptive to gaming assistance.

The gaming context614may indicate, for example, the type of gaming application (e.g., single-player vs. multi-player game), the gaming genre or category of the gaming application, the recommended age rating of the gaming application, and the type of gaming environment (e.g., a local gaming environment vs. an online or cloud-based gaming environment) supported by the gaming application. The gaming context614for the gaming application may be determined based on the gaming cloud data606acquired from an online or cloud data repository associated with a cloud gaming service, as described above. In some implementations, the online data repository may also serve as a resource or reference guide for relevant gaming advice, tips, or assistance information for the gaming application. that may be used to provide the in-game assistance622or the mobile application feedback624. Accordingly, the gaming cloud data606may further include information that may be used to provide gaming assistance as an output of the analysis performed by the ML model612.

In some embodiments, the gaming assistance may correspond to one or more output options620. The output options620may include, for example, in-game assistance622provided via an on-screen display with assistance information superimposed on the gaming content rendered for the gaming application during the gaming session, mobile application feedback624provided via an interface of a mobile application executable at the user's mobile device (e.g., the user device420), and/or controller feedback626provided via a haptic or lighting interface of the user's game controller or other peripheral device (e.g., via the haptic interface416or the lighting interface418of the user device410).

FIG.7is a flow diagram of an example method700for providing assistance with a user's personal wellness during a gaming session of a gaming application according to some embodiments of the disclosure. Method700may be performed by an information handling system or hub device, such as the server430ofFIG.4, as described above. In some implementations, the hub device may be part of a gaming system that includes a display device (or “gaming display”) for rendering gaming content generated by the gaming application and one or more peripheral devices (e.g., the game controller ofFIGS.5A and5B) associated with the user.

Method700begins at block702, which includes acquiring telemetry data associated with a user during a gaming session for a gaming application (e.g., the gaming application432, as described above) executed by the information handling system or hub device. The telemetry data may include session data, biometric data, and relationship data associated with the user. The session data may be indicative of one or more gaming characteristics of the user, such as a skill level of the user with respect to the gaming application (or relevant portion thereof). The biometric data may be indicative of one or more physiological characteristics of the user, such as a stress level of the user, during the gaming session. In some embodiments, the biometric data may include sensor measurements received from a peripheral device (such as a game controller) of the user (e.g., the user device410ofFIG.4) during the gaming session. The peripheral device may include, for example, one or more sensors for collecting the sensor measurements. In some embodiments, the one or more sensors may include one or more of: a moisture sensor to measure hand moisture; a temperature sensor to measure hand temperature; a hear rate sensor to measure heart rate; at least one location sensor (e.g., a GPS and/or a wireless antenna) to measure a location of the user; and at least one motion sensor (e.g., an accelerometer and/or a gyroscope) to detect motion or measure a movement of the user.

At block704, a wellness level of the user is monitored during the gaming session, based, at least in part, on the telemetry data received at block702. Based on this monitoring, a determination is made at block706as to whether the user needs personal wellness assistance during a portion of the gaming session, e.g., corresponding to a difficult stage or other challenging portion of the gaming application. In some embodiments, the determination at block706may include determining whether a stress or excitement level of the user is above a threshold stress level and determining that the user needs wellness assistance when the stress level of the user is above the threshold stress level during the portion of the gaming session. The threshold stress level may be based on, for example, a previous stress level of the user during corresponding portions of one or more previous gaming sessions. Alternatively, the threshold stress level may be based on an average stress level of others users associated with the gaming application.

In some implementations, a wellness score representing the wellness level and/or stress level may be calculated based on the telemetry data acquired for the user during the gaming session. For example, an AI engine (e.g., AI engine444ofFIG.4) or a machine learning (ML) model thereof (e.g., ML model452ofFIG.4or ML model612ofFIG.6) may identify one or more biometric data points (e.g., hand temperature data604a, hand moisture604b, heart rate604c, and motion data604dofFIG.6) relevant to the user's stress level and overall wellness level. The AI engine may then assign weights to each biometric data point based on its importance in assessing stress and/or wellness. The data may be normalized to ensure that different data points are on the same scale for comparison purposes. Normalization may involve, for example, scaling the data to a common range or using standard deviation normalization. The scaled data may then be used to define scoring ranges for one or more physiological characteristics of the user, such as different excitement levels, stress levels and/or wellness levels. For example, the scoring ranges for different stress levels on a scale of 0-10 may include or correspond to a low stress level (0-3), a moderate stress level (4-6), a high stress level (7-9), and very high stress level (10). The scoring ranges or values for different wellness levels may include or correspond to, for example, poor (0-3), fair (4-6), good (7-9), and excellent (10). Similar scoring ranges may be used for different excitement levels, e.g., a low excitement level (0-3), a moderate excitement level (4-6), a high excitement level (7-9), and a very high excitement level (10). Such ranges may then be used to calculate intermediate or sub-scores for each biometric data point by mapping the normalized data point onto the defined scoring ranges. For example, a sub-score of 5 may be assigned for heart rate if the user's measured heart rate during the gaming session falls within the range associated with moderate stress. In some implementations, each sub-score may be assigned a weight based on its relative importance to the user's stress level or overall wellness and the weighted sub-scores may be added together to obtain an aggregated wellness score. The scores may be interpreted or compared against the threshold stress level to determine at block708whether the user is in need of personal wellness assistance. The threshold stress level may be, for example, a predetermined maximum stress level (e.g., corresponding to the moderate stress range described above).

If it is determined at block708that no assistance is needed, the method700returns to block704to continue monitoring the user's wellness (or stress) level. However, if it is determined at block708that assistance is needed (e.g., when the stress level of the user is above the threshold stress level), the method700proceeds to block710. At block710, a wellness assistance session for the user is initiated via an interface of at least one device associated with the user. For example, block710may include initiating an in-game wellness assistance session via an on-screen display interface associated with the gaming application. Additionally or alternatively, the wellness assistance session at block710may be initiated by providing wellness feedback (e.g., wellness advice or suggestions) to the user via a user interface of a mobile application (e.g., application422ofFIG.4) executable at a mobile device of the user (e.g., the user device420ofFIG.4), as described above. Additionally or alternatively, the wellness assistance session at block710may be initiated by triggering haptic feedback via a haptic interface of the user's peripheral device and/or by triggering feedback via a lighting interface of the peripheral device.

In some implementations, the in-game wellness assistance session may be initiated only after the user agrees or confirms the user's willingness to receive wellness assistance. For example, the wellness assistant may first display a prompt via a display interface of a display device (or gaming display) coupled to the information handling system or hub device (e.g., the server430ofFIG.4). In some implementations, the prompt may be displayed in a dialog or pop-up window (e.g., as an overlay displayed with the rendered gaming content on the gaming display), which asks or requests the user to confirm that wellness assistance is needed during the gaming session and includes selectable control buttons (marked “yes” and “no”). In response to receiving user input confirming that the user needs assistance (e.g., based on the user's selection of the button marked “yes”), the in-game wellness assistance may be provided by the wellness assistant, for example, by providing wellness assistance information (e.g., relevant health advice or tips) via an on-screen display interface of the gaming display (e.g., within the same or a different pop-up window as the prompt displayed earlier). The wellness assistant may also trigger haptic feedback via a haptic interface of a peripheral device (e.g., wireless game controller) associated with the user and/or lighting feedback via a lighting interface of the user's game controller (e.g., by triggering different light sources on the controller in a predefined pattern or sequence via the lighting interface418of the user device410shown inFIG.4) based on the user's response (e.g., to acknowledge receiving the requested confirmation from the user). Additionally or alternatively, the wellness assistant may provide wellness feedback via a user interface of a mobile application executable at a mobile device of the user (e.g., via a GUI of the application422or other messaging interface at the user device420ofFIG.4). The wellness feedback provided via the mobile device may include, for example, one or more health tips or warnings for the user (e.g., if an abnormal heart rate or other high stress level is detected).

In some embodiments, the wellness (or stress) level of the user after the wellness assistance session may be assessed and feedback data from the assessment may be used to retrain the ML model used by the AI engine and improve any wellness assistance provided to the user during one or more subsequent gaming sessions.

FIG.8is a table800including different examples of scenarios for providing wellness assistance to a user (e.g., using method700ofFIG.7) based on telemetry data (including session data and biometric data) collected for the user during a gaming session. The gaming session may be for a game or gaming application (e.g., gaming application432ofFIG.4) executed by an information handling system or hub device (e.g., system430ofFIG.4). In some implementations, the hub device may be part of a gaming system that includes a display device (or gaming display) for rendering gaming content generated by the gaming application and one or more peripheral devices (e.g., the game controller ofFIGS.5A and5B) associated with the user. As described above, the telemetry data may include session data indicating one or more gaming characteristics of the user, biometric data indicating one or more physiological characteristics of the user (such as the user's level of stress during the gaming session), and relationship data indicating social connections between the user and other users associated with one or more gaming environments (e.g., one or more rooms of the user's house) in which the gaming session (or similar gaming sessions take place). The telemetry data for the user in each scenario may be analyzed by an AI engine (e.g., the AI engine444ofFIG.4) of a wellness assistant (e.g., wellness assistant434ofFIG.4or wellness assistant610ofFIG.6) executed by the information handling system alongside the gaming application, as described above.

As shown inFIG.8, the first row of table800corresponds to a scenario810, where the session data indicates that the user has been playing one or more games in the same gaming environment (e.g., living room) for an extended period of time (e.g., 5 hours). The AI engine in scenario810may initiate a personal wellness assistance session for the user by suggesting that the user take a break, e.g., by displaying a message within an overlay window via on-screen display (OSD) interface of the gaming display and/or by triggering lighting and/or haptic feedback via respective lighting and haptic interfaces of the user's game controller (e.g., the haptic interface416and the lighting interface418of the user device410shown inFIG.4).

In another scenario820corresponding to the second row of table800, the AI engine may determine that the user has been playing the game(s) in the gaming environment (e.g., living room) for over 15 minutes while standing (e.g., based on height measurements collected by accelerometer and gyroscope sensors of the user's game controller). The AI engine in scenario820may initiate a personal wellness assistance session for the user by displaying a message via the OSD interface of the gaming display with a suggestion for the user to take a seat. The AI engine in this scenario may also have knowledge of the user's location relative to other devices and objects (e.g., the living room sofa) in the user's gaming environment (e.g., based on a confidence-based knowledge graph built using available telemetry from various sources, as described above). Therefore, the wellness assistance provided by the AI engine may also include triggering lighting feedback via the lighting interface of the user's controller, which guides or directs the user to the location of the sofa.

In yet another scenario830, the AI engine may determine from the session data that the user has initiated the gaming session on a Wednesday night (e.g., after working hours). Based on the biometric data (e.g., voice data and heart rate), the AI engine in this scenario may also determine that the user is in a state of tiredness or a very low excitement level (e.g., based on a low heart rate, little-to-no movement or motion detected, and no voice data recorded). In this scenario, the AI engine may suggest a relaxing game for the user to play and also, display health tips for the user when the screen is idle (e.g., between gaming sessions).

FIG.9is a table900including additional examples of scenarios for providing wellness assistance to one or more users based on telemetry data (including biometric data, session data, and relationship data) received for the user during a gaming session according to some embodiments of the disclosure. In a first scenario910, the biometric data may provide a wellness indication for a first user in the form of a heart rate recorded by a heart rate sensor (e.g., heart rate sensor512of the game controller shown inFIG.5B). The session data may include historical gameplay data from previous gaming sessions indicating that the first user plays often. The session data may include the time of day and duration of each session associated with the first user. Furthermore, the session data may be recorded with relationship data indicating the identity, age, and gender of any other user who may also be playing at the same time as the first user as well as the type or genre of the game each user is playing.

The AI engine in scenario910may analyze the first user's heart rate pattern based on gaming cloud data (e.g., gaming cloud data606ofFIG.6). For example, the AI engine may attempt to match the first user's heart rate pattern or pattern of heart rate variability with that of other known users of the same gender and in the same age bracket to identify any patterns that correspond to a particular physiological state. This may allow the AI engine to determine or estimate the physiological state of the first user and provide appropriate wellness assistance to the first user. For example, a wellness assistance session may be initiated via an interface of a mobile application executable at a mobile device (e.g., the user device420ofFIG.4) associated with the first user. The wellness assistance session may be used to indicate a cardio health score and/or provide wellness advice, health tips or warnings, e.g., if an abnormal heart rate is detected (such as an extremely high heart rate during a combat game). The wellness assistance may also include triggering feedback via a lighting and/or haptic interface of the first user's game controller as a reminder for the first user to take a rest or slow down. In some implementations, various rewards (e.g., digital wallpaper, points or other in-game rewards) may be provided if the first user follows the wellness advice (e.g., by taking a break, such as in a scenario920).

In scenario920, the biometric data may indicate that the first user's location corresponds to the living room. The session data and relationship data may indicate that the first user and a second user have been playing for really long hours. The AI engine may also have learned the location of seats within the living room (e.g., based on the knowledge graph described above). The AI engine may analyze the game content to determine the best timing for providing assistance. For example, the AI engine may suggest taking a bio break after detecting that the first and second users have passed a stage of the game. The AI engine in this scenario may display a message via the OSD interface of the gaming display and if necessary, also trigger controller lighting and/or haptic feedback as reminders. The lighting may be used to guide each user to a seat if required (e.g., if the AI engine detects one or both of the users are standing during the gaming session). If either user is a minor (e.g., below a minimum age threshold, as indicated by corresponding user profile information included in the session data), the AI engine may optionally inform the user's parents or other designated contact related to the user (e.g., another adult family member) of any wellness suggestions intended for the user (e.g., based on parental settings associated with the gaming application and/or relationship data associated with the user or user profile that identifies one or more designated contacts).

In yet another scenario930, which is similar to the scenario830in table800ofFIG.8, the AI engine may determine from the biometric data (e.g., voice data and heart rate) that the first user is in a state of tiredness or a very low excitement level (e.g., based on a low heart rate, little-to-no movement or motion detected, and no voice data recorded). The session data may also indicate that the first user has initiated the gaming session late at night after working hours. Based on the first user's gaming preferences, the AI engine may suggest one or more relaxing game options for the user to play (e.g., a puzzle game or other less competitive games with shorter periods of gameplay). The wellness assistance in this case may include one or more links to the suggested game(s) (e.g., within a game library, such as the library106ofFIG.1described above) or to a gaming website for the user to purchase or download the suggested game(s). The AI engine may also display health tips for the user, for example, when the screen is idle for a certain period of time or between gaming sessions.

In addition to the types of wellness assistance in the example scenarios ofFIGS.8and9described above, it should be appreciated that the wellness assistant (or AI engine thereof) may be used to provide other types of wellness assistance, such as recommendations for collaborative-based games rather than competition-based games to promote social connections or family relationships. This type of wellness assistance may be based on analysis of various telemetry data relating to a user's gaming preferences or gameplay characteristics, such as time information (e.g., preferred day and duration of gameplay) and location information (e.g., preferred gaming environment, such as one or more rooms of the user's house). The AI engine may analyze such information together with other telemetry data, such as relationship data, session data, and/or biometric data, collected for each user (e.g., each family member within a household) to provide appropriate wellness suggestions or recommendations to a user detected within a particular gaming environment (e.g., living room or other room of the house), as shown by the various scenarios in table1000ofFIG.10.

If the session data indicates that the user is a minor (or below a minimum threshold age), such as in a scenario1060of the table1000, a recommendation may be sent to one or more other users related to the user, such as the minor's parents or other designated contact identified by the relationship data associated with the user, as described above. The recommendation in this scenario may be sent to the user's (or kid's) parents (e.g., based on the relationship data) and the kid's gaming preferences or habits (e.g., based on other session data). For example, the assistance provided in the scenario1060may be a recommendation for a multi-user racing game sent to the parents of the user/kid (or corresponding devices thereof). The recommendation in this example may be based on session data from previous gaming sessions that indicates the kid has played racing games and therefore, may prefer to play a multi-user racing game with other users (e.g., one or more other family members who also like to play racing games).

FIG.11illustrates an example information handling system1100. Information handling system1100may include a processor1102(e.g., a central processing unit (CPU)), a memory (e.g., a dynamic random-access memory (DRAM))1104, and a chipset1106. In some embodiments, one or more of the processor1102, the memory1104, and the chipset1106may be included on a motherboard (also referred to as a mainboard), which is a printed circuit board (PCB) with embedded conductors organized as transmission lines between the processor1102, the memory1104, the chipset1106, and/or other components of the information handling system. The components may be coupled to the motherboard through packaging connections such as a pin grid array (PGA), ball grid array (BGA), land grid array (LGA), surface-mount technology, and/or through-hole technology. In some embodiments, one or more of the processor1102, the memory1104, the chipset1106, and/or other components may be organized as a System on Chip (SoC).

The processor1102may execute program code by accessing instructions loaded into memory1104from a storage device, executing the instructions to operate on data also loaded into memory1104from a storage device, and generate output data that is stored back into memory1104or sent to another component. The processor1102may include processing cores capable of implementing any of a variety of instruction set architectures (ISAs), such as the x86, POWERPC®, ARM®, SPARC®, or MIPS® ISAs, or any other suitable ISA. In multi-processor systems, each of the processors1102may commonly, but not necessarily, implement the same ISA. In some embodiments, multiple processors may each have different configurations such as when multiple processors are present in a big-little hybrid configuration with some high-performance processing cores and some high-efficiency processing cores. The chipset1106may facilitate the transfer of data between the processor1102, the memory1104, and other components. In some embodiments, chipset1106may include two or more integrated circuits (ICs), such as a northbridge controller coupled to the processor1102, the memory1104, and a southbridge controller, with the southbridge controller coupled to the other components such as USB1110, SATA1120, and PCIe buses1108. The chipset1106may couple to other components through one or more PCIe buses1108.

Some components may be coupled to one bus line of the PCIe buses1108, whereas some components may be coupled to more than one bus line of the PCIe buses1108. One example component is a universal serial bus (USB) controller1110, which interfaces the chipset1106to a USB bus1112. A USB bus1112may couple input/output components such as a keyboard1114and a mouse1116, but also other components such as USB flash drives, or another information handling system. Another example component is a SATA bus controller1120, which couples the chipset1106to a SATA bus1122. The SATA bus1122may facilitate efficient transfer of data between the chipset1106and components coupled to the chipset1106and a storage device1124(e.g., a hard disk drive (HDD) or solid-state disk drive (SDD)) and/or a compact disc read-only memory (CD-ROM)1126. The PCIe bus1108may also couple the chipset1106directly to a storage device1128(e.g., a solid-state disk drive (SDD)). A further example of an example component is a graphics device1130(e.g., a graphics processing unit (GPU)) for generating output to a display device1132, a network interface controller (NIC)1140, and/or a wireless interface1150(e.g., a wireless local area network (WLAN) or wireless wide area network (WWAN) device) such as a Wi-Fi® network interface, a Bluetooth® network interface, a GSM® network interface, a 3G network interface, a 4G LTE® network interface, and/or a 5G NR network interface (including sub-6 GHz and/or mmWave interfaces).

The chipset1106may also be coupled to a serial peripheral interface (SPI) and/or Inter-Integrated Circuit (I2C) bus1160, which couples the chipset1106to system management components. For example, a non-volatile random-access memory (NVRAM)1170for storing firmware1172may be coupled to the bus1160. As another example, a controller, such as a baseboard management controller (BMC)1180, may be coupled to the chipset1106through the bus1160. BMC1180may be referred to as a service processor or embedded controller (EC). Capabilities and functions provided by BMC1180may vary considerably based on the type of information handling system. For example, the term baseboard management system may be used to describe an embedded processor included at a server, while an embedded controller may be found in a consumer-level device. As disclosed herein, BMC1180represents a processing device different from processor1102, which provides various management functions for information handling system1100. For example, an embedded controller may be responsible for power management, cooling management, and the like. An embedded controller included at a data storage system may be referred to as a storage enclosure processor or a chassis processor.

System1100may include additional processors that are configured to provide localized or specific control functions, such as a battery management controller. Bus1160can include one or more busses, including a Serial Peripheral Interface (SPI) bus, an Inter-Integrated Circuit (I2C) bus, a system management bus (SMBUS), a power management bus (PMBUS), or the like. BMC1180may be configured to provide out-of-band access to devices at information handling system1100. Out-of-band access in the context of the bus1160may refer to operations performed prior to execution of firmware1172by processor1102to initialize operation of system1100.

Firmware1172may include instructions executable by processor1102to initialize and test the hardware components of system1100. For example, the instructions may cause the processor1102to execute a power-on self-test (POST). The instructions may further cause the processor1102to load a boot loader or an operating system (OS) from a mass storage device. Firmware1172additionally may provide an abstraction layer for the hardware, such as a consistent way for application programs and operating systems to interact with the keyboard, display, and other input/output devices. When power is first applied to information handling system1100, the system may begin a sequence of initialization procedures, such as a boot procedure or a secure boot procedure. During the initialization sequence, also referred to as a boot sequence, components of system1100may be configured and enabled for operation and device drivers may be installed. Device drivers may provide an interface through which other components of the system1100can communicate with a corresponding device. The firmware1172may include a basic input-output system (BIOS) and/or include a unified extensible firmware interface (UEFI). Firmware1172may also include one or more firmware modules of the information handling system. Additionally, configuration settings for the firmware1172and firmware of the information handling system1100may be stored in the NVRAM1170. NVRAM1170may, for example, be a non-volatile firmware memory of the information handling system1100and may store a firmware memory map namespace1100of the information handling system. NVRAM1170may further store one or more container-specific firmware memory map namespaces for one or more containers concurrently executed by the information handling system.

Information handling system1100may include additional components and additional busses, not shown for clarity. For example, system1100may include multiple processor cores (either within processor1102or separately coupled to the chipset1106or through the PCIe buses1108), audio devices (such as may be coupled to the chipset1106through one of the PCIe busses1108), or the like. While a particular arrangement of bus technologies and interconnections is illustrated for the purpose of example, one of skill will appreciate that the techniques disclosed herein are applicable to other system architectures. System1100may include multiple processors and/or redundant bus controllers. In some embodiments, one or more components may be integrated together in an integrated circuit (IC), which is circuitry built on a common substrate. For example, portions of chipset1106can be integrated within processor1102. Additional components of information handling system1100may include one or more storage devices that may store machine-executable code, one or more communications ports for communicating with external devices, and various input and output (I/O) devices, such as a keyboard, a mouse, and a video display.

In some embodiments, processor1102may include multiple processors, such as multiple processing cores for parallel processing by the information handling system1100. For example, the information handling system1100may include a server comprising multiple processors for parallel processing. In some embodiments, the information handling system1100may support virtual machine (VM) operation, with multiple virtualized instances of one or more operating systems executed in parallel by the information handling system1100. For example, resources, such as processors or processing cores of the information handling system may be assigned to multiple containerized instances of one or more operating systems of the information handling system1100executed in parallel. A container may, for example, be a virtual machine executed by the information handling system1100for execution of an instance of an operating system by the information handling system1100. Thus, for example, multiple users may remotely connect to the information handling system1100, such as in a cloud computing configuration, to utilize resources of the information handling system1100, such as memory, processors, and other hardware, firmware, and software capabilities of the information handling system1100. Parallel execution of multiple containers by the information handling system1100may allow the information handling system1100to execute tasks for multiple users in parallel secure virtual environments.

Machine learning models, as described herein, may include logistic regression techniques, linear discriminant analysis, linear regression analysis, artificial neural networks, machine learning classifier algorithms, or classification/regression trees in some embodiments. In various other embodiments, machine learning systems may employ Naive Bayes predictive modeling analysis of several varieties, learning vector quantization artificial neural network algorithms, or implementation of boosting algorithms such as adaptive boosting (AdaBoost) or stochastic gradient boosting systems for iteratively updating weighting to train a machine learning classifier to determine a relationship between an influencing attribute, such as received device data, and a system, such as an environment or particular user, and/or a degree to which such an influencing attribute affects the outcome of such a system or determination of environment.