Patent Description:
Modern automotive vehicles can include cabin monitoring systems to detect and classify in-vehicle occupants. These systems can include one or more components, including cameras, radar sensors, microphones, speakers, and/or display consoles. Although these systems can display a rear-seat occupant (e.g., a baby in a car seat) on a console display in response to driver input, a driver may still be distracted as they try to determine whether the occupant is distressed or as they soothe a fussy child.

<CIT> discloses a vehicle display device and system for monitoring a rear seat occupant. The system is configured to apply a facial expression recognition algorithm to the image of the face of the rear seat occupant, and to notify the driver of a change of facial expression. For example, when a child is riding in the rear seat of the vehicle, detection of the child crying will result in a notification being displayed to the driver along with live video of the child. <CIT> discloses a system for monitoring a rear seat passenger. The system includes motion and sound intensity detectors for monitoring the rear seat passenger. When motion of the passenger and/or sound generated by the passenger is above a predetermined threshold, the system displays live video of the passenger to the driver so the driver can monitor the passenger.

This document describes one or more aspects of an in-vehicle occupant monitoring and calming system. In one example, the system includes a processor that receives occupant data from occupancy-monitoring sensors (e.g., microphones, cameras, radar sensors, ultrasonic sensors) of a vehicle. Based on the occupant data, the processor can determine whether the occupant is distressed and provide an image or video of the occupant to the driver. The processor also displays driver-selectable options to calm the distressed occupant. The options can include playing an audio or video file for the occupant, adjusting the ambient lighting of the vehicle, or rocking or gently vibrating the occupant's seat. In this way, the described system can monitor vehicle occupants and automatically display a video of the distressed occupant to the driver. In addition, the driver can calm the distressed occupant without removing their attention from driving.

This document also describes methods performed by the above-summarized system and other configurations of the system set forth herein and means for performing these methods.

This Summary introduces simplified concepts related to an in-vehicle occupant monitoring and calming system, which are further described in the Detailed Description and Drawings. This Summary is not intended to identify essential features of the claimed subject matter, nor is it intended for use in determining the scope of the claimed subject matter.

The details of one or more aspects of an in-vehicle occupant monitoring and calming system are described in this document with reference to the following drawings. The same numbers are used throughout the drawings to reference like features and components:.

As described above, some modem vehicles include cabin monitoring systems to detect in-vehicle occupants. These systems can include one or more components, including cameras, radar sensors, microphones, speakers, and/or display consoles. Although these systems can display a rear-seat occupant (e.g., a baby in a car seat), a driver may still be distracted as they try to determine whether the occupant is distressed (e.g., a fussy child) or as they try to soothe the occupant.

The techniques of this disclosure relate to an in-vehicle occupant monitoring and calming system. An example system can include a processor that receives occupant data from occupancy-monitoring sensors of a vehicle. The occupancy-monitoring sensors can include audio sensors that detect sounds made by an occupant, cameras that track the movement of the occupant, and other sensors that monitor the occupant's biometric data. Based on the occupant data, the processor can determine whether the occupant is distressed and provide an image or video of the occupant to the driver. The processor also displays driver-selectable options to calm the distressed occupant. The options can include playing an audio or video file for the occupant, adjusting the ambient lighting of the vehicle, or rocking or gently vibrating the occupant's seat. In this way, the described system can monitor vehicle occupants and automatically display an image or video of the distressed occupant to the driver without requiring the driver to request the image or video. In addition, the driver can soothe the distressed occupant without removing their attention from the operation of the vehicle.

This is just one example of the described in-vehicle occupant monitoring and calming system. This document describes other examples and implementations.

<FIG> illustrates an example environment <NUM> in which an in-vehicle occupant monitoring and calming system can be implemented. In the depicted environment <NUM>, a vehicle <NUM> is driving along a roadway with a driver <NUM> and an occupant <NUM> (e.g., an infant) in a rear seat of the vehicle <NUM>. The occupant <NUM> is illustrated in <FIG> as an infant. In other implementations, the occupant <NUM> can be a child, adult, pet, or another object. Although illustrated as an automobile, the vehicle <NUM> can represent other types of motorized vehicles (e.g., a passenger truck, a car, a motorcycle, a bus, a tractor, a semi-trailer truck), watercraft (e.g., a boat), or aircraft (e.g., an airplane).

The vehicle <NUM> includes one or more interior sensors <NUM>, an occupant calming system <NUM>, and a display <NUM>. The interior sensors <NUM> are mounted to, or integrated within, an interior portion of the vehicle <NUM> to detect aspects of the occupant <NUM> and other passengers. The interior sensors <NUM> can include one or more cameras, microphones, radar sensors, ultrasonic sensors, and/or infrared cameras that monitor the occupant <NUM>. In particular, the interior sensors <NUM> can be positioned to have a field of view that includes one or more occupants <NUM>.

The occupant calming system <NUM> can include an occupant monitoring system <NUM>, a display controller <NUM>, and a feedback controller <NUM>. Using data from one or more interior sensors <NUM>, the occupant monitoring system <NUM> can monitor the occupant <NUM>. In particular, the occupant monitoring system <NUM> can determine whether the occupant <NUM> is distressed or may soon become distressed. Although the term "distressed" is used in this description, it should be understood that the disclosed techniques and apparatuses can monitor for and address any forms of discomfort, stress, agitation, fussiness, sadness, or pain.

As described in greater detail below, the occupant monitoring system <NUM> can use camera data to track key body points of the occupant <NUM> and determine whether the occupant <NUM> is distressed or will soon become distressed (e.g., be awakened). The occupant monitoring system <NUM> can, for example, provide the camera data to a machine-learned model (e.g., a deep neural network) to recognize actions of the occupant <NUM> that indicate distress or potential distress soon. For example, the occupant monitoring system <NUM> can monitor camera data for certain facial expressions or strong or sudden arm or leg movements.

As another example, the occupant monitoring system <NUM> can use audio data from one or more microphones to monitor the occupant <NUM>. A machine-learned model can be trained to recognize typical sounds associated with a baby or young child that is distressed or will soon be distressed. The occupant monitoring system <NUM> can also use the audio data to determine from which seat the distress sounds are coming. The occupant monitoring system <NUM> can also use thermographic camera data to monitor the body temperature of the occupant <NUM>.

The display controller <NUM> can control data and user interface options provided to the driver <NUM> via the display <NUM>. For example, the display controller <NUM> can process camera data to provide a cropped and/or enhanced image or video of the occupant <NUM> on the display <NUM>. The image or video can be automatically provided to the driver in response to the occupant monitoring system <NUM> determining that the occupant <NUM> is distressed. The display controller <NUM> can also determine one or more user-selectable options to provide to the driver <NUM> via the display <NUM>. For example, the display controller <NUM> can present one or more touchscreen buttons to the driver <NUM> to soothe or calm the occupant <NUM>.

The feedback controller <NUM> can control feedback operations to calm or soothe the occupant <NUM>. For example, the feedback controller <NUM> can adjust the interior lighting of the vehicle <NUM> by dimming or slightly illuminating the vehicle's interior. The feedback controller <NUM> can also adjust the color or hue of the interior lights to provide a more calming effect. Similarly, the feedback controller <NUM> can play music (e.g., white noise, nursery songs) to calm the occupant <NUM>. The music can be loaded from the driver's smartphone, an infotainment system of the vehicle <NUM>, or a remote computer system. The feedback controller <NUM> can also control vibration motors and/or other motors to provide a soothing motion or rocking sensation to the seat occupied by the occupant <NUM>.

The display <NUM> provides the driver <NUM> with information regarding the occupant <NUM> and feedback control options. For example, the display <NUM> can be integrated into a center console or dashboard of the vehicle <NUM>. As described above, the display <NUM> can provide an image or video of the occupant <NUM> when the occupant monitoring system <NUM> determines that the occupant <NUM> is distressed or may soon become distressed. The display <NUM> also provides options for the driver <NUM> to soothe the occupant <NUM> without having to remove their attention from the roadway. In this way, the driver <NUM> can be alerted to the occupant's distress and calm the occupant <NUM> without removing their attention from the operation of the vehicle <NUM>.

<FIG> illustrates an example configuration <NUM> of the vehicle <NUM> with the in-vehicle occupant monitoring and calming system. As described for <FIG>, the vehicle includes the interior sensors <NUM>, the occupant calming system <NUM>, and the display <NUM>. In addition, the vehicle <NUM> can include one or more processors <NUM>, computer-readable storage media (CRM) <NUM>, interior lights <NUM>, one or more speakers <NUM>, a rear entertainment system <NUM>, and one or more seat motors <NUM>.

The interior sensors <NUM> can include one or more cameras <NUM>, microphones <NUM>, infrared (IR) cameras <NUM>, and radar sensors <NUM>. The cameras <NUM> can capture an image or video of the vehicle cabin. For example, the cameras <NUM> can provide color images of the vehicle cabin to provide on the display <NUM>. The IR cameras <NUM> can detect whether the occupant <NUM> is a living being. The IR cameras <NUM> can use an infrared light source (e.g., a vertical-cavity surface-emitting laser (VCSEL) or IR light-emitting diode (LED)) to provide lighting for interior sensing during low ambient light conditions. The IR cameras <NUM> can provide monochrome IR images that the occupant monitoring system <NUM> can process. The vehicle <NUM> can position the cameras <NUM> and the IR cameras <NUM> to have fields of view for regions of interest that correspond to the seating positions within the vehicle cabin. The vehicle <NUM> can include the cameras <NUM> (e.g., red-green-blue (RGB) cameras) and the IR cameras <NUM> capturing the same seating position(s). In other implementations, an RGB camera and IR camera can be integrated into a single camera device. The cameras <NUM> can also include three-dimensional time-of-flight cameras that measure the time for light pulses to leave the camera and reflect back on the camera's imaging array.

The microphones <NUM> can detect audio content in the vehicle <NUM>, including sounds associated with an infant waking up or becoming agitated. The vehicle <NUM> can also include thermographic cameras or thermal cameras that detect a temperature of the occupant <NUM> based on a thermal image. The radar sensor <NUM> or another sensor (e.g., an ultrasonic sensor, a time-of-flight camera) can detect the heart rate of the occupant <NUM> by monitoring radio waves, including monitoring for an irregular heart rate of the occupant <NUM>.

The processor <NUM> (e.g., an electronic control unit or control circuit) can be a microprocessor or a system-on-chip configured to perform the techniques described in this disclosure. The processor <NUM> can include one or more application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or one or more general-purpose hardware processors programmed to perform the techniques described herein. The processor <NUM> can execute computer-executable instructions stored in the CRM <NUM>. The CRM <NUM> can be a memory or storage media, including non-volatile memory, such as electrically erasable programmable read-only memory (EEPROM) for storing one or more computer-executable instructions, routines, thresholds, and captured data. The CRM <NUM> may include other examples of non-volatile memory, such as flash memory, read-only memory (ROM), programmable read-only memory (PROM), and erasable PROM (EPROM). The processor <NUM> or the CRM <NUM> may also include volatile memory (e.g., dynamic random-access memory (DRAM), static random-access memory (SRAM)).

The processor <NUM> processes sensor data from the interior sensors <NUM> and determine whether the occupant <NUM> is distressed or about to become distressed. In response to driver input, the processors <NUM> control the operation of the interior lights <NUM>, the speakers <NUM>, the rear entertainment system <NUM>, and/or the seat motors <NUM> to calm the occupant <NUM>.

The occupant calming system <NUM> can be stored in the CRM <NUM>. As described for <FIG>, the occupant calming system <NUM> can include the occupant monitoring system <NUM>, the display controller <NUM>, and the feedback controller <NUM>. The occupant monitoring system <NUM> can use data from the interior sensors <NUM> to determine that the occupant <NUM> is distressed or about to become distressed. For example, the occupant monitoring system <NUM> can use video data from the cameras <NUM> or the IR cameras <NUM> to recognize facial expressions associated with agitation or potential agitation of the occupant <NUM>. Similarly, the position of key body points (e.g., arms, shoulders, hips, legs, feet) can be tracked to recognize distress or potential distress of the occupant <NUM>. In some implementations, the occupant monitoring system <NUM> can use a machine-learned model (e.g., a deep neural network) to process data from the cameras <NUM> or the IR cameras <NUM> and recognize the distress or potential distress of the occupant <NUM>. The occupant monitoring system <NUM> can, for example, use a configurable threshold (e.g., seventy-five percent) that the occupant <NUM> will become distressed in the next few minutes before making a determination that the occupant <NUM> is distressed.

The occupant monitoring system <NUM> can similarly use data from the microphones <NUM>, the IR cameras <NUM>, and the radar sensors <NUM> to determine occupant distress. The audio data from the microphones <NUM> can be used to monitor for sounds associated with a fussy child or a child about to become fussy. The thermographic camera data can indicate raising body temperatures or irregular body temperatures. Likewise, the occupant monitoring system <NUM> can identify irregular or raised heart rates or breathing that indicate occupant distress. Like the camera data, the occupant monitoring system <NUM> can use a machine-learned model to analyze the data from the microphones <NUM>, the IR cameras <NUM>, the radar sensors <NUM>, and/or other sensors to detect occupant distress.

In response to the occupant monitoring system <NUM> determining that the occupant <NUM> is distressed or about to become distressed, the display controller <NUM> controls the display <NUM> to provide an image or video of the occupant <NUM> to the driver <NUM> The display controller <NUM> can also control the interior lights <NUM> to improve the visibility of the occupant <NUM> in the captured image or video. For example, the display controller <NUM> can slowly or partially illuminate the interior lights above or near the occupant <NUM> to improve the clarity of the captured image or video. The interior lights <NUM> can include cabin lighting in the ceiling, door panels, or other areas of the vehicle <NUM>. The display controller <NUM> also provides selectable options for the driver <NUM> to calm the occupant <NUM>. The selectable options can include changing the interior lighting, playing music, playing a video, or rocking the occupant's seat. The driver <NUM> can select a particular option via a touchscreen of the display <NUM>, audio input, a button near the display <NUM>, or similar input means.

In response to the driver <NUM> selecting one of the selectable options, the feedback controller <NUM> performs certain functions to try to calm the occupant <NUM>. For example, the feedback controller <NUM> can control the interior lights <NUM> to dim or brighten the ambient lighting. The feedback controller <NUM> can also control a central entertainment system or the rear entertainment system <NUM> to play music or multimedia files. The rear entertainment system <NUM> can include the speakers <NUM> and/or a display dedicated to the occupant's seat or the rear portion of the vehicle <NUM>. The music can include white noise, ocean noises, nursery songs, or any other audio file. The multimedia files can include a movie, content from a streaming service, a live video of the driver <NUM>, or any other video data.

The feedback controller <NUM> can also control the seat motors <NUM> (e.g., actuators) in the occupant's seat to try to calm the occupant <NUM>. The seat motors <NUM> can be integrated into the bottom and/or backrest of the seat. The feedback controller <NUM> can activate the seat motors <NUM> to mimic a rocking movement, vibrations, or other soothing motions to calm the occupant <NUM>.

<FIG> illustrates an example conceptual diagram <NUM> for performing in-vehicle occupant monitoring and calming. The conceptual diagram <NUM> illustrates example inputs, outputs, and operations of the occupant calming system <NUM>, but the occupant calming system <NUM> is not necessarily limited to the order or combinations in which the inputs, outputs, and operations are shown herein. Further, any of one or more of the operations (or acts) may be repeated, combined, or reorganized to provide other functionality. For example, some or all the operations may be repeated and/or combined for each occupant of a vehicle.

At <NUM>, the occupant monitoring system <NUM> obtains audio data <NUM> from the microphones <NUM> in the vehicle <NUM>. The occupant monitoring system <NUM> or a machine-learned model (e.g., a deep neural network) thereof can use the audio data <NUM> to detect sounds associated with the occupant <NUM> becoming or being distressed. For example, the occupant monitoring system <NUM> can determine that a child is awakening from a change in their breathing, soft or intermittent crying sounds, or similar sounds. The distress sounds can also include soft speaking or statements directed to the driver (e.g., "mom") from a toddler or young child. The occupant monitoring system <NUM> can also utilize the audio data <NUM> to determine from which seat the distress sounds are coming. In response to detecting distressed sounds, the occupant monitoring system <NUM> can forward the audio data <NUM> or the distress sounds to another machine-learned model or component thereof.

At <NUM>, the occupant monitoring system <NUM> obtains camera data <NUM> (e.g., images or video) from the cameras <NUM> in the vehicle <NUM>. The occupant monitoring system <NUM> or a machine-learned model thereof can use the camera data <NUM> to track key body points of the occupant <NUM>. The key body points can include arms, shoulders, hips, head, eyes, or other body locations. For example, the occupant monitoring system <NUM> can analyze the camera data <NUM> to determine if the occupant <NUM> has extensive or strong body movements that indicate distress.

At <NUM>, the occupant monitoring system <NUM> obtains the audio data <NUM> or the distress sounds identified in operation <NUM>, along with the tracking of key body points and biometric data <NUM>. The biometric data <NUM> can include the body temperature of the occupant <NUM> from a thermographic camera or heart rate data from the radar sensor <NUM> or a time-of-flight camera.

The occupant monitoring system <NUM> or a machine-learned model can fuse the input data (or a subset thereof) and analyze it to detect whether the occupant <NUM> is distressed or becoming distressed. For example, the machine-learned model can be trained to associate certain movements of one or more key body points with the occupant <NUM> becoming or being distressed. Certain facial expressions or rapid arm or leg movements can be associated with distress or rising distress. Similarly, the biometric data <NUM> can be used to identify or complement a determination that the occupant <NUM> is distressed. For example, changes in heart rate or body temperature can be monitored to improve the occupant monitoring. To reduce false detections, the occupant monitoring system <NUM> or the machine-learned model can determine a probability that the occupant <NUM> is distressed or about to become distressed and compare the probability to a configurable threshold (e.g., seventy-five percent) before determining that the occupant <NUM> is distressed. The occupant monitoring system <NUM> or the machine-learned model can also use the input data to locate the occupant <NUM> in the vehicle <NUM>.

The input data to operation <NUM>, including the audio data <NUM>, the biometric data <NUM>, and the camera data <NUM>, can be periodically updated by the occupant monitoring system <NUM> to ensure accurate and timely determinations of occupant distress. For example, the occupant monitoring system <NUM> can update the input data at fifteen-second intervals to account for temporary changes in occupant distress.

At <NUM>, the occupant monitoring system <NUM> or the display controller <NUM> captures video with a region of interest (ROI) associated with the seat in which the occupant <NUM> is located. In particular, the display controller <NUM> can crop the camera data <NUM> to focus on the distressed occupant. For example, the audio data <NUM> can be used to determine the region of interest and which occupant has become distressed to assist with the cropping. In this way, the irrelevant background is removed from an ROI video <NUM> displayed to the driver <NUM>.

At <NUM>, the display controller <NUM> can also adjust the interior lights <NUM> in the vehicle <NUM> to improve the clarity of the video by improving the image exposure. The display controller <NUM> can also perform other image processing (e.g., enhancement, contrast adjustment) to improve the ROI video <NUM> of the occupant <NUM>.

The display controller <NUM> can then output the ROI video <NUM> to the display <NUM>. For example, the ROI video <NUM> can be shown on a center console display. The display controller <NUM> also causes one or more calming options <NUM> to be displayed with the ROI video <NUM> on the display <NUM>. The calming options <NUM> are driver-selectable user input (UI) options that the feedback controller <NUM> can cause to be performed to try to calm the occupant <NUM>. For example, the calming options <NUM> can include playing an audio or multimedia file for the occupant <NUM> using the speakers <NUM> and/or the rear entertainment system <NUM>. The audio or multimedia file can be obtained from a local memory in the vehicle, a remote computer system (e.g., an online music service), or the driver's smartphone or another electronic device (e.g., a music app installed into an entertainment system of the vehicle <NUM>). The audio file can be nursery music, white noise, or a favorite song of the occupant. The multimedia file can be a movie, television show episode, or other multimedia content.

The calming options <NUM> can also include displaying a live video of the driver's face to the rear entertainment system <NUM> so the driver can talk, sing, or otherwise calm the occupant <NUM>. The calming options <NUM> can also include adjusting the ambient lighting by adjusting the interior lights <NUM> or raising or lowering automatic sunshades for the rear windows of the vehicle <NUM>. Further, the calming options <NUM> can include applying a soothing motion pattern to the seat occupied by the occupant <NUM>. For example, the feedback controller <NUM> can activate actuators or motors in or attached to the seat to induce a rocking or vibration sensation to the occupant <NUM>. Similarly, the feedback controller <NUM> can activate motors to move the seat forward and backward to gently rock the occupant <NUM>.

<FIG> illustrates an example method <NUM> of in-vehicle monitoring and calming. Method <NUM> is shown as sets of operations (or acts) performed, but not necessarily limited to the order or combinations in which the operations are shown herein. Further, any of one or more of the operations may be repeated, combined, or reorganized to provide other methods. In portions of the following discussion, reference may be made to the environment <NUM> of <FIG>, and entities detailed in <FIG>, reference to which is made for example only. The techniques are not limited to performance by one entity or multiple entities.

At <NUM>, occupant data from one or more occupancy-monitoring sensors of a vehicle is obtained. For example, the occupant monitoring system <NUM> can obtain the audio data <NUM> from the microphones <NUM>, the camera data <NUM> from the cameras <NUM> or the IR cameras <NUM>, or the biometric data <NUM>. The vehicle <NUM> can include additional interior sensors <NUM> to obtain occupant data for the rear passengers.

The occupant data can include the audio data <NUM> from the rear seat(s) of the vehicle or the camera data <NUM> of the rear seat(s) of the vehicle. The biometric data <NUM> can be obtained from the IR cameras <NUM>, radar sensor <NUM>, a thermographic camera, an ultrasonic sensor, or a time-of-flight camera. The biometric data <NUM> can include a body temperature, a breathing pattern, or the heart rate of the occupant <NUM>.

At <NUM>, it is determined whether an occupant of the vehicle is distressed based on the occupant data. The occupant is seated in a rear seat of the vehicle and can be an infant, toddler, or young child. For example, the occupant monitoring system <NUM> can determine whether the occupant <NUM> is distressed based on the occupant data. The occupant monitoring system <NUM> can provide the occupant data as an input to a machine-learned model. The machine-learned model can then determine whether changes in the occupant data are associated with distress. For example, the distress determination can be based on the audio data <NUM> and whether sounds or changes in breathing from the occupant are associated with distress. It can also include tracking, using the camera data <NUM>, positions of one or more key body points of the occupant <NUM>, and determining changes in the positions of the key body points are above a movement threshold. The key body points can include the head, shoulders, eyes, arms, shoulders, legs, or hips of the occupant <NUM>. Similarly, the distress determination can be based on whether changes in the biometric data <NUM> associated with the occupant <NUM> are associated with distress.

The machine-learned model can also track changes in the occupant data and determine a probability that the occupant is distressed based on those changes or a current data cycle of occupant data. The distress probability can then be compared to a distress threshold, which can be a configurable confidence value. The machine-learned model can determine that the occupant <NUM> is distressed based on a determination that the distress probability is greater than the distress threshold.

At <NUM>, in response to determining that the occupant is distressed, an image or video of the occupant is displayed on a display located in a field of view of a driver of the vehicle. For example, the display controller <NUM> can display the ROI video <NUM> on the display <NUM> in response to a determination by the occupant monitoring system <NUM> that the occupant <NUM> is distressed. The display controller <NUM> can apply a processing action to the ROI video <NUM> before displaying it to improve its clarity. For example, the processing action can include cropping the ROI video <NUM> to focus on the occupant <NUM>, adjusting the brightness or contrast of the ROI video <NUM>, and/or adjusting the ambient lighting of the vehicle <NUM>.

At <NUM>, one or more selectable options for the driver to select are displayed on the display to calm or soothe the occupant. The display controller <NUM> provides selectable calming options <NUM> on the display <NUM> for calming the occupant <NUM>. The calming options <NUM> can include playing an audio or multimedia file on the rear entertainment system <NUM> of the vehicle. Similarly, the calming options <NUM> can also include displaying an image or video of the driver <NUM> to the occupant via the rear entertainment system <NUM>. The driver <NUM> can also be given the option to adjust the ambient lighting of the vehicle <NUM> by dimming or brightening the interior lights of the vehicle or raising or lowering sunshades of the vehicle <NUM>. The calming options <NUM> can further include activating actuators or motors in or attached to the rear seat to introduce a vibration or motion pattern to the occupant <NUM>. Similarly, the feedback controller <NUM> can activate motors in or attached to the rear seat to move the rear seat forward and backward to introduce a rocking motion.

<FIG> illustrates an example of a display of an in-vehicle monitoring and calming system in accordance with techniques of this disclosure. In illustrated environment <NUM>, the driver <NUM> is driving the vehicle <NUM> along a roadway and the occupant <NUM> is in a rear seat of the vehicle <NUM>. The occupant <NUM> in the illustrated environment <NUM> is an infant in a car seat. The vehicle <NUM> includes the display <NUM> as part of a center console in or near the dashboard. In other implementations, the display <NUM> can be in another portion of the field of view of the driver <NUM>.

As described with respect to <FIG> and <FIG>, the occupant calming system <NUM> can monitor the occupant <NUM> and determine whether the occupant <NUM> is or may soon become distressed. For example, the occupant monitoring system <NUM> can determine that the occupant <NUM> may soon awaken based on a change in the infant's breathing, facial expressions, or rapid body movements. Similarly, the occupant monitoring system <NUM> can monitor sounds or biometric data associated with the infant indicating that they are becoming distressed. In response to determining that the occupant <NUM> is distressed, the display controller <NUM> can display the ROI video <NUM> on the display <NUM>. The ROI video <NUM> provides a real-time video feed of the occupant <NUM> to the driver <NUM>. As described above, the display controller <NUM> can crop the camera data <NUM> to focus the ROI video <NUM> on the distressed occupant <NUM>. Similarly, the display controller <NUM> can enhance the brightness or contrast of the camera data <NUM> to improve the clarity of the ROI video <NUM> for the driver <NUM>. In this way, the occupant calming system <NUM> can alert the driver <NUM> of a distressed occupant (e.g., potentially before the driver <NUM> becomes aware) and display a real-time video of the occupant <NUM> on the display <NUM> without requiring the driver <NUM> to request the video and reducing the driver's distraction.

The display controller <NUM> also displays one or more calming options <NUM> on the display <NUM>. In the illustrated environment <NUM>, four calming options <NUM> are presented to the driver <NUM>. In other implementations, fewer or additional calming options <NUM> can be presented. The types of calming options <NUM> are described with respect to <FIG> and <FIG>. In addition, the calming options <NUM> can be selected based on the driver's preferences, the current environment within the vehicle <NUM> (e.g., music is currently playing, night-time driving), or most recent selections by the driver <NUM>.

In response to the driver <NUM> selecting one or more of the calming options <NUM>, the feedback controller <NUM> can perform the calming action associated with the selected calming option <NUM>. For example, the feedback controller <NUM> can cause a real-time video of the driver <NUM> to be displayed on the rear entertainment system <NUM> of the vehicle <NUM> so the occupant <NUM> can see the driver's face. As another example, the feedback controller <NUM> can cause a particular multimedia file to be played on the speakers <NUM> or the rear entertainment system <NUM>. In this way, the occupant calming system <NUM> can assist the driver <NUM> in monitoring and calming the distress of the occupant <NUM> with minimal distraction from operation of the vehicle <NUM>.

Claim 1:
A method comprising:
obtaining (<NUM>) occupant data from one or more occupancy-monitoring sensors (<NUM>) of a vehicle (<NUM>);
determining (<NUM>), based on the occupant data, whether an occupant (<NUM>) of the vehicle (<NUM>) is distressed, the occupant being seated in a rear seat of the vehicle (<NUM>);
in response to determining that the occupant is distressed, displaying (<NUM>), on a display (<NUM>) located in a field of view of a driver (<NUM>) of the vehicle (<NUM>), an image or video (<NUM>) of the occupant (<NUM>); and
displaying (<NUM>), on the display (<NUM>), one or more selectable options (<NUM>) for the driver to select to calm or soothe the occupant (<NUM>), wherein the one or more selectable options comprise:
playing an audio file in the vehicle (<NUM>);
playing a multimedia file on a rear entertainment system of the vehicle (<NUM>);
displaying an image or video of the driver (<NUM>) to the occupant (<NUM>) via the rear entertainment system;
adjusting ambient lighting of the vehicle (<NUM>) by dimming or brightening interior lights of the vehicle (<NUM>);
raising or lowering sunshades of the vehicle (<NUM>);
activating actuators or motors in or attached to the rear seat to introduce a vibration or motion pattern; or
activating additional motors in or attached to the rear seat to move the rear seat forward and backward to introduce a rocking motion.