Driving condition detection and indication to a remote device during a communication session

An electronic system, a method, and a computer program product support wireless communication with reduced distraction for a driver experiencing challenging driving conditions. The method includes connecting to a communication session with second electronic device(s) via a communication subsystem of the electronic device and via a communication network. The method includes receiving an indication of a challenging driving condition from a first image stream from image capturing device(s) positionable to have a field of view encompassing a face of a driver of a vehicle and/or a detected condition from sensor(s) that detects external conditions to the vehicle that can require the driver to be more attentive while the vehicle is in motion. In response to receiving the indication of the challenging driving condition, the method includes communicating a notification over the communication session to the second electronic device(s), the notification indicating that the driver is in the challenging driving condition.

RELATED APPLICATIONS

The present application is related to the following co-pending U.S. Patent Applications: U.S. Ser. No. 17/816,307 entitled “Reducing Distractions Caused By A Communication Session With A Remote Device That Is Used During Driving” and U.S. Ser. No. 17/816,325 entitled “Reducing Distractions Caused By Setting Up A Communication Session With A Remote Device That Is Being Used During Driving”, both filed concurrently herewith, with relevant content of cach related application being incorporated herein by reference.

BACKGROUND

1. Technical Field

The present disclosure relates generally to a mobile communication system and more particularly to video communication systems used in a vehicle.

2. Description of the Related Art

Electronic devices such as mobile phones, network servers, desktop workstations, laptops, and tablets are often used to participate in or to enable online human communication, such as a video communication session. So as to simulate an in-person conversation, meeting, or lecture, each participant in a video communication session generally positions themself within the field of view of the local camera and within detection range of the microphone. The respective image stream and audio stream captured by a first electronic device are communicated to the video communication session for presentation by one or more second electronic devices. Recent trends are for increased use of video communications in a wide range of settings including homes and public spaces. Additionally, some participants use their electronics devices to participate in a video communication session from within a vehicle and while driving.

When a driver is driving a vehicle with a passenger, in-person conversations are typically conducted in a holistic manner by understanding and reacting to the environment the vehicle is in. For example, if the vehicle in front slams on the brakes, the passenger would not continue speaking to avoid disturbing the driver who needs to be attentive to driving the vehicle. Similarly, the passenger would be aware when a challenging driving condition exists due to road conditions, impaired visibility, and proximity to pedestrians. By contrast, when the driver is participating in audio or video communication session via an electronic device, the other participants in the communication session would have no knowledge of the challenging driving conditions and may unknowingly distract the driver.

DETAILED DESCRIPTION

According to a first aspect of the present disclosure, an electronic system, a method, and a computer program product enable wireless communication sessions with reduced distraction for a driver experiencing challenging driving conditions. The electronic system includes at least one image capturing device positionable to have a field of view encompassing at least a face and/or posture of a driver of a vehicle. The electronic system includes or is communicatively coupled to at least one sensor that detects external conditions to the vehicle that can require the driver to be more attentive while the vehicle is in motion. The electronic system includes a communication subsystem connectable to a communication network. The electronic system includes at least one output device that provides a communication output received from a network-connected communication session with at least one second electronic device. The electronic system includes a memory that stores a communication application comprising a driver distraction notification and communication reduction (DNCR) module. A controller of the electronic system is communicatively connected to the at least one image capturing device, the at least one sensor, the at least one output device, the communication subsystem, and the memory. The controller executes the communication application to connect, via the communication subsystem and the communication network, with the at least one second electronic device during the communication session. The controller receives an indication of a challenging driving condition from at least one of a first image stream from the at least one image capturing device and a detected condition from the at least one sensor. In response to receiving the indication of the challenging driving condition, the controller communicates, via the communication subsystem and the communication network, a notification over the communication session to the at least one second electronic device, the notification indicating that the driver is in the challenging driving condition.

In one or more embodiments, the electronic system supports sharing of distraction indications by a user's mobile device providing a dual camera based dashcam. The controller determines that a user is driving the vehicle and is using the mobile device as a dashcam. The controller additionally determines that the user (driver) is on an active communication (audio or video) with a remote user. Using image analysis from the visual data received from the rear camera system of the mobile device, the controller further determines whether there are any pending or ongoing distractions on the road in front of the vehicle. Additionally, the controller invokes the front camera system of the device and analyzes a face and posture of the driver to determine any signs of stress while the driver is looking at the road. With this example, the rear camera is positioned to view the roadway and can detect pedestrians and other traffic. The front camera can supplement the “distraction” worthiness of the pedestrians or traffic conditions by monitoring the driver's face. As an example, the front camera would serve as validation of the distraction based on the expression or stress level detected on the user's face. However, if the controller monitoring the front camera determines that the driver is looking at the electronic device instead of the road, the controller may rely solely on the rear camera to determine the distraction worthiness. When distractions are detected, the controller informs an attendee on the call that the driver is currently occupied. This information serves as a cue for the attendee to stop speaking to allow the driver to concentrate on the road. Alternatively, the attendee (or background monitoring system) may inform the driver to pay attention to the road. In situations in which ongoing weather phenomena are detected (e.g., heavy rain, snow, wind, etc.), the controller may provide a notification of the condition to the attendee on the call. In one embodiment, the controller may communicate an indication that is intended to be visually displayed to the attendee, prompting the attendee to not disturb the driver during the challenging driving conditions. In one or more embodiments, the controller recognizes that an extended period of time that the driver not looking at the road creates a challenging driving condition and presents an alert (e.g., visual, tactile, and/or aural). In an example, a display screen either presents an indication to look at the road or goes blank to remove distractions that would otherwise encourage the driver to look away from the road.

In the following detailed description of exemplary embodiments of the disclosure, specific exemplary embodiments in which the various aspects of the disclosure may be practiced are described in sufficient detail to enable those skilled in the art to practice the invention, and it is to be understood that other embodiments may be utilized and that logical, architectural, programmatic, mechanical, electrical, and other changes may be made without departing from the spirit or scope of the present disclosure. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims and equivalents thereof. Within the descriptions of the different views of the figures, similar elements are provided similar names and reference numerals as those of the previous figure(s). The specific numerals assigned to the elements are provided solely to aid in the description and are not meant to imply any limitations (structural or functional or otherwise) on the described embodiment. It will be appreciated that for simplicity and clarity of illustration, elements illustrated in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements are exaggerated relative to other elements.

As further described below, implementation of the functional features of the disclosure described herein is provided within processing devices and/or structures and can involve use of a combination of hardware, firmware, as well as several software-level constructs (e.g., program code and/or program instructions and/or pseudo-code) that execute to provide a specific utility for the device or a specific functional logic. The presented figures illustrate both hardware components and software and/or logic components.

Those of ordinary skill in the art will appreciate that the hardware components and basic configurations depicted in the figures may vary. The illustrative components are not intended to be exhaustive, but rather are representative to highlight essential components that are utilized to implement aspects of the described embodiments. For example, other devices/components may be used in addition to or in place of the hardware and/or firmware depicted. The depicted example is not meant to imply architectural or other limitations with respect to the presently described embodiments and/or the general invention. The description of the illustrative embodiments can be read in conjunction with the accompanying figures. Embodiments incorporating teachings of the present disclosure are shown and described with respect to the figures presented herein.

FIG.1depicts a functional block diagram of communication environment100including a mobile electronic system101used by first participant (“driver”)102in vehicle103to communicate, via radio access networks (RANs)104a-104z, with stationary electronic system105. Stationary electronic system105includes one or more network electronic devices107(e.g., network servers) that provide network communication hosting services includes second electronic devices109a-109mused by corresponding second participants106a-106m. Communication environment100provides an operating environment within which the features of the present disclosure are advantageously implemented. In particular, these features include reducing distractions to driver102during challenging driving conditions that may exist during setup of or ongoing communication session. In one or more embodiments, mobile electronic system101consists essentially of first electronic device108, which is capable of communicating with RANs104a-104z. Each participant (first participant or driver102, and second participants106a-106m) may participate in the communication session via a corresponding electronic device (first electronic device108and second electronic devices109a-109m).

First electronic device108can be one of a host of different types of devices, including but not limited to, an infant monitoring system, a mobile cellular phone, satellite phone, or smart-phone, a laptop, a net-book, an ultra-book, a networked smart watch, networked sports/exercise watch, and/or a tablet computing device or similar device. As more completely presented as communication device208ofFIG.2, which is described hereafter, electronic device108can also be a device supporting wireless communication. In these implementations, electronic device108can be utilized as, and also be referred to as, a system, device, subscriber unit, subscriber station, mobile station (MS), mobile, mobile device, remote station, remote terminal, user terminal, terminal, user agent, user device, a Session Initiation Protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), computer workstation, a handheld device having wireless connection capability, a computing device, or other processing devices connected to a wireless modem. Most importantly, it is appreciated that the features described herein can be implemented with a display device of various other types of electronic devices that are not necessarily a communication device. The specific presentation or description herein of a mobile communication device in addition to a data processing system as different examples of electronic device108are for example only, and not intended to be limiting on the disclosure.

Referring now to the specific component makeup and the associated functionality of the presented components, in one or more embodiments, mobile electronic system101is augmented by additional devices such as a (“dashcam”) dash image capturing device (ICD)110, vehicle access point (AP)112, and external memory such as network memory114. In an example, first electronic device108is configured with the ability to communicatively connect via communication subsystem116to the vehicle AP112via one or more network interfaces117, such as low power local wireless communication module118and local wired communication module119.

In one or more embodiments, mobile electronic system101includes one or more sensors120that can detect challenging driving conditions created by adverse weather122(e.g., wind and precipitation), poor road conditions124(e.g., incline, slick, rough, etc.), crowded traffic conditions126, and pedestrian traffic conditions128. In one embodiment, the one or more sensors120include dash ICD110that has forward field of view (FOV)121. In another embodiment, the one or more sensors120include back ICD130of first electronic device108that has forward FOV131. In yet another embodiment, the one or more sensors120are in-vehicle sensors that a communicatively connected to the vehicle computer, which communicates the sensor detected data to controller140of mobile electronic system101via a wireless or USB connection (e.g., vehicle access point (AP)112). Dash ICD110and back ICD130may provide two-dimensional or three-dimensional image data in the visual or infrared spectrums. Dash ICD110and back ICD130may include range finding, such as provided by stereo cameras, a radar or lidar sensor. In one or more embodiments, mobile electronic system101includes at least one ICD positioned to capture a face and posture of driver102. In an example, dash ICD110may include dual cameras, with a rear camera facing the rear of the dash cam device towards the driver. In another example, first electronic device108includes front ICD132having FOV134that is positioned toward driver102.

Back ICD130and front ICD132are part of input/output subsystem136of first electronic system101. In addition to communication subsystem116and input/output subsystem136, first electronic device108includes memory subsystem138and data storage subsystem139that are managed by controller140. Controller140may be communicatively connected to communication subsystem116and input/output subsystem136, memory subsystem138and data storage subsystem139via system interlink141. In an example, memory subsystem138includes network memory114that is external to first electronic device108and includes device memory142that is internal to first electronic device108. In one or more embodiments, input/output subsystem provides user interface devices144including one or more output devices146and one or more input devices148. User interface devices144may enable user interaction with first electronic device108using inputs and outputs that are one or more of visual, haptic, touch, sound, gesture, etc.

Device memory142includes program code for applications, such as communication application150that includes driver distraction notification and communication reduction (DNCR) module152and other applications154. In one or more embodiments, DNCR module152is a suite of applications, utilities, components, or modules that configure first electronic device108to monitor and/or detect and/or determine challenging driving conditions and provide mitigations for reducing distractions to driver102. Device memory142further includes operating system (OS)156, firmware interface158, such as basic input/output system (BIOS) or Uniform Extensible Firmware Interface (UEFI), and firmware160. Device memory142stores computer data162that is used by DNCR module152. Computer data162includes visual object library164that stores object images166and facial expressions168for recognizing challenging driving conditions. Computer data162may also include challenging driving conditions data170that is provided to stationary electronic system105.

In one or more embodiments, the communication session may be one-on-one between driver102using electronic device108of mobile electronic system101and one second participant106ausing (“second”) electronic device109a. The RANs104a-104zconnect the (“first”) mobile electronic system101to the (“second”) stationary system105via external network145. Electronic system101and stationary system105may access other systems such as navigation service176and weather service178via external networks145. The communication session may be multiparty between driver102using electronic device108and more than one second participant106a-106m, each using a corresponding (“second”) electronic device109a-109m. In one or more embodiments, the communication session may be assisted by network electronic device(s)146of communication network147. Each second electronic device109a-109mmay include the same or similar components described for first electronic device108or communication device208(FIG.2). Each electronic device108and109a-109mmay originate audio and/or video that is provided to the communication session. Each electronic device108and109a-109mmay receive audio and/or video from the communication session that originates at one or more other electronic devices108and109a-109m.

FIG.2is a functional block diagram of communication environment200in which communication device208, operating as a host device for a communication session, communicates with an in-vehicle communication system209that is integrated within and moves with vehicle103and with external communication system211that is external to vehicle103. Communication device208is an implementation of first electronic device108(FIG.1). Communication device208includes communication subsystem116, input/output subsystem136, data storage subsystem139, controller140, and device memory142as previously described but having additional functionality.

Controller140includes processor subsystem220, which executes program code to provide operating functionality of communication device208. Controller140manages, and in some instances directly controls, the various functions and/or operations of communication device208. These functions and/or operations include, but are not limited to including, application data processing, communication with second communication devices, navigation tasks, image processing, and signal processing. In one or more alternate embodiments, communication device208may use hardware component equivalents for application data processing and signal processing. For example, communication device208may use special purpose hardware, dedicated processors, general purpose computers, microprocessor-based computers, micro-controllers, optical computers, analog computers, dedicated processors and/or dedicated hard-wired logic.

The software and/or firmware modules executed by processor subsystem220have varying functionality when their corresponding program code is executed by data processor(s)222or secondary processing devices within communication device208such as digital signal processor224. Processor subsystem220can include other processors that are communicatively coupled internally or externally to data processor222. Data processor222is communicatively coupled, via system interlink141, to data storage subsystem139and device memory142. System interlink141represents internal components that facilitate internal communication by way of one or more shared or dedicated internal communication links, such as internal serial or parallel buses. As utilized herein, the term “communicatively coupled” means that information signals are transmissible through various interconnections, including wired and/or wireless links, between the components. The interconnections between the components can be direct interconnections that include conductive transmission media or may be indirect interconnections that include one or more intermediate electrical components. Although certain direct interconnections (system interlink141) are illustrated inFIG.1, it is to be understood that more, fewer, or different interconnections may be present in other embodiments.

Processor subsystem220of controller140can execute program code of DNCR module152to configure communication device208to perform specific functions for recognizing challenging driving conditions. In an example, DNCR module152can include or utilize camera control application230, object recognition application232, look gaze detect application234, facial expression/posture recognition application235, navigation application236, and setup delay timer237. Processor subsystem220receives data from certain components of input/output subsystem136and presents data on certain components of input/output subsystem136. In an example, input/output subsystem136includes back ICDs130, front ICDs132, touch display240, microphone242, and audio output device(s)244.

Data storage subsystem139of communication device208includes data storage device(s)160. Controller140is communicatively connected, via system interlink141, to data storage device(s)160. Data storage subsystem139provides applications, program code, and stored data on nonvolatile storage that is accessible by controller140. For example, data storage subsystem139can provide a selection of applications and computer data, such as communication application150that includes DNCR module152and other application(s)154. These applications can be loaded into device memory142for execution by controller140. In one or more embodiments, data storage device(s)250can include hard disk drives (HDDs), optical disk drives, and/or solid-state drives (SSDs), etc. Data storage subsystem139of communication device208can include removable storage device(s) (RSD(s))252, which is received in RSD interface254. Controller140is communicatively connected to RSD252, via system interlink141and RSD interface254. In one or more embodiments, RSD164is a non-transitory computer program product or computer readable storage device. Controller140can access RSD164or data storage device(s)160to provision communication device208with program code, such as code for communication application150and other application(s)154, and with computer data162such as visual object library164(FIG.1).

Communication device208further includes communication subsystem116for communicating, using a cellular connection, with network node(s)260of external communication system211and for communicating, using a wireless connection, with vehicle access point112of local communication system209. Communication subsystem116includes antenna subsystem262. Communication subsystem116includes radio frequency (RF) front end263and communication module264. RF front end263includes transceiver(s)266, which includes transmitter(s)268and receiver(s)270. RF front end263further includes modem(s)272. Communication module264of communication subsystem116includes baseband processor274that communicates with controller140and RF front end263. Baseband processor274operates in a baseband frequency range to encode data for transmission and decode received data, according to a communication protocol. Modem(s)272modulate baseband encoded data from communication module264onto a carrier signal to provide a transmit signal that is amplified by transmitter(s)268. Modem(s)272demodulates each signal received from external communication system211using by antenna subsystem262. The received signal is amplified and filtered by receiver(s)270, which demodulate received encoded data from a received carrier signal.

In one or more embodiments, controller140, via communication subsystem116, performs multiple types of cellular OTA or wireless communication with local communication system209. Communication subsystem116can communicate via an over-the-air (OTA) connection276with local wireless devices278. In an example, OTA connection276is a peer-to-peer connection, Bluetooth connection, or other personal access network (PAN) connection. In one or more embodiments, communication subsystem116communicates with one or more locally networked devices via a wireless local area network (WLAN) link279supported by vehicle access point112. In one or more embodiments, vehicle access point112supports communication using one or more IEEE 802.11 WLAN protocols. Vehicle access point112is connected to external networks145via a cellular connection. In one or more embodiments, communication subsystem116communicates with GPS satellites280via downlink channel282to obtain geospatial location information. Communication subsystem116can communicate via an over-the-air (OTA) cellular connection284with network node(s)260.

According to aspects of the present disclosure, mobile electronic system101includes at least one image capturing device132positionable to have a field of view encompassing at least a face and/or posture of driver102of vehicle103. At least one sensor120(FIG.1) of mobile electronic system101detects external conditions to vehicle103that can require driver102to be more attentive while vehicle103is in motion. Mobile electronic system101includes communication subsystem116that is connectable to a communication network147(FIG.1). At least one output device146(FIG.1) of mobile electronic system101provides a communication output from a network-connected communication session with at least one second electronic device109a-109m(FIG.1).

Memory subsystem138(FIG.1) of mobile electronic system101stores communication application150including DNCR module152. Controller140of mobile electronic system101is communicatively connected to communication subsystem116, at least one sensor120, at least one image capturing device132, memory subsystem138, and at least one output device146. Controller140executes communication application150to connect, via communication subsystem116and communication network147, with at least one second electronic device109a-109m(FIG.1) during the communication session. Controller140receives an indication of a challenging driving condition from at least one of a first image stream from at least one image capturing device132and a detected condition from at least one sensor120. In response to receiving the indication of the challenging driving condition, controller140communicates, via communication subsystem116and communication network147, a notification over the communication session to at least one second electronic device109a-109m(FIG.1), the notification indicating that driver102is in the challenging driving condition.

In one or more embodiments, controller140receives the first image stream from first image capturing device132. Controller140analyzes one or more of an expression of the face of driver102and a posture of driver102contained in the first image stream. Controller140determines that the challenging driving condition exists in response to identifying a corresponding one or more of a stressed expression and a stressed posture.

In one or more embodiments, prior to determining the challenging driving condition, controller140communicates the first image stream to the communication session. Then controller140replaces at least a portion of the first image stream with the notification in response to determining the challenging driving conditions exist.

In one or more embodiments, controller140determines that the challenging driving condition exists by identifying one or more of hazardous road conditions, proximity of at least one pedestrian, and proximity of another vehicle, at least in part based on the sensed condition from the at least one sensor. In one or more particular embodiments, controller140identifies a look gaze direction of eyes of the face of the driver. Controller140presents a local notification to at least one output device146in response to determining that the look gaze direction indicates that driver102is not looking out at a roadway and surrounding driving conditions and that the challenging driving condition exists. The local notification prompts driver102to pay attention to the challenging driving conditions.

In one or more embodiments, at least one sensor120includes a second image capturing device (110,130) that captures a second image stream of conditions external to vehicle103. Controller140receives the second image stream from the second image capturing device (110,130). Controller140identifies the external conditions contained in the second image stream as representative of at least one challenging driving condition. Controller140communicates the notification in response to the second image stream including images representative of at least one challenging driving condition, to visually indicate to a user (participant106) of second electronic device109a-109mthat driver102is in the challenging driving condition.

In one or more embodiments, controller140determines that the challenging driving conditions exist based on: (i) receiving information from one or more of navigation service176and weather service178(FIG.1) that vehicle103is driving in an area subject to a corresponding one of hazardous traffic conditions and hazardous road conditions; and (ii) analyzing the first image stream to verify the one of hazardous traffic conditions or hazardous road conditions.

In one or more embodiments, mobile electronic system101includes microphone242communicatively coupled to controller140and positioned to detect sounds spoken by driver102. Controller140compares current speech detected by microphone242to baseline speech characterized as unstressed. Controller140identifies that driver102is experiencing challenging driving conditions at least in part based on a change, by more than a baseline threshold amount, in a sound characterization of the current speech, which indicates stressed speech.

In one or more embodiments, mobile electronic system101is, or consists essentially of, a mobile user device such as communication device208. At least one output device146(FIG.1) includes a display device such as touch display240. At least one sensor120(FIG.1) is one of (a) embedded within a housing of the mobile user device and (b) communicatively connected to a local communication interface of the mobile user device.

In one or more embodiments, while disconnected from a communication session, controller140receives an indication of a challenging driving condition from at least one of the first image stream from at least one image capturing device132and a detected condition from the at least one sensor (110,130). Controller140stores a status of the driving conditions for the vehicle corresponding to mobile electronic system101as a challenging driving condition in memory subsystem138. In one or more embodiments, memory subsystem is accessible to communication application150at one or more of device memory142at mobile electronic system101and network memory114(FIG.1) at communication network145. In response to detecting a request to initiate a new communication session, controller140transmits the status to requesting second electronic device(s)109a-109mwithout connecting to the new communication session. In one embodiment, the status can be tracked via a driving condition entry stored in a particular memory location/address, the entry being one or more bits settable by the controller to at least two different values, a first default value (e.g., 0/00) indicating that no challenging condition currently exist and a second value (e.g., 1/01) indicating when a challenging driving condition is detected. It is appreciated that other embodiments may provide for different levels of challenging driving conditions being uniquely identified using a larger number of bits enabling additional distinct values (e.g., 00, 01, 10, 11, with 2 bits). Controller periodically detects/identifies the current driving conditions and updates the value of the bit accordingly.

FIG.3depicts communication environment300of stationary electronic system105that includes network electronic device107such as a network server that is configured to facilitate communication between first electronic device108used by driver102and second electronic device109used by second participant106and to perform driver distraction notification and resulting communication reduction functions. Network electronic device107includes memory subsystem138, communication subsystem116, which includes one or more network interfaces117connected to external networks145via fiber305, and data storage subsystem116managed by controller140via system interlink141. Network electronic device107may have identical or similar components and functionality as described above for electronic device108ofFIG.1and communication device208ofFIG.2. Controller140executes applications stored in memory subsystem138to support communication. In an example, enterprise communication management application320can host an audio and/or video communication session as well as communicate data associated with the session between user devices. Enterprise communication management application320creates and manages communication subscriber database322that is also stored in memory subsystem138. Controller140also executes applications stored in memory subsystem138to support aspects of the present disclosure. Enterprise driver distraction notification and communication reduction (EDNCR) application330detects and responds to instances of challenging driving conditions for one or more participants in a communication session. EDNCR application330creates and maintains driving status reports332from a plurality of electronic devices to prompt notifications and reductions in communication content pushed to particular user devices described below forFIG.6.

According to aspects of the present disclosure, stationary electronic system105includes communication subsystem116connectable to communication network145(FIG.1) and controller140communicatively connected to communication subsystem116. With stationary electronic system105operating as a host for communication sessions, controller140initiates setup of a communication session, via communication subsystem116and communication network145(FIG.1), with at least one first electronic device108and one second electronic device109. In response to determining that first electronic device108is being used by driver102in vehicle103that is potentially in a challenging driving condition, controller140pauses or halts setup of the communication session to reduce distractions to driver102while the vehicle is in the challenging driving condition. Controller140requests data about/from first electronic device108that indicates whether current driving conditions allow driver102to communicate without degrading driving performance. Controller140completes setup of the communication session in response to verifying that driver102is able to communicate while safely operating the vehicle under the current driving conditions.

In one or more embodiments, in response to initiating the setup of the communication session, controller140receives, from communication network145(FIG.1), information indicating that first electronic device108is being used by user or driver102who is driving based on one or more of: (i) a rate of movement of first electronic device108; (ii) geographic location of first electronic device108indicating movement aligned with a roadway; and (iii) a report from navigation application236(FIG.2) being executed by first electronic device108(FIG.1).

In one or more embodiments, to verify that driver102is able to communicate while safely operating vehicle103under the current driving conditions, controller140identifies a geographic location of first electronic device108and queries one or more of navigation service176and weather service178(FIG.1) for information of a corresponding one of hazardous traffic conditions and hazardous road conditions at the location.

In one or more embodiments, to verify that driver102is able to communicate while safely operating vehicle103under the current driving conditions, controller140transmits a request, to first electronic device108, for a user input that indicates whether driver102is able to communicate while safely operating vehicle103under the current driving conditions. Controller140completes setup of the communication session in response to receiving, from driver102via first electronic device108, an affirmative response to the request for the user input.

In one or more embodiments, to verify that driver102is able to communicate while safely operating vehicle103under the current driving conditions, controller140transmits a request, to first electronic device108, for a user input that indicates whether driver102is able to communicate while safely operating vehicle103under the current driving conditions. Controller140discontinues setup of the communication session and activates a timer with a time period for a delayed setup of a communication session in response to receiving, from the driver via the first electronic device, a negative response to the request for the user input.

In one or more embodiments, to verify that driver102is able to communicate while safely operating vehicle103under the current driving conditions, controller140transmits a request, to the driver via the first electronic device, for a user input that indicates whether the driver is able to communicate while safely operating the vehicle under the current driving conditions. Controller140completes setup of the communication session and pauses communication of content to the first electronic device in response to receiving a negative user input from the driver using the first electronic device. In one or more particular embodiments, controller140initiates communicating content to the first electronic device in response to subsequently receiving an affirmative user input from the driver using the first electronic device.

In one or more embodiments, stationary electronic system105includes at least one output device146(FIG.1) at the at least one second electronic device109that is communicatively coupled to controller140. The at least one output device146(FIG.1) presents one or more of an audio output and a visual output. Controller140presents a notification via at least one output device146(FIG.1) indicating that first electronic device108is being used by driver102in vehicle103.

In one or more embodiments, to verify that driver102is able to communicate while safely operating vehicle103under the current driving conditions, controller140transmits a request, to a navigation application that is either executed by first electronic device108or communicating with first electronic device108, for data that indicates whether driver102is able to communicate while safely operating vehicle103under the current driving conditions. Controller140completes setup of the communication session in response to receiving, from navigation application236(FIG.2), an affirmative response to the request for the user input. Controller140discontinues setup of the communication session and activates timer237(FIG.2) with a time period for a delayed setup of a communication session in response to receiving, from navigation application236(FIG.2), a negative response to the request for the user input.

According to one or more aspects of the present disclosure, stationary electronic system105includes communication subsystem116connectable to communication network147(FIG.1) and controller140communicatively connected to communication subsystem116. With stationary electronic system105operating as a host for communication sessions, controller140communicates, via communication subsystem116and communication network147(FIG.1), with at least one first electronic device108participating in a communication session with at least one second electronic device109. In response to determining that first electronic device108is being used by driver102in vehicle103that is potentially in a challenging driving condition, controller140receives data from/about either directly or indirectly from first electronic device108that indicates whether current driving conditions allow driver102to communicate without degrading driving performance. When the received data indicates driver102is in the challenging driving condition, controller140restricts transmission of new communication inputs from the communication session to first electronic device108to reduce distractions to driver102.

In one or more embodiments, controller140receives, from first electronic device108, a first image stream that captures at least one of a face of driver102or a forward view of surrounding conditions of vehicle103. Controller140analyzes the first image stream to determine whether first electronic device108is being used by a driver in a vehicle that is moving and whether driver102is potentially in a challenging driving condition. Controller140restricts the communication session to reduce distractions to driver102in response to determining that first electronic device108is being used by driver102in moving vehicle103and that driver102is potentially in the challenging driving condition.

In one or more embodiments, controller140receives, from first electronic device108, a first image stream that captures at least a face of driver102. Controller140analyzes the first image stream for one or more of a stressed expression or tense posture of driver102. Controller140determines whether the current driving condition allow driver102to communicate without degrading driving performance based on whether the stressed expression or tense posture is identified. Controller140restores full functionality of the communication session in response to verifying that driver102is able to communicate without restrictions under the current driving conditions.

In one or more embodiments, to determine whether driver102is able to communicate in the current driving condition without degrading driving performance, controller140receives information identifying a geographic location and direction of movement of first electronic device108. Controller140queries one or more of navigation service176and weather service178for information of a corresponding one of hazardous traffic conditions and hazardous road conditions at the location.

In one or more embodiments, to determine whether the current driving condition allows driver102to communicate without degrading driving performance, controller140transmits a request, to first electronic device108, for a user input/response that indicates whether current driving conditions allow driver102to communicate without degrading driving performance. Controller140pauses the communication session in response to receiving, from driver102via first electronic device108, a negative user response to the request for the user input. Controller140restores the communication session in response to receiving, from driver102via first electronic device108, an affirmative user response to the request for the user input.

In one or more embodiments, to determine whether the current driving conditions allow driver102to communicate without degrading driving performance, controller140transmits a request, to first electronic device108, for a user input that indicates whether current driving conditions allow driver102to communicate without degrading driving performance. Controller140disconnects first electronic device108from the communication session in response to receiving, from driver102via first electronic device108, a negative user response to the request for the user input. Controller140activates timer237with a time period for rechecking the driving conditions prior to reconnecting first electronic device108to the communication session.

In one or more embodiments, stationary electronic system105includes at least one output device146(FIG.1) that presents one or more of an audio output and a visual output and that is communicatively coupled to controller140. Controller140presents a prompt at the at least one output device146(FIG.1) indicating that first electronic device108is being used by driver102in vehicle103. In one or more particular embodiments, controller140communicates the prompt, via communication subsystem116and communication network147(FIG.1), to at least one second electronic device109mof the at least one second electronic devices109.

In one or more embodiments, controller140receives an audio stream from first electronic device108. Controller140compares current speech detected in the audio stream to baseline speech characterized as unstressed speech. Controller140restricts the communication session to reduce distractions to driver102at least in part based on a change in a sound characterization of the current speech by more than a baseline threshold amount indicating stressed speech, while driver102is potentially in a challenging driving condition.

FIG.4depicts an example layout diagram of communication environment400with mobile electronic system101positioned in vehicle103that is in a challenging driving condition, which is mitigated by driver distraction notification and communication reduction features, according to one or more embodiments. Mobile electronic system101consists essentially of first electronic device108that is positioned by fixture401on dash403of vehicle103. Fixture401positions FOV131of back ICD130to view driving conditions405and positions FOV134of front ICD132to view face407and posture409of driver102. First electronic device108is positioned on dashboard of vehicle103to provide easy viewing by driver102without blocking the view forward. Look gaze direction411of eyes413of driver102are looking straight ahead and not at first electronic device108. First electronic device108is communicating with stationary electronic system105of network electronic devices107and second electronic devices109via network nodes260and communication network147.

First electronic device108of mobile electronic system101may detect urgent challenging driving conditions of short duration, such as appearance of pedestrian415in the path of vehicle103. In response, first electronic device108may immediately trigger cessation of presenting content from the communication session. In one or more embodiments, the first electronic device108may determine that the presence of the pedestrian415walking or crossing the road and/or presence in school zones, high traffic areas, etc., is a challenging driving condition in response to determining that the look gaze direction411of driver102is away from pedestrian415.FIG.5Ais first image501athat depicts driver102looking at a camera (132,FIG.4).FIG.5Bis second image501bdepicts driver102looking away from the camera (132,FIG.4) and out towards a front of vehicle103(FIG.4). With continued reference toFIG.4, first electronic device108of mobile electronic system101may integrate information from multiple sources to determine challenging driving conditions of longer duration. In an example, first electronic device108may visually detect precipitation417(e.g., rainfall, snowfall, sleet, haze).FIG.5Cis third image501cthat depicts example normal driving conditions in front of vehicle103(FIG.4).FIG.5Dis fourth image501dthat depicts example challenging driving conditions in front of vehicle103(FIG.4). With continued reference toFIG.4, the severity of the precipitation417may be verified at least in part based on determining sensed or reported temperatures at a location of vehicle103. Other sensed or identified severity factors may include vehicle speed, type of road surface, inclines/declines or turns of the road, wind speed and direction, and other factors. First electronic device108may be communicatively connected to a vehicle management system (VMS)419via wired or wireless connection to vehicle AP112to receive data such as detected tire slippage. First electronic device108may also detect driving proficiency of driver102to adjust thresholds for a challenging driving condition determination.

In one or more embodiments, first electronic device108detects visual and/or audible responses by driver102to the driving conditions to determine whether the driver102is being challenged.FIG.5Eis fifth image501ethat depicts driver102having facial expression503aand posture505athat is relaxed and unstressed. Driver102is also producing unstressed speech507a.FIG.5Fis sixth image501fthat depicts driver102having facial expression503band posture505bthat is tense and stressed. Driver102is also producing stressed speech507b.

FIG.6depicts example user interface device144for a communication session between participant A (driver102a) using first device601a, second participant B (driver102b) using second device601b, and local participant C106cusing third device601c. User interface device144presents mitigating features for driver distraction notification to participant C106cto prompt communication reduction for drivers102a-102b. In an example, each participant has respective avatar or video stream window602a,602b, and602c. Both first and second drivers102a-102bare identified as being in a challenging driving condition. To notify local participant C106c, user interface device144may present a dashcam view603ain window602athat is streamed from first device601a, enabling local participant C106cto receive the visual experience as a passenger in the vehicle with driver102a, thus experiencing the severity of the challenging driving conditions intuitively and in real-time. To notify local participant C106c, user interface device144may present a notification image603bin window602bto inform local participant C106cof the challenging driving conditions. Due to communication limitations or privacy considerations, second device601bmay not stream an image stream of a forward view that depicts the challenging driving conditions to the communication session. Alternatively, the forward view may not enable participant C106cto readily discern the challenging driving condition from the forward view and thus the forward view may not be selected for notification. Avatar, still image, or image stream604of local participant C106cis shared to the communication session and lacks a notification or indication of being in a challenging driving condition. In one or more embodiments, in addition to visually notifying other participants that particular participants are encountering challenging driving conditions, user interface device144may present auditory notifications to participant C106c. In an example, user interface device144may revert to auditory notifications if participant C106cis not receiving or viewing visual content of the communication session. In one or more embodiments, the user interface device144that local to the drivers102a-102bmay present audio inputs from remote participants, such as participant C106c, are automatically muted or attenuated in volume to reduce distractions.

Other information shared during the communication session may include notifications of the status of drivers102a-102bwho are in challenging driving conditions. In an example, chat box610can include messages611from participants as well as automated notification messages612. A visual alert box613may pop-up, visually presenting notification614to non-driving participant C106c, which may be accompanied by a verbal notification as appropriate for the type of communication session. Screen share window620may be accompanied by a notification622of which participants are prevented from viewing the content due to being in a challenging driving condition.

FIGS.7A-7B(collectively “FIG.7”) present a flow diagram of a method for reducing driver distractions by a mobile electronic system to a driver of a vehicle experiencing challenging driving conditions while engaged in a communication session.FIG.8presents a flow diagram of a method of selectively setting up and connecting a communication session with a remote mobile electronic system used by a driver to enable reduction in distractions based on challenging driving conditions.FIG.9presents a flow diagram of a method of reducing driver distractions during a communication session with a remote mobile electronic system used by a driver in challenging driving conditions. The descriptions of method700(FIGS.7A-7B), method800(FIG.8), and method900(FIG.9) are provided with general reference to the specific components illustrated within the precedingFIGS.1-4and5A-5F. Specific components referenced in method700(FIGS.7A-7B), method800(FIG.8), and method900(FIG.9) may be identical or similar to components of the same name used in describing precedingFIGS.1-4and5A-5F. In one or more embodiments, controller140(FIGS.1-3) respectively of mobile electronic system101(FIG.1), communication device208(FIG.2) and stationary electronic system105(FIG.3) provides functionality of method700(FIGS.7A-7B), method800(FIG.8) and method900(FIG.9).

With reference toFIG.7A, method700includes connecting, via a communication subsystem of an electronic system, to a communication session with at least one second electronic device, the electronic system being in a vehicle operated by a driver (block702). Method700includes determining whether the communication session is locally or remotely terminated (decision block704). In response to determining that the communication session is terminated, method700ends. In response to determining that the communication session is not terminated, in one or more embodiments, method700includes communicating an audio stream detected by a microphone of the electronic system to the communication session (block706). In one or more embodiments, method700includes communicating to the communication session a first image stream from at least one image capturing device positionable to have a field of view encompassing at least a face and/or posture of a driver of a vehicle (block708). Method700includes monitoring at least one sensor that detects external conditions (e.g., hazardous road conditions, proximity of at least one pedestrian, and proximity of another vehicle) to a vehicle, where the external conditions can require a driver of the vehicle to be more attentive while the vehicle is in motion (block710). Method700includes monitoring a first image stream from at least one image capturing device for a stressed facial expression and/or a stressed posture of the driver of the vehicle, which correlates to or is an indication of a challenging driving condition (block712). Then method700proceeds to block714ofFIG.7B.

With reference toFIG.7B, in one or more embodiments, method700includes comparing current speech detected by a microphone, which is positioned to detect sounds spoken by the driver, to baseline speech characterized as unstressed in order to identify an indication of the challenging driving condition (block714). In one or more embodiments, method700includes receiving information from one or more of a navigation service and a weather service that the vehicle is driving in an area subject to a corresponding one of hazardous traffic conditions and hazardous road conditions that verify or corroborate inconclusive visual indications (block716). Method700include determining whether the indication of the challenging driving condition is received and identified (decision block718). In response to determining that no challenging driving condition is received or identified, method700returns to block704(FIG.7A). In response to determining that the challenging driving condition is received and identified, method700may include communicating, via the communication subsystem and the communication network, a notification over the communication session to the at least one second electronic device by: (i) replacing at least a portion of the first image stream with the notification in response to determining the challenging driving conditions exists; or (ii) streaming a second image stream from a roadway facing/oriented image capturing device, utilized as one of the at least one sensor (block720). Then method700returns to block710(FIG.7A).

In one or more embodiments, method700includes identifying a look gaze direction of eyes of the face of the driver. Method700includes presenting a local notification at the at least one output device in response to determining that the look gaze direction indicates that the driver is not looking out at a roadway and surrounding driving conditions and that the challenging driving condition exists. The local notification prompts the driver to pay attention to the challenging driving conditions.

In one or more embodiments, method700includes, while disconnected from a communication session, receiving an indication of a challenging driving condition from at least one of the first image stream from the at least one image capturing device and a detected condition from the at least one sensor. Method700includes storing, in memory accessible to the communication application at one or more of the memory at the electronic system and the communication network, a driving condition status of the electronic system as being in a challenging driving condition. Method700includes transmitting the status to a requesting second electronic device without connecting a new communication session.

With reference toFIG.8, method800includes initiating setup of a communication session by the electronic system as host, with a first electronic device and at least one second electronic device (block802). Method800includes determining that the first electronic device is being used by a driver in a vehicle by receiving, from the communication network, information indicating that the first electronic device is being used by a user who is driving, based on one or more of: (i) a rate of movement of the first electronic device; (ii) geographic location of the first electronic device indicating movement aligned with a roadway; and (iii) a report from a navigation application being executed by the first electronic device (block804). Method800includes pausing setup of the communication session to reduce possible distractions to the driver when communicating while in potentially in a challenging driving condition (block806). Method800includes requesting data about/from the first electronic device that indicates whether current driving conditions allow the driver to communicate without degrading driving performance by one of: (i) identifying a geographic location of the first electronic device and querying one or more of a navigation service and weather service for information of a corresponding one of hazardous traffic conditions and hazardous road conditions at the location; and (ii) transmitting a request, to the first electronic device, for a user input or an automated response from a navigation application that indicates whether the driver is able to communicate while safely operating the vehicle under the current driving conditions (block808). Method800includes determining whether an indication is received indicating it is safe for the driver to communicate via a communication session (decision block810). In response to determining that the indication of safe to communicate is received, method800includes completing setup of the communication session (block812). Then method800ends. In response to determining that the indication of safe to communicate is not received, method800includes discontinuing setup of the communication session to reduce distractions to the driver by (block814). In one embodiment, method800also includes optionally activating a timer with a time period for a delayed setup of a communication session. Then method800ends.

In one or more embodiments, determining that the first electronic device is being used by the driver in the vehicle includes: in response to initiating the setup of the communication, receiving, from the communication network, information indicating that the first electronic device is being used by a user who is driving based on one or more of: (i) a rate of movement of the first electronic device; (ii) geographic location of the first electronic device indicating movement aligned with a roadway; and (iii) a report from a navigation application being executed by the first electronic device.

In one or more embodiments, verifying that the driver is able to communicate while safely operating the vehicle under the current driving conditions includes: (i) identifying a geographic location of the first electronic device; and (ii) querying one or more of a navigation service and weather service for information of a corresponding one of hazardous traffic conditions and hazardous road conditions at the location.

In one or more embodiments, verifying that the driver is able to communicate while safely operating the vehicle under the current driving conditions includes: (i) transmitting a request, to the first electronic device, for a user input that indicates whether the driver is able to communicate while safely operating the vehicle under the current driving conditions; and (ii) receiving, from the driver via the first electronic device, an affirmative response to the request for the user input. Method800then includes completing setup of the communication session in response to receiving the affirmative user input from the driver.

In one or more embodiments, in verifying whether the driver is able to communicate while safely operating the vehicle under the current driving conditions, the method includes: (i) transmitting a request, to the first electronic device, for a user input that indicates whether the driver is able to communicate while safely operating the vehicle under the current driving conditions; and (ii) discontinuing setup of the communication session and activates a timer with a time period for a delayed setup of a communication session in response to receiving, from the driver via the first electronic device, a negative response to the request for the user input.

In one or more embodiments, verifying that the driver is able to communicate while safely operating the vehicle under the current driving conditions includes: (i) transmitting a request, to the driver via the first electronic device, for a user input that indicates whether the driver is able to communicate while safely operating the vehicle under the current driving conditions; and (ii) completing setup of the communication session and pausing communication of content to the first electronic device in response to receiving a negative user input from the driver using the first electronic device. In one or more particular embodiments, later initiating communication of content to the first electronic device occurs in response to subsequently receiving an affirmative user input from the driver using the first electronic device. In one or more embodiments, method800includes presenting a notification of one or more of an audio output and a visual output via at least one output device at the at least one second electronic device indicating that the first electronic device is being used by a driver in a vehicle that is navigating in potentially difficult/hazardous driving conditions.

In one or more embodiments, verifying that the driver is able to communicate while safely operating the vehicle under the current driving conditions includes: (i) transmitting, to a navigation application executed by or communicating with the first electronic device, a request for data that indicates whether the driver is able to communicate while safely operating the vehicle under the current driving conditions; (iii) completing setup of the communication session in response to receiving, from the navigation application, an affirmative response to the request for the user input; and (iii) discontinuing setup of the communication session and activating a timer with a time period for a delayed setup of a communication session in response to receiving, from the navigation application, a negative response to the request for the user input.

With reference toFIG.9, method900includes an electronic system communicating, via a communication subsystem and a communication network, with a first electronic device participating in a communication session with at least one second electronic device, the communication session hosted by the electronic system (block902). Method900includes identifying that the first electronic device is being used by a driver in a vehicle (block904). Method900includes receiving data (e.g., navigation/weather service data, first image stream of the driver that included stressed or unstressed facial condition and/or stressed or unstressed posture, second image stream of road conditions in front of the vehicle, etc.) about the driver, vehicle, external conditions to the vehicle, and first electronic device that indicates whether current driving conditions allow the driver to communicate with the established communication session without degrading driving performance (block906). Method900includes restricting transmission of new communication inputs from the communication session to the first electronic device while determining whether the current driving condition allows the driver to communicate without degrading driving performance (block908). Method900includes transmitting, to the first electronic device, a request for a user input that indicates whether current driving conditions allow the driver to communicate without degrading driving performance (block910). Method900includes determining whether an affirmative user response is received (decision block912). In response to determining that an affirmative user response is received verifying that the driver is able to communicate without restrictions under the current driving conditions, method900includes restoring the communication session (block914). Then method900returns to block902. In response to determining that an affirmative user response is not received, method900includes continuing to reduce distractions for the driver by one of: (i) disconnecting the first electronic device from the communication session and activating a timer with a time period for performing a next check prior to reconnecting the first electronic device to the communication session; or (ii) pausing the communication session until an affirmative user response is received (block916). Then method900ends.

In one or more embodiments, method900includes receiving, from the first electronic device, a first image stream that captures at least one of a face of the driver or a forward view of surrounding conditions of the vehicle. Method900includes analyzing the first image stream to determine whether the first electronic device is being used by a driver in a vehicle that is moving, and that the driver is potentially in a challenging driving condition. Method900includes restricting the communication session to reduce distractions to the driver in response to determining that the first electronic device is being used by the driver in the vehicle that is moving, and that the driver is potentially in the challenging driving condition.

In one or more embodiments, method900includes receiving, from the first electronic device, a first image stream that captures at least a face of the driver. Method900includes analyzing the first image stream for one or more of a stressed expression or tense posture of the driver. Method900includes determining whether the current driving condition allow the driver to communicate without degrading driving performance based on whether the stressed expression or tense posture is identified. Method900includes restoring the communication session in response to verifying that the driver is able to communicate without restrictions under the current driving conditions.

In one or more embodiments, determining whether the driver is able to communicate in the current driving condition without degrading driving performance includes: (i) receiving information identifying a geographic location and direction of movement of the first electronic device; and (ii) querying one or more of a navigation service and weather service for information of a corresponding one of hazardous traffic conditions and hazardous road conditions at the location.

In one or more embodiments, determining whether the current driving condition allows the driver to communicate without degrading driving performance includes: (i) transmitting, to the first electronic device, a request for a user input that indicates whether current driving conditions allow the driver to communicate without degrading driving performance; (ii) pausing the communication session in response to receiving, from the driver via the first electronic device, a negative user response to the request for the user input; and (iii) restoring the communication session in response to receiving, from the driver via the first electronic device, an affirmative user response to the request for the user input.

In one or more embodiments, determining whether the current driving conditions allow the driver to communicate without degrading driving performance includes: (i) transmitting, to the first electronic device, a request for a user input that indicates whether current driving conditions allow the driver to communicate without degrading driving performance; (ii) disconnecting the first electronic device from the communication session in response to receiving, from the driver via the first electronic device, a negative user response to the request for the user input; and (iii) activating a timer with a time period for reconnecting the first electronic device to the communication session.

In one or more embodiments, method900includes presenting a prompt of one or more of an audio output and a visual output at one or more output devices to indicate that the first electronic device is being used by a driver in a vehicle. In one or more particular embodiments, method900includes communicating the prompt, via the communication subsystem and the communication network, to at least one of the at least one second electronic devices.

In one or more embodiments, method900includes: (i) receiving an audio stream from the first electronic device; (ii) comparing current speech detected in the audio stream to baseline speech characterized as unstressed; and (iii) restricting the communication session to reduce distractions to the driver at least in part based on a change in a sound characterization of the current speech by more than a baseline threshold amount indicating stressed speech, while the driver is potentially in a challenging driving condition.

As will be appreciated by one skilled in the art, embodiments of the present innovation may be embodied as a system, device, and/or method. Accordingly, embodiments of the present innovation may take the form of an entirely hardware embodiment or an embodiment combining software and hardware embodiments that may all generally be referred to herein as a “circuit,” “module” or “system.”

While the innovation has been described with reference to exemplary embodiments, it will be understood by those skilled in the art that various changes may be made, and equivalents may be substituted for elements thereof without departing from the scope of the innovation. In addition, many modifications may be made to adapt a particular system, device, or component thereof to the teachings of the innovation without departing from the essential scope thereof. Therefore, it is intended that the innovation is not limited to the particular embodiments disclosed for carrying out this innovation, but that the innovation will include all embodiments falling within the scope of the appended claims. Moreover, the use of the terms first, second, etc. do not denote any order or importance, but rather the terms first, second, etc. are used to distinguish one element from another.