Methods, devices, and systems pertaining to in-vehicle tutorials are described. A method may involve receiving a request for an in-vehicle tutorial of an operational feature of a vehicle from a user and simulating expected driving behavior corresponding to the operational feature in the vehicle. The method may further include monitoring operational behavior of the user, comparing the operational behavior with the expected driving behavior, and providing a feedback to the user based on the comparison.

TECHNICAL FIELD

The present disclosure generally relates to automotive technologies and, more particularly, to methods and systems for in-vehicle tutorials.

BACKGROUND

New and valuable technologies are continually being introduced into today's cars. Technologies that are not well communicated often go unused. Also, users may not use the technologies as intended by the manufacturer. While car manuals may contain detailed instructions, users may find consultation with the manual unintuitive and inconvenient. Sometimes, instructions on how to use the technologies may be provided to the users at the point of sale (e.g., dealerships). However, explanation of technologies at the point of sale tends to be limited (e.g., as it is a one-time event) and the effectiveness depends largely on the degree to which the salesperson has been educated on the technologies.

DETAILED DESCRIPTION

Implementations herein relate to methods and systems for in-vehicle tutorials. The implementations present technologies to a user in a vehicle, e.g., the driver or a passenger of the vehicle, by leveraging multiple modes of communications, such as audio and lighting cues, as well as animations on an in-vehicle display system. Implementations in accordance with the present disclosure help introducing new features of the vehicle to the user when the introduction is requested or when a certain condition is triggered. Advantageously, the user may follow an in-vehicle tutorial at a pace of the user and therefore the user is provided an intuitive and immersive experience that enables a deeper understanding, learning and appreciation of the features.

FIG. 1illustrates a computing environment100that enables in-vehicle tutorials. The computing environment100includes a vehicle102and a user104of vehicle102, e.g., a driver or passenger of vehicle102. Vehicle102includes an in-vehicle tutorial system106that provides a tutorial of one or more operational features of vehicle102to user104. In accordance with various embodiments, in-vehicle tutorial system106, in a basic configuration, may include a multimedia user interface108and a tutorial application110, each discussed in turn below.

Multimedia user interface108may be configured to enable multiple modes of communications between in-vehicle tutorial system106and user104via, for example, audio and lighting cues as well as animations on an in-vehicle display system. For example, multimedia user interface108may provide an in-vehicle tutorial112and other information, indications and messages to user104in the form of an audible, audio, voice, visual, video and/or haptic signal. Through multimedia user interface108, tutorial application110may interact with user104to provide in-vehicle tutorial112.

Tutorial application110may be configured to provide tutorial112in vehicle102to user104by leveraging multiple modes of communications. In some implementations, tutorial application110may receive a request from user104, e.g., via multimedia user interface108, for tutorial112on one or more of a number of operational features associated with vehicle102. The request may be in the form of voice command, a touch to a touch-sensing panel of multimedia user interface108, a wireless signal, or any other user-input mechanisms. For example, an operational feature may be an adaptive cruise control (ACC) feature, a hill descent control (HDC) feature, a lane keeping assistance (LKA) feature, a basic park assistance (PA) feature, a lane departure warning (LDW) feature of vehicle102, a change in one or more settings related to a component of vehicle102(e.g., any change in the settings of an infotainment system of vehicle102), a learning feature about an advanced driver assistance system (ADAS), or an autonomous vehicle feature (e.g., autonomous vehicle hand-off between vehicle102and user104). The tutorial112may include multiple tutorial instructions that indicate or suggest expected driving behavior of user104to use one or more operational features of vehicle102.

In some implementations, tutorial application110may simulate expected driving behavior corresponding to an operational feature using the multiple modes of communications, and monitor actual operational behavior of user104. For example, the expected driving behavior may be indicated or suggested by a tutorial instruction of tutorial112, e.g., via an audible, audio, voice, visual, video or haptic indication. Further, tutorial application110may compare the operational behavior of user104with the expected driving behavior of user104. Based on the comparison, tutorial application110may provide feedback to user104using the multiple modes of communications. For example, in an event that the monitored or actual operational behavior of user104differs or deviates from the expected driving behavior, tutorial application110may provide an indication, e.g., via an audible, audio, voice, visual, video or haptic indication, to alert user104of such difference or deviation. Thus, user104may be made aware of the difference or deviation and, in response, make adjustment to his/her operational behavior with respect to the particular operational feature in concern. Accordingly, user104may learn from this experience and improve his/her proficiency and knowledge in terms of using the particular operational feature of vehicle102.

In some implementations, tutorial application110may detect a driving context corresponding to the operational feature of vehicle102, e.g., HDC. Tutorial application110may initiate tutorial112of an operational feature to provide a corresponding tutorial to user104, e.g., via multimedia user interface108. Further, tutorial application110may monitor driving behavior of user104during the proceeding of tutorial112, and compare the driving behavior of user104with expected driving behavior corresponding to the operational feature. Based on the comparison, tutorial application110may provide feedback to user104using the multiple modes of communications.

In-vehicle tutorial system106may include processor(s)202and memory204. The memory204may store various modules, applications, programs, or other data. The memory204may store instructions that, when executed by the processor(s)202, cause the processor(s)202to perform the operations described herein for in-vehicle tutorial system106. In some implementations, processor(s)202may include one or more graphics processing units (GPU) and one or more central processing units (CPU).

In-vehicle tutorial system106may have additional features and/or functionalities. For example, in-vehicle tutorial system106may include additional data storage devices (removable and/or non-removable) including computer-readable media. Computer-readable media may include, at least, two types of computer-readable media, namely computer storage media and communication media. Computer storage media may include volatile and non-volatile, removable, and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, program data, or other data. The system memory, the removable storage and the non-removable storage are all examples of computer storage media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and which can be accessed by in-vehicle tutorial system106. Any such computer storage media may be part of in-vehicle tutorial system106. Moreover, the computer-readable media may include computer-executable instructions that, when executed by the processor(s), perform various functions and/or operations described herein.

Memory204may store an operating system206as well as various components including, for example, tutorial application110and program data208. For example, program data208may store one or more sets of instructions and/or multimedia interaction data pertaining to tutorial112. Tutorial112may include a number of specific tutorials on a corresponding number of operational features of vehicle102.

Tutorial application110may include various modules including, for example, a tutorial module210, an interaction module212and a context module214.

Interaction module212may be configured to receive a request from user104for tutorial112of an operational feature of vehicle102. The operational feature may include ordinary in-vehicle capabilities such as climate and radio adjustment and may further include other vehicle features that involve multiple stages of operation. For example, operational features of vehicle102may include an ACC feature, a HDC feature, an LKA feature, a PA feature, an LDW feature, a change in one or more settings related to a component of vehicle102(e.g., any change in the settings of an infotainment system of vehicle102), a learning feature about an ADAS, or an autonomous vehicle feature (e.g., autonomous vehicle hand-off between vehicle102and user104). Depending on the operational feature being demonstrated, tutorial112may explain, for example in simple terms, the steps of operation for the particular operational feature (e.g., expected driving behavior), applicability of the feature, and expected benefits to user104deriving from the particular operational feature (e.g., safety and/or comfort). For some features (e.g., changing certain settings in vehicle102), tutorial112may present step-by-step guidance towards a goal. For example, a series of animations may be presented via multimedia user interface108to user104to demonstrate an operational feature of vehicle102and associated benefit(s). In some implementations, interaction module212may be capable of speech recognition and natural language processing to receive voice commands and inquiries from user104for tutorial help such as, for example, “How do I change the tires on my car”, “How do I fill gas?”, “How do I check the tire pressure?” and the like.

Tutorial module210may be configured to simulate the expected driving behavior corresponding to the operational feature of vehicle102. For example, tutorial module210may simulate the expected driving behavior corresponding to the operational feature of vehicle102with one or more notifications in the form of one or more audible, audio, voice, visual, video and/or haptic signals including lighting, audio, and video signals.

Tutorial module210may further monitor operational behavior of user104, and compare the operational behavior of user104with the expected driving behavior corresponding to the operational feature. Based on the comparison, tutorial module may generate and provide a feedback to user104, e.g., via multimedia user interface108.

Multimedia user interface108may be configured to display an animation with synchronized audio to user104and present one or more notifications including lighting, audio, and video signals to simulate the expected driving behavior corresponding to the operational feature. Multimedia user interface108may also be configured to provide a haptic signal, e.g., vibration on the steering wheel or another component of vehicle102, as a form of notification to user102. Further, multimedia user interface108may activate or otherwise turn on one or more lights of vehicle102in a demonstration indicating to user104the expected driving behavior corresponding to the operational feature of vehicle102. In these instances, in-vehicle tutorial system106may rely on simple intuitive interactions between user104and multimedia user interface108. For example, in-vehicle tutorial system106may implement initiation and continuation of dialogues between user104and multimedia user interface108using speech recognition and machine learning techniques. Accordingly, the dialogues may be established such as to be simple and precise as close as possible to a naturalistic dialogue between two people.

In some implementations, context module214may determine a status mode of vehicle102. For example, the status mode of vehicle102may be either a parking mode or a drive mode. In response to a determination that the status mode is the parking mode (e.g., vehicle102is parked and not in motion), tutorial module210may simulate the expected driving behavior corresponding to the operational feature of vehicle102. In response to a determination that the status mode is the drive mode (e.g., vehicle102is not parked and may or may not be in motion), tutorial module210may determine whether tutorial112is allowed in the drive mode due to safety concerns and/or relevant traffic rules. If tutorial112is not allowed in the drive mode, interaction module212may generate an error message and present the error message to user104, e.g., in the form of an audible, audio, voice, visual, video and/or haptic signal. If tutorial112is allowed in the drive mode, interaction module212may present an instruction of tutorial112to user104via multimedia user interface108. In some implementations, tutorial module210may initiate tutorial112based on a user request, and then terminate tutorial112either in response to a termination request from user104or without receiving a response from user104within a predetermined time period, e.g., 30 seconds, 1 minute, 3 minutes or any user-definable period.

Suppose that the operational feature includes an ACC. In a parking mode, tutorial application110may receive a request for a tutorial on operational feature ACC via multimedia user interface108. Tutorial application110may present an animation with synchronized audio to user104via multimedia user interface108. The animation may contain small icons that user104may “click on” to resolve further questions or acquire further technical detail regarding the operational feature. For example, similar support may be included with an audio dialog tree. The backlights on the dashboard of vehicle102may also light up or blink along with the demonstration to indicate to user104the expected driving behavior corresponding to the operational feature of ACC on vehicle102while the ACC feature is activated or otherwise turned on. Accordingly, user104may expect certain behaviors in vehicle102while using and being aware of the operational feature of ACC.

In some implementations, context module214may detect a driving context corresponding to an operational feature of a vehicle102. In some implementations, the driving context may include a status mode of vehicle102and an operational context. The status mode may be either a parking mode or a drive mode. The driving context may include, for example, a landform that extends above a surrounding terrain, and the operational feature may include a HDC. The operational context may include, for example, at least one of the following: an environmental condition, a driving condition, a traffic condition and a geographic condition.

In some implementations, tutorial module210may initiate tutorial112of the operational feature to provide tutorial112to user104in vehicle102. Tutorial module210may further monitor driving behavior of user104, determine expected driving behavior corresponding to the operational feature, and compare the driving behavior of user104with the expected driving behavior. Based on the comparison, interaction module212may provide a feedback to user104.

In some implementations, tutorial module210may request an approval for proceeding tutorial112from user104after initiating tutorial112. If interaction module212receives the approval, tutorial module210may proceed with providing an instruction of tutorial112.

Suppose that the operational feature includes HDC. In a drive mode, context module214may detect that vehicle102is traveling on a steep hill without HDC and that the steepness or slope of the hill is, for example, greater than a predetermined value. Over audio instruction, in-vehicle tutorial system106may begin a HDC tutorial. The HDC tutorial may instruct user104to brake at the top of the hill, enable the HDC (e.g., by depressing the button on the gear selector of vehicle102), and release the brake slowly. At the bottom of the hill, the HDC tutorial may instruct user104to turn off the HDC. In some instances, the HDC tutorial may ask user104about whether HDC concept has been learned or if further an instruction is needed.

FIG. 3illustrates an example process300in accordance with the present disclosure. Example process300may be one of various implementation scenarios based on example environment100, and is provided solely for illustrative purpose so that those skilled in the art may better appreciate benefits and advantages provided by the present disclosure. Therefore, the scope of the present disclosure is not limited by example process300. For illustration purpose and not limiting the scope of the present disclosure, the description of example process300is provided below in the context of implementation using processor202of computing apparatus200with respect to vehicle102. Example process300may begin at302.

At302, processor202may receive a request for tutorial112of an operational feature of vehicle102from user104. The operational feature may include ordinary in-vehicle capabilities such as climate and radio adjustment and may further include one or more other vehicle features that involve multiple stages of operation. Example process300may proceed from302to304.

At304, processor202may determine a status mode of vehicle102. For example, the status mode of vehicle102may be either a parking mode or a drive mode. Example process300may proceed from304to306. Alternatively, example process300may proceed from304to308.

At306, processor202may determine one or more features in context of (and hence applicable to) the determined status mode of the vehicle, whether it is the parking mode or drive mode, and proceed to312and simulate an expected driving behavior corresponding to the operational feature of the vehicle with respect to the determined feature. Example process300may then proceed from312to314.

At308, processor202may determine whether the status mode is a parking mode. Processor202may determine the status mode using various sensors associated with vehicle102.

In response to a determination that the status mode is the parking mode (i.e., the “YES” branch of operation308), processor202may proceed to312and simulate expected driving behavior corresponding to the operational feature of vehicle102. For example, processor202may simulate expected driving behavior corresponding to the operational feature of vehicle102with one or more notifications including at least one of lighting, audio, and video signals. Example process300may then proceed from312to314.

In response to a determination that the status mode is not the parking mode (i.e., the “NO” branch of operation308), processor202may proceed to310and generate an error message. For example, processor202may determine that the status mode is a drive mode, processor202may generate an error message. In other implementations, the processor202may further determine whether tutorial112corresponding to the operational feature is allowed in status modes other than the parking mode. If the operational feature (e.g., HDC) is allowed in the status mode (e.g., a drive mode), processor202may proceed with tutorial112.

At314, processor202may monitor operational behavior of user104. In some implementations, after processor202initiates tutorial112, processor202may terminate tutorial112in response to failure of receiving a response from user104for a predetermined time period. Example process300may then proceed from314to316.

At316, processor202may compare the operational behavior with the expected driving behavior corresponding to the operational feature. For example, the expected driving behavior may be indicated or suggested by a tutorial instruction of tutorial112. Example process300may then proceed from316to318.

At318, processor202may provide a feedback to user104based on the comparing. For example, processor202may present an animation with synchronized audio to user104in vehicle102, and turn on one or more lights of the vehicle in a demonstration indicating to the expected driving behavior of user104corresponding to the operational feature of vehicle102.

FIG. 4is a diagram depicting an example process400in accordance with the present disclosure. Example process400may be one of various implementation scenarios based on example environment100, and is provided solely for illustrative purpose so that those skilled in the art may better appreciate benefits and advantages provided by the present disclosure. Therefore, the scope of the present disclosure is not limited by example scenario400. For illustration purpose and not limiting the scope of the present disclosure, the description of example process400is provided below in the context of implementation using processor202of computing apparatus200with respect to vehicle102.

At402, processor202may detect a driving context corresponding to an operational feature of vehicle102. For example, the driving context may include a status mode of vehicle102and an operational context, and the status mode may be either a parking mode or a drive mode. In some implementations, the operational context may include at least one of an environmental condition, a driving condition, a traffic condition, or a geographic condition. For example, the driving context may include a landform that extends above a surrounding terrain, and the operational feature may include a HDC. Example process400may proceed from402to404.

At404, processor202may initiate tutorial112of the operational feature to provide tutorial112to user104. For example, processor202may request an approval for tutorial112from user104. In response to receiving the approval from the user104, processor202may provide an instruction of tutorial112to user104. Example process400may proceed from404to406.

At406, processor202may monitor driving behavior of user104. For example, processor202may monitor driving behavior of user104by collecting driving data from various sensors (e.g., pressure or proximity sensors) and/or other devices (e.g., Global Positioning System (GPS)) associated with vehicle102. In some implementations, one or more sensors may be utilized under the OpenXC architecture and connected to processor202and connected to an in-vehicle network such as, for example, a controller area network (CAN) bus, e.g., through on-board diagnostics (OBD) ports such as OBD2 ports, to monitor various parameters. Example process400may proceed from406to408.

At408, processor202may determine expected driving behavior corresponding to the operational feature. For example, the expected driving behavior may be stored in program data208of in-vehicle tutorial system106. Example process400may proceed from408to410.

At410, processor202may compare the driving behavior with expected driving behavior corresponding to the operational feature. Example process400may proceed from410to412.

At412, processor202may provide a feedback to user104based on the comparing. The feedback may be presented via multimedia user interface108.

The articles “a” and “an” are used herein to refer to one or to more than one (i.e., to at least one) of the grammatical object of the article. By way of example, “a user” means one user or more than one users. Reference throughout this specification to “one embodiment,” “an embodiment,” “one example,” or “an example” means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment of the present disclosure. Thus, appearances of the phrases “in one embodiment,” “in an embodiment,” “one example,” or “an example” in various places throughout this specification are not necessarily all referring to the same embodiment or example. Furthermore, the particular features, structures, databases, or characteristics may be combined in any suitable combinations and/or sub-combinations in one or more embodiments or examples. In addition, it should be appreciated that the figures provided herewith are for explanation purposes to persons ordinarily skilled in the art and that the drawings are not necessarily drawn to scale.

The flow diagrams and block diagrams in the attached figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It will also be noted that each block of the block diagrams and/or flow diagrams, and combinations of blocks in the block diagrams and/or flow diagrams, may be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. These computer program instructions may also be stored in a computer-readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instruction means which implement the function/act specified in the flow diagram and/or block diagram block or blocks.

Although the present disclosure is described in terms of certain embodiments, other embodiments will be apparent to those of ordinary skill in the art, given the benefit of this disclosure, including embodiments that do not provide all of the benefits and features set forth herein, which are also within the scope of this disclosure. It is to be understood that other embodiments may be utilized, without departing from the scope of the present disclosure.