Patent Description:
In the field of automotive technology, in addition to the two major issues of safety and comfort, more and more attention is paid to user experience (UX). In order to provide a better UX, the ongoing research focuses on human-machine interaction technologies and can deliver basic interaction functions between a vehicle and a user of the vehicle, for example, the user can learn vehicle status information and road condition information from an HMI of the vehicle, and also can activate vehicle functions such as cruise control, self-driving in high-way, and automatic parking by interacting with the HMI. There is increased expectation that the vehicle should provide customized services for the user.

<CIT> and <CIT> both specify an editor executed on an external device for generating rules in a rule-based system (IF condition for state variable THEN action) for a vehicle.

Further preferred embodiments are defined by the dependent claims.

The disclosed aspects will hereinafter be described in connection with the appended drawings that are provided to illustrate but not to limit the scope of the disclosure.

Examples of the disclosure relate to vehicle-user interaction systems and methods, which are implemented based on an on-board system having an Android operating system such as AAOS (Android Automotive Operation System). Using the vehicle-user interaction systems and methods, a user can program his/her vehicle. For example, the user can edit a custom rule on one or more of a mobile phone, a web page, and an on-board unit (OBU). Then, the rule is synchronized to the on-board system (e.g., the infotainment system of the vehicle). Then, the rule is executed at the on-board system. In this way, the custom rule can be edited inside or outside the vehicle, and then executed inside the vehicle. This allows the vehicle to provide customized services.

According to embodiments of the disclosure, on one hand, the user is provided with a visual programming platform such that the user can program the rule on the platform, and on the other hand, the content of the rule that the user can program is controlled. Specifically, the content that the user can input to the visual programming platform is limited due to the consideration of vehicle safety. In this way, the vehicle can provide user-customized services while ensuring vehicle safety, and to a certain extent, customizing a vehicle from the perspective of user experience is realized.

Embodiments of the disclosure will be described below in connection with the drawings.

<FIG> schematically illustrates a vehicle-user interaction system <NUM> for a vehicle V according to an embodiment of the disclosure. The vehicle-user interaction system <NUM> includes an editor <NUM>, a storage unit <NUM>, a parser <NUM>, and an actuator <NUM>.

The editor <NUM> is provided in at least one of an electronic device of the user and an infotainment system of the vehicle V. The operating system of the electronic device can be either an Android operating system or an IOS operating system (see 10A, 10B, 10C in <FIG>). That is to say, the editing process can be performed on an electronic device having an Android operating system or an Apple iOS operating system. The electronic device, for example, includes one or more of an Android phone, an Apple phone, a laptop computer, an Android or Apple tablet, an Android or Apple wearable device (such as i-watch), and a desktop computer.

In some examples, the editor <NUM> includes one or more editors disposed at one or more of the electronic devices and the infotainment system. For example, the editor <NUM> is implemented as software (e.g., an APP), and the software is disposed in the mobile phone, the tablet, the wearable device, the laptop computer, and the desktop computer. In this way, the user can edit a custom rule on one or more of an APP, a web page, and an OBU (on-board unit).

It is noted that the electronic device refers to an electronic device that is not part of the vehicle V. The physical position of the electronic device is not limited. The electronic device can be within the vehicle V, for example, the electronic device such as a smartphone is carried by the user who is inside the vehicle V. The position of the electronic device also can be outside the vehicle, for example, the electronic device such as a smartphone is carried by the user who is outside the vehicle V.

The editor <NUM> receives one or more inputs from the user and edits the inputs into a rule script that includes a user-defined rule. The user-defined rule includes a trigger condition and a vehicle operation that will be performed if the trigger condition is satisfied. In an example, the trigger condition includes environment information and/or vehicle status information, and the vehicle operation includes vehicle functions that will be activated if the trigger condition is met. For example, a user-defined rule could be "if an ambient temperature is above <NUM>, close vehicle windows and turn on air conditioning".

It is noted that, as described above, personalization and customization can be realized for individual users; however, the content of the personalization and customization is controlled due to vehicle safety concerns, that is to say, the user is not allowed to program any rule as he or she wishes, which is due to the consideration of vehicle safety.

The storage unit <NUM> is provided in the infotainment system of the vehicle V for storing the rule script which has been edited at the editor. The edited rule script is transmitted to the storage unit <NUM> via a wired or wireless communication link. In an example, the edited rule script is directly transmitted to the storage unit <NUM> through a near-field communication link. In another example, the edited rule script is uploaded to a cloud server and then transferred to the storage unit <NUM> from the cloud server.

The parser <NUM> is provided in the infotainment system. The parser <NUM> acquires the edited rule script from the storage unit <NUM> and creates a list of monitoring elements and a list of functional elements based on the acquired rule script. The list of monitoring elements includes sensor elements for directly and/or indirectly sensing the trigger condition. The list of functional elements includes functional elements for executing the vehicle operation.

The actuator <NUM> is provided in the infotainment system. The actuator <NUM> monitors sensor detection information associated with the sensor elements. For example, if the trigger condition includes an ambient temperature, the corresponding sensor detection information may include detected ambient temperature information and/or other detected information based on which the ambient temperature can be calculated. The actuator <NUM> determines whether the trigger condition is satisfied based on the sensor detection information. If it is determined the trigger condition is satisfied, the actuator <NUM> executes corresponding functional elements to implement the user-defined rule in the vehicle V such as turning on air conditioning and closing the windows.

In some examples, each of the editor <NUM>, the parser <NUM>, and the actuator <NUM> is each implemented as an individual software module that communicates with each other. In another example, the editor <NUM>, the parser <NUM>, and the actuator <NUM> are combined together to form one integrated software module.

In some examples, the editor <NUM>, the parser <NUM>, and the actuator <NUM> are all be provided in the infotainment system of the vehicle V.

In some examples, the editor <NUM> includes two or more editors provided in the infotainment system and the electronic device respectively. The parser <NUM> is provided in the infotainment system, and the actuator <NUM> is provided in the infotainment system.

In some examples, the operating system of the infotainment system is the Android Automotive OS, i.e., Android Automotive Operation System (AAOS).

<FIG> schematically illustrate embodiments of the editor <NUM>, the parser <NUM>, and the actuator <NUM>.

With reference to <FIG>, the editor <NUM> includes a database <NUM>, an editing interface <NUM>, and an editing module <NUM>.

The database <NUM> is used to store interface elements. The interface elements include elements that will be presented on the editing interface <NUM> and can receive user inputs. The interface elements, for example, include elements such as borders, colors, symbols, values, and the like.

The editing interface <NUM> includes four sub-interfaces, i.e., a first sub-interface <NUM>, a second sub-interface <NUM>, a third sub-interface <NUM>, and a fourth sub-interface <NUM>.

In some examples, interface elements presented on the editing interface <NUM> can be touch-sensitive, not touch-sensitive, or controlled by voice or hand gestures.

In the example of elements that are touch-sensitive, those elements are capable of understanding finger gestures and actions such as touching, tapping, sliding, touching with handwriting, or combinations thereof.

In the example of elements that are not touch-sensitive, those elements can receive an input signal from another device such as a mouse or keyboard. For example, an icon on the editing interface <NUM> that is not touch-sensitive can be selected by receiving a mouse click.

In the example of elements that are controlled by voice, the user can read out an interface element presented on the editing interface <NUM> to select that interface element.

The editing module <NUM> edits the user inputs into a rule script according to a pre-defined expression rule and a predetermined format. The rule script having the predetermined format is transmitted to the storage unit and stored in the storage unit.

With reference to <FIG>, the parser <NUM> includes an acquiring module <NUM>, a parsing module <NUM>, and a management module <NUM>. The acquiring module <NUM> acquires the rule script from the storage unit <NUM>. The parsing module <NUM> converts the acquired rule script into an expression configured to run on operating system of the infotainment system, such as the AAOS and then inputs the expression into the management module <NUM>. The management module <NUM> creates the list of monitoring elements and the list of functional elements according to the expression.

With reference to <FIG>, the actuator <NUM> includes a monitoring module <NUM> and a triggering module <NUM>. The monitoring elements created in the management module <NUM> are registered into the monitoring module <NUM> so that the monitoring module <NUM> monitors corresponding sensor detection information. The monitoring module <NUM> determines whether the trigger condition is satisfied based on the sensor detection information. In an example, if the trigger condition is satisfied, the triggering module <NUM> triggers the management module <NUM> to make a response, and then the triggering module <NUM> triggers a corresponding vehicle function to realize the user-defined rule. In another example, if the trigger condition is satisfied, the triggering module <NUM> triggers the corresponding vehicle function directly without triggering the management module <NUM> to make a response.

It is noted that the editor <NUM>, the parser <NUM> and the actuator <NUM> can be implemented by means of hardware or software or a combination of hardware and software, including code stored in a non-transitory computer-readable medium and implemented as instructions executed by a processor. Regarding implementation by means of hardware, it may be implemented in an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a data signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a processor, a controller, microcontroller, a microprocessor, an electronic unit, or a combination thereof. The implementation by software may include microcode, program code or code segments. The software may be stored in a machine-readable storage medium, such as a memory.

It is noted that the editor <NUM>, parser <NUM> and actuator <NUM> are named based on their functions, and their physical forms or arrangements are not limited by their names.

<FIG> shows an editing process <NUM> according to an embodiment of the disclosure. This editing process <NUM> can be implemented by the editor <NUM>.

In an example, an initialization process (not shown) can be performed before performing the process <NUM>. For example, an APP in which the editor <NUM> is disposed is activated in the initialization process.

With reference to <FIG>, at block <NUM>, a first sub-interface <NUM> is presented on the editing interface <NUM>. With reference to <FIG>, the first sub-interface <NUM> includes a status element window <NUM> in which one or more status elements are presented. Those status elements can be implemented by means of one or more visual programming components encapsulated by the AAOS. In the status element window <NUM>, one of the one or more status elements is selected by receiving a user input. For example, the user selects a status element of humidity by touching the status element of humidity.

Each status element represents a class of status. In some examples, the status elements include one or more of the following elements: time, location, alarm time, email, ambient temperature, cabin temperature, calendar, Bluetooth, instant message, IoT device (e.g., a smart home device, a wearable device, or a road-side unit), vehicle status information (e.g., vehicle speed, vehicle acceleration/deceleration, energy consumption, or remaining range), and environmental status information (e.g., rainfall, brightness, visibility, or wind speed).

At block <NUM>, a first user input is received via the first sub-interface <NUM> such that the user selects one of the status elements. For example, the user touches a status element on the first sub-interface <NUM> to select that status element, or the user reads out a status element on the first sub-interface <NUM> to select that status element.

At block <NUM>, the second sub-interface <NUM> is presented on the editing interface <NUM> in response to the user selection. With reference to <FIG>, the second sub-interface <NUM> includes a condition setting window <NUM>. The condition setting window <NUM> includes interface elements for setting the trigger condition. For example, the interface elements for setting the trigger condition include an element J for setting a judgment condition, an equation symbol S for setting an equality or inequity expression and an element V for setting a threshold value. The equation symbol includes a selection of "greater than", "equal to", "less than," "greater than," "less than or equal to", "greater than or equal to", "not equal to," "inclusive" and "exclusive" listed in a drop-down menu of the element S for user selection. The user can select a condition from the drop-down menu of element J as the specific judgment condition, such as temperature and select "greater than or equal to" as the equation symbol. The threshold value is a value corresponding to the selected status element. For example, the threshold value corresponding to a status element of temperature is a temperature value such as <NUM>.

At block <NUM>, a second user input is received at the second sub-interface <NUM> so that the user sets the trigger condition. For example, user inputs are received at the elements J, S and V, thereby completing the setting of the trigger condition. For example, a user input of "T" is received at element J, "≥" is received at the element S, another user input of "<NUM>" is received at the element T, and a trigger condition of "if an ambient temperature being greater than <NUM>" is completed.

The second sub-interface <NUM> can also include an "OK" button. When the "OK" button receives a user input, it indicates the setting of the trigger condition has been completed, and subsequent operations will be performed.

At block <NUM>, the third sub-interface <NUM> is presented on the editing interface <NUM>. Referring to <FIG>, the third sub-interface <NUM> includes an operation element window <NUM>. The operation element window <NUM> includes one or more operation elements. The one or more operation elements are implemented by means of one or more visual programming components encapsulated by the AAOS. The user can select one of the operation elements by touching or reading out said operation element.

Each operation element represents a class of vehicle operation/function. In some examples, the operation elements include one or more of the following elements: turning on interior or exterior lighting, turning on air conditioning, locking vehicle doors and windows, controlling the vehicle speed, adjusting the rearview mirror, adjusting the seat position, changing the display on HUD (head-up display) or dashboard, starting navigation, connecting to an IoT device, playing multimedia, and activating lane centering.

At block <NUM>, a third user input is received at the third sub-interface <NUM> for selecting an operation element. For example, the user selects an operation element on the third sub-interface <NUM> by touching or reading out that operation element.

At block <NUM>, the fourth sub-interface <NUM> is presented on the editing interface <NUM>. With reference to <FIG>, the fourth sub-interface <NUM> includes an operation setting window <NUM>. The operation setting window <NUM> includes one or more interface elements, for example, interface elements A-C.

At block <NUM>, a fourth user input is received at the fourth sub-interface <NUM> for setting the vehicle operation. For example, the user input to element A is "closing vehicle windows", the user input to element B is "setting the air conditioning", and the user input to element C is "<NUM>".

The fourth sub-interface <NUM> also includes an "OK" button. When the "OK" button receives a user input, it indicates the setting of the vehicle operation has been completed, and subsequent operations will be performed.

It is noted that the user interacts with interface elements on the sub-interfaces in sequence. For example, when the user interaction on the first sub-interface is completed, the user is directed to the second sub-interface. Only when the user interaction on the second sub-interface is completed is the user directed to the third sub-interface. After the user interaction with interface elements on the third sub-interface is complete, the user can navigate to the fourth sub-interface.

It is noted that interface elements presented on each of the sub-interfaces and how they are configured on the sub-interface can be personalized and flexibly adjusted. For example, one or more interface elements on a sub-interface can be hidden, added, or deleted, and the configuration, such as position, color or size of the one or more interface elements can be changed.

At block <NUM>, the editing module <NUM> edits the user inputs into a rule script using a predetermined expression rule, for example, a regular expression rule.

In some examples, the predetermined expression rule includes an expression consisting of one or more meta-characters and custom characters. The expression supports nested functions and recursive functions. The one or more meta-characters include operators and factors. The operators and factors are predefined in the editing module. The one or more custom characters can be defined by the user.

The operators include one or more of "IF", "THEN", "ELSE", "AND", "OR", and "NOT". The factors include one or more visual programming components encapsulated by the AAOS. In some examples, the factors include visual programming components of "Status", "Action", and "Preset (e.g., Pre-process)".

The custom characters include characters input by the user and expression names such as names of other pre-defined expressions.

In some examples, the editing module <NUM> converts the rule script into text information with a predetermined format for transmission.

The predetermined format includes three portions, i.e., a first portion and a second portion having the same prefixed data length, and a third portion having a variable data length. The length of both the first and second portions is <NUM> bits. The length of the third portion is variable. The length information of the third portion is recited in a field of "Data Packet Length". The length of third portion (i.e., a data portion) is in a range from <NUM> to <NUM> (i.e., <NUM>^<NUM>-<NUM>).

The first portion includes two or more fields. For example, the first portion sequentially includes a header having a length of <NUM> bits, an editor version having a length of <NUM> bits, an AAOS version having a length of <NUM> bits, and an identification having a length of <NUM> bits.

The second portion includes two or more fields. For example, the second portion sequentially includes a data packet having a length of <NUM> bits and a reservation having a length of <NUM> bits.

The third portion includes a data portion having a variable length.

Definitions and examples of the predetermined format for the expression are described in the tables below. Table <NUM> illustrates an exemplary format of the expression. Table <NUM> illustrates an exemplary format of the data portion of the expression in Table <NUM>. Tables <NUM>-<NUM> illustrate exemplary formats of sub-portions of the data portion in Table <NUM>. Table <NUM> illustrates an exemplary format of the beginning portion of the expression. Table <NUM> illustrates an exemplary format of the ending portion of the expression. Table <NUM> illustrates an exemplary format of operators of the expression. Table <NUM> illustrates an exemplary format of names of the expression, which can be defined by the user. Table <NUM> illustrates an exemplary format of preset operations of the expression, which can be defined by the user. Table <NUM> illustrates an exemplary format of trigger conditions of the expression, which can be defined by the user. Table <NUM> illustrates an exemplary format of actions of the expression, which can be defined by the user. The values in parentheses in the tables below are numbers of bits.

<FIG> illustrates a parsing and execution process <NUM> according to an embodiment of the disclosure. This process <NUM> can be implemented by the parser <NUM> and the actuator <NUM>.

Before performing the process <NUM>, the rule script having the predetermined format is stored in the storage unit <NUM>. Then, the parser <NUM> is notified that there is an update in the storage unit <NUM>.

With reference to <FIG>, at block <NUM>, the acquiring module <NUM> acquires the edited rule script having the predetermined format from the storage unit <NUM>.

At block <NUM>, the parsing module <NUM> translates the rule script into an expression that can run on the AAOS.

At block <NUM>, the management module <NUM> creates a list of monitoring elements and a list of functional elements based on the expression. The list of monitoring elements includes sensor elements that directly or indirectly sensing the user-defined trigger condition. The list of functional element includes functional elements associated with the user-defined vehicle operation.

At block <NUM>, the management module <NUM> inputs the monitoring elements into the monitoring module <NUM>.

At block <NUM>, the monitoring module <NUM> monitors sensor detection information.

At block <NUM>, the monitoring module <NUM> determines whether the user-defined trigger condition is satisfied based on the sensor detection information.

If it is determined that the trigger condition is not satisfied, the process <NUM> turns back to block <NUM> and continues to monitor the sensor detection information.

If it is determined that the trigger condition is satisfied, the process <NUM> proceeds to block <NUM>. At block <NUM>, the management module <NUM> makes a response to the determination of the trigger condition being satisfied.

At block <NUM>, the trigger module <NUM> triggers the vehicle operation so that the user-defined rule is executed.

For clarity, an example of blocks <NUM>-<NUM> is described with an example of "monitoring an ambient temperature and performing operations of closing vehicle windows and turning on the air conditioning. " In this example, the monitoring module <NUM> determines whether the user-defined condition is satisfied, i.e., the module <NUM> determines whether the ambient temperature detected by an environmental sensor is equal to or greater than <NUM>. If it is the ambient temperature is below <NUM>, the monitoring module <NUM> continues to monitor the ambient temperature. If it is determined the ambient temperature is equal to or greater than <NUM>, the management module <NUM> makes a response. Then, in response to the response of the management module <NUM>, the trigger module <NUM> triggers a vehicle operation of closing vehicle windows and turning on the air conditioning.

<FIG> shows a flowchart of a vehicle-user interaction method <NUM> according to an embodiment of present disclosure. The method can be executed by the system <NUM>. Therefore, the above descriptions about the system <NUM> are also applicable here.

Referring to <FIG>, in step S610, one or more inputs are received from a user of the vehicle at the editor <NUM>.

In step S620, the one or more inputs are edited into a rule script at the editor. The rule script includes a user-defined rule based on the one or more inputs. The user-defined rule has a trigger condition and a vehicle operation performed when the trigger condition is satisfied.

In step S630, a list of monitoring elements and a list of functional elements are created at the parser <NUM> based on the rule script. The list of monitoring elements includes sensor elements that directly or indirectly sense the trigger condition. The list of functional elements includes functional elements that are associated with the vehicle operation.

In step S640, sensor detection information associated with the sensor elements is monitored at the actuator <NUM>.

In Step S650, it is determined whether the trigger condition is satisfied at the actuator based on the monitored sensor detection information.

In Step S660, if it is determined the trigger condition is satisfied, the functional elements are executed at the actuator <NUM>, to implement the user-defined rule in the vehicle.

The disclosure provides a non-transitory computer-readable medium with instructions stored therein which, when executed, causes a processor to carry out the steps of the method <NUM> described above.

It is noted that all the operations in the method and process described above are examples for putting the claimed subject-matter into practice.

The processors can be implemented using electronic hardware, computer software, or any combination thereof. Whether these processors are implemented as hardware or software will depend on the specific application and the overall design constraints imposed on the system. By way of example, a processor, any portion of a processor, or any combination of processors presented in this disclosure may be implemented as a microprocessor, a micro-controller, a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a state machine, a gate logic, a discrete hardware circuitry, and other suitable processing components configured to perform the various functions described in this disclosure. The functions of a processor, any portion of a processor, or any combination of processors presented in this disclosure may be implemented as software executed by a microprocessor, a micro-controller, a DSP, or other suitable platforms.

Software should be interpreted broadly to represent instructions, instruction sets, code, code segments, program code, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, running threads, processes, functions, and the like. Software can reside on a non-transitory computer-readable medium. Such non-transitory computer-readable medium may include, for example, a memory, which may be, for example, a magnetic storage device (e.g., a hard disk, a floppy disk, a magnetic strip), an optical disk, a smart card, a flash memory device, a random access memory (RAM), a read only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a register, or a removable disk. Although a memory is shown as being separate from the processor in various aspects presented in this disclosure, a memory may also be internal to the processor (e.g., a cache or a register).

Claim 1:
A vehicle-user interaction system for a vehicle, comprising:
an editor (<NUM>) configured to receive one or more inputs from a user of the vehicle and edit the one or more inputs into a rule script, the rule script comprising a user-defined rule based on the one or more inputs, the user-defined rule defining a trigger condition and a vehicle operation performed when the trigger condition is satisfied;
a parser (<NUM>) configured to create a list of monitoring elements and a list of functional elements based on the rule script, the list of monitoring elements comprising sensor elements that directly or indirectly sense the trigger condition, and the list of functional elements comprising functional elements that are associated with the vehicle operation; and
an actuator (<NUM>) configured to monitor sensor detection information associated with the sensor elements, determine whether the trigger condition is satisfied based on the monitored sensor detection information, and execute the functional elements to implement the user-defined rule in the vehicle if it is determined the trigger condition is satisfied,
wherein the editor (<NUM>) comprises an editing module configured to edit the one or more inputs according to a regular expression rule and a predetermined format to generate the rule script,
wherein the predetermined format comprises first and second portions each having a fixed data length, and a third portion having a variable data length, and
wherein the first portion sequentially includes a header having a length of <NUM> bits, an editor version having a length of <NUM> bits, an infotainment operating system version having a length of <NUM> bits, and an identification having a length of <NUM> bits;
the second portion sequentially includes a data packet having a length of <NUM> bits and a reserved field having a length of <NUM> bits; and
the third portion comprises a data field having a variable length.