VEHICLE COMPONENT IDENTIFICATION SYSTEM

A vehicle component is identified based on a gaze direction of an occupant in a vehicle seat. A first direction of the vehicle seat is determined relative to a forward-facing direction of a vehicle. A second direction is determined from the vehicle seat to the vehicle component. Then, the vehicle seat is rotated based on an angle between the first direction and the second direction.

BACKGROUND

A vehicle may include components that allow occupants to face one another during operation of the vehicle. As one example, a vehicle may be autonomously operated, allowing occupants of the vehicle to ride in the vehicle without any of them operating the vehicle. For example, the autonomous vehicle may include seats free to rotate between rides of the vehicle between forward-facing and rearward-facing positions.

A number and complexity of vehicle components that are available to occupants have increased due to vehicle systems becoming more advanced. For example, an instrument panel often includes a multitude of buttons, knobs, touch screens, etc. As the number and complexity of the vehicle components increases, the instrument panel may be unable to support all of the vehicle components such that at least some of the vehicle components may be mounted at various positions throughout the vehicle cabin.

DETAILED DESCRIPTION

A system includes a computer including a processor and a memory, the memory storing instructions executable by the processor to identify a vehicle component based on a gaze direction of an occupant in a vehicle seat. The instructions further include instructions to determine a first direction of the vehicle seat relative to a forward-facing direction of a vehicle. The instructions further include instructions to determine a second direction from the vehicle seat to the vehicle component. The instructions further include instructions to then rotate the vehicle seat based on an angle between the first direction and the second direction.

The instructions can further include instructions to, upon determining a gaze point based on the gaze direction, determine a third direction from the gaze point to the vehicle component.

The instructions can further include instructions to actuate a speaker including an audio cue specifying the third direction.

The instructions can further include instructions to actuate the vehicle component towards the gaze point.

The instructions can further include instructions to actuate an indicator including a visual cue specifying the third direction.

The instructions can further include instructions to, upon identifying the vehicle component, actuate a light on the vehicle component.

The instructions can further include instructions to identify the vehicle component based further on receiving an occupant input specifying the vehicle component.

The instructions can further include instructions to identify the vehicle component based further on detecting an occupant gesture.

The instructions can further include instructions to actuate the vehicle component towards the first direction to align the first direction and the second direction.

The instructions can further include instructions to actuate a haptic output device upon detecting a hand of the occupant within a distance threshold of the vehicle component, wherein the haptic output device is on at least one of an occupant device and the vehicle seat.

The instructions can further include instructions to actuate at least one of a speaker and an indicator upon detecting a hand of the occupant on the vehicle component via a capacitive sensor, wherein the speaker and the indicator each includes a cue specifying one or more controls of the vehicle component.

The instructions can further include instructions to actuate a haptic output device on the vehicle component upon detecting a hand of the occupant on the vehicle component.

The instructions can further include instructions to prevent rotation of the vehicle seat based on the angle being less than a threshold.

The instructions can further include instructions to prevent rotation of the vehicle seat based on the vehicle component being on the vehicle seat.

A method including identifying a vehicle component based on a gaze direction of an occupant in a vehicle seat. The method further including determining a first direction of the vehicle seat relative to a forward-facing direction of a vehicle. The method further including determining a second direction from the vehicle seat to the vehicle component. The method further including then rotating the vehicle seat based on an angle between the first direction and the second direction.

The method can further include, upon determining a gaze point based on the gaze direction, determining a third direction from the gaze point to the vehicle component.

The method can further include actuating a speaker including an audio cue specifying the third direction.

The method can further include identifying the vehicle component based further on receiving an occupant input specifying the vehicle component.

The method can further include preventing rotation of the vehicle seat based on the angle being less than a threshold.

The method can further include preventing rotation of the vehicle seat based on the vehicle component being on the vehicle seat.

Further disclosed herein is a computing device programmed to execute any of the above method steps. Yet further disclosed herein is a computer program product, including a computer readable medium storing instructions executable by a computer processor, to execute an of the above method steps.

With initial reference toFIGS. 1-4, an example vehicle control system100includes a vehicle computer110programmed to identify a vehicle component125based on a gaze direction G of an occupant in a vehicle seat200. The vehicle computer110is further programmed to determine a first direction A of the vehicle seat200relative to a forward-facing direction B of a vehicle105. The vehicle computer110is further programmed to determine a second direction C from the vehicle seat200to the vehicle component125. The vehicle computer110is further programmed to then rotate the vehicle seat200based on an angle β between the first direction A and the second direction C.

A vehicle105occupant may provide input to the vehicle computer110via a vehicle component125. Typically, a vehicle component125is associated with one or more specific operations, e.g., a knob for adjusting a volume of a radio, a knob for adjusting the climate in a vehicle105cabin, a switch for adjusting a window position, etc. With increasing number of vehicle components125in vehicles, the vehicle components125may be mounted at various positions throughout a vehicle105cabin, e.g., on an instrument panel, on a door panel, on an overhead console, etc. Advantageously, upon determining the vehicle component125based on the gaze direction G of the occupant, the vehicle computer110can rotate the seat200to face the vehicle component125, which can reduce packaging constraints for the vehicle components125by allowing the occupant to access vehicle components125throughout the vehicle105cabin while the occupant is seated in the seat200.

Turning now toFIG. 1, the vehicle105includes the vehicle computer110, sensors115, actuators120to actuate various vehicle components125, and a vehicle communications module130. The communications module130allows the vehicle computer110to communicate with an occupant device140, e.g., via a messaging or broadcast protocol such as Dedicated Short Range Communications (DSRC), cellular, and/or other protocol that can support vehicle-to-vehicle, vehicle-to infrastructure, vehicle-to-cloud communications, or the like, and/or via a packet network135.

The vehicle computer110includes a processor and a memory such as are known. The memory includes one or more forms of computer-readable media, and stores instructions executable by the vehicle computer110for performing various operations, including as disclosed herein.

The vehicle computer110may operate the vehicle105in an autonomous, a semi-autonomous mode, or a non-autonomous (or manual) mode. For purposes of this disclosure, an autonomous mode is defined as one in which each of vehicle105propulsion, braking, and steering are controlled by the vehicle computer110; in a semi-autonomous mode the vehicle computer110controls one or two of vehicle105propulsion, braking, and steering; in a non-autonomous mode a human operator controls each of vehicle105propulsion, braking, and steering.

The vehicle computer110may include programming to operate one or more of vehicle105brakes, propulsion (e.g., control of acceleration in the vehicle105by controlling one or more of an internal combustion engine, electric motor, hybrid engine, etc.), steering, transmission, climate control, interior and/or exterior lights, horn, doors, etc., as well as to determine whether and when the vehicle computer110, as opposed to a human operator, is to control such operations.

The vehicle computer110may include or be communicatively coupled to, e.g., via a vehicle communications network such as a communications bus as described further below, more than one processor, e.g., included in electronic controller units (ECUs) or the like included in the vehicle105for monitoring and/or controlling various vehicle components125, e.g., a transmission controller, a brake controller, a steering controller, etc. The vehicle computer110is generally arranged for communications on a vehicle communication network that can include a bus in the vehicle105such as a controller area network (CAN) or the like, and/or other wired and/or wireless mechanisms.

Via the vehicle105network, the vehicle computer110may transmit messages to various devices in the vehicle105and/or receive messages (e.g., CAN messages) from the various devices, e.g., sensors115, an actuator120, ECUs, etc. Alternatively, or additionally, in cases where the vehicle computer110actually comprises a plurality of devices, the vehicle communication network may be used for communications between devices represented as the vehicle computer110in this disclosure. Further, as mentioned below, various controllers and/or sensors115may provide data to the vehicle computer110via the vehicle communication network.

Vehicle105sensors115may include a variety of devices such as are known to provide data to the vehicle computer110. For example, the sensors115may include Light Detection And Ranging (LIDAR) sensor(s)115, etc., disposed on a top of the vehicle105, behind a vehicle105front windshield, around the vehicle105, etc., that provide relative locations, sizes, and shapes of objects surrounding the vehicle105. As another example, one or more radar sensors115fixed to vehicle105bumpers may provide data to provide locations of the objects, second vehicles, etc., relative to the location of the vehicle105. The vehicle sensors115may further include camera sensor(s)115, e.g. front view, side view, rear view, etc., providing images from a field of view inside and/or outside the vehicle105. In the context of this disclosure, an object is a physical, i.e., material, item that has mass and that can be represented by physical phenomena (e.g., light or other electromagnetic waves, or sound, etc.) detectable by sensors115. Thus, the vehicle105, as well as other items including as discussed below, fall within the definition of “object” herein.

The vehicle105actuators120are implemented via circuits, chips, or other electronic and or mechanical components that can actuate various vehicle subsystems in accordance with appropriate control signals as is known. The actuators120may be used to control components125, including braking, acceleration, and steering of a vehicle105.

In the context of the present disclosure, a vehicle component125is one or more hardware components adapted to perform a mechanical or electro-mechanical function or operation—such as moving the vehicle105, slowing or stopping the vehicle105, steering the vehicle105, etc. Non-limiting examples of components125include a propulsion component (that includes, e.g., an internal combustion engine and/or an electric motor, etc.), a transmission component, a steering component (e.g., that may include one or more of a steering wheel, a steering rack, etc.), a suspension component125(e.g., that may include one or more of a damper, e.g., a shock or a strut, a bushing, a spring, a control arm, a ball joint, a linkage, etc.), a brake component, a park assist component, an adaptive cruise control component, an adaptive steering component, one or more passive restraint systems (e.g., airbags), one or more occupant input devices (e.g., knobs, buttons, switches, levers, touchscreens, etc.), a movable seat etc.

In addition, the vehicle computer110may be configured for communicating via a vehicle-to-vehicle communication module130or interface with devices outside of the vehicle105, e.g., through a vehicle-to-vehicle (V2V) or vehicle-to-infrastructure (V2X) wireless communications (cellular and/or DSRC., etc.) to another vehicle and/or to an occupant device140. The communications module130could include one or more mechanisms, such as a transceiver, by which the computers110of vehicles105may communicate, including any desired combination of wireless (e.g., cellular, wireless, satellite, microwave and radio frequency) communication mechanisms and any desired network topology (or topologies when a plurality of communication mechanisms are utilized). Exemplary communications provided via the communications module130include cellular, Bluetooth, IEEE 802.11, dedicated short range communications (DSRC), and/or wide area networks (WAN), including the Internet, providing data communication services.

The network135represents one or more mechanisms by which a vehicle computer110may communicate with remote computing devices, e.g., the occupant device140, another vehicle computer, etc. Accordingly, the network135can be one or more of various wired or wireless communication mechanisms, including any desired combination of wired (e.g., cable and fiber) and/or wireless (e.g., cellular, wireless, satellite, microwave, and radio frequency) communication mechanisms and any desired network topology (or topologies when multiple communication mechanisms are utilized). Exemplary communication networks include wireless communication networks (e.g., using Bluetooth®, Bluetooth® Low Energy (BLE), IEEE 802.11, vehicle-to-vehicle (V2V) such as Dedicated Short Range Communications (DSRC), etc.), local area networks (LAN) and/or wide area networks (WAN), including the Internet, providing data communication services.

An occupant device140can be a conventional computing device, i.e., including one or more processors and one or more memories, programmed to provide operations such as disclosed herein. The occupant device140can be a portable device. A portable device can be any one of a variety of computers that can be used while carried by a person, e.g., a smartphone, a tablet, a personal digital assistant, a smart watch, etc. Further, the occupant device140can be accessed via the network135, e.g., the Internet or some other wide area network.

The vehicle computer110can identify an occupant approaching the vehicle105via image data received from one or more sensors115, e.g., by using computer vision techniques. Additionally, the vehicle computer110can identify cargo carried by the occupant, e.g., to be transported in the vehicle105. For example, the vehicle computer110can apply object recognition techniques, such as are known, to image data to identify the cargo, e.g., a package, luggage, etc. As another example, the vehicle computer110can identify the occupant and/or cargo based on occupant input, e.g., to a keypad on the vehicle105. The occupant input may specify an identifier for the occupant (e.g., an alphanumeric string of characters that identifies the occupant and/or specifies cargo for the occupant). As yet another example, the vehicle computer110can receive the identifier from the occupant device140, e.g., via the network.

Upon identifying the occupant and/or cargo, the vehicle computer110can actuate one or more indicators to direct the occupant to a door of the vehicle105. The vehicle computer110can determine the door based on, e.g., seat availability, available cargo space, etc. For example, the vehicle computer110can actuate an audio indicator including a cue, e.g., a message specifying the door, a tone on the door, etc., to direct the occupant to a door. Additionally, or alternatively, the vehicle computer110can actuate a visual indicator including a cue, e.g., lights projected on a ground surface indicating a path to the door, a visual message displayed on windows, a light on the door, etc., to direct the occupant to the door.

The vehicle computer110can store, e.g., in a memory, location data of objects, e.g., vehicle components125, an instrument panel, door panels, etc., in the vehicle105cabin. The location data of the objects can be specified in a vehicle coordinate system, e.g., a multi-dimensional Cartesian coordinate system having a predetermined origin point included in the vehicle105cabin. For example, the location data may represent boundaries, i.e., three-dimensional (3D) surfaces, of the objects. In other words, the vehicle computer110can store, e.g., in a memory, a 3D map of the vehicle105cabin.

The vehicle105can include one or more sensors115, e.g., camera sensors115, mounted inside a cabin of the vehicle105, e.g., to provide images of occupants in the vehicle105cabin. That is, the sensors115may be oriented to capture images of occupants in the vehicle105cabin and provide the images to the vehicle computer110.

The vehicle computer110can determine a location of an occupant's eyes based on image data including the occupant. For example, upon receiving image data including an occupant from a camera sensor115, the vehicle computer110can generate a depth map based on the received image data. A depth map is a set of data specifying 3D locations of points in an image. For example, a depth map may include a set of 3D coordinates for each pixel of the received camera image, e.g., with respect to the vehicle coordinate system. In other words, a depth map provides 3D location coordinates of real-world surface points represented in respective pixels of the image. For example, the depth map may include 3D location coordinates of the points visible in the image, e.g., on occupants, seats, etc., in the vehicle105cabin. Thus, a depth map may include 3D coordinates corresponding to each pixel of the image. The vehicle computer110may be programmed, using computer vision techniques, such as are known, to generate a depth map for the received image of the occupant. The vehicle computer110may be programmed to generate a depth map by processing image data received from two or more camera sensors115viewing the occupant from different locations while having an overlapping field of view. Alternatively, other techniques of generating a depth map may include use of a camera capable of detecting light fields, use of photometric stereo methods, or monocular depth estimation techniques which typically utilize a neural network based transformation of 2D image data.

Additionally, the vehicle computer110is programmed to determine a gaze direction G of the occupant based on the captured images, e.g., using computer vision techniques. A gaze direction G is a direction defined by a line that is an axis of an eye's lens, i.e., a direction in which a person's eyes are looking. For example, the vehicle computer110can determine the gaze direction G by determining a location and pose of the occupant's head and the location of the occupant's pupils with respect to the occupant's head.

Further, the vehicle computer110can be programmed to determine a gaze point P of the occupant based on the gaze direction G. A gaze point P is a point at which a line extending from an occupant's eyes in the gaze direction G intersects an object, i.e., a point on which the occupant's eyes are focusing. The vehicle computer110can determine the gaze point based on location data of the occupant's eyes, the gaze direction, and location data of the objects in the vehicle105cabin. For example, the vehicle computer110can determine a line extending in the gaze direction G from the location of the occupant's eyes and based on including the line in the map of the vehicle105cabin, can determine an object in the vehicle105cabin intersected by the line (SeeFIG. 4). Based on determining the object intersected by the line, the gaze point P can be determined.

Upon determining the gaze point P, the vehicle computer110can identify the vehicle component125, e.g., an occupant input device. For example, the vehicle computer110can compare the gaze point P to area thresholds415. Each area threshold415encloses an area around one respective vehicle component125, e.g., occupant input device. The area threshold415may be determined by empirical testing based on, e.g., a minimum area around a vehicle component125that allows for a determination that an occupant is looking at the vehicle component125. The vehicle computer110can store, e.g., in a memory, an area threshold415associated with each vehicle component125. The vehicle computer110can identify the vehicle component125based on the gaze point P being within the area threshold415associated with the vehicle component125.

Additionally, or alternatively, the vehicle computer110can identify the vehicle component125based on an occupant input. For example, the vehicle computer110may be programmed to receive occupant input specifying the vehicle component125. For example, the occupant input may be a vocal request. In such an example, the vehicle computer110may obtain audio data including the request via a microphone, and be programmed to determine the occupant input, e.g., by using data processing techniques, such as Natural Language Processing. For example, the vehicle computer110may receive audio data including an occupant request “how to change the cabin temperature?” The vehicle computer110may be programmed to determine the vehicle component125, e.g., occupant input device, associated with the received request, e.g., based on a look-up table or the like associating the vehicle component125with identified words or phrases.

As another example, the occupant input may be an occupant gesture, e.g., a hand reaching for a vehicle component125. In such an example, the vehicle computer110may obtain image data including the occupant gesture via a camera sensor115and be programmed to determine the vehicle component125. For example, the vehicle computer110can determine a direction in which an occupant is reaching relative to the sensor115lens using computer vision techniques, such as are known. In such an example, the vehicle computer110can determine the vehicle component125based on a location of the occupant's hand (e.g., determined via a depth map, as discussed above), location data of the vehicle components125, and the direction the occupant is reaching. That is, the vehicle computer110can determine a point at which a line extending from the occupant's hand in the direction the occupant is reaching intersects an object in the vehicle105cabin (e.g., similar to determining the gaze point P discussed above). The vehicle computer110can then compare the point to the area thresholds415to determine the vehicle component125, as discussed above.

Upon identifying the vehicle component125, the vehicle computer110may be programmed to actuate a light on the vehicle component125. The vehicle computer110may actuate the light to output a signal indicating the vehicle component125to the occupant. For example, the vehicle computer110can actuate the light to, e.g., turn on, to flash, to adjust a brightness, etc.

As shown inFIG. 2, the vehicle105includes a plurality of seats200. Each seat includes a respective seatback205and seat bottom210. The seatback205may be supported by the seat bottom210and may be stationary or movable relative to the seat bottom210. The seatback205and the seat bottom210may be adjustable in multiple degrees of freedom. Each seat200is supported by a vehicle floor215. Each seat200is independently rotatable relative to the vehicle floor215.

Each seat200includes a rotator220. The rotator220is disposed between the seat bottom210and the floor215. The rotator220may be mounted to the seat bottom210. The rotator220may be designed to rotate the seat200. For example, the rotator220may include a first ring fixed to the floor215and a second ring fixed to the bottom210and rotatable relative to the first ring. In another example not shown in the Figures, the rotator220may include a post mounted to the seat bottom210, the post rotatable about an axis. Each seat200includes a motor225. The vehicle computer110actuates the motor225to rotate the rotator220and move the seat200.

As shown inFIG. 3, the vehicle computer110is programmed to determine a first direction A of the seat200relative to the forward-facing direction B of the vehicle105. The first direction A extends forward relative to the seat200. For example, the first direction A extends from the seatback205forward relative to an occupant of the seat200. The vehicle computer110can determine the first direction A of the seat200based on sensor115data. For example, an angular sensor115can provide data to the vehicle computer110specifying an angular position of the seat200. An angular position is an angle α between an axis extending in the first direction A and a forward axis about a point D on the seat bottom210, e.g., at a center of rotation of the rotator, measured in degrees. The forward axis extends in the forward-facing direction B of the vehicle105regardless of the rotation of the seat200. Based on the angular position, the vehicle computer110can determine the first direction A of the seat200relative to the forward-facing direction B of the vehicle105.

Upon identifying the vehicle component125and the first direction A of the seat200, the vehicle computer110can determine a second direction C based on the location of the vehicle component125. The second direction C is a line starting at the origin in the seat200and ending at the vehicle component125in a horizontal plane defined with respect to the vehicle floor215in the vehicle coordinate system. The vehicle computer110can determine an angle β between the first direction A and the second direction C, i.e., that the axis extending in the first direction A defines with an axis extending in the second direction C about the point D on the seat bottom210.

The vehicle computer110can then compare the angle β to an angle threshold. The angle threshold specifies a minimum angle, e.g., 5 degrees, below which the vehicle computer110prevents rotation of the seat. The angle threshold may be determined by empirical testing based on, e.g., a minimum angle within which occupants can identify the vehicle component125. In the case that the angle β is less than the angle threshold, the vehicle computer110prevents rotation of the seat200. Additionally, the vehicle computer110may be programmed to prevent rotation of the seat200when the vehicle component125is on, i.e., fixed to a movable with, the seat200(e.g., a seat position control switch). In the case that the angle β is equal to or greater than the angle threshold, the vehicle computer110can then actuate the rotator to rotate the seat200based on the angle β, e.g., to align the first direction A and the second direction C.

Additionally, upon determining the gaze point P based on the gaze direction G, the vehicle computer110can determine a third direction. The third direction is a line starting at the gaze point P and ending at the vehicle component125in a vertical plane defined with respect to the vehicle floor215in the vehicle coordinate system. Upon determining the third direction, the vehicle computer110can actuate at least one of an audio indicator, i.e., a speaker, and a visual indicator. The speaker may be mounted in the vehicle105cabin, e.g., on a seat200. When the vehicle computer110actuates the speaker, the speaker includes an audio cue405specifying the third direction. The audio cue405directs the occupant to look in the third direction, e.g., down and right. As one example, the audio cue405can be a message (e.g., generated using Natural Language Processing techniques) output from the speaker specifying the third direction. As another example, the audio cue405can be a directional audio signal corresponding to the third direction (SeeFIG. 4). In such an example, the vehicle computer110can actuate a plurality of speakers in a speaker array to adjust parameters, e.g., phase, amplitude, etc., of the audio signals output from the respective speakers to direct an audio signal heard by the occupant in the third direction, e.g., by using signal interference techniques. Alternatively, the vehicle computer110can actuate a sound dome based on the third direction. A sound dome extends partially around a speaker to direct audio signals output by the speaker. In such an example, the vehicle computer110can actuate the sound dome to face the third direction, which can direct the audio signal output from the speaker in the third direction.

The visual indicator may be mounted in the vehicle105cabin, e.g., to a roof, a floor, a seatback, etc. When the vehicle computer110actuates the visual indicator, the visual indicator includes a visual cue410specifying the third direction. The visual cue410directs the occupant to look in the third direction, e.g., down and right. The visual cue410may be displayed by the visual indicator, e.g., on a touchscreen, or projected by the visual indicator, e.g., onto a window, a seatback, an instrument panel, etc. The visual cue410may include alphanumeric characters, symbols, lines, etc. For example, the visual cue410may be an arrow pointing in the third direction and/or a message specifying the third direction, e.g., “look down and right” (SeeFIG. 4).

Additionally, or alternatively, the vehicle computer110may be programmed to actuate the vehicle component125towards the first direction A of the seat200. For example, the vehicle computer110can actuate the vehicle component125to move the vehicle component125to align the second direction C and the first direction A. Additionally, or alternatively, the vehicle computer110may be programmed to actuate the vehicle component125to align the vehicle component125and the gaze point P, e.g., by moving the vehicle component125towards the gaze point P. As one example, the vehicle component125, e.g., a display screen, may be supported by a telescopic support pivotally connected to the instrument panel. The vehicle computer110can actuate a motor to pivot and extend the telescopic support to move the vehicle component125towards the first direction A and/or the gaze point P.

The vehicle computer110can be programmed to actuate a haptic output device. The haptic output device may be on at least one of the seat200, the identified vehicle component125, and the occupant device140. The vehicle computer110can actuate the haptic output device to provide haptic output based on the vehicle computer110detecting a hand of the occupant on the identified vehicle component125and/or within a distance threshold of the vehicle component125. For example, the vehicle computer110may be programmed to actuate the haptic output device, e.g., on the seat and/or the occupant device140, to provide haptic output at first parameters, e.g., frequency, intensity, etc., when the occupant's hand is within the distance threshold. Additionally, the vehicle computer110may be programmed to actuate the haptic output device, e.g., on the seat and/or the occupant device140, to provide haptic output at a second parameters, e.g., frequency, intensity, etc., when the occupant's hand is on the vehicle component125.

The vehicle computer110can be programmed to detect a hand of the occupant within a distance threshold of the vehicle component125based on image data. For example, the vehicle computer110can receive image data including the occupant's hand and the vehicle component125and can detect the occupant's hand, e.g., using image processing techniques, such as Canny edge detection. Upon detecting the hand, the vehicle computer110can determine a location of the occupant's hand, e.g., via a depth map (as discussed above). The vehicle computer110can then determine a distance from the occupant's hand to the vehicle component125based on a length of a line from the location of the occupant's hand to the vehicle component125. The vehicle computer110can then compare the distance to the distance threshold. The distance threshold can be determined empirically based on, e.g., determining a minimum distance between an occupant's hand and the vehicle component125that an occupant can identify the vehicle component125. When the distance is equal to or less than the distance threshold, the vehicle computer110may be programmed to actuate the haptic output device, e.g., on the seat and/or the occupant device140, to provide haptic output. When the distance is greater than the distance threshold, the vehicle computer110may be programmed to actuate the haptic output device to stop providing haptic output.

The vehicle computer110can detect the hand of the occupant on the vehicle component125based on a capacitive sensor on the vehicle component125. For example, the vehicle computer110can determine the hand of the occupant is on the vehicle component125when the capacitive sensor detects a change in capacitance above a threshold, e.g., determined by empirically testing occupant hands on test capacitive sensors. When the occupant's hand is on the vehicle component125, the vehicle computer110may be programmed to actuate the haptic output device, e.g., on the seat, the vehicle component125, and/or the occupant device140, to provide haptic output. Conversely, the vehicle computer110may be programmed to actuate the haptic output device to stop providing haptic output when the occupant's hand is not on the vehicle component125. Additionally, or alternatively, the vehicle computer110can actuate at least one of the speaker and the indicator, e.g., to output a cue specifying one or more controls of the vehicle component125. For example, the cue can specify operational instructions for the vehicle component125.

FIG. 5is a diagram of an example process500for rotating a seat200upon identifying a vehicle component125. The process500begins in a block505.

In the block505, a vehicle computer110receives sensor115data, e.g., image data, from one or more sensors115, e.g., via the vehicle network. The image data includes an occupant seated in the seat200. The process500continues in a block510.

In the block510, the vehicle computer110determines a gaze direction G of the occupant in the seat200. For example, the vehicle computer110can determine the gaze direction G by applying conventional computer vision techniques to the image data including the occupant. For example, the vehicle computer110can determine the gaze direction G by determining a location and pose of the occupant's head and the location of the occupant's pupils with respect to the occupant's head via the image data. The process500continues in a block515.

In the block515, the vehicle computer110identifies a vehicle component125, e.g., a user input device, based on the gaze direction G. For example, the vehicle computer110can determine a location of the occupant's eye via a depth map, as discussed above. The vehicle computer110stores location data of objects, e.g., an instrument panel, vehicle components125, etc., in a vehicle105cabin. The vehicle computer110can then determine a gaze point P based on determining a line extending in the gaze direction G from the location of the occupant's eyes and an object intersected by the line, as discussed above. The vehicle computer110can then compare the gaze point P to area thresholds415(as discussed above). The vehicle computer110identifies the vehicle component125based on the gaze point P being within an area threshold415associated with the vehicle component125. The vehicle computer110can then, for example, actuate a visual cue such as a light on the vehicle component125.

Additionally or alternatively, the vehicle computer110can identify the vehicle component125based on an occupant input. For example, the vehicle computer110may be programmed to receive occupant input specifying the vehicle component125. The occupant input may be a vocal request or a user gesture, as discussed above. For example, the vehicle computer110may obtain audio data including the request via a microphone, and be programmed to determine the occupant input, e.g., by using data processing techniques, such as Natural Language Processing. The vehicle computer110may be programmed to the determine the vehicle component125associated with the received vocal request.

As another example, the vehicle computer110may obtain image data including the occupant gesture via a camera sensor115and execute programming to determine the vehicle component125. For example, the vehicle computer110can determine a direction in which an occupant is reaching relative to the sensor115lens using computer vision techniques, such as are known. In such an example, the vehicle computer110can determine the vehicle component125based on a location of the occupant's hand (e.g., determined via a depth map, as discussed above), location data of the vehicle components125, and the direction the occupant is reaching, as discussed above. The process500continues in a block520.

In the block520, the vehicle computer110determines a first direction A of the seat200relative to a forward-facing direction B of the vehicle105. For example, the vehicle computer110can receive sensor115data, e.g., from an angular position sensor115, specifying an angular position of the seat200. As discussed above, the angular position is an angle α that an axis extending in the first direction A defines with a forward axis extending in the forward-facing direction B of the vehicle105about a point D on a seat bottom210of the seat200, measured in degrees. Based on the angular position, the vehicle computer110can determine the first direction A of the seat200relative to the forward-facing direction B of the vehicle105.

Additionally, the vehicle computer110can then determine a second direction C based on the location of the vehicle component125. As discussed above, the second direction C is a line starting at the origin in the seat200and ending at the vehicle component125in a horizontal plane defined with respect to the vehicle floor215in the vehicle coordinate system. The vehicle computer110can determine an angle β between the first direction A and the second direction C, i.e., that the axis extending in the first direction A defines with an axis extending in the second direction C about the point D.

Additionally, upon determining the vehicle component125and the gaze point P, the vehicle computer110can determine a third direction based on the location of the vehicle component125. As discussed above, the third direction is a line starting at the gaze point P and ending at the vehicle component125in a vertical plane defined with respect to the vehicle floor215in the vehicle coordinate system. The process500continues in a block525.

In the block525, the vehicle computer110rotates the seat200towards the vehicle component125. For example, the vehicle computer110can actuate a motor225of the seat200to rotate a rotator220based on the angle β between the first direction A and the second direction C. That is, the vehicle computer110can rotate the seat200to align the first direction A and the second direction C.

Additionally or alternatively, the vehicle computer110can actuate at least one of a speaker and a visual indicator. The speaker includes an audio cue405specifying the third direction. For example, the audio cue410can be a verbal message and/or a directional audio signal, as discussed above. The visual indicator includes a visual cue410specifying the third direction. The visual cue410may include alphanumeric characters, symbols, lines, etc. The vehicle computer110can actuate the speaker and/or the visual indicator to direct the occupant to look in the third direction, i.e., move the gaze point P towards the vehicle component125.

Additionally, or alternatively, the vehicle computer110can actuate the vehicle component125to move towards at least one of the first direction A (e.g., to align the first direction A and the second direction C) and the gaze point P (e.g., to align the vehicle component125and the gaze point P), as discussed above. The process500continues in a block530.

In a block530, the vehicle computer110can determine whether a hand of the occupant is within a distance threshold of the vehicle component125. For example, the vehicle computer110can identify the hand of the occupant by using convention object recognition techniques, as discussed above. The vehicle computer110can then determine a distance from the hand of the occupant to the vehicle component125, as discussed above. The vehicle computer110compares the distance to the distance threshold. In the case that the distance is less than the distance threshold, the process500continues in a block535. Otherwise the process500remains in the block530.

In the block535, the vehicle computer110can actuate a haptic output device to provide haptic output. The haptic output device may be on at least one of the seat200, the identified vehicle component125, and an occupant device140. For example, the vehicle computer110may be programmed to actuate the haptic output device, e.g., on the seat200and/or the occupant device140, to provide haptic output at first parameters, e.g., frequency, intensity, etc., a hand of the occupant is within the distance threshold (as discussed above). Additionally or alternatively, the vehicle computer110may be programmed to actuate the haptic output device, e.g., on the seat200and/or the occupant device140, to provide haptic output at a second parameters, e.g., frequency, intensity, etc., when the occupant's hand is on the vehicle component125. For example, the vehicle computer110can detect the hand of the occupant on the vehicle component125when a capacitive sensor detects a change in capacitance above a threshold. The process500ends after the block535.

With regard to the media, processes, systems, methods, heuristics, etc. described herein, it should be understood that, although the steps of such processes, etc. have been described as occurring according to a certain ordered sequence, such processes may be practiced with the described steps performed in an order other than the order described herein. It further should be understood that certain steps may be performed simultaneously, that other steps may be added, or that certain steps described herein may be omitted. In other words, the descriptions of processes herein are provided for the purpose of illustrating certain embodiments and should in no way be construed so as to limit the claims.