Patent Description:
The autonomous-driving logic can determine the appropriate action based on whether the object is stationary or moving and a correct estimation of the object's orientation or heading. In some instances, the time available for determining the appropriate action can be short, and in that time, the sensors may fail to obtain sufficient observations of the object to determine its motion classification and accurately predict future movements. It can be challenging for the sensors to classify the object quickly and accurately for enabling autonomous-driving applications.

<CIT> discloses a radar apparatus comprising a decision unit which changes a method of deciding whether a target is true or false in accordance with a determination result of a reliability by a determination unit. <CIT> discloses a radar apparatus which is configured to determine whether or not a target is a standstill object. <CIT> discloses a moving subject recognizing system for recognizing a subject, such as a pedestrian, moving toward a forward path of a vehicle.

It is an object of the invention to enable for more accurate and robust motion classification in radar systems.

The object is solved by the method according to claim <NUM>.

Techniques and apparatuses are described that implement motion classification using low-level detections. In particular, a radar system, which is mounted to a moving platform, extracts detections associated with an object based on radar data. The radar system identifies fused detections among the detections that are associated with a particular object and determines whether the fused detections indicate that the particular object is moving. In response to determining that the fused detections indicate that the particular object is moving, the radar system increments a current motion counter for the particular object. The radar system also increments a perpendicular motion counter in response to determining that the fused detections indicate that the particular object is moving perpendicular to the host vehicle. The radar system then sets a current motion flag and/or a perpendicular motion flag as true in response to determining that the current motion counter or the perpendicular motion counter has a value greater than a threshold value, respectively. In response to setting either flag as true, the radar system increments a historical motion counter for the particular object as true. The host vehicle is then operated based on the current motion flag, the perpendicular motion flag, and the historical motion counter. In this way, the radar system introduces hysteresis in its motion classification to provide a reliable and stable motion classification output to downstream vehicle-based systems. The described radar system can also introduce flag reset mechanisms to prevent false positives and erroneous vehicle operation reactions.

This document also describes methods performed by the above-summarized radar system and other configurations of the radar system set forth herein and means for performing these methods.

This Summary introduces simplified concepts related to motion classification using low-level detections, which are further described in the Detailed Description and Drawings. This Summary is not intended to identify essential features of the claimed subject matter, nor is it intended for use in determining the scope of the claimed subject matter.

The details of one or more aspects of motion classification using low-level detections are described in this document with reference to the following figures. The same numbers are used throughout the drawings to reference like features and components:.

Many assisted-driving and autonomous-driving applications rely on information from sensors (e.g., radar sensors, cameras) to visualize a surrounding environment. Correct classification of objects (e.g., stationary, moving) and estimation of the movement heading (e.g., cross-traffic, receding, oncoming) are necessary elements of many assisted-driving and autonomous-driving applications. These applications often use a multi-sensor architecture (e.g., a combination of radar sensors and cameras) to provide improved motion classification and heading for nearby objects, enabling driving decisions with a greater degree of confidence.

Sensors in a multi-sensor architecture have relative strengths and weaknesses. For example, radar sensors can provide improved performance in the presence of different environmental conditions, such as low lighting and fog, or with moving or overlapping objects. Radar sensors can also provide accurate range and range-rate estimations of objects in the environment. In contrast, cameras can provide improved identification of objects (e.g., identifying vulnerable road users (VRUs) like pedestrians and bicyclists) and azimuth angle estimations. In particular, cameras can determine the motion and orientation states of cross-moving objects with a higher degree of precision than radar sensors.

To correctly classify the motion classification and heading of objects, multi-sensor systems should utilize a computationally-efficient algorithm with a high degree of confidence. Some multi-sensor systems use a speed-based determination to classify object motion. For example, these systems can determine whether any tracklets or fused objects associated with a target satisfy threshold criteria and mark a corresponding flag accordingly. If an object is determined to be moving, then the systems can determine whether the object is moving away or toward the host vehicle.

Because the sensors can often detect negligible velocity components associated with stationary objects, these systems can find it difficult to differentiate slow-moving objects from stationary objects. Similarly, it can be difficult for these systems to track the motion states of objects in urban driving environments where objects can repeatedly toggle between moving and stationary. These systems can also struggle to correctly classify objects across a range of object types (e.g., pedestrians versus vehicles).

In contrast, this document describes techniques that perform motion classification and heading determination more robustly and accurately. For example, the described system can directly use sensor data to reduce noise output and overprocessing. The system can use camera data to identify the type of object and dynamically apply the corresponding threshold to classify its motion more accurately. Hysteresis techniques are used to provide stable motion and heading classifications, especially for urban environments and slow-moving objects. In this way, the described system provides a more reliable and stable motion and heading classification for all types of objects with different headings and orientations. The system also utilizes reset mechanisms to reduce false positives and avoid erroneous reactions by assisted-driving and autonomous-driving applications.

<FIG> illustrates an example environment <NUM> in which a radar system <NUM> capable of performing motion classification using low-level detections can be implemented in accordance with the techniques of this disclosure. In the depicted environment <NUM>, the radar system <NUM> is mounted to, or integrated within, a vehicle <NUM>. The radar system <NUM> is capable of detecting one or more objects <NUM> that are within proximity of the vehicle <NUM> by transmitting and receiving electromagnetic (EM) signals. The vehicle <NUM> can also include a camera <NUM> (or vision system) configured to capture vision data regarding the objects <NUM>.

Although illustrated as a truck, the vehicle <NUM> can represent other types of motorized vehicles (e.g., a car, a motorcycle, a bus, a tractor, a semi-trailer truck, or construction equipment), types of non-motorized vehicles (e.g., a bicycle), types of railed vehicles (e.g., a train or a trolley car), watercraft (e.g., a boat or a ship), aircraft (e.g., an airplane or a helicopter), or spacecraft (e.g., satellite). In general, the radar system <NUM> and the camera <NUM> can be mounted to any type of moving platform, including moving machinery or robotic equipment.

In the depicted implementation, the radar system <NUM> and the camera <NUM> are mounted near the front of the vehicle <NUM> and each provides an instrumental field of view <NUM>. In other implementations, the radar system <NUM> and the camera <NUM> can be mounted to the backside, left side, or right side of the vehicle <NUM>. In some cases, the vehicle <NUM> includes multiple radar systems <NUM> (or cameras <NUM>), such as a first front-mounted radar system <NUM> positioned near the left side of the vehicle <NUM> and a second front-mounted radar system <NUM> positioned near the right side of the vehicle <NUM>. In general, locations of the radar systems <NUM> and cameras <NUM> can be designed to provide a particular field of view <NUM> that encompasses a region of interest in which the object <NUM> may be present. Example fields of view <NUM> include a <NUM>-degree field of view, one or more <NUM>-degree fields of view, one or more <NUM>-degree fields of view, and so forth, which can overlap (e.g., four <NUM>-degree fields of view).

The radar system <NUM> can include at least one antenna array and at least one transceiver to transmit and receive radar signals. The antenna array includes at least one transmit antenna element and at least one receive antenna element. In some situations, the antenna array includes multiple transmit antenna elements and multiple receive antenna elements to implement a multiple-input multiple-output (MIMO) radar capable of transmitting multiple distinct waveforms at a given time (e.g., a different waveform per transmit antenna element). The antenna elements can be circularly polarized, horizontally polarized, vertically polarized, or a combination thereof.

Using the antenna array, the radar system <NUM> can form beams that are steered or un-steered and wide or narrow. The steering and shaping can be achieved through analog beamforming or digital beamforming. The transmitting antenna elements can have, for instance, an un-steered omnidirectional radiation pattern or can produce a wide steerable beam to illuminate a large volume of space. To achieve target angular accuracies and angular resolutions, the receiving antenna elements can be used to generate hundreds of narrow steered beams with digital beamforming. In this way, the radar system <NUM> can efficiently monitor an external environment and detect one or more objects <NUM> within the field-of-view <NUM>.

The transceiver includes circuitry and logic for transmitting and receiving radar signals via the antenna array. Components of the transceiver can include amplifiers, mixers, switches, analog-to-digital converters, or filters for conditioning the radar signals. The transceiver also includes logic to perform in-phase/quadrature (I/Q) operations, such as modulation or demodulation. A variety of modulations can be used, including linear frequency modulations, triangular frequency modulations, stepped frequency modulations, or phase modulations. The transceiver can be configured to support continuous-wave or pulsed radar operations. A frequency spectrum (e.g., range of frequencies) that the transceiver uses to generate the radar signals can encompass frequencies between one and four hundred gigahertz (GHz), between four and one hundred GHz, or between approximately seventy and eighty GHz. The transceiver can employ a spread spectrum technique, such as code-division multiple access (CDMA), to support MIMO operations.

In general, the object <NUM> is composed of one or more materials that reflect radar signals. The object <NUM> can be a non-living object, such as a four-wheel vehicle or a two-wheel vehicle. In other cases, the object <NUM> is a living object, such as a pedestrian or an animal. The objects <NUM> can be moving or stationary. If moving, the objects <NUM> can be moving perpendicular and/or parallel with the travel path of the vehicle <NUM>. Other types of objects <NUM> can include a continuous or discontinuous road barrier, a traffic cone, a concrete barrier, a guard rail, a fence, or a tree.

Information regarding the object type, its motion classification, and its heading that is detected by the radar system <NUM> and/or the camera <NUM> can enable an assisted-driving or autonomous-driving application of the vehicle <NUM> to determine an appropriate action to avoid a potential collision with the object <NUM>. Example actions can include braking to stop, veering, or reducing speed. In particular, the object type can indicate an appropriate speed threshold for classifying the motion of the object <NUM> (e.g., moving, stationary). Based on this information, the assisted-driving or autonomous-driving application can determine an appropriate safety margin (e.g., an appropriate distance and/or speed) to maintain while approaching the object <NUM> in order to reserve adequate time to avoid a potential collision.

The vehicle <NUM> also includes a processor <NUM> (e.g., a hardware processor, a processing unit) and computer-readable storage media (CRM) <NUM> (e.g., a memory, long-term storage, short-term storage) that stores computer-executable instructions for a motion classifier <NUM> and a heading classifier <NUM>. The processor <NUM> can include multiple processing units or a single processing unit (e.g., a microprocessor). The processor <NUM> can also be a system-on-chip of a computing device, a controller, or an electronic control unit. The processor <NUM> executes computer-executable instructions stored within the CRM <NUM>. As an example, the processor <NUM> can execute the motion classifier <NUM> to classify the motion of the object <NUM> as moving or stationary and maintain a history of the object's motion classification. Similarly, when the heading classifier <NUM> is executed by the processor <NUM>, a heading (e.g., cross-traffic, oncoming, receding) of the object <NUM> can be determined relative to the vehicle <NUM>. The motion classifier <NUM> can directly use data from the radar system <NUM> and the camera <NUM> to reduce overprocessing by the vehicle <NUM> and/or the processor <NUM> and noise from the motion classifications.

<FIG> illustrates an example configuration of the vehicle <NUM> with the radar system <NUM> that can perform motion classification using low-level detections. As described with respect to <FIG>, the vehicle <NUM> can include the radar system <NUM>, the camera <NUM>, the processor <NUM>, the CRM <NUM>, the motion classifier <NUM>, and the heading classifier <NUM>. The vehicle <NUM> can also include one or more communication devices <NUM> and one or more vehicle-based systems <NUM>.

The communication devices <NUM> can include a sensor interface and a vehicle-based system interface. The sensor interface and the vehicle-based system interface can transmit data over a communication bus of the vehicle <NUM>, for example, when the individual components of the motion classifier <NUM> and the heading classifier <NUM> are integrated within the vehicle <NUM>. The communication devices <NUM> can provide raw or processed sensor data from the radar system <NUM> and the camera <NUM> to the motion classifier <NUM> and the heading classifier <NUM>.

The motion classifier <NUM> can include a flag tracker <NUM> and a motion threshold module <NUM>. The flag tracker <NUM> can provide historical tracking of flags and counters associated with the movement and/or heading associated with individual objects <NUM>. For example, the flag tracker <NUM> can maintain both instantaneous and historical flag data for different motion and heading classifications (e.g., moving, receding, moveable, oncoming, and/or stationary flags) associated with the objects <NUM>. In this way, the motion classifier <NUM> can avoid inconsistent or flickering motion classifications for a particular object, especially in urban or congested driving environments, by introducing hysteresis into the motion tracking. In addition, the described techniques decrease the occurrence of false positives in identifying a particular object as moving or stationary.

The motion threshold module <NUM> can use vision data from the camera <NUM> to determine an object type for each object <NUM>. Based on the object type, the motion threshold module <NUM> can use different motion thresholds to classify a particular object <NUM> as moving or stationary. In this way, the motion classifier <NUM> can adjust motion thresholds based on the type of object detected and provide improved motion and heading classifications for the objects <NUM>. For example, the motion threshold module <NUM> can use a reduced motion threshold for pedestrians to account for the uncertainty and ambiguity inherent in recognizing motion patterns for pedestrians and other VRUs.

The vehicle <NUM> also includes the vehicle-based systems <NUM>, such as an assisted-driving system <NUM> and an autonomous-driving system <NUM>, that rely on data from the motion classifier <NUM> and/or the heading classifier <NUM> to control the operation of the vehicle <NUM> (e.g., braking, lane changing). Generally, the vehicle-based systems <NUM> can use data provided by the motion classifier <NUM> and/or the heading classifier <NUM> to control operations of the vehicle <NUM> and perform certain functions. For example, the assisted-driving system <NUM> can output information (e.g., a message, a sound, a visual indicator) to alert a driver of a stationary object and perform evasive maneuvers to avoid a collision with the stationary object. As another example, the autonomous-driving system <NUM> can navigate the vehicle <NUM> to a particular location to avoid a collision with a moving object.

<FIG> illustrates an example method <NUM> to perform motion classification using low-level detections. In portions of the following discussion, reference may be made to the environment <NUM> of <FIG> and entities detailed in <FIG> and <FIG>, reference to which is made for example only. The techniques are not limited to performance by one entity or multiple entities.

At <NUM>, radar signals are obtained from a radar system for a data cycle. The radar signals are reflected by one or more objects in a vicinity of a host vehicle. For example, the radar system <NUM> may be installed in the vehicle <NUM> and provides a field-of-view <NUM> of the roadway <NUM>. The radar system <NUM> transmits and receives radar signals, including radar signals reflected by the objects <NUM> that in the environment <NUM> around the vehicle <NUM>.

At <NUM>, one or more detections associated with each of the one or more objects are identified using the radar signals. The detections associated with a particular object of the one or more objects represent a fused detection for the particular object. For example, the processor <NUM> or the radar system <NUM> can determine, using the radar signals, detections associated with the objects <NUM>. The processor <NUM> or the radar system <NUM> can relate detections associated with a particular object <NUM> as a fused detection for that particular object <NUM>. In this way, the processor <NUM> or the radar system <NUM> can provide fused detections to the motion classifier <NUM> for each of the objects <NUM>.

At <NUM>, for each fused detection, it is determined whether the fused detection indicates that the particular object is moving. For example, the motion classifier <NUM> can determine, for each fused detection, whether a particular fused detection indicates that the particular object <NUM> associated with that fused detection is moving. The motion classifier <NUM> can determine a particular object <NUM> is moving if at least half of the detections associated with that particular object <NUM> indicate movement. If less than half of the detections associated with the particular object <NUM> indicate movement, then the motion classifier <NUM> can determine that movement is ambiguous.

The motion classifier <NUM> can also determine, for each fused detection, whether the fused detection indicates that the particular object <NUM> is moving perpendicular (e.g., has an orthogonal velocity component) to the vehicle <NUM>. The motion classifier <NUM> can determine perpendicular movement by determining that the orthogonal velocity component associated with the fused detection is greater than zero.

Method <NUM> includes performing operations <NUM> through <NUM> for each fused detection (associated with each object <NUM>) in each data cycle (or fused data cycle). Although the operations <NUM> through <NUM> may be written in the singular for a particular fused detection or particular object, it should be understood that the operations <NUM> through <NUM> are performed for each fused detection of each data cycle.

At <NUM>, in response to determining that the fused detection indicates that the particular object is moving, a consecutive motion counter associated with the particular object is incremented. For example, the motion classifier <NUM> can increment (e.g., increase a value by one) a consecutive motion counter associated with the particular object <NUM> in response to determining that the fused detection indicates that the object <NUM> is moving. Similarly, the motion classifier <NUM> can increment a consecutive ambiguity counter associated with the particular object <NUM> in response to determining that movement of the object <NUM> is ambiguous. In response to incrementing the consecutive ambiguity counter associated with the particular object <NUM>, the motion classifier <NUM> can reset a value of the consecutive motion counter associated with the particular object <NUM> to zero. Similarly, in response to incrementing the consecutive motion counter associated with the particular object <NUM>, the motion classifier <NUM> can reset a value of the consecutive ambiguity counter associated with the particular object <NUM> to zero. The motion classifier <NUM> can also increment a consecutive perpendicular motion counter associated with the particular object <NUM> in response to determining that the fused detection indicates that the object <NUM> is moving perpendicular to the vehicle <NUM>.

At <NUM>, a consecutive motion flag associated with the particular object is set as true in response to determining that the consecutive motion counter associated with the particular object has a value greater than a threshold counter value. For example, the motion classifier <NUM> can set a consecutive motion flag associated with the particular object as true in response to determining that the consecutive motion counter associated with the particular object <NUM> has a value greater than a threshold counter value (e.g., having a value of three). Similarly, the motion classifier <NUM> can set a consecutive perpendicular motion flag associated with the particular object <NUM> as true in response to determining that the consecutive perpendicular motion counter has a value greater than the threshold counter value.

At <NUM>, a moving flag associated with the particular object is set as true if the consecutive motion flag associated with the particular object is set as true and the fused detection associated with the particular object has a velocity component greater than a motion threshold. For example, the motion classifier <NUM> can set a moving flag associated with the particular object <NUM> as true if the consecutive motion flag associated with the object <NUM> is set as true and the fused detection associated with the object <NUM> has a velocity component (e.g., a longitudinal velocity component) greater than a motion threshold. The motion classifier <NUM> determines if the fused detection associated with the particular object <NUM> has a longitudinal velocity component greater than the motion threshold by first determining a corresponding motion threshold. In particular, the motion classifier <NUM> determines, , for each fused detection and using vision data from the camera <NUM>, an object type (e.g., pedestrian, vehicle, bicyclist) of the particular object <NUM>. The motion classifier <NUM> then sets the motion threshold based on the object type of the particular object <NUM> as described in greater detail with respect to <FIG> and <FIG>.

The motion classifier <NUM> compares the longitudinal velocity component associated with the particular object <NUM> to this motion threshold. This comparison can be performed by first comparing a radar longitudinal velocity component associated with the particular object <NUM> to the motion threshold. The radar longitudinal velocity component is obtained from the radar signals. If the radar data is insufficient to make this comparison, the motion classifier <NUM> can then compare a vision velocity component associated with the particular object <NUM> to the motion threshold. The vision longitudinal velocity component is obtained from the vision data. If neither the radar data nor vision data can support this comparison, then the motion classifier <NUM> can use a previous state of the moving flag associated with the particular object <NUM> determined during the previous data cycle. To reduce processing requirements, the motion classifier <NUM> can determine whether the fused detection associated with the particular object <NUM> has a longitudinal velocity component greater than a clearly moving threshold. If the velocity component is greater than this clearly moving threshold, the motion classifier <NUM> need not compare the radar or vision data to the motion threshold.

Similarly, the motion classifier <NUM> can set the moving flag associated with the particular object <NUM> as true if the consecutive perpendicular motion flag associated with the object <NUM> is set as true and the fused detection associated with the object <NUM> has an orthogonal velocity component greater than a perpendicular motion threshold. The perpendicular motion threshold can be set based on the object type of the particular object <NUM>. The motion classifier <NUM> can determine that the orthogonal velocity component of the fused detection is greater than the perpendicular motion threshold by determining that a radar orthogonal velocity component of the fused detection based on the radar signals is greater than the perpendicular motion threshold. Or the motion classifier <NUM> can determine that a vision orthogonal velocity component of the fused detection based on the vision data is greater than the perpendicular motion threshold. If the motion classifier <NUM> determines that the orthogonal velocity component associated with the fused detection is not greater zero (e.g., no perpendicular motion detected for that data cycle), it can reduce the value of the consecutive perpendicular motion counter associated with the particular object <NUM> by one, as opposed to resetting its value to zero.

If the moving flag associated with the particular object is set as true, the heading classifier <NUM> can then determine a heading classification (e.g., crossing, east, west, northeast, etc.) associated with the particular object from a heading associated with the fused detection. If the heading classification associated with the particular object is one that indicates perpendicular movement relative to the vehicle <NUM>, the heading classifier <NUM> can set a crossing flag as true.

At <NUM>, in response to setting the moving flag associated with the particular object as true, a historical motion counter associated with the particular object is incremented for each fused detection. For example, the motion classifier <NUM> can increment a historical motion counter associated with the particular object <NUM> if its moving flag is set as true. The motion classifier <NUM> can also set a moveable flag associated with the particular object <NUM> as true if the moving flag associated with the particular object <NUM> is true. The moveable flag indicates whether the moving flag associated with the particular object <NUM> has been set as true in this data cycle or any previous data cycle. The motion classifier <NUM> can also reset the moveable flag associated with the particular object <NUM> to false if the historical motion counter associated with the object <NUM> has a value less than ten.

At <NUM>, the host vehicle is operated in the roadway based on a status of the moving flag and the historical motion counter associated with each of the one or more objects. For example, the assisted-driving system <NUM> or the autonomous-driving system <NUM> can operate the vehicle <NUM> based on a status of the moving flag and the historical motion counter associated with each of the objects <NUM>. The assisted-driving system <NUM> or the autonomous-driving system <NUM> can also operate the vehicle <NUM> based on a status of the crossing flag and/or the moveable flag associated with each of the objects <NUM>.

<FIG> and <FIG> illustrate an example conceptual diagram <NUM> for performing motion classification using low-level detections. In particular, the conceptual diagram <NUM> illustrates the motion classification process of the motion classifier <NUM> and the heading classification process of the heading classifier <NUM>. The conceptual diagram <NUM> illustrates example inputs, outputs, and operations of the motion classifier <NUM> and the heading classifier <NUM>.

At operation <NUM>, the motion classifier <NUM> determines motion attributes for objects <NUM> associated with fused detections. The motion classifier <NUM> receives radar data <NUM> from the one or more radar systems <NUM>. The radar data <NUM> can be at the detection level and include one or more detections associated with each of the objects <NUM> within the field-of-view <NUM> of the radar system <NUM>. Multiple detections associated with a particular object <NUM> can represent a fused detection (or "a fused object"). The motion classifier <NUM> determines the number or percentage of radar detections associated with each fused detection to indicate that the particular object <NUM> is moving.

At operation <NUM>, the motion classifier <NUM> determines counter values for consecutive motion and consecutive ambiguity. In particular, the motion classifier <NUM> determines a value for a consecutive motion counter <NUM> (e.g., "cntConsecutiveMoving") and a consecutive ambiguity counter <NUM> (e.g., "cntConsecutiveAmbiguous"). For each frame of radar data, the motion classifier <NUM> can classify the fused detection associated with each object <NUM> as moving or ambiguous. For example, if more than fifty percent of the radar detections associated with the fused detection indicate that the particular object <NUM> is moving, the consecutive motion counter <NUM> is incremented linearly. The consecutive motion counter <NUM> can have a maximum value of one hundred. The motion classifier <NUM> can use a different percentage threshold to classify each fused detection as moving or ambiguous in other implementations. If the consecutive motion counter <NUM> is incremented, the motion classifier <NUM> reinitializes the consecutive ambiguity counter <NUM> to zero.

Similarly, if less than fifty percent of the radar detections associated with the fused detection indicate that the particular object <NUM> is moving, the consecutive ambiguity counter <NUM> is incremented linearly. The consecutive ambiguity counter <NUM> can also have a maximum value (e.g., one hundred). If the consecutive ambiguity counter <NUM> is incremented, the motion classifier <NUM> reinitializes the consecutive motion counter <NUM> to zero. The motion classifier <NUM> stores the historical values of the consecutive motion counter <NUM> and the consecutive ambiguity counter <NUM> for each object <NUM>.

At operation <NUM>, the motion classifier <NUM> determines motion thresholds for the fused detections based on an object class of a particular object <NUM>. In particular, the motion classifier <NUM> can account for the uncertainty in VRU movements and the difficulty in recognizing their motion patterns by using lower threshold values to determine their motion classification. For example, the motion classifier <NUM> can use a default minimum moving speed of <NUM> meters/second (m/s) as a motion threshold to classify an object <NUM> as moving. A motion threshold of <NUM>/s can be used to classify a pedestrian as moving. The motion classifier <NUM> can use vision data <NUM> to identify a particular object <NUM> as a pedestrian. The vision data <NUM> can be image data from the camera <NUM>.

Similarly, the motion classifier <NUM> can use different cross-direction (e.g., perpendicular) motion thresholds based on the object type associated with each fused detection. For example, a vehicle can be classified as having perpendicular motion using a perpendicular motion threshold of <NUM>/s. In contrast, a pedestrian can be classified as moving perpendicular to the vehicle <NUM> using a perpendicular motion threshold of <NUM>/s.

At operation <NUM>, the motion classifier <NUM> determines whether the fused detections can be classified as cross-traffic. The motion classifier <NUM> can use the radar data <NUM> to determine radial and orthogonal velocities for each fused detection and its heading, which are then used to determine a cross-traffic motion status for each object <NUM> for each data cycle. Based on the appropriate perpendicular motion threshold, the motion classifier <NUM> determines whether to set a perpendicular motion (radar) flag <NUM> (e.g., "f_cur_perp_motion") as true. The perpendicular motion (radar) flag <NUM> is a Boolean flag.

Similarly, the motion classifier <NUM> can use the vision data <NUM> to determine radial and orthogonal velocities for each fused detection and its heading. Based on the appropriate perpendicular motion threshold, the motion classifier <NUM> determines whether to set a perpendicular motion (vision) flag <NUM> (e.g., "f_vis_data_perp_motion") as true. The perpendicular motion (vision) flag <NUM> is also a Boolean flag and allows the motion classifier <NUM> to add redundancy to the motion classification by discretely including the vision data <NUM>. This redundancy in the perpendicular motion classification adds robustness to the motion classification algorithm of the motion classifier <NUM>.

The motion classifier <NUM> also tracks the cyclical history of the perpendicular motion classification for each fused detection using a consecutive perpendicular motion counter <NUM> (e.g., "cntConsecutivePerp"). The consecutive perpendicular motion counter <NUM> is incremented by one if either the perpendicular motion (radar) flag <NUM> or the perpendicular motion (vision) flag <NUM> are true for the fused detection for that data cycle. The consecutive perpendicular motion counter <NUM> can have a maximum value of twenty. The motion classifier <NUM> stores the historical values of the consecutive perpendicular motion counter <NUM> for each object <NUM>.

At operation <NUM>, the motion classifier <NUM> determines historical motion classifications for each fused detection. If the consecutive motion counter <NUM> for a fused detection has a value greater than or equal to three, the particular object <NUM> is determined to have motion at that data cycle and a consecutive motion flag <NUM> (e.g., "f_ConsecutiveMoving_ok") is marked as true for that data cycle. The motion classifier <NUM> can use a different value than three to determine the state value of the consecutive motion flag <NUM> in other implementations.

Similarly, if the consecutive perpendicular motion counter <NUM> for a fused detection has a value greater than or equal to three, the particular object <NUM> is determined to have perpendicular motion at that data cycle and a consecutive perpendicular motion flag <NUM> (e.g., "f_perp_motion") is marked as true for that data cycle. The motion classifier <NUM> can use a different value than three to determine the state value of the consecutive perpendicular motion flag <NUM> in other implementations. To account for the difficulty in detecting perpendicular motion, the consecutive perpendicular motion counter <NUM> can be linearly reduced. The linear reduction of the consecutive perpendicular motion counter <NUM> can avoid erratic driving operations from the vehicle-based systems <NUM> that would be caused by an immediate drop in the counter value.

At operation <NUM>, the motion classifier <NUM> determines motion classification for potentially-moving fused detections. If the speed for a fused detection is above a clearly moving threshold (e.g., <NUM>/s or <NUM> miles per hour (mph)), then the motion classifier <NUM> can confidently classify the object <NUM> as moving. In this way, computational power can be conserved for fused detections that are safely deemed as moving. The clearly moving threshold can be adjusted based on the object type or driving environment as well.

For fused detections below the clearly moving threshold, the motion classifier <NUM> can determine if the radar data <NUM> indicates that the fused detection is moving using the motion threshold and the perpendicular motion threshold from operation <NUM>. If the object is determined to be moving, a moving flag <NUM> (e.g., "f_moving") is set as true. For pedestrians and other VRUs that have a true or positive value for the consecutive motion flag <NUM> and the consecutive perpendicular motion flag <NUM>, the motion classifier <NUM> sets the moving flag <NUM> as true if the corresponding motion threshold and perpendicular motion thresholds are also satisfied. The additional constraints or checks for VRUs help to reduce incorrect motion classifications for such objects.

If the motion classifier <NUM> cannot perform the motion classification using the radar data <NUM> for a fused detection for a particular data cycle, then the motion classifier <NUM> can use the vision data <NUM> to perform motion classification. The motion classifier <NUM> can ensure the vision data <NUM> is accurate by using it only if the difference between the velocity determined from the vision data <NUM> is less than <NUM>/s (or some other threshold value) in both the longitudinal and lateral directions. The vision data <NUM> can be used to update the consecutive motion counter <NUM> and the consecutive ambiguity counter <NUM>. In addition, the motion classifier <NUM> can use the vision data <NUM> to recompute the consecutive motion flag <NUM> to determine if the moving flag <NUM> should be set as true for the fused detection.

If the motion classification using the radar data <NUM> or the vision data <NUM> cannot be performed, the motion classifier <NUM> can retain the value of the moving flag <NUM> from the previous data cycle. This scenario can occur when updated sensor data is not available for a particular data cycle.

If the moving flag <NUM> is updated for a fused detection in a particular data cycle, the motion classifier <NUM> can also update a stationary flag, moveable flag, fast-moving flag, slow-moving flag, oncoming flag, oncomeable flag, receding flag, and recedable flag for the fused detection. The update of these other motion flags ensures consistency across the motion flags. The motion classifier <NUM> can use a threshold of <NUM>/s to differentiate between fast-moving and slow-moving objects, regardless of the object type. The stationary flag can be set as true if the consecutive motion counter <NUM> is equal to zero and the fused detection has a speed lower than the corresponding threshold. If the stationary flag is not true, then the movable flag is set to true. The moveable flag represents a historical version of the moving flag <NUM> and defines whether a particular object has ever been classified as moving. The oncoming flag is set as true if the fused detection indicates that the particular object <NUM> is moving away from the vehicle <NUM>. The receding flag is set as true if the fused detection indicates that the particular object <NUM> is moving towards the vehicle <NUM>. Similar to the movable flag, the oncomeable flag and the recedable flag are historical versions of the oncoming flag and the receding flag, respectively.

At operation <NUM>, the motion classifier <NUM> can determine whether to increment historical motion counters for each fused detection. For example, if the moving flag is set as true for a fused detection, then a consecutive any motion counter <NUM> for the particular object <NUM> is incremented by one. The consecutive any motion counter <NUM> represents the number of data cycles in which a particular fused detection has had the moving flag <NUM> activated or set as true.

The motion classifier <NUM> can also update a consecutive any motion ever counter <NUM> based on the value of the consecutive any motion counter <NUM>. In particular, the consecutive any motion ever counter <NUM> represents the maximum value of the consecutive any motion counter <NUM> for a particular fused detection and can be used to determine potentially incorrect moveable classifications as described in relation to operation <NUM>. The motion classifier <NUM> stores the historical values of the consecutive any motion counter <NUM> and the consecutive any motion ever counter <NUM> for each object <NUM>.

At operation <NUM>, the motion classifier <NUM> determines whether to correct for stale information among the flags and/or counters. For example, the motion classifier <NUM> can determine whether to reset the moveable flag, the oncoming flag, and the receding flag. If a particular object <NUM> is a recent fused detection (e.g., it has been detected for less than ten data cycles), more than <NUM> meters out with a maximum observed speed lower than <NUM>/s, and the value for the consecutive any motion counter is less than ten, then the moveable flag, the oncomeable flag, and the recedable flag are reset to zero. Other cycle, distance, and speed thresholds can be used to determine whether to reset these flags. An exception to resetting these flags is made for VRUs considering the challenges surrounding their motion classification.

At operation <NUM>, the heading classifier <NUM> can determine heading classification for the fused detections. The heading classifier <NUM> can classify the heading based on the detected heading of the fused detections. The heading sectors can be defined by the heading classifier <NUM> into a north, northeast, east, southeast, south, southwest, west, and northwest heading. If a heading classification is set as either east or west for a fused detection, the heading classifier <NUM> can set a crossing flag <NUM> (e.g., "f_crossing") as true for the particular object <NUM>. The crossing flag <NUM> can be provided to vehicle-based systems <NUM> to improve decision-making related to straight-cross path scenarios.

<FIG> and <FIG> illustrate conceptual diagrams <NUM>-<NUM> and <NUM>-<NUM>, respectively, in which motion classification is performed using low-level detections. The conceptual diagrams <NUM>-<NUM> and <NUM>-<NUM> illustrate the motion classification process of the motion classifier <NUM> for two different driving scenarios. The conceptual diagrams <NUM>-<NUM> and <NUM>-<NUM> illustrate example motion flags <NUM> and <NUM> provided by the motion classifier <NUM> to the vehicle-based systems <NUM>, but the motion flags <NUM> and <NUM> are not necessarily limited to this order or combination of flags. In other implementations, fewer or additional flags and counters can be provided to the vehicle-based systems <NUM> to improve operation of the vehicle <NUM>.

In <FIG>, the vehicle <NUM> is driving along a roadway and another vehicle <NUM> is driving in front of it. The vehicle <NUM> includes the radar system <NUM> and the camera <NUM> with respective fields-of-view that include the other vehicle <NUM>. The radar system <NUM> can transmit radar signals <NUM> that are reflected by the other vehicle <NUM> and processed by the radar system <NUM> and/or the motion classifier <NUM>.

The motion classifier <NUM> can perform a motion and heading classification process according to the conceptual diagram <NUM> and set corresponding values of the motion flag <NUM> for the other vehicle <NUM>. In particular, the motion classifier <NUM> can set a moving flag <NUM>-<NUM> as true, which is represented by a value of "<NUM>" in <FIG>. The moving flag <NUM>-<NUM> indicates that the other vehicle <NUM> is moving. Because the moving flag <NUM>-<NUM> is set as true, the motion classifier <NUM> sets a stationary flag <NUM>-<NUM> as false, which is represented by a value of "<NUM>" in <FIG>. In addition, because the moving flag <NUM>-<NUM> was set as true in this data cycle or in a previous data cycle, a moveable flag <NUM>-<NUM> is also set as true. The radar system <NUM> or the motion classifier <NUM> also determines that the other vehicle <NUM> is moving away from the vehicle <NUM> and sets a receding flag <NUM>-<NUM> as true. Because the receding flag <NUM>-<NUM> was set as true in this data cycle or in a previous data cycle, a recedable flag <NUM>-<NUM> is also set as true. Because the other vehicle <NUM> is moving away from the vehicle <NUM>, the motion classifier <NUM> sets an oncoming flag <NUM>-<NUM> as false. Similar to the recedable flag <NUM>-<NUM>, the motion classifier <NUM> also sets an oncomeable flag <NUM>-<NUM> as false because the oncoming flag <NUM>-<NUM> has not been true for this data cycle or a previous data cycle.

In <FIG>, the vehicle <NUM> is driving along a roadway and a pedestrian <NUM> is crossing the roadway in front of it. The radar system <NUM> can transmit the radar signals <NUM> that are reflected by the pedestrian <NUM> and processed by the radar system <NUM> and/or the motion classifier <NUM>.

The motion classifier <NUM> can perform a motion and heading classification process according to the conceptual diagram <NUM> and set corresponding values of the motion flag <NUM> for the pedestrian <NUM>. In particular, the motion classifier <NUM> can set a moving flag <NUM>-<NUM> as true. The moving flag <NUM>-<NUM> indicates that the pedestrian <NUM> is moving (e.g., perpendicular to the vehicle <NUM>). Because the moving flag <NUM>-<NUM> is set as true, the motion classifier <NUM> sets a stationary flag <NUM>-<NUM> as false. In addition, because the moving flag <NUM>-<NUM> was set as true in this data cycle or in a previous data cycle, a moveable flag <NUM>-<NUM> is also set as true. The radar system <NUM> or the motion classifier <NUM> also determines that the pedestrian <NUM> is moving towards the vehicle <NUM> and sets an oncoming flag <NUM>-<NUM> as true. Because the oncoming flag <NUM>-<NUM> was set as true in this data cycle or in a previous data cycle, an oncomeable flag <NUM>-<NUM> is also set as true. Because the pedestrian <NUM> is moving towards the vehicle <NUM>, the motion classifier <NUM> also sets a receding flag <NUM>-<NUM> as false. Similar to the oncomeable flag <NUM>-<NUM>, the motion classifier <NUM> also sets a recedable flag <NUM>-<NUM> as false because the receding flag <NUM>-<NUM> has not been true for this data cycle or a previous data cycle.

Claim 1:
A method (<NUM>) comprising:
obtaining (<NUM>), from a radar system (<NUM>) for a data cycle, radar signals reflected by one or more objects (<NUM>) in a vicinity of a host vehicle (<NUM>);
identifying (<NUM>), using the radar signals, one or more detections associated with each of the one or more objects (<NUM>), the one or more detections associated with a particular object of the one or more objects (<NUM>) representing a fused detection for the particular object;
determining (<NUM>), for each fused detection, whether the fused detection indicates that the particular object is moving;
in response to determining that the fused detection indicates that the particular object is moving, incrementing (<NUM>) a consecutive motion counter (<NUM>) associated with the particular object;
in response to determining that the consecutive motion counter (<NUM>) associated with the particular object has a value greater than a threshold counter value, setting (<NUM>) a consecutive motion flag (<NUM>) associated with the particular object as true;
setting (<NUM>) a moving flag (<NUM>) associated with the particular object as true if the consecutive motion flag (<NUM>) associated with the particular object is set as true and the fused detection associated with the particular object has a longitudinal velocity component greater than a motion threshold by:
determining whether the fused detection associated with the particular object has a longitudinal velocity component greater than the motion threshold by:
determining, for each fused detection and using vision data (<NUM>) from one or more cameras of the host vehicle, an object type of the particular object;
setting the motion threshold based on the object type of the particular object; and
comparing the longitudinal velocity component associated with the particular object to the motion threshold;
in response to setting the moving flag (<NUM>) associated with the particular object as true, incrementing (<NUM>) a historical motion counter associated with the particular object; and
operating (<NUM>), based on a status of the moving flag (<NUM>) and the historical motion counter associated with each of the one or more objects, the host vehicle (<NUM>) in a roadway.