Evaluating risk factors of proposed vehicle maneuvers using external and internal data

Apparatuses and methods for evaluating the risk factors of a proposed vehicle maneuver using remote data are disclosed. In embodiments, a computer-assisted/autonomous driving vehicle communicates with one or more remote data sources to obtain remote sensor data, and process such remote sensor data to determine the risk of a proposed vehicle maneuver. A remote data source may be authenticated and validated, such as by correlation with other remote data sources and/or local sensor data. Correlation may include performing object recognition upon the remote data sources and local sensor data. Risk evaluation is performed on the validated data, and the results of the risk evaluation presented to a vehicle operator or to an autonomous vehicle navigation system.

TECHNICAL FIELD

Embodiments described herein generally relate to vehicle navigation and driving assistance. In particular, apparatuses and systems for evaluating the risk factors of proposed vehicle maneuvers using a remote data source are described.

BACKGROUND

Modern vehicles, such as automobiles, may be equipped with various safety systems, such as blind spot detection, adaptive cruise control, and automatic emergency braking. These systems may be part of a computer assisted or fully autonomous driving (CA/AD) vehicle. A CA/AD vehicle may be configured with systems that assist a driver (such as lane keeping and automatic emergency braking), and/or partially or fully allow the vehicle to navigate autonomously (e.g. self driving vehicles). Assist and safety systems, as well as navigation and/or driving systems for fully autonomous vehicles, may use object detection and recognition to help ensure safe navigation and obstacle avoidance. To accomplish object detection and recognition, CA/AD vehicles may be equipped with a variety of sensors that provide data to various vehicle navigation systems. CA/AD vehicles may also include other vehicle types such as unmanned aerial vehicles (commonly referred to as “drones”), which likewise may use object detection and recognition as part of navigation and obstacle avoidance.

Along with various sensors, CA/AD vehicles may be configured for vehicle to vehicle (V2V) and/or vehicle to any (V2X) communications, such as with remote servers, to allow CA/AD vehicles to coordinate movements to help ensure safety. Such communications may be accomplished via radio links of various types, which allows the exchange of data between vehicles, including data from each vehicle's sensors, such as video and range finding.

DESCRIPTION OF EMBODIMENTS

Modern vehicles are often equipped with increasingly sophisticated computer assistance features to aid vehicle operators in situational awareness and safe operation. Such features include lane keeping assistance, blind spot detection, cross-traffic detection, adaptive cruise control, and emergency automatic braking. These various assistance features rely upon a variety of sensors placed upon the vehicle being operated to detect obstacles that are relatively proximate to the vehicle. By expanding the number and types of sensors, and with improvement of on-board processing power, the capabilities of assistance features may range up to fully autonomous driving systems, e.g. self-driving vehicles, with other vehicles offering greater or lesser degrees of autonomous driving.

Such assistance features are constrained to what sensors local to (e.g. on-board) the vehicle can detect, and so are largely ineffective and detecting potential obstacles and hazards that are either beyond the range of the local sensors or are obscured by an intervening obstruction, such as terrain or another vehicle. Consequently, the safety of maneuvers that require a knowledge of relatively distant obstacles, such as on-coming and/or upcoming traffic, cannot be readily ascertained in all scenarios. Knowledge of such distant obstacles could enable evaluation of the safety of vehicle maneuvers such as passing a vehicle, changing lanes, making a turn, or any other maneuver where the presence and relative speed of vehicles or objects ahead of the vehicle must be assessed.

Vehicles may be equipped for communications between vehicles or with remote servers (e.g., V2V or V2X communications) and configured to receive information via wireless communication links with other proximate vehicles. Depending upon the communications technology employed, these wireless communication links may allow vehicles to exchange sensor information, including data streams from video cameras, LIDAR, radar, range fingers, ultrasonic detectors, etc. By exchanging such data streams, a CA/AD vehicle can be enabled to evaluate the safety of proposed maneuvers that may be impacted by oncoming obstacles that are beyond the range of local sensors, or even obstructed by terrain, such as a hill or curve in the road. Proximate vehicle and stationary sources located further down the road from a CA/AD vehicle may be able to see and relay upcoming obstructions to the CA/AD vehicle prior to such obstructions being visible to the CA/AD vehicle's sensors, and so provide the CA/AD vehicle with sufficient time to evaluate the safety of a proposed vehicle maneuver. The evaluated risk can then either be presented to the vehicle operator to enhance the vehicle operator's situational awareness, or, in the context of an autonomous/self-driving vehicle, allow the CA/AD vehicle to determine when safe to perform a maneuver that may be prescribed by the vehicle's navigation system.

As used herein, the term semi-autonomous driving is synonymous with computer-assisted driving. The terms do not mean exactly 50% of the driving functions are automated. The percentage of driving functions automated may be a fraction of a percent to almost 100%.

For the purposes of the present disclosure, the phrase “A or B” means (A), (B), or (A and B). For the purposes of the present disclosure, the phrase “A, B, or C” means (A), (B), (C), (A and B), (A and C), (B and C), or (A, B, and C).

FIG.1depicts a system100for a vehicle to evaluate the risk factors of a proposed maneuver by the vehicle. In various embodiments, a first vehicle102, equipped with an apparatus200that implements a method300as will be described herein, is configured to communicate with one or more remote data sources external to first vehicle102, such as a second vehicle104and/or a stationary source106(which may be proximally located or remotely located). As used herein, references to first vehicle102include functionality provided by apparatus200as installed in first vehicle102. Communications may be handled via any suitable wireless technology for vehicle to vehicle (V2V) or vehicle to any (V2X) communications. As will be described in greater detail herein, such technologies may include millimeter wave, cellular communications, Wi-Fi, or other such technologies that provide sufficient bandwidth to support the exchange of necessary data between the one or more remote data sources and first vehicle102. In some embodiments, communications are direct point-to-point, e.g. first vehicle102and second vehicle104directly exchange data with each other via transceivers, or in other embodiments may be indirect, e.g. first vehicle102and second vehicle104exchange data using some intermediary. For example, first vehicle102and second vehicle104may rely upon a stationary source106to act as a local relay, as depicted inFIG.1. In another example, first vehicle102and second vehicle104may relay data via a cloud or another remote service, such as using a cellular network data link. Other embodiments may utilize a combination of the foregoing, with some sources directly accessible and other sources requiring either a local or remote relay or intermediary.

A person skilled in the relevant art will appreciate that, in various embodiments, the selection of communications technology will depend upon the nature of the data to be exchanged with the one or more remote data sources. For example, remote data sources that offer high-definition video streams will necessitate a comparatively higher bandwidth communications technology compared to remote data sources that offer only standard-definition video. In embodiments, first vehicle102is equipped with communications technology for apparatus200with sufficient bandwidth to handle any expected driving scenario, e.g. the equipped communications technology is selected with respect to the maximum bandwidth required to communicate with any anticipated remote data source. Further, in embodiments, the equipped communication technology is selected with respect to the anticipated physical range of any proposed vehicle maneuvers. In other words, first vehicle102may be equipped with communication technology capable of exchanging data with any second vehicle104or stationary source106that is within the distance required to execute any proposed vehicle maneuver.

In embodiments, first vehicle102is a computer-assisted or autonomous driving (CA/AD) vehicle, with apparatus200providing at least some of the computer-assisted or autonomous driving functionality. Computer assistance may range from driver assist systems similar to blind spot and cross-traffic warnings (and including risk evaluation from apparatus200), may include a level of automatic control, such as an emergency braking system or lane keeping system, or may range to full autonomous driving (a/k/a self driving or driverless) vehicles, which may be assisted by apparatus200in providing an evaluation to the autonomous driving systems of whether to execute a proposed maneuver. Thus, system100in conjunction with apparatus200(described herein below) can provide a risk assessment of a proposed maneuver to a driver or, in some embodiments, may work with an autonomous vehicle's navigation and steering systems to evaluate whether and when it is safe for the vehicle to execute proposed maneuvers. In embodiments, first vehicle102is equipped with apparatus200. Apparatus200may be interfaced with first vehicle102's computer assistance or autonomous driving system. For example, and as will be discussed in greater detail herein, apparatus200may receive input from first vehicle102's navigation system, and/or may provide output via an infotainment system installed within first vehicle102. In addition to passenger CA/AD vehicles, first vehicle102may comprise trucks, vans, busses, and commercial vehicles that may be equipped for computer assisted or autonomous driving. In some embodiments, first vehicle102may include boats, planes, drones, motorcycles, or other suitable conveyances that could be autonomously controlled.

Second vehicle104may be any other vehicle or conveyance located within communications range of first vehicle102. In some examples, second vehicle104may be ahead of first vehicle102, and either traveling in the same general direction as first vehicle102, as depicted inFIG.1, or traveling in the opposite direction, such as third vehicle108. In other examples, second vehicle104may be laterally adjacent to or behind first vehicle104, or traveling the opposite direction. First vehicle102may be in concurrent communication with multiple vehicles, such as second vehicle104as well as a third vehicle108, in some embodiments. As used here, “concurrent communication” does not necessarily mean active transmitting/receiving from multiple vehicles at the same time, but rather that first vehicle102may have ongoing established and authenticated communication sessions with several vehicles at the same time, with actual data exchange occurring either simultaneously or in a sequential order.

Stationary source106may include any relatively fixed location adjacent to, over, or on the roadway that may be equipped with one or more sensors useable by first vehicle102with the apparatus and methods described herein. For example, stationary source106may include light stanchions, street signs, highway signs, overpasses, trees, buildings, terrain, or any other structure or feature proximate to the roadway where sensors can be mounted with a view of the roadway. Stationary source106may include sensors that would not ordinarily be found upon a vehicle, such as weather condition sensors to determine the presence of fog, haze, smoke, or other visual impediments, and roadway conditions such as wet, dry, snowy, icy, under construction, accident blockages/closures, etc., that may become factors in ascertaining the risk of a vehicle maneuver. Stationary, as used here, means stationary relative to the roadway. As a first vehicle102moves along the roadway, the position and distance of a stationary source106with respect to first vehicle102will change, with a stationary source106potentially coming into range and leaving range of first vehicle102.

In embodiments, first vehicle102receives one or more types of data from the various data sources of second vehicle104, stationary source106, and/or third vehicle108. The various data sources may include video, audio, LIDAR, radar, ultrasonic, and similar types of detectors or sensors of second vehicle104, stationary source106, and/or third vehicle108for the external environment of first vehicle102, as well as internal sensors such as speed, acceleration, and equipment status such as whether the brakes are applied, the throttle is applied, turn signal actuation, anti-lock brake system status, rain sensors, wiper actuation, and/or any other similar data from second vehicle104, stationary source106, and/or third vehicle108, that may be used by first vehicle102to assess risk of a maneuver by vehicle102. It will be understood that the types of data will depend upon the nature of the data source. For example, where first vehicle102communicates directly with each source (no intermediary relay), stationary source106would not provide vehicle-specific data points such as speed and brake status. Some data sources may not be equipped with certain sensors, e.g. second vehicle104may only be equipped with a video camera, but not have any range finding sensors such as LIDAR or radar, or vice-versa. Further, other data sources may be equipped with a range of sensors, but may be able to selectively enable or disable broadcasting and/or receiving information to or from one or more of the range of sensors.

FIG.1further depicts the general operation of system100according to embodiments. Per operation150, first vehicle102collects real-time data from the various remote sources external to first vehicle102, which may include second vehicle104, stationary source106, and third vehicle108. Data sources may further include remote sources such as weather and traffic data (e.g. as may be provided by services such as WAZE®). The data is aggregated, merged, reconciled, or otherwise combined, and, in operation152is authenticated as will be discussed in greater detail herein. In operation154, the various remote sources may be cross-correlated, both with each other (e.g. where first vehicle102obtains data from multiple remote sources) as well as with any local sensors on-board first vehicle102. Using authenticated and cross-correlated data, first vehicle102can build a predictive model for predicting/estimating the risk associated with executing a proposed vehicle maneuver in operation156.

Proposed maneuvers may include any maneuvers where the condition of the roadway ahead of first vehicle102is relevant to determining the risk presented by the proposed maneuver. Proposed vehicle maneuvers may include maneuvers such as passing a vehicle, changing lanes, making a turn, or any other maneuver where the presence and relative speed of vehicles or objects ahead of the vehicle must be assessed to determine whether the maneuver may be safely executed.

FIG.2depicts a block diagram of an apparatus200, which may be equipped to first vehicle102, for assessing the risk of a proposed maneuver by first vehicle102, according to various embodiments. In some embodiments, apparatus200includes an authenticator202, a validator204, and a risk evaluator206. Apparatus200may include an object recognizer208to assist the risk evaluator206in processing data from the various remote data sources for risk evaluation. Apparatus200receives data from one or more sensors210, in some embodiments, that are also equipped to first vehicle102. Apparatus200may exchange data from remote sources such as second vehicle104, stationary source106, and third vehicle108via a transceiver220. Apparatus200further may be in communication with a vehicle infotainment system250, which may provide navigation and interaction with a user/operator of the vehicle.

In embodiments, authenticator202handles verifying the source of remote data received through transceiver220, such as data received from second and third vehicles104and108and/or stationary source106. Authentication may include, in some embodiments, receipt or exchange of keys with a remote data source to ensure the data is not coming from a spoofed source. For example, a remote source may utilize a public-private key arrangement (e.g. asymmetric encryption) where the remote source provides a public key to authenticator202and uses its private key to encrypt some or all of the data provided to authenticator202. Other methods may be employed for verification, such as third party authentication, where apparatus200may verify credentials from a remote data source with a third party (e.g. cloud-based or remote) service. Still other embodiments may employ any suitable technique now known or later developed for authenticating data from a remote source. Authenticator202may perform an authentication process for each remote data source received by apparatus200. Authentication may occur on a per-session basis, e.g. each time a remote data source enters into range and apparatus200initiates communication, on a per transmission basis, where the remote data source is authenticated upon each transmission of a data stream, or at such other frequency as appropriate and/or may be programmed for a given implementation.

Validator204, according to embodiments, receives authenticated data from the remote data source, such as via authenticator202, to ensure data reliability. Validator204may receive data from multiple remote data sources, and may further receive data from sensors210. In embodiments, received data may be correlated across sources to ensure that each remote data source is providing not only accurate data, but also data that is directed to the same physical region relevant to the proposed vehicle maneuver. Correlation may include comparing data from a first remote data source, such as second vehicle104, with a second data source, such as data from sensors210local to first vehicle102, or a second remote source, such as stationary source106and/or third vehicle108. For example, where second vehicle104and stationary source106are each equipped with video cameras, their respective video streams may be compared against each other to ascertain whether objects that appear in one video stream likewise appear in the other, with adjustments/transformations performed as necessary to compensate for different camera angles. Similarly, different types of data may be compared, such as objects in a video stream from one remote source correlated with detected objects in a LIDAR or radar scan from a second remote source.

As shown inFIG.2, an object recognizer208may interface with validator204to perform object recognition on data streams from the various data sources, including both remote data sources and data from sensors210. Object recognition, as described above, may allow objects detected in each data source to be correlated to validate each of the various data sources. The particular algorithms employed by object recognizer208may depend upon the nature of the data source being analyzed. Video streams may require a different type of object recognition than data from a LIDAR or radar sensor, for example. Object recognizer208may employ any algorithm now known or later developed that is suitable for use with a given data source.

Once the remote data source or sources have been validated, in various embodiments they are provided to risk evaluator206, which also receives data from various sensors210local to first vehicle102as described above. Risk evaluator206receives a proposed vehicle maneuver and, using the remote data sources and sensors210, determines the risk of the maneuver. As the risk evaluation is based upon object detection, risk is expressed in terms of the likelihood that a proposed maneuver will result in collision with at least one of a detected object. As will be described further herein, risk may be classified into at least three categories: no risk of collision, a moderate risk of collision, and a high risk or imminent collision. Determining between no/low risk, moderate risk, and high risk may be expressed based upon a distance threshold, i.e. how close any portion of vehicle102will come within one or more detected objects. A predetermined distance threshold may be established. Where first vehicle102will likely be further from any detected object than the distance threshold, the risk may be determined as low or none. If first vehicle102will likely travel within the distance threshold to any detected object (viz. closer to the object than the distance threshold), then the risk may be determined as moderate. If first vehicle102will likely collide or otherwise contact a detected object, then the risk may be determined as high, More than one threshold may be established in various embodiments; for example, a second threshold, closer in distance to a detected object, may be used to establish a high risk, rather than requiring a likely collision as the criteria for high risk. In still other embodiments, other variables may be factored into assessing risk, i.e. the likelihood that the maneuver might result in injuries to a pedestrian, an occupant of the vehicle, damages to the vehicle or other vehicles or structures, which can depend upon the speed of first vehicle102.

Local sensor data may include measurements from sensors210relevant to vehicle status and condition such as vehicle speed, acceleration, engine speed (e.g. revolutions per minute), gear, throttle position, brake position, antilock brake system status, traction control status, and any other measurements relevant to evaluating a vehicle maneuver. Further, sensors210may include external sensors appropriate to CA/AD vehicles, such as LIDAR, radar, ultrasonic sensors, range finders, video cameras, and other similar sensors that provide input to a CA/AD vehicle's driver assist and/or navigation systems. Risk evaluator206, as shown inFIG.2, may be in communication with object recognizer208, similar to validator204, and may utilize objects recognized from each remote data source as well as sensors210in determining the risk and risk factors of a proposed vehicle maneuver. Sensors210may be integrated into apparatus200in some embodiments, or in other various embodiments, may be partially or wholly separate from apparatus200.

In embodiments, risk evaluator206evaluates the remote data source or sources, such as via object recognizer208, to locate objects such as vehicles, plants, signs, and other such potential obstacles that are proximate to the path to be taken by first vehicle102in executing the proposed maneuver. In locating such objects, the movement of any such objects, e.g. speed, acceleration, and/or direction of travel, as detected in the remote data sources may also be evaluated. With this evaluation, risk evaluator206may further factor in data from sensors210relevant to the status and condition of first vehicle102, in particular, data indicating first vehicle102′s path and rate of travel. Risk evaluator206may employ geometric algorithms across the data and project the path of first vehicle102in executing the proposed maneuver, as well as the projected path of any detected objects, to determine whether it will result in a collision with any of the detected objects, result in first vehicle102passing unacceptably close to any detected object, or if first vehicle102may travel the projected path without conflict from any detected object.

Whether first vehicle102passes unacceptably close to a detected object may depend upon whether first vehicle102passes within a predetermined threshold for maintaining separation between detected objects and first vehicle102. This predetermined threshold may itself vary depending upon the speed of first vehicle102relative to any obstacles, and may vary on a per-obstacle basis. For example, an obstacle moving towards first vehicle102may have a higher predetermined threshold than a non-moving obstacle, which may in turn have a higher threshold than an obstacle moving away from first vehicle102(which may have no threshold or be a non-factor), as the relative speed of an obstacle moving towards first vehicle102is greater than the speed at which first vehicle102may be approaching a stationary obstacle. The predetermined threshold may vary based upon road conditions, with a threshold being greater where road conditions are adverse, such as wet, icy or snowy pavement, in areas of rain, fog or smoke, or any other environmental conditions that may impact visibility and/or vehicle traction and control. The predetermined threshold may be calculated dynamically by apparatus200, in real-time based upon input from remote data sources and local sensors210, with some embodiments calculating multiple thresholds on a per-object basis.

Where first vehicle102is a computer-assisted vehicle (as opposed to autonomous driving), the proposed maneuver may be determined on the basis of signal inputs from the vehicle operator, as well as detection of road markings. For example, risk evaluator206may detect or be informed that the vehicle operator has actuated the left turn signal. If first vehicle102is maintaining speed, then risk evaluator206may examine lane markings. If the lane marking indicate a two lane road, then risk evaluator206can conclude that the vehicle operator intends to pass a vehicle ahead of first vehicle102. This may be confirmed if first vehicle102is equipped with sensors210that can detect the presence and closing range of a vehicle, such as second vehicle104, in the lane immediately in front of first vehicle102. Apparatus200may initiate communications with second vehicle104to obtain a remote data feed from second vehicle104, that may further include a video feed that apparatus200can use to determine the presence of any vehicles (or other obstacles) ahead of second vehicle104, as well as the possible presence of oncoming traffic. Risk evaluator206can use this information, such as the rate of closure of any oncoming traffic, to determine the risk of attempting a passing maneuver.

In another example, risk evaluator206may determine that the lane markings indicate a multi-lane highway, and with the left turn signal, conclude that the vehicle operator is simply intending to make a lane change. As with a passing maneuver, apparatus200may utilize remote data sources from vehicles and stationary sources ahead of first vehicle102to determine whether the proposed lane change can be made safely, or if an obstruction is present ahead in the lane.

In yet another example, sensors210may indicate to risk evaluator206that the vehicle operator has applied the brakes and is slowing down. In conjunction with the left turn signal, risk evaluator206may conclude that the vehicle operator intends to make a left hand turn. As with a lane change, remote data sources may be used to ascertain whether the left turn can be made safely, or whether there is oncoming traffic that poses a risk of collision. The foregoing are only examples; other possible signals may be ascertained by apparatus200.

In embodiments where first vehicle102is an autonomous driving vehicle, apparatus200may receive input directly from the vehicle's navigation and/or autosteering systems in the form of intended maneuvers, as will be described further herein. In such embodiments, reference to external cues such as lane markings may be unnecessary. In other embodiments, a vehicle operator may directly signal apparatus200of the intended maneuver, such as by voice cues or via an interface.

In some embodiments, apparatus200may not establish contact with remote data sources until it receives an indication of a proposed vehicle maneuver. In other embodiments, apparatus200may establish contact with remote data sources whenever they are sufficiently proximate to apparatus200to establish reliable communications, regardless of whether a proposed vehicle maneuver is indicated. In still other embodiments, contact may be established with remote data sources on the basis of other CA/AD vehicle operations that require or are facilitated by an exchange of data with other vehicles, and so apparatus200may advantageously use such established communications sessions. In some such examples, authenticator202may not need to authenticate remote data sources when the remote data sources are authenticated by another aspect or function of the CA/AD vehicle. In other such examples, authentication may be carried out by authenticator202and relied upon by the other aspect(s) or function(s) of the CA/AD vehicle. It should be understood that any of the foregoing scenarios may be realized by a single apparatus200, viz. some remote data sources may be established and authenticated outside of apparatus200, while apparatus200may handle establishing and authenticating other remote data sources, which may then be used by other aspect(s) or function(s) of the CA/AD vehicle.

Once the risk of a proposed maneuver has been calculated by risk evaluator206, in various embodiments, the assessment is communicated to infotainment system250, and, for embodiments in computer-assisted vehicles, for presentation on a display258to a vehicle operator. In some such embodiments, the risk assessment may be presented to the vehicle operator in a “traffic light” fashion, with relative risks presented as red (indicating that the proposed vehicle maneuver will, barring an evasive maneuver, result in a collision), yellow (indicating that the proposed vehicle maneuver will bring the vehicle within the predetermined threshold distance discussed above with at least one obstacle and so is potentially dangerous), or green (indicating that, barring an unexpected change in direction by any obstacles ahead in the road, the proposed vehicle maneuver may be safely executed without risk of collision). In various embodiments, the red/yellow/green presentation may be conveyed to the vehicle operator via a heads-up display, dash panel/instrument cluster, vehicle navigation screen, dedicated lights, or via any other aspect of display258so that a vehicle operator can readily understand the risk assessment without being unduly distracted from driving. Other embodiments may present the risk assessment in a different fashion; any method of conveying the risk assessment to a vehicle operator without unduly distracting the operator may be employed. In some embodiments, the display of the risk assessment may be customized by the vehicle operator to suit the operator's preferred method of information presentation. Some embodiments may engage in periodic or continuous assessment of risk factors, such as from updated data received from both local sensors210and one or more remote data sources. As the risk assessment is updated, the vehicle operator may likewise be provided with periodic or continuous updates of the risk assessment, such as when conditions change.

It will further be appreciated that, in embodiments where first vehicle102is either an autonomous driving vehicle or a CA/AD vehicle in autonomous mode, such as where apparatus200receives indications of a proposed vehicle maneuver through a navigation unit256, the risk assessment may not be presented to the vehicle operator, as the vehicle operator is not actively controlling first vehicle102in such embodiments. Moreover, in such embodiments apparatus200may forego a “yellow” assessment, and instead simply inform a drive assist/autosteer unit254whether/when it is safe to execute the proposed vehicle maneuver (e.g. a “green” assessment). Risk evaluator206, in such embodiments, may simply calculate a safe/unsafe determination, based upon the predetermined thresholds discussed above; any assessment that would fall into the “red” (e.g. imminent collision) or “yellow” (e.g. high risk of collision) category would result in an unsafe determination.

In addition to the aforementioned display258, drive assist/autosteer unit254, and navigation unit256, infotainment system250may, in various embodiments, include one or more processors252. Drive assist/autosteer unit254and navigation unit256may be any appropriate technology now known or later developed for a CA/AD vehicle that assists a vehicle operator in navigation and control, up to complete autonomous driving. Drive assist/autosteer unit254may provide for little to full control of a first vehicle102, depending upon a given implementation. Navigation unit256may include a GPS receiver along with various other sensors for determining vehicle orientation and position, and may provide input to the drive assist/autosteer unit254to guide first vehicle102to a selected destination. Both drive assist/autosteer unit254and navigation unit256may provide feedback to a vehicle operator via display258, such feedback including driver assist cues and navigational information, e.g. turn by turn navigation instructions.

Display258may be any suitable display technology, such as a touch-screen flat panel LCD, LED, or OLED display in some embodiments. As suggested above, display258need not be a single display; display258may encompass several different devices for providing feedback to a vehicle operator. For example, in some embodiments, display258include both a conventional screen as well as a heads-up display and/or instrument cluster display. Some embodiments may include the instrument cluster as part of display258.

Processor252may be a general purpose or application specific processor. For example, processor252may be a processor504that is part of a computer device500, as described herein with respect toFIG.5. As seen inFIG.2, processor252is in data communication with at least drive assist/autosteer unit254, navigation unit256, and display258. As mentioned above, depending upon the particular embodiment of apparatus200and infotainment system250, processor252may comprise multiple processors. Each block of apparatus200(authenticator202, validator204, risk evaluator206, object recognizer208) and/or block of infotainment system250may utilize one or more of its own processors252. In other embodiments, a single processor252may coordinate the operations of all blocks of apparatus200and/or infotainment system250.

One or more components of apparatus200, including authenticator202, validator204and/or risk evaluator206, in embodiments, include or is implemented using, for example, one or more processors situated in separate components, or alternatively one or more processing cores embodied in a component (e.g., in a System-on-a-Chip (SoC) configuration), and any processor-related support circuitry (e.g., bridging interfaces, etc.). Example processors may include, but are not limited to, various microprocessors such as general-purpose processors that may be used for general-purpose computing, and/or microprocessors that are purpose-built, such as specifically for processing of digital signals, and more specifically for processing of digital audio signals. Examples may include processors of the iAPX family, ARM family, MIPS family, SPARC family, PA-RISC family, POWER family, or any other suitable processor architecture now known or later developed. Still other embodiments may use an application-specific integrated circuit (ASIC) or field-programmable gate array (FPGA).

It should be understood that in other embodiments of apparatus200, authenticator202, validator204, risk evaluator206, and object recognizer208may use a different arrangement of components, including different types. For example, in one particular embodiment, apparatus200is implemented as software, such as instructions604stored on a medium602(described further with respect toFIG.6) to be executed by a computer device500(described herein with respect toFIG.5). In other embodiments, portions of apparatus200may be implemented as software, with other portions implemented in hardware. It will be appreciated the various blocks inFIG.2, including those of apparatus200as well as infotainment system250, are simply logical depictions of functions; the actual implementation of the blocks can vary from embodiment to embodiment, with functions of different blocks potentially being split or combined into one or more software and/or hardware modules. Some of the components may be omitted or moved to other locations, depending upon a given implementation.

Embodiments of apparatus200, as discussed above, are in communication with one or more radios capable of transmitting and receiving signals using various suitable wireless communications techniques. One possible embodiment is transceiver220, which includes receiver222circuitry and transmitter224circuitry. Other embodiments may use a discrete receiver222that is separate from a discrete transmitter224. Communications techniques may involve communications across one or more wireless networks. Some example wireless networks include (but are not limited to) wireless local area networks (WLANs), wireless personal area networks (WPANs), wireless metropolitan area network (WMANs), cellular networks, and satellite networks. In communicating across such networks, the transceiver220(which term contemplates embodiments with integrated receive/transmit circuitry as well as embodiments with either or both discrete receiver222/transmitter224circuitry) may operate in accordance with one or more applicable standards in any version. To this end, the transceiver220may include, for instance, hardware, circuits, software, or any combination thereof that allows communication with external computer systems.

In some specific non-limiting examples, the transceiver220may comport with the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard (e.g., Wi-Fi), a Bluetooth®, ZigBee®, near-field communication, or any other suitable wireless communication standard. In addition, the transceiver220may comport with cellular standards such as 3G (e.g., Evolution-Data Optimized (EV-DO), Wideband Code Division Multiple Access (W-CDMA)) and/or 4G wireless standards (e.g., High Speed Packet Access (HSPA), Worldwide Interoperability for Microwave Access (WIMAX), Long-Term Evolution (LTE)).

Apparatus200, in some embodiments, is a standalone unit, capable of being attached or interfaced with a vehicle. In other embodiments, apparatus200is provided as an integral part of a vehicle, and may be in communication with a vehicle's navigation systems, if so equipped. In still other embodiments, apparatus200may be a part of a vehicle's autonomous driving, computer assistance, or navigation systems.

InFIG.3, a flowchart of the various operations of a method300that may be carried out in whole or in part by an apparatus200is depicted. In operation302, apparatus200may establish communications sessions with, and receive remote data from, one or more remote data sources, such as second vehicle104, third vehicle108, and stationary source106, as described above with respect toFIG.1. The remote data from each of the one or more remote data sources is authenticated in operation304, such as by authenticator202as described above.

In operation306, the authenticated remote data from each of the one or more remote data source is validated and correlated with other remote data sources (where there are a plurality of remote data sources) and/or with local sensor data. The logical operations of operation306will be discussed in more detail below with respect toFIG.4.

In operation308, the correlated and validated remote data is used to evaluate the risk of a proposed vehicle maneuver, as discussed above with respect to the functions of risk evaluator206. In embodiments/implementations where a vehicle operator is controlling first vehicle102, the results of the risk evaluation are displayed or otherwise provided to the vehicle operator in operation310, as described above with respect to risk evaluator206andFIG.2.

FIG.4is a flowchart of the operations that may be executed in whole or in part by an apparatus200that is performing or executing operation306, to validate and correlate the one or more remote data sources. In operation402, object recognition may be performed on each remote data source in the one or more remote data sources, such as by using object recognizer208, as described above. In operation404, object recognition may be performed, such as by object recognizer208, on local data obtained from local sensors, such as sensors210. If local data need not be used, e.g. where there are sufficient authenticated remote data sources to perform correlation and validation, operation404may be omitted. Finally, in operation406, the recognized objects from the remote data sources and/or local data are correlated, with any necessary transforms or mapping of reference frames performed to facilitate correlation.

The operations depicted inFIG.4may be carried out wholly or partially by validator204of apparatus200, or, depending upon the particulars of a given implementation, by another block, such as authenticator202and/or risk evaluator206. Some embodiments may execute operation306of method300using different operations than those depicted inFIG.4, viz. some embodiments may not need to use object recognition, or may use data types that are not well suited to object recognition, and so may be correlated using other techniques, such as pattern matching.

In some embodiments, apparatus200, and in particular, risk evaluator206, may include one or more trained neural networks in performing its determinations and/or assessments.FIG.5illustrates an example neural network, in accordance with various embodiments. As shown, example neural network500may be a multilayer feedforward neural network (FNN) comprising an input layer512, one or more hidden layers514and an output layer516. Input layer512receives data of input variables (xi)502. Hidden layer(s)514processes the inputs, and eventually, output layer516outputs the determinations or assessments (yi)504. In one example implementation the input variables (xi)502of the neural network are set as a vector containing the relevant variable data, while the output determination or assessment (yi)504of the neural network are also as a vector.

Multilayer feedforward neural network (FNN) may be expressed through the following equations:
hoi=f(∈j=1R(iwi,jxj)+hbi), fori=1, . . . ,N
yi=f(∈k=1N(hwi,khok)+obi), fori=1, . . . ,S
where hoiand yiare the hidden layer variables and the final outputs, respectively. f( ) is typically a non-linear function, such as the sigmoid function or rectified linear (ReLu) function that mimics the neurons of the human brain. R is the number of inputs. N is the size of the hidden layer, or the number of neurons. S is the number of the outputs.

The goal of the FNN is to minimize an error function E between the network outputs and the desired targets, by adapting the network variables iw, hw, hb, and ob, via training, as follows:
E=∈k=1m(Ek), whereEk=∈p=1S(tkp−ykp)2
where ykpand tkpare the predicted and the target values of pth output unit for sample k, respectively, and m is the number of samples.

In some embodiments, apparatus200may include a pre-trained neural network500to evaluate various factors such as vehicle speed, speed relative to detected objects, other factors such as vehicle and road conditions, and whether the vehicle will approach a detected object within the predetermined distance threshold. The input variables (xi)502may include objects recognized from the images of the outward facing cameras as well as remote data sources, distance and speed vectors for the recognized objects; the readings of various vehicles sensors, such as accelerometer, gyroscopes, IMU, and so forth, from both local sensors210as well as sensor data received from remote data sources; and parameters of the proposed maneuver to be executed by first vehicle102. The output variables (yi)504may include values indicating whether first vehicle102will collide with any of the recognized objects, and the expected minimum distance that first vehicle102will come within each vehicle while executing the proposed maneuver. The network variables of the hidden layer(s) for the neural network of apparatus200for determining whether first vehicle102will collide with a recognized object and/or the minimum distance first vehicle102will come within each recognized object, are determined by the training data.

In the example ofFIG.5, for simplicity of illustration, there is only one hidden layer in the neural network. In some other embodiments, there can be many hidden layers. Furthermore, the neural network can be in some other types of topology, such as Convolution Neural Network (CNN), Recurrent Neural Network (RNN), and so forth.

Referring now toFIG.6, a possible software component view of a vehicle computer system1000, which may implement apparatus200in software according to various embodiments, is illustrated. As shown, for the embodiments, vehicle computer system1000includes hardware1002which executed software1010in whole or in part. Vehicle computer system1000may provide some or all functionality of infotainment system250described above with respect toFIG.2. Software1010includes hypervisor1012hosting a number of virtual machines (VMs)1022-1028. Hypervisor1012is configured to host execution of VMs1022-1028. The VMs1022-1028include a service VM1022and a number of user VMs1024-1028. Service machine1022includes a service OS hosting execution of a number of instrument cluster applications1032, which may include an instrument cluster display, such as display258for displaying the risk evaluation from apparatus200. User VMs1024-1028may include a first user VM1024having a first user OS hosting execution of front seat infotainment applications1034, a second user VM1026having a second user OS hosting execution of rear seat infotainment applications1036, a third user VM1028having a third user OS hosting execution of a vehicle maneuver risk evaluator1038(implementing apparatus200), and so forth.

Except for apparatus200providing a vehicle maneuver risk evaluator1038of the present disclosure incorporated, elements1012-1036of software1010may be any one of a number of these elements known in the art. For example, hypervisor1012may be any one of a number of hypervisors known in the art, such as KVM, an open source hypervisor, Xen, available from Citrix Inc, of Fort Lauderdale, Fla., or VMware, available from VMware Inc of Palo Alto, Calif., and so forth. Similarly, service OS of service VM1022and user OS of user VMs1024-1028may be any one of a number of OS known in the art, such as Linux, available e.g., from Red Hat Enterprise of Raliegh, N.C., or Android, available from Google of Mountain View, Calif.

Referring now toFIG.7, an example computing platform that may be suitable for use to practice the present disclosure, according to various embodiments, is illustrated. As shown, computing platform1100, which may be hardware1002ofFIG.6, may include one or more system-on-chips (SoCs)1102, ROM1103and system memory1104. Each SoCs1102may include one or more processor cores (CPUs), one or more graphics processor units (GPUs), one or more accelerators, such as computer vision (CV) and/or deep learning (DL) accelerators. ROM1103may include basic input/output system services (BIOS)1105. CPUs, GPUs, and CV/DL accelerators may be any one of a number of these elements known in the art. Similarly, ROM1103and BIOS1105may be any one of a number of ROM and BIOS known in the art, and system memory1104may be any one of a number of volatile storage known in the art.

Additionally, computing platform1100may include persistent storage devices1106. Example of persistent storage devices1106may include, but are not limited to, flash drives, hard drives, compact disc read-only memory (CD-ROM) and so forth. Further, computing platform1100may include one or more input/output (I/O) interfaces1108to interface with one or more I/O devices, such as sensors1120. Other example I/O devices may include, but are not limited to, display, keyboard, cursor control and so forth. Computing platform1100may also include one or more communication interfaces1110(such as network interface cards, modems and so forth). Communication devices may include any number of communication and I/O devices known in the art. Examples of communication devices may include, but are not limited to, networking interfaces for Bluetooth®, Near Field Communication (NFC), WiFi, Cellular communication (such as LTE 4G/5G) and so forth. The elements may be coupled to each other via system bus1112, which may represent one or more buses. In the case of multiple buses, they may be bridged by one or more bus bridges (not shown).

Each of these elements may perform its conventional functions known in the art. In particular, ROM1103may include BIOS1105having a boot loader. System memory1104and mass storage devices1106may be employed to store a working copy and a permanent copy of the programming instructions implementing the operations associated with hypervisor1012, service/user OS of service/user VM1022-1028, and components of apparatus200(such as authenticator202, validator204, risk evaluator206, object recognizer208, and so forth), collectively referred to as computational logic1122. The various elements may be implemented by assembler instructions supported by processor core(s) of SoCs1102or high-level languages, such as, for example, C, that can be compiled into such instructions.

As will be appreciated by one skilled in the art, the present disclosure may be embodied as methods or computer program products. Accordingly, the present disclosure, in addition to being embodied in hardware as earlier described, may take the form of an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to as a “circuit,” “module” or “system.” Furthermore, the present disclosure may take the form of a computer program product embodied in any tangible or non-transitory medium of expression having computer-usable program code embodied in the medium.FIG.8illustrates an example computer-readable non-transitory storage medium that may be suitable for use to store instructions that cause an apparatus, in response to execution of the instructions by the apparatus, to practice selected aspects of the present disclosure. As shown, non-transitory computer-readable storage medium1202may include a number of programming instructions1204. Programming instructions1204may be configured to enable a device, e.g., computing platform1100, in response to execution of the programming instructions, to implement (aspects of) hypervisor1012, service/user OS of service/user VM1022-1028, and components of vehicle maneuver risk evaluator1038/apparatus200(such as authenticator202, validator204, risk evaluator206, object recognizer208, and so forth). In alternate embodiments, programming instructions1204may be disposed on multiple computer-readable non-transitory storage media1202instead. In still other embodiments, programming instructions1204may be disposed on computer-readable transitory storage media1202, such as, signals.

EXAMPLES

The following examples pertain to further embodiments.

Example 1 is an apparatus for computer-assisted or autonomous driving (CA/AD) vehicle, comprising an authenticator, disposed in a CA/AD vehicle, to authenticate a remote data source external to the CA/AD vehicle; a validator, disposed in the CA/AD vehicle, to validate at least one video stream received from the authenticated remote data source, through correlation of the at least one video stream with sensor data from one or more sensors in communication with the apparatus; and a risk evaluator, disposed in the CA/AD vehicle, to evaluate risk factors of a proposed vehicle maneuver, based at least in part on the at least one validated video stream and sensor data.

Example 2 includes the subject matter of example 1, or some other example herein, wherein the validator is to validate the at least one video stream through correlation of one or more objects recognized via object recognition from the at least one video stream, with one or more objects recognized from the sensor data.

Example 3 includes the subject matter of example 1 or 2, or some other example herein, wherein the risk evaluator is to evaluate the risk factors of the proposed vehicle maneuver based at least in part on the one or more objects recognized from the at least one video stream, and the sensor data.

Example 4 includes the subject matter of any of examples 1-3, or some other example herein, wherein the risk evaluator is to evaluate the risk factors of the proposed vehicle maneuver based at least in part on one or more of: location of each of the one or more recognized objects, path being traveled by each of the one or more recognized objects, speed of the one or more recognized objects, road surface conditions, and weather conditions.

Example 5 includes the subject matter of any of examples 1-4, or some other example herein, wherein the risk evaluator is in communication with a display device, and is to cause the display device to present an indication of the evaluated risk factors to an operator of the CA/AD vehicle.

Example 6 includes the subject matter of any of examples 1-5, or some other example herein, wherein the validator is to further validate the at least one video stream with a second video stream from a second authenticated remote data source.

Example 7 includes the subject matter of any of examples 1-6, or some other example herein, wherein the one or more sensors are sensors located on the CA/AD vehicle, and the validator is further to validate remote sensor data received from the authenticated remote data source.

Example 8 includes the subject matter of any of examples 1-7, or some other example herein, wherein the risk evaluator is to evaluate the risk factors based at least in part on the remote sensor data.

Example 9 includes the subject matter of any of examples 1-8, or some other example herein, wherein the one or more sensors include one or more of a LIDAR sensor, a video camera, a radar, or a depth sensor.

Example 10 includes the subject matter of any of examples 1-9, or some other example herein, wherein the remote data source comprises another CA/AD vehicle, or a stationary camera external to the CA/AD vehicle.

Example 11 includes the subject matter of any of examples 1-10, or some other example herein, wherein the apparatus is part of an in-vehicle infotainment system of the CA/AD vehicle.

Example 12 is a method, comprising authenticating a remote data source external to a computer-assisted or autonomous driving (CA/AD) vehicle; receiving remote sensor data from the authenticated remote data source; correlating the remote sensor data with local sensor data from one or more sensors located on the CA/AD vehicle to obtain validated sensor data; and evaluating risk factors of a proposed vehicle maneuver based at least in part upon the validated sensor data.

Example 13 includes the subject matter of example 12, or some other example herein, wherein the remote sensor data comprises at least one video stream.

Example 14 includes the subject matter of example 12 or 13, or some other example herein, wherein correlating further comprises performing object recognition upon the at least one video stream; and validating the local sensor data with recognized objects from the at least one video stream.

Example 15 includes the subject matter of any of examples 12-14, or some other example herein, wherein evaluating risk factors of the proposed vehicle maneuver comprises evaluating the risk factors based at least in part on the one or more objects recognized from the at least one video stream, and the sensor data.

Example 16 includes the subject matter of any of examples 12-15, or some other example herein, wherein evaluating the risk factors comprises evaluating the risk factors of the proposed vehicle maneuver based at least in part on one or more of: location of each of the one or more recognized objects, path being traveled by each of the one or more recognized objects, speed of the one or more recognized objects, road surface conditions, and weather conditions.

Example 17 includes the subject matter of any of examples 12-16, or some other example herein, further comprising displaying a risk assessment based upon the evaluated risk factors to an operator of the CA/AD vehicle.

Example 18 includes the subject matter of any of examples 12-17, or some other example herein, further comprising determining whether to execute the proposed vehicle maneuver based upon the evaluated risk factors.

Example 19 includes the subject matter of any of examples 12-18, or some other example herein, further comprising validating the at least one video stream with a second video stream from a second authenticated remote data source.

Example 20 includes the subject matter of any of examples 12-19, or some other example herein, wherein the one or more sensors are sensors located on the CA/AD vehicle, and further comprising validating remote sensor data received from the authenticated remote data source.

Example 21 includes the subject matter of any of examples 12-20, or some other example herein, further comprising evaluating the risk factors based at least in part on the remote sensor data.

Example 22 includes the subject matter of any of examples 12-21, or some other example herein, wherein the one or more sensors include one or more of a LIDAR sensor, a video camera, a radar, or a depth sensor.

Example 23 includes the subject matter of any of examples 12-22, or some other example herein, wherein the remote data source comprises another CA/AD vehicle, or a stationary camera external to the CA/AD vehicle.

Example 24 includes the subject matter of any of examples 12-23, or some other example herein, wherein the method is performed in whole or in part by an in-vehicle infotainment system of the CA/AD vehicle.

Example 25 is a transitory or non-transitory computer readable medium (CRM) containing instructions executable by a processor in a computer-assisted or autonomous driving (CA/AD) vehicle, that when executed cause the processor to authenticate a remote data source external to the CA/AD vehicle; validate remote sensor data received from the authenticated remote data source, the remote sensor data including at least one video stream, through correlation of the remote sensor data with local sensor data from one or more sensors located upon the CA/AD vehicle in communication with the apparatus; and evaluate risk factors of a proposed vehicle maneuver based at least in part upon the validated remote sensor data and local sensor data.

Example 26 includes the subject matter of example 25, or some other example herein, wherein the instructions are further to cause the processor to evaluate the risk factors of the proposed vehicle maneuver based at least in part on one or more of: location of each of one or more objects recognized from the validated remote sensor data and local sensor data, a path being traveled by each of the one or more recognized objects, a speed of the one or more recognized objects, road surface conditions, and weather conditions.

Example 27 includes the subject matter of example 25 or 26, or some other example herein, wherein the instructions are further to cause the processor to display to an operator of the CA/AD vehicle a visual indication of a risk assessment based upon the evaluated risk factors.

Example 28 includes the subject matter of any of examples 25-27, or some other example herein, wherein the processor is part of an in-vehicle infotainment system.

Example 29 includes the subject matter of any of examples 25-28, or some other example herein, wherein the remote sensor data comprises at least one video feed from a video camera external to the CA/AD vehicle.

Example 30 includes the subject matter of any of examples 25-29, or some other example herein, wherein the instructions are further to cause the processor to evaluate the risk factors of the proposed vehicle maneuver based at least in part on the one or more objects recognized from the at least one video stream, and the sensor data.

Example 31 includes the subject matter of any of examples 25-30, or some other example herein, wherein the instructions are further to cause the processor to validate the at least one video stream with a second video stream from a second authenticated remote data source.

Example 32 includes the subject matter of any of examples 25-31, or some other example herein, wherein the one or more sensors are sensors located on the CA/AD vehicle, and the instructions are further to cause the processor to validate remote sensor data received from the authenticated remote data source.

Example 33 includes the subject matter of any of examples 25-32, or some other example herein, wherein the instructions are further to cause the processor to evaluate the risk factors based at least in part on the remote sensor data.

Example 34 includes the subject matter of any of examples 25-33, or some other example herein, wherein the one or more sensors include one or more of a LIDAR sensor, a video camera, a radar, or a depth sensor.

Example 35 includes the subject matter of any of examples 25-34, or some other example herein, wherein the remote data source comprises another CA/AD vehicle, or a stationary camera external to the CA/AD vehicle.

Example 36 includes the subject matter of any of examples 25-35, or some other example herein, wherein the instructions on the CRM are executed in whole or in part by an in-vehicle infotainment system of the CA/AD vehicle.

Example 37 is an apparatus for a computer-assisted or autonomous driving (CA/AD) vehicle, comprising means to authenticate each of a plurality of remote data sources external to the CA/AD vehicle; means to validate remote sensor data received from each of the plurality of authenticated remote data sources, through correlation of the remote sensor data received from each of the plurality of authenticated remote data sources with the remote sensor data received from the remaining authenticated remote data sources; and means to evaluate risk factors of a proposed vehicle maneuver based at least in part upon the validated remote sensor data.

Example 38 includes the subject matter of example 37, or some other example herein, wherein the remote sensor data comprises at least two video streams.

Example 39 includes the subject matter of example 37 or 38, or some other example herein, wherein the means to validate is to correlate the remote sensor data through: object recognition performed on a first of the at least two video streams; object recognition performed on a second of the at least two video streams; and a comparison of objects recognized in the first video stream with objects recognized in the second video stream.

Example 40 includes the subject matter of example 39, or some other example herein, wherein the means to evaluate risk factors is to evaluate the risk factors of the proposed vehicle maneuver based at least in part on the one or more objects recognized from the at least two video stream, and the remote sensor data.

Example 41 includes the subject matter of any of examples 37-40, or some other example herein, wherein the means to evaluate risk factors is to evaluate the risk factors of the proposed vehicle maneuver based at least in part on one or more of: location of each of the one or more recognized objects, path being traveled by each of the one or more recognized objects, speed of the one or more recognized objects, road surface conditions, and weather conditions.

Example 42 includes the subject matter of any of examples 37-41, or some other example herein, wherein the means to evaluate risk is in communication with a display device, and is to cause the display device to present an indication of the evaluated risk factors to an operator of the CA/AD vehicle.

Example 43 includes the subject matter of any of examples 37-42, or some other example herein, further comprising one or more sensors located on the CA/AD vehicle, and wherein the means to validate is further to validate the remote sensor data with sensor data from the one or more sensors located on the CA/AD vehicle.

Example 44 includes the subject matter of any of examples 37-43, or some other example herein, wherein the means to evaluate risk is to evaluate the risk factors based at least in part on the sensor data from the one or more sensors located on the CA/AD vehicle.

Example 45 includes the subject matter of example 43 or 44, or some other example herein, wherein the one or more sensors include one or more of a LIDAR sensor, a video camera, a radar, or a depth sensor.

Example 46 includes the subject matter of any of examples 37-45, or some other example herein, wherein at least one of the plurality of remote data source comprises another CA/AD vehicle, or a stationary camera external to the CA/AD vehicle.

Example 47 includes the subject matter of any of examples 37-46, or some other example herein, wherein the apparatus is part of an in-vehicle infotainment system of the CA/AD vehicle.