Vehicle networking system testing and characterization

Disclosed are systems, methods, and non-transitory computer-readable media for testing and characterization of computing systems implemented in vehicles. A testing system causes packet injectors in a vehicle networking system to execute a testing mode in which the packet injectors transmit synchronized sequences of data packets within the vehicle networking system. The testing system gathers, from logging mechanisms located within the vehicle networking system, diagnostic data describing data packet transmissions in the vehicle computer network during the testing mode. The diagnostic data includes data identifying data packets detected by the logging mechanisms and timestamps indicating times at which the data packets were detected by the logging mechanisms. The testing system generates, based on the diagnostic data and an expected diagnostic data, a testing report describing performance of the vehicle networking system during the testing mode.

TECHNICAL FIELD

An embodiment of the present subject matter relates generally to computing and networking systems implemented in vehicles and, more specifically, to testing and characterization of computer networking systems implemented in vehicles.

BACKGROUND

Modern vehicles include many computer managed features. For example, vehicles include computers that monitor and/or control engine emissions, tire pressure, throttle position, engine temperature, spark plugs, fuel injection, automatic transmission, anti-lock brakes, automated driving, etc. Vehicles also include computers that manage non-critical luxury functions, such as keyless entry, climate control, motorized seats and mirrors, entertainment systems (e.g., radio, compact disk player), cruise control, etc.

To provide these computer managed features, vehicles are equipped with multiple sensors that continuously gather data and provide the data to computers (e.g., Electronic Control Unit (ECU)) included in the vehicle which in turn provide control command for actuators. For example, a computer that manages the anti-lock brake system uses data gathered from sensors located on the tires that read wheel speed and control the brake system. As another example, a computer that manages the climate control system gathers data from temperature sensors. Computers that manage advanced functions, such as automated driving, gather data from multiple sensors located at various points on the vehicle to control steering wheel, brakes, warning systems, etc.

One technical issue faced when implementing computer managed features in vehicles is available bandwidth for transmitting data. Vehicles are equipped with limited networking ability (e.g., wiring) to transmit data between the various computers and sensors located within the vehicle. This limited bandwidth may cause system latency through various portions of the network. This becomes an important issue when providing mission critical features, such as automated driving, that rely on time-sensitive data to make real-time decisions. For example, an automated driving system uses real-time sensor data gathered from multiple sensors (e.g, camera, sonar, radar, lidar, etc.) to determine the current surroundings of the vehicle and guide the direction of the vehicle accordingly. Network latency results in the automated driving system functioning without the data needed to accurately determine the vehicles surrounding, which may lead to a collision.

Current systems address this issue with network features that prioritize data for mission critical functions. For example, the network features provide priority to data packets being used for mission critical features such as automated driving over data packets being used for non-critical functions such as the entertainment system. While these network features provide a solution, determining whether they are functioning properly or as desired is difficult. Accordingly, improvements are needed.

DETAILED DESCRIPTION

In the following description, for purposes of explanation, various details are set forth in order to provide a thorough understanding of some example embodiments. It will be apparent, however, to one skilled in the art, that the present subject matter may be practiced without these specific details, or with slight alterations.

Disclosed are systems, methods, and non-transitory computer-readable media for testing and characterization of computing systems and networks implemented in vehicles (e.g., vehicle networking systems). Vehicle networking systems have limited available bandwidth because vehicles are equipped with limited wiring for transmitting data between computers, sensors, actuators, etc., located within the vehicle. Accordingly, a vehicle networking system is designed to prioritize transmission of certain types of data packets to ensure optimal performance of mission critical features. There are various functions in time-sensitive networking (TSN) standards (e.g. IEEE 802.1) that, if deployed and configured properly, can guarantee an upper limit in the latency of packet transmission in specific flows for pairs of source and destination nodes in an Ethernet network and provide the required quality of service (QoS) in an automotive network environment.

It is important to have a mechanism to monitor and log the flow of traffic within a vehicle networking system to characterize the nature of data traffic within the vehicle networking system which provides the information to identify the proper configuration of TSN and other networking functions for optimal network performance. It is also important to monitor the state of hardware and software elements of the vehicle networking system as data packets flow within the vehicle networking system. This provides valuable information in refining the configuration of network elements (e.g., nodes) for improved performance. It also provides the means to test the networking elements to ensure they behave as expected. This can be a very important mechanism to provide the high level of functional safety that is needed within vehicle computing networks. Some networking functions are stressed under certain traffic patterns while in a specific configuration state (e.g., specific configuration of hardware and software elements of the vehicle networking system) which may happen only under very rare scenarios. It is important to have the means to create those traffic patterns along with the proper state of networking elements in order to validate the proper operation of the key network functionalities. It is important to have programmable test traffic generators in the networking nodes. It is also important to have a mechanism to preset the state of some or all of the network elements in order to monitor the behavior of the elements under desired configuration states. Since the relative timing of data packets plays an important role in how the network elements behave, it is important to have the capability to synchronize the test traffic generators with each other and with the configuration state of the network elements.

The vehicle networking system includes a testing system that periodically tests the vehicle networking system to ensure that the included network features, such as network traffic prioritization, are performing as expected. For example, the testing system initiates a testing mode during which controlled network traffic is transmitted within the vehicle networking system to exercise and stress various critical network features. The features under test may be features related to quality of service (QoS) such as scheduling, policing, shaping. These features may also be other network functions such as packet encapsulation, editing, forwarding, filtering, etc. The testing system gathers diagnostic data describing data packet transmissions during the testing mode and analyzes the diagnostic data to determine whether the hardware and/or software in the vehicle networking system is performing as expected. For example, the testing system compares the diagnostic data to expected diagnostic data based on the controlled network traffic to determine whether the exercised network features performed as expected. In some embodiments, each element (e.g., hardware and/or software function) of the vehicle networking system is capable of logging key information about the data packet that traverse through the element, as well as information describing the state of the function at that time. The logged information may include the time of arrival of the data packet, the time of departure of the data packet, the size of the data packet, the header of the data packet, the encapsulation, etc. The state of the network function may be the depth of the queues, the credit or token of the traffic shapers, the state of the gates, etc.

The testing system includes preset mechanisms that configure the vehicle networking system into a predetermined configuration state for a testing mode and programmable packet injectors that can be instructed to transmit synchronized sequences of predetermined data packets during a testing mode. The packet injectors transmit data packets according to a programmable list. Each entry of this list provides key information about each test packet to be sent. The information may include the size of the packet, the destination for the packet, other header components of the packet, and any other fields of the packet, the time to send the packet (which may be an absolute time for the first entry but may be incremental time on the following entries), etc. Once the list is exhausted, the packet injector may be programmed to halt or repeat the list from the top after a programmable delay. Similarly, the functional elements of the network can have a list of preset configurations. Each entry of the list identifies which state should be set with what value and at what time. Again, once the list is exhausted, the element may be programmed to stop presetting its state or continue from the top of the list. The programmable packet injectors and the preset mechanisms operate according to internal clocks that are synchronized using a timing protocol, such as the Precision Timing Protocol (PTP) or its generalized version gPTP.

Each programmable packet injector receives a sequence of data packets to be transmitted by the respective packet injector, as well as times at which the data packets are to be transmitted. The programmable packet injectors transmit the data packets at the corresponding times based on the synchronized internal clocks, thereby creating the desired network traffic to test network features of the vehicle networking system.

The testing system further includes logging mechanisms that detect and log transmission of data packets within the vehicle networking system. For example, the logging mechanisms log relevant hardware and software information during the testing mode. The logged diagnostic data describes data packet transmissions through the vehicle networking system during the testing mode. For example, each logging mechanism logs data packets as they travel through the logging mechanism (e.g., packet arrival time, departure time, buffer backlog, credit, gate states, route, etc.). Note that each functional element of the network may include a logging mechanism so that the entire history of a packet from the source to destination is logged.

The testing system gathers the logged diagnostic data from the logging mechanisms and analyzes the diagnostic data to characterize performance of the vehicle networking system and determine whether the vehicle networking system is performing as desired. For example, the testing system may use the logged diagnostic data to determine the time and order in which data packets were received and transmitted by various nodes in the vehicle networking system and compare the determined time and order to an expected order in which the data packers were to be received and transmitted. The testing system may further generate a testing report describing the determined performance of the vehicle networking system.

FIG. 1shows a vehicle networking system100, according to some example embodiments. To avoid obscuring the inventive subject matter with unnecessary detail, various functional components (e.g., modules, mechanisms, devices, nodes, etc.) that are not germane to conveying an understanding of the inventive subject matter have been omitted fromFIG. 1. However, a skilled artisan will readily recognize that various additional functional components may be supported by the vehicle networking system100to facilitate additional functionality that is not specifically described herein.

The vehicle networking system100is a collection of nodes distributed within a vehicle (e.g., automobile) and interconnected via a communication network102comprising communication links and segments for transporting data between end points, such as sensors104, actuators, and computing devices106. Each node in the vehicle networking system100may be a redistribution point or an endpoint that can receive, create, store or send data along distributed network routes. Each node, whether an endpoint or a redistribution point, has either a programmed or engineered capability to recognize, process and forward data transmissions to other nodes in the vehicle networking system100. Examples of nodes include sensors104, displays, actuators, computing devices106, routers (not shown), switches (not shown), etc.

The communication network102is implemented using any number of nodes and communications links, including one or more wired communication links, one or more wireless communication links, or any combination thereof. Additionally, the communication network102is configured to support the transmission of data formatted using any number of protocols.

Multiple sensors104, computing devices106, and actuators110can be connected to the communication network102. A computing device106is any type of general computing device capable of network communication with other computing devices. For example, a computing device106can include some or all of the features, components, and peripherals of the computing system600shown inFIG. 6.

To facilitate communication with other computing devices106, a computing device106includes a communication interface configured to receive a communication, such as a request, data, and the like, from another computing device106or sensor104in network communication with the computing device106and pass the communication along to an appropriate module running on the computing device106. The communication interface also sends a communication to another computing device106in network communication with the computing device106.

The sensors104may be any type of sensors used to capture data. For example, the sensors104may include engine speed sensors, fuel temperature sensors, voltage sensors, pressure sensors, radar sensors, light detection and ranking (LIDAR) sensors, imaging sensors (e.g., camera, video camera), etc. The sensors104capture data describing performance of a vehicle and its surroundings and provide the captured data to one or more of the computing devices106.

The computing devices106use the captured sensor data to provide various computer managed features. For example, the computing devices106may use the gathered sensor data to monitor and/or control engine emissions, tire pressure, throttle position, engine temperature, spark plugs, fuel injection, automatic transmission, anti-lock brakes, automated driving, etc. The computing devices106may also use the gathered sensor data to provide non-critical luxury functions, such as keyless entry, climate control, motorized seats and mirrors, entertainment system (e.g., radio, compact disk player), cruise control, etc.

The actuators110are hardware components that are responsible for executing an a mechanical/electrical action, such as moving and controlling a mechanism or system. Examples of actuators110include an on/off switch (e.g. door locks, lights, etc.), electric motors (e.g. side mirror, seat and steering wheel control), etc. The computing devices106transmit commands to the actuators to perform a specified action. This category of network devices also include any device that mostly consume data, such as video displays and audio speakers.

As previously explained, the communication network102may have limited available bandwidth for transmitting data. Accordingly, the vehicle networking system100is designed to ensure quality of service (QoS) for certain types of data packets to ensure optimal performance of mission critical features. For example, the various nodes included in the communication network102may provide priority to sensor data used for providing automated driving to ensure that the computing device106facilitating the automated driving function is concurrently receiving sensor data captured from each sensor at a given time.

As shown, the vehicle networking system100includes a testing system108. The testing system108periodically tests the vehicle networking system100to ensure that the included network features, such as network traffic prioritization, are performing as expected. One critical component of testing system108is the capability to monitor and log diagnostic data about packet traffic along with the state of functional elements of the network as the packets traverse them. This diagnostic data may be gathered for all or some packets as they traverse through all of some of the functional elements of the network. The diagnostic data that the functional elements record may include: arrival and departure time of the packets, header and other critical components of the packet, the critical state of the functional element (such as the depth of queues, the credit or tokens of the traffic shapers, etc.) as the packet was processed by them. The diagnostic data may be forwarded through dedicated diagnostic and management ports to the testing system108. Alternatively, the diagnostic data may be transmitted over a logical channel (e.g., Operation Administration and Maintenance (OAM)) over the main data port. The testing system108may continuously monitor the network for this diagnostic data. Alternatively, the testing system108may periodically command (e.g., through an OAM channel) the network nodes to log and send their diagnostic data. The testing system108may monitor the network as the network is operating normally with its typical data traffic. Alternatively, the testing system108may initiate (e.g., through a multicast request over an OAM channel) a special testing mode during which controlled network traffic is generated and transmitted within the vehicle networking system100to exercise and stress various critical network features. In this special testing mode, the testing system108may also send a request to preset some or all states of some or all function elements of the network. For instance, the testing system108may request to preset the credit of a certain credit-based traffic shaper. The request to generate the test traffic and preset the state of function elements may be carried over an OAM channel. The request can include the details of test packet generation and the state of elements across the network. The request may also carry time information so that the test packet generators may be synchronized with each other and/or with the time of preset of the function elements. The synchronization of the disjoint elements of the network may be achieved through a timing mechanism such as Precision Timing Protocol (PTP). The testing system108gathers diagnostic data describing data packet transmissions during the testing and monitoring modes. This information can be used to characterize the network traffic in different modes of operation of the vehicle. This characterization can help identify an optimized configuration of the network elements in each mode of operation of the vehicle. The gathered diagnostic data can also be analyzed to determine whether the hardware and/or software in the vehicle networking system100is performing as expected. For example, the testing system108compares the diagnostic data to expected values based on the controlled network traffic to determine whether the exercised network features performed as expected. This may provide additional means to guarantee a target functional safety of the network operation in the vehicle. Additionally, the diagnostic data may also be used to identify atypical traffic patterns which may indicate a malicious network intrusion, providing additional level of security for the automotive networking system.

While the testing system108is shown separately from the sensors104, computing devices106and communication network102, this is just for ease of explanation and is not meant to limiting. The testing system108may be incorporated throughout the vehicle networking system100, meaning that the various components of the testing system108may be incorporated into any of the nodes of the vehicle networking system100.

The testing system108includes preset mechanisms that configure the vehicle networking system into a predetermined configuration state for a testing mode and programmable packet injectors that can be instructed to transmit synchronized sequences of predetermined data packets during a testing mode. The programmable packet injectors operate according to internal clocks that are synchronized using a timing protocol, such as the Precision Timing Protocol (PTP). Each programmable packet injector receives a sequence of data packets to be transmitted by the respective packet injector, as well as times at which the data packets are to be transmitted. The programmable packet injectors transmit the data packets at the corresponding times based on the synchronized internal clocks, thereby creating the desired network traffic to test the vehicle networking system100.

The testing system108further includes logging mechanisms that detect and log transmission of data packets within the vehicle networking system100. For example, the logging mechanisms log relevant hardware information during the testing mode. The logged diagnostic data describes data packet transmissions through the vehicle networking system100during the testing mode. For example, each logging mechanism logs data packets as they travel through the logging mechanism (e.g., packet arrival time, departure time, buffer backlog, credit, gate states, etc.).

The testing system108gathers the logged diagnostic data from the logging mechanisms and analyzes the diagnostic data to characterize performance of the vehicle networking system100and determine whether the vehicle networking system100is performing as desired. For example, the testing system108may use the logged diagnostic data to determine an order in which data packets were received and transmitted by various nodes in the vehicle networking system100and compare the determined order to an expected order in which the data packers were to be received and transmitted. The testing system108may further generate a testing report describing the determined performance of the vehicle networking system.

FIG. 2is a block diagram of the testing system108, according to some example embodiments. To avoid obscuring the inventive subject matter with unnecessary detail, various functional components (e.g., modules, mechanisms, devices, nodes, etc.) that are not germane to conveying an understanding of the inventive subject matter have been omitted fromFIG. 2. However, a skilled artisan will readily recognize that various additional functional components may be supported by the testing system108to facilitate additional functionality that is not specifically described herein. Furthermore, the various functional modules depicted inFIG. 2may reside on a single computing device or may be distributed across several computing devices in various arrangements such as those used in cloud-based architectures.

As shown, the testing system108includes a management module202, programmable packet injectors204, preset mechanisms206, logging mechanisms208, and a synchronization mechanism210. The management module202manages the functionality of the testing system108. For example, the management module202causes execution of testing modes in which various network features of the vehicle networking system100are exercised and stressed to ensure that the network features are performing as expected. During a testing mode, the management module202may cause the vehicle networking system100to be configured into a predetermined configuration state corresponding to the testing mode. Configuring the vehicle networking system100into a configuration state includes configuring the state of specified nodes in the vehicle networking system100to properly test one or more network functions of the vehicle networking system100. For example, configuring the vehicle networking system100into a configuration state may include configuring specified hardware first in first outs (FIFOs), data buffers, etc.

The management module202configures the vehicle networking system100using preset mechanisms206. The present mechanisms206are implemented within the vehicle networking system100and are configured to reconfigure the state of nodes in the vehicle networking system100. The management module202transmits commands to one or more of the preset mechanisms206to configure the vehicle networking system100. The commands include instructions on how each respective preset mechanism206should configure one or more nodes of the vehicle networking system100for a given testing mode. The functionality of the preset mechanisms206are described in greater detail below.

In addition to presetting the functional elements of the network in the vehicle networking system100into the appropriate configuration state for a testing mode, the management module202causes transmission of sequences of data packets within the vehicle networking system100to stress and exercise specified features of the vehicle networking system100. For example, the management module202transmits commands to one or more of the programmable packet injectors204to initiate transmission of the sequence of data packets. The command may include data describing the sequence of data packets, such as the size of the individual data packets, header information (e.g., destination address, priority, etc.), payload data, etc., as well as data describing how the data packets are to be transmitted. For example, the command may include a time at which the programmable packet injectors204should initiate transmission of the sequence of data packets and data identifying the sequence in which the data packets are to be transmitted and/or the times at which individual data packets are to be transmitted.

The management module202may transmit the command to initiate the testing mode and data describing the sequence of data packets as a single message or as separate messages. Further, the management module202may transmit the data describing the sequence of data packets using a messaging protocol such as Operation Administration and Maintenance (OAM). For example, the data describing the sequence of data packets may be included in the payload of an OAM message. The functionality of the programmable data packet injectors is described in greater detail below.

In addition to causing execution of the testing modes, the management module202gathers diagnostic data describing behavior and performance of the vehicle networking system100during the testing mode. The diagnostic data describes how the data packet traversed through the functional elements of the network in the vehicle networking system100during the testing mode. The diagnostic data may also include information about the state of the functional elements as each packet was traversing through them. The management module202analyzes the diagnostic data to determine whether the tested traffic and networking elements are behaving as expected. For example, the management module202compares the diagnostic data to an expected data. The management module202further generates a testing report describing performance of the vehicle networking system100during the testing mode. The management module202may also transmit notifications indicating any detected issues with the tested network features of the vehicle networking system100. The functionality of the management module202is described in greater detail below in references toFIG. 3.

The programmable packet injectors204transmit sequences of data packets during a testing mode. Each programmable packet injector204may be implement in a node in the vehicle networking system100. That is, a programmable packet injector204may be implemented as part of any node in the network such a sensor node, a computing node, or switching (or bridging, routing) node, etc. The programmable packet injectors204receive commands from the management module202to execute a testing mode and execute the testing mode in accordance with data included in the command. For example, the programmable packet injectors204transmit a sequence of data packets described in a command received from the management module202. As explained earlier, the commands include data describing the size, payload, header, transmission time, etc., for each data packet in the sequence of data packets. The programmable packet injectors204transmit the sequence of data packets according to the provided description in the command.

The preset mechanisms206configure the state of some or all functional elements of some or all networking nodes in the vehicle networking system100. Each preset mechanism206may be implement as additional software or hardware features in some or all nodes (e.g., sensors, switches, etc.) in the vehicle networking system100. The preset mechanisms206receive commands from the management module202to configure the vehicle networking system100into a configuration state for a testing mode. The command includes data identifying the nodes to be configured and any corresponding value. For example, the command may identify a buffer to be configured as well as token values for configuring the identified buffer. The command may also include data identifying a time at which a preset mechanism206is to perform the configuration. The present mechanisms206configure the functional elements of the network of the vehicle networking system100according to the provided description in the command.

During a testing mode, synchronization amongst the various programmable packet injectors204and preset mechanisms206is crucial to properly test a specified network feature of the vehicle networking system100. To create the desired network traffic that causes the network feature to be triggered, configuration of the affected nodes of the vehicle networking system100and transmission of the sequences of data packets should be synchronized precisely. Accordingly, the programmable packet injectors204and preset mechanisms206support a time-based triggering mechanism that allows for synchronization. For example, the programmable packet injectors204and preset mechanisms206may include or have access to internal clocks that have been synchronized. For example, the internal clocks may be synchronized by the synchronization mechanism210using a timing protocol, such as the Precision Timing Protocol (PTP). Accordingly, the programmable packet injectors204and preset mechanisms206use the internal clocks available to them to ensure that configuration of the vehicle networking system100and transmission of the sequences of data packets are synchronized.

The logging mechanisms208record the diagnostic data captured during a testing mode. Each logging mechanisms208may be a standalone hardware or software feature in a networking node in the vehicle networking system100. Each logging mechanism208logs relevant hardware information as a data packet travels through the respective logging mechanism208. For example, the logging mechanisms208may log the packet arrival time, departure time, buffer backlog, credit, gate states, etc.

The synchronization mechanism210synchronizes multiple internal clocks in the vehicle networking system100. Various hardware nodes in the vehicle networking system100may have access to their own internal clock. Accordingly, functions performed by the node may be synchronized using their respective internal clock. However, the internal clocks on the varying nodes may not be synchronized with each other. This may lead to issues when conducting a testing mode. Accordingly, the synchronization mechanism210synchronizes the internal clocks using a timing protocol, such as PTP or gPTP.

FIG. 3is a block diagram of the management module202, according to some example embodiments. To avoid obscuring the inventive subject matter with unnecessary detail, various functional components (e.g., modules, mechanisms, devices, nodes, etc.) that are not germane to conveying an understanding of the inventive subject matter have been omitted fromFIG. 3. However, a skilled artisan will readily recognize that various additional functional components may be supported by the management module202to facilitate additional functionality that is not specifically described herein. Furthermore, the various functional modules depicted inFIG. 3may reside on a single computing device, node, switch, etc., or may be distributed across several computing devices, nodes, switches, etc., in various arrangements. For example, instances of the management module202may be implemented on multiple switches included in the vehicle networking system100.

As shown, the management module202includes a testing determination module302, a testing mode initiation module304, an diagnostic data collection module306, an diagnostic data analysis module308, and a report generation module310.

The testing determination module302determines when to initiate a testing mode in the vehicle networking system100. The test may be non-intrusive and consists solely of collection of diagnostic data during normal data traffic. It may be intrusive and include additional test traffic for better characterization of the network. Execution of the intrusive testing mode may be preferable during periods of time when the vehicle is stationary, idle, or in low use. The testing determination module302monitors use to the vehicle to determine when the vehicle is in an appropriate state to execute a testing mode. For example, the testing determination module302receives data from one or more sensors104of the vehicle networking system100to determine the current state of the vehicle, such as whether the vehicle is idle, moving, etc.

The testing determination module302then determines whether the current state of the vehicle is an appropriate state in which a testing mode may be initiated. For example, the testing determination module302may have access to a listing of predetermined states in which a testing mode may be initiated. The listing mode may include varying states for different tests of the features of the vehicle networking system100. Accordingly, a suitable state in which to execute a testing mode to test one feature of the vehicle networking system100may not be a suitable state in which to execute a testing mode to test another feature of the vehicle networking system100.

The testing determination module302may further determine when to execute testing modes based on a predetermined timing schedule and/or prioritization of the various testing modes. That is, the testing determination module302may determine whether to execute a testing mode or which testing mode to execute based on a period of time that has elapsed since previous execution of the testing modes and/or relative importance of the feature to be tested. Accordingly, the testing determination module302may determine to execute a testing mode if both the vehicle is in a suitable state to execute the testing mode and at least a threshold period of time has elapsed since the testing mode was previously executed. Further, in situations in which the testing determination module302may have to select which testing mode to initiate, the testing determination module302may use the predetermined prioritization to select the testing mode that tests the feature that is determined to have greater importance. Accordingly, testing of mission critical features may be given higher priority than testing of other features.

In response to determining that a testing mode should be initiated, the testing determination module302notifies the testing mode initiation module304to execute the testing mode. The notification provided by the testing determination module302to the testing mode initiation module304may include data identifying the testing modes that should be executed. For example, the notification may include a unique identifier that identifies the testing modes to be executed.

The testing mode initiation module304initiates a testing mode in response to receiving a notification from the testing determination module302. To initiate the testing mode, the testing mode initiation module304initially configures the vehicle networking system100for execution of the testing mode. For example, the testing module initiation module304identifies a configuration state in which to configure the nodes of the vehicle networking system100to execute the testing mode and communicates with the appropriate preset mechanisms206to configure the vehicle networking system100accordingly. For example, the configuration state may include specified configurations for the hardware first in first outs (FIFOs), data buffers, etc.

The testing initiation module304identifies the configuration state in which to configure the vehicle networking system100for a given testing mode based on a predetermined listing of configuration states that correspond to the various testing modes. For example, the predetermined listing of configuration states may list the unique identifiers associated with the various testing modes along with the configuration state corresponding to the testing mode. The configuration states listed in the predetermined listing of configuration states may include data identifying the preset mechanisms206that the testing module initiation module304should communicate with to configure the vehicle networking system100into the appropriate configuration state, as well as data describing the configuration state, such as the nodes to be configured and any values used to configure the nodes. Additionally, the predetermined listing of configuration states may include command messages to be transmitted to configure the vehicle networking system100into the configuration state. Accordingly, the testing module initiation module304uses the data included in the predetermined listing of configuration states to generate and transmit commands to the preset mechanisms206or causes transmission of the commands included in the predetermined listing of configuration states.

The testing mode initiation module304also identifies the programmable packet injectors204to use to initiate the testing mode as well as the sequences of data packets to be transmitted by each of the identified programmable packet injectors204. For example, the testing mode initiation module304uses a listing of data packet sequences that correspond to the various testing modes. For example, the predetermined listing of data packet sequences may list the unique identifiers associated with the various testing modes along with the data packet sequences corresponding to the testing mode. The data packet sequences listed in the predetermined listing of data packet sequences may include data identifying the programmable packet injectors204that the testing module initiation module304should communicate with to initiate the testing mode, as well as data describing the data packet sequences that should be transmitted by each identified programmable packet injector during the testing mode. Additionally, the predetermined listing of data packet sequences may include command messages to be transmitted to the programmable data packet injectors204to initiate the testing mode. Accordingly, the testing module initiation module304uses the data included in the predetermined listing of data packet sequences to generate and transmit commands to the programmable packet injectors204or causes transmission of the commands included in the predetermined listing of data packet sequences.

In some embodiments, the testing mode initiation module304determines times at which the preset mechanisms206are to configure the vehicle networking system100and the data packet injectors204are to initiate the testing mode (e.g., begin transmission of the sequence of data packets). The testing mode initiation module304includes the determined times in the commands sent to the preset mechanisms206and the data packet injectors204. As explained earlier, the preset mechanisms206and the data packet injectors204may have access to internal clocks that are synchronized using a timing protocol. Accordingly, the preset mechanisms206and the data packet injectors204use the times included in the commands along with the internal clocks to synchronize configuration of the vehicle networking system100and execution of the testing mode.

The output collection module306gathers diagnostic data captured by the logging mechanisms206during the testing mode. The diagnostic data describes transmission of data packets, such as packet arrival times, packet departure times, components of packet header, buffer backlog, credit, gate states, etc.

The diagnostic data analysis module308analyzes the diagnostic data to determine whether the tested network functions are performing as expected. For example, the diagnostic data analysis module308may use the timestamp value (e.g., packet arrival and/or departure times) to determine and characterize performance of the vehicle networking system100during the testing mode, such as determining an order in which the sequence of data packets was transmitted and/or received by the various nodes of the vehicle networking system100. The diagnostic data analysis module308may compare this determined sequence to an expected sequence based on the testing mode. The diagnostic data analysis module308identifies discrepancies between the detected order in which data packets were received and the expected order in which the data packets were expected to be received to determine whether the tested network feature is performing as desired or expected.

The report generation module310generates a testing report based on the analysis of the diagnostic data analysis module308. The testing report may include data describing the performance of the tested feature (e.g., whether the tested feature performed as expected) as well as the underlying diagnostic data used during the analysis. For example, the testing report may include data characterizing performance of the vehicle networking system100during the testing mode. The testing report may further include data indicating discrepancies between the determined performance and an expected performance of the vehicle networking system100. In instances in which a tested network feature is determined to be faulty, the report generation module310may transmit or cause presentation of a warning notification indicating that the network feature is not performing as expected.

FIG. 4is a flowchart showing a method400of initiating a testing mode in a vehicle networking system100, according to certain example embodiments. The method400may be embodied in computer readable instructions for execution by one or more processors such that the operations of the method400may be performed in part or in whole by the testing system108; accordingly, the method400is described below by way of example with reference thereto. However, it shall be appreciated that at least some of the operations of the method400may be deployed on various other hardware configurations and the method400is not intended to be limited to the testing system108.

At operation402, the testing determination module302gathers sensor data describing a current state of the vehicle. The testing determination module302determines when to initiate a testing mode in the vehicle networking system100. Execution of the testing mode may be preferable during periods of time when the vehicle is stationary, idle, or in low use. The testing determination module302monitors use to the vehicle to determine when the vehicle is in an appropriate state to execute a testing mode. For example, the testing determination module302receives data from one or more sensors104of the vehicle networking system100to determine the current state of the vehicle, such as whether the vehicle is idle, moving, etc.

At operation404, the testing determination module302determines that the current state is a suitable state to execute a testing mode. For example, the testing determination module302may have access to a listing of predetermined states in which a testing mode may be initiated.

At operation406, the testing mode initiation module304transmits commands to present mechanisms206to configure the vehicle networking system100to a configuration state. To initiate the testing mode, the testing mode initiation module304initially configures the vehicle networking system100for execution of the testing mode. For example, the testing module initiation module304identifies a configuration state in which to configure the nodes of the vehicle networking system100to execute the testing mode and communicates with the appropriate preset mechanisms206to configure the vehicle networking system100accordingly. For example, the configuration state may include specified configurations for the hardware first in first outs (FIFOs), data buffers, etc.

The testing module initiation module304identifies the configuration state in which to configure the vehicle networking system100for a given testing mode based on a predetermined listing of configuration states that correspond to the various testing modes. For example, the predetermined listing of configuration states may list the unique identifiers associated with the various testing modes along with the configuration state corresponding to the testing mode. The configuration states listed in the predetermined listing of configuration states may include data identifying the preset mechanisms206that the testing module initiation module304should communicate with to configure the vehicle networking system100into the appropriate configuration state, as well as data describing the configuration state, such as the nodes to be configured and any values used to configure the nodes. Additionally, the predetermined listing of configuration states may include command messages to be transmitted to configure the vehicle networking system100into the configuration state. Accordingly, the testing module initiation module304uses the data included in the predetermined listing of configuration states to generate and transmit commands to the preset mechanisms206or causes transmission of the commands included in the predetermined listing of configuration states.

At operation408, the testing mode initiation module304transmits commands to packet injectors204to execute a testing mode. The testing mode initiation module304identifies the programmable packet injectors204to use to initiate the testing mode as well as the sequences of data packets to be transmitted by each of the identified programmable packet injectors204. For example, the testing mode initiation module304uses a listing of data packet sequences that correspond to the various testing modes. For example, the predetermined listing of data packet sequences may list the unique identifiers associated with the various testing modes along with the data packet sequences corresponding to the testing mode. The data packet sequences listed in the predetermined listing of data packet sequences may include data identifying the programmable packet injectors204that the testing module initiation module304should communicate with to initiate the testing mode, as well as data describing the data packet sequences that should be transmitted by each identified programmable packet injector during the testing mode. Additionally, the predetermined listing of data packet sequences may include command messages to be transmitted to the programmable data packet injectors204to initiate the testing mode. Accordingly, the testing module initiation module304uses the data included in the predetermined listing of data packet sequences to generate and transmit commands to the programmable packet injectors204or causes transmission of the commands included in the predetermined listing of data packet sequences.

In some embodiments, the testing mode initiation module304determines times at which the preset mechanisms206are to configure the vehicle networking system100and the data packet injectors204are to initiate the testing mode (e.g., begin transmission of the sequence of data packets). The testing mode initiation module304includes the determined times in the commands sent to the preset mechanisms206and the data packet injectors204. As explained earlier, the preset mechanisms206and the data packet injectors204may include of have access to internal clocks that are synchronized using a timing protocol. Accordingly, the preset mechanisms206and the data packet injectors204use the times included in the commands along with the internal clocks to synchronize configuration of the vehicle networking system100and execution of the testing mode.

FIG. 5is a flowchart showing a method500of analyzing diagnostic data of a testing mode in a vehicle networking system100, according to certain example embodiments. The method500may be embodied in computer readable instructions for execution by one or more processors such that the operations of the method500may be performed in part or in whole by the testing system108; accordingly, the method500is described below by way of example with reference thereto. However, it shall be appreciated that at least some of the operations of the method500may be deployed on various other hardware configurations and the method500is not intended to be limited to the testing system108.

At operation502, the diagnostic data collection module306gathers diagnostic data from logging mechanisms208. The output collection module306gathers diagnostic data captured by the logging mechanisms206during the testing mode. The diagnostic data describes transmission of data packets, such as packet arrival times, packet departure times, packet payloads, buffer backlog, credit, gate states, etc.

At operation504, the diagnostic data analysis module308analyzes the gathered diagnostic data. The diagnostic data analysis module308analyzes the diagnostic data to determine whether the tested network functions are performing as expected. For example, the diagnostic data analysis module308may use the timestamp value (e.g., packet arrival and/or departure times) to determine and characterize performance of the vehicle networking system100during the testing mode, such as determining an order in which the sequence of data packets was transmitted and/or received by the various nodes of the vehicle networking system100. The diagnostic data analysis module308may compare this determined sequence to an expected sequence based on the testing mode. The diagnostic data analysis module308identifies discrepancies between the detected order in which data packets were received and the expected order in which the data packets were expected to be received to determine whether the tested network feature is performing as desired or expected.

At operation506, the report generation module310generated a testing report. The testing report may include data describing the performance of the tested feature (e.g., whether the tested feature performed as expected) as well as the underlying diagnostic data used during the analysis. For example, the testing report may include data characterizing performance of the vehicle networking system100during the testing mode. The testing report may further include data indicating discrepancies between the determined performance and an expected performance of the vehicle networking system100. In instances in which a tested network feature is determined to be faulty, the report generation module310may transmit or cause presentation of a warning notification indicating that the network feature is not performing as expected.

Note that the methods described above for diagnostic logging and network testing and characterization may be done in a hierarchical fashion. It is possible and may be advantageous in certain conditions that each node or a group of nodes of a network execute their own test (using the methods described above) and log their diagnostic information (using the schemes described above) to characterize and validate the network traffic and the behavior of functional elements of the network within that node or group of network nodes. The test information may be elevated up to a bigger subset of the network nodes and subnetworks for expanded test coverage and broader network characterization, which among other things, would include an additional test coverage of the connectivity between the nodes and subnetworks. This hierarchy can extend up until it covers the entire network within a vehicle. The mechanisms explained in this document are not limited to network within a vehicle. The same hierarchical approach may also extend the coverage beyond the network within a vehicle and covers vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2X) networks and vehicle-to-cloud as well.

Modules, Components and Logic

Electronic Apparatus and System

Machine Architecture

FIG. 6is a diagrammatic representation of a machine in the example form of a computer system600within which a set of instructions for causing the machine to perform any one or more of the methodologies discussed herein may be executed. The computer system600may include instructions for causing the machine to perform any one or more of the methodologies discussed herein. In alternative embodiments, the machine operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine may operate in the capacity of a server or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.

The machine may, for example, be a personal computer (PC), a PDA, a cellular telephone, a smart phone (e.g., iPhone®), a tablet computer, a web appliance, a handheld computer, a desktop computer, a laptop or netbook, a set-top box (STB) such as provided by cable or satellite content providers, a wearable computing device such as glasses or a wristwatch, a multimedia device embedded in an automobile, a Global Positioning System (GPS) device, a data enabled book reader, a video game system console, a network router, switch or bridge, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

The example computer system600includes a processor602(e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both), a main memory604, and a static memory606, which communicate with each other via a bus608. The computer system600may further include a video display610(e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)). The computer system600also includes one or more input/output (I/O) devices612, a location component614, a drive unit616, a signal generation device618(e.g., a speaker), and a network interface device620. The I/O devices612may, for example, include a keyboard, a mouse, a keypad, a multi-touch surface (e.g., a touchscreen or track pad), a microphone, a camera, and the like.

The location component614may be used for determining a location of the computer system600. In some embodiments, the location component614may correspond to a GPS transceiver that may make use of the network interface device620to communicate GPS signals with a GPS satellite. The location component614may also be configured to determine a location of the computer system600by using an internet protocol (IP) address lookup or by triangulating a position based on nearby mobile communications towers. The location component614may be further configured to store a user-defined location in main memory604or static memory606. In some embodiments, a mobile location enabled application may work in conjunction with the location component614and the network interface device620to transmit the location of the computer system600to an application server or third-party server for the purpose of identifying the location of a user operating the computer system600.

In some embodiments, the network interface device620may correspond to a transceiver and antenna. The transceiver may be configured to both transmit and receive cellular network signals, wireless data signals, or other types of signals via the antenna, depending on the nature of the computer system600.

The drive unit616includes a machine-readable medium622on which is stored one or more sets of data structures and instructions624(e.g., software) embodying or used by any one or more of the methodologies or functions described herein. The instructions624may also reside, completely or at least partially, within the main memory604, the static memory606, and/or the processor602during execution thereof by the computer system600, with the main memory604, the static memory606, and the processor602also constituting machine-readable media.

Consistent with some embodiments, the instructions624may relate to the operations of an operating system (OS). Depending on the particular type of the computer system600, the OS may, for example, be the iOS® operating system, the Android® operating system, a BlackBerry® operating system, the Microsoft® Windows® Phone operating system, Symbian® OS, or webOS®. Further, the instructions624may relate to operations performed by applications (commonly known as “apps”), consistent with some embodiments. One example of such an application is a mobile browser application that displays content, such as a web page or a user interface using a browser.

Transmission Medium

The instructions624may further be transmitted or received over a network626using a transmission medium. The instructions624may be transmitted using the network interface device620and any one of a number of well-known transfer protocols (e.g., HTTP). Examples of communication networks include a LAN, a WAN, the Internet, mobile telephone networks, plain old telephone service (POTS) networks, and wireless data networks (e.g., WiFi and WiMax networks). The term “transmission medium” shall be taken to include any intangible medium that is capable of storing, encoding, or carrying the instructions624for execution by the machine, and includes digital or analog communications signals or other intangible media to facilitate communication of such software.

Reference in the specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present subject matter. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment” appearing in various places throughout the specification are not necessarily all referring to the same embodiment.

For purposes of explanation, specific configurations and details are set forth in order to provide a thorough understanding of the present subject matter. However, it will be apparent to one of ordinary skill in the art that embodiments of the subject matter described may be practiced without the specific details presented herein, or in various combinations, as described herein. Furthermore, well-known features may be omitted or simplified in order not to obscure the described embodiments. Various examples may be given throughout this description. These are merely descriptions of specific embodiments. The scope or meaning of the claims is not limited to the examples given.