Patent Description:
Automotive safety organizations and insurance claim adjustors have a great interest in understanding the events surrounding a vehicle crash. Typically, experts have attempted to reconstruct a crash using measurements taken at the crash site after the crash. Reconstructing and understanding the details of a crash after the fact is difficult, time-consuming, and often inaccurate. Advances in vehicle telematics make it possible to collect vehicle motion data before, during, and after a crash.

<CIT>describes a telematics system that prepares event reports, e.g., for vehicle crashes.

<CIT> describes a device for automatic crash notification.

<CIT>describes detecting airbag deployment resulting from a vehicle crash.

The present invention is directed to the method for automatically generating a documentation of a crash involving a vehicle defined in claim <NUM> and the vehicle telematics system defined in claim <NUM>. The dependent claims depict advantageous embodiments of the present invention.

We use the term "vehicle crash" broadly to include, for example, any impact or set of impacts between a vehicle and one or more objects such as another vehicle, a pedestrian, an animal, a stationary object, or road debris, and any impact or set of impacts between a vehicle and the ground, such as during a vehicle rollover.

We use the term "vehicle crash period" broadly to include, for example, any period of time that spans some or all vehicle impacts occurring in the course of a vehicle crash. In some examples, the vehicle crash period can also include periods of time prior to and leading to an initial vehicle impact and periods of time after and resulting from a final vehicle impact.

We use the term "trip data" broadly to include, for example, a segment of a vehicle's telematics data accumulated during a trip between two locations. In some examples, data accumulated during a vehicle crash period represents a part or segment of trip data.

We use the term "vehicle" broadly to include, for example, any kind of ground conveyance such as a car, truck, bicycle, motorcycle, or recreational vehicle, among others.

We use the term "telematics data" broadly to include, for example, any kind of information about vehicle motion, vehicle state, operator state or behavior, or other information captured at the vehicle, for instance, and communicated wirelessly to another device or location at or remote from the vehicle. In some cases, telematics data includes information that has been captured at the vehicle and processed at the vehicle to derive other telematics data, such as summary telematics data.

We use the term "crash metrics" broadly to include, for example, any measure, figure of merit, or other value that characterizes any aspect or feature of a vehicle impact, vehicle crash, or a vehicle motion, vehicle state, operator state or behavior, or other information associated with a vehicle crash. In some cases, the crash metrics are based on telematics data or other information for a vehicle during a vehicle crash period. The crash metrics can be determined from the telematics data using algorithms, formulas, translations, or any other computational process.

In some implementations, the technology described here uses telematics data or an analysis of telematics data (and in some cases other information) to detect a vehicle crash, a vehicle crash period, a start and an end of the vehicle crash period, one or more crash metrics for the vehicle crash period, and one or more crash scores for a vehicle crash. Based on the telematics data, the results of the analysis, the crash metrics, or the scores, or combinations of them, the technology automatically, for example, generates and provides one or more human-readable documentations (e.g., descriptions) of the vehicle crash that present telematics data, the results of the analysis, the crash metrics, or the scores, or other information, or combinations of them. Other activities can also be performed. We sometimes refer to the technology (hardware, software, or both) as a "vehicle crash system" or simply as "the system.

As shown in <FIG>, a vehicle crash during a trip may be considered to occur during a vehicle crash period beginning at a time T0 prior to a first impact <NUM> at a time T1 of a vehicle <NUM> that is involved in the vehicle crash. The vehicle's location, speed, acceleration, or trajectory, the state of the vehicle, the state and behavior of an operator of the vehicle, and a variety of other parameters beginning at time T0, may have at least partially led directly and within a short period of time (from T0 to T1) to the first impact. Although in theory activities and events prior to T0 may have led to or contributed to the accident indirectly, they are of less interest to a user trying to understand the causes and nature of the vehicle crash. Although these earlier periods could also be taken into account and used by the system, in the examples that we discuss below, the focus is on the vehicle crash period beginning at time T0.

Typically the period from T0 to T1 has a duration less than <NUM> seconds (for example, a duration in the range from <NUM> second to <NUM> seconds), but in some cases could be longer than <NUM> seconds. In the illustrated example, the first impact <NUM> of the vehicle <NUM> could be an impact against a guard rail. After the first impact, a second impact <NUM>, say against a second vehicle <NUM> in the example, occurs at a time T2. After the second impact, the vehicle <NUM> may travel further to a stopping point <NUM> at the end T3 of the vehicle crash period. Even though, in some sense, the crash has ended just after the second impact, activities and events during the period from T2 to T3 can provide important information about the nature and cause of the vehicle crash. For this reason, in typical examples, the technology is applied to the period from T2 to T3 in addition to the period from T0 to T2. The vehicle crash period is typically only a part of the time that elapses during a complete vehicle trip. In some cases the total duration of a vehicle crash can be as short as five seconds (or even shorter) and can be as long as <NUM> seconds (or even longer).

The details of the motion of the vehicle (e.g., locations, speeds, and accelerations in various directions) during the vehicle crash period and information about the state of the vehicle, the state and behavior of the operator of the vehicle, and other factors, some of which can be acquired as telematics data are indicative of the timing and occurrence of each of the impacts of the vehicle crash, the effects of the vehicle crash, and the causes of the vehicle crash, among other things. The system acquires and uses the telematics data and other information to detect the vehicle crash, a vehicle crash period, a start and an end of the vehicle crash period, one or more crash metrics for the vehicle crash period, and one or more crash scores for a vehicle crash. As noted earlier, based on the telematics data, the results of the analysis, the crash metrics, or the scores, or combinations of them, the technology automatically, for example, generates and provides one or more human-readable documentations (e.g., descriptions) of the vehicle crash that present telematics data, the results of the analysis, the crash metrics, or the scores, or other information, or combinations of them.

As shown in <FIG>, the automated crash documentation system <NUM> may include hardware components <NUM>, software applications <NUM>, and data communication channels <NUM> for communicating data between the hardware components and software applications of devices <NUM> that are part of the system. Some of the components and devices may be implemented as computer-readable storage mediums <NUM> containing computer-readable instructions for execution by one or more processors <NUM> within the system <NUM>. The system <NUM> shown in <FIG> may include additional, fewer, or alternate components, including those discussed above and below. Note that the elements shown at the upper left of <FIG> are representative of components and devices that may be distributed among the vehicle,.

the server, the other computer, and other components shown in <FIG> and described here.

As shown in <FIG>, the system <NUM> may include a variety of devices that cooperate to perform the activities and functions described here. The devices can include a telematics device <NUM> in or on a vehicle <NUM>, a mobile device <NUM>, a server <NUM>, and a computer <NUM>. The telematics device <NUM> may include sensors and modules <NUM> to measure, process, and communicate telematics data associated with the vehicle <NUM>. For example, the telematics device <NUM> may include (as the sensors and specialized modules <NUM>) one or more position sensors, such as a Global Positioning System (GPS), to detect locations and speeds of a vehicle, accelerometers to detect accelerations of a vehicle in one or more dimensions, gyroscopes, speed sensors, or barometric sensors, among others. Generally the telematics device <NUM> may include any number of other sensors or modules to detect data related to a state of a vehicle or a state or behavior of an operator of the vehicle, such as one or more weight sensors, engine sensors, alternator sensors, vibration sensors, voltage sensors, oxygen sensors, biometric sensors, electronic control unit (ECU) devices, cameras, or microphones, or combinations of them, among others. The telematics device <NUM> can also include memory <NUM> and one or more processors <NUM> to process and store data and a transceiver <NUM> to enable wired or wireless communications with other components or devices of the system <NUM>, as discussed below.

In some cases, the telematics device <NUM> may be an aftermarket telematics device connected, for example, through an On-Board Diagnostics (OBD) port <NUM> of the vehicle <NUM> or an OEM telematics device that is installed during manufacture of the vehicle <NUM>. In some examples, the telematics device <NUM> may be a tag device placed or affixed in the vehicle <NUM>, such as tags of the kind described in <CIT>. In some implementations, the telematics device <NUM> may include a smartphone, wearable device, or other device that is portable <NUM> and may not necessarily be a dedicated telematics device and may not necessarily be connected to an OBD port of the vehicle. The telematics device <NUM> can be battery-powered, connected into the electrical system of the vehicle <NUM>, or both.

In some implementations, the telematics device <NUM> may communicate telematics data over a wireless channel <NUM> to a mobile device <NUM> or through a mobile device to a server. The mobile device <NUM> may be a portable computing device, such as a smartphone, tablet computer, laptop computer, or wearable computing device, brought into the vehicle <NUM> temporarily, for example, by an operator of the vehicle. To facilitate communications, a wired or wireless communication channel <NUM> such as Bluetooth, WiFi, Radio-Frequency Identification (RFID), or Near-Field Communication (NFC), or combinations of them, for example, may be established between the telematics device <NUM> and the mobile device <NUM>. The mobile device <NUM> is not required to be present in the vehicle <NUM> at all times, however, since the telematics device <NUM> can collect and store data for later transfer to the mobile device <NUM> (and through the mobile device to a server) when the mobile device is present and communicatively coupled. Furthermore, although the telematics device <NUM> is described as being separate from the mobile device <NUM>, in some implementations the functions of the telematics device <NUM> and the mobile device <NUM> are combined by, for example, using GPS, accelerometers, barometers, gyroscopes, or other sensors and modules built into the mobile device <NUM>. In this manner, the mobile device <NUM> can obtain telematics data associated with the vehicle <NUM> in place of or as a supplement to the data obtained by the telematics device <NUM>.

After receiving or otherwise obtaining telematics data associated with the vehicle <NUM>, the mobile device <NUM> can, in some implementations, transmit <NUM> the telematics data to a central server <NUM> over a network <NUM>, which may be the Internet, a cellular network, a local area network, a wide area network, a satellite network, or any other suitable data transmission network, or combinations of them. The central server <NUM> may then store the telematics data, along with other user and vehicle information, in a database <NUM> that can communicate with the central server <NUM>. The database <NUM> may be implemented using one or more non-transitory computer-readable storage mediums including, but not limited to, hard disk drives, solid state drives, optical storage drives, or any combination thereof.

Referring to <FIG>, the central server <NUM> may include hardware and software components, such as one or more processors <NUM>, a memory <NUM>, and a communication interface <NUM>, which are interconnected by a data bus <NUM>. The memory <NUM> can be any non-transitory computer-readable storage medium and may store computer-readable instructions executable by the processor(s) <NUM>. The memory <NUM> may store executable instructions associated with an impact and crash detection module <NUM> (which we sometimes call more simply a "crash detection module"), a crash period module <NUM>, an impact and crash metrics module <NUM> (which we sometimes call more simply a "crash metrics module"), a crash scoring module <NUM>, and an impact and crash documentation module <NUM> (which we sometimes call more simply a "crash documentation module") to enable the central server <NUM> or other components and devices to carry out the techniques described here, such as processing telematics data and other information to detect whether the vehicle <NUM> experienced a vehicle impact or a vehicle crash, determining crash metrics for the vehicle crash period, and automatically providing a human-readable documentation of the vehicle impact or vehicle crash. We sometimes use the phrase "vehicle crash" alone or with other words to refer to a vehicle crash or a vehicle impact or both.

We use the term "module" broadly to include, for example, any code, program, software object, or other software device or arrangement that can be executed by processor to perform one or more activities, functions, or facilities.

The central server <NUM> or other components or devices can use the communication interface <NUM> to transmit and receive raw or processed data, such as the telematics data, the crash metrics, and the human-readable documentation of the vehicle crash, among other information, to and from other components or devices of the system <NUM>. For instance, the central server <NUM> or other devices or components may transmit or receive data to or from a database <NUM> for storage purposes, to or from the mobile device <NUM> using the network <NUM>, to or from the telematics device <NUM> or to or from a remote computing device <NUM> using a network <NUM>, which may be the Internet or any other suitable data transmission network.

The remote computing device <NUM> may include one or more computing devices or servers or both associated with, for example, an automotive safety organization, an insurance company, an emergency service, a user of the mobile device <NUM>, or an owner of the vehicle <NUM>, or combinations of these parties. The central server <NUM> can also provide the mobile device <NUM>, the telematics device <NUM>, the remote computing device <NUM>, or combinations of them with software or an interface, such as an application programming interface (API) or a web service, which allows for visualization, interaction, or further processing of the data, for example.

Although the central server <NUM> is described as processing the telematics data associated with the vehicle <NUM>, other components and devices of the system <NUM>, such as the mobile device <NUM> or the remote computing device <NUM>, may process the telematics data in addition to, in combination with, or instead of the central server <NUM> to carry out the techniques described here. Although only one central server <NUM>, one database <NUM>, and one remote computing device <NUM> are illustrated in <FIG>, the system <NUM> may include any number of computing devices and data storage devices (located in a single place or distributed and) communicatively connected using any number of networks.

As shown in <FIG>, a process for automatically generating a human-readable documentation of a vehicle crash can include activities, for example a sequence of activities, that include detecting one or more vehicle impacts <NUM> individually or as part of a vehicle crash, determining from the one or more vehicle impacts that a vehicle crash has occurred <NUM>, calculating a vehicle crash period <NUM>, calculating crash metrics related to the vehicle crash <NUM>, calculating vehicle crash scores <NUM>, and automatically generating human readable documentation of the vehicle crash <NUM>. In some instances, the automated crash documentation system <NUM> may provide a human-readable documentation of a vehicle crash for a given vehicle crash period, such as vehicle crash period T0-T3 shown in <FIG>. In some cases, the system may use, for example, the crash detection module <NUM> to first detect one or more likely vehicle impacts or vehicle crashes over an entire vehicle trip (step <NUM>). The system may then provide a human-readable documentation of some or all of the likely vehicle crashes including vehicle impacts during the trip.

Referring to <FIG>, detecting vehicle impacts, vehicle crashes, and vehicle crash periods over an entire vehicle trip (only a portion of which is shown in <FIG>) includes causing the vehicle crash detection module <NUM> of the system <NUM> to identify by computation one or more likely vehicle impacts <NUM> based on telematics data <NUM> (and other information) at each point in time or during brief segments of time spanning multiple time points within the time period of the vehicle trip, to determine if the telematics data indicates a vehicle impact or a vehicle crash (including one or more impacts). The telematics data in the example shown in <FIG> includes acceleration data <NUM> (along three axes, longitudinal, latitudinal, and vertical) at successive times <NUM> during the vehicle trip (the upper graph) and the GPS speeds <NUM> determined at successive times <NUM> during the vehicle trip, including during portions of the vehicle crash period.

For example, the system (e.g. the server or other devices or components) may split the telematics data <NUM> associated with the portion of the trip <NUM> into segments, such as <NUM>-second segments (although the length of each segment can be within a range from about one second to about <NUM> seconds in various implementations), and then compute and store one or more of the following features with respect to each segment using the telematics data <NUM>:.

The following tables (and <FIG>) illustrate a specific example of the computations described above.

The tables are based on a trip start and a trip end represented by the following time stamps:.

In this example, the trip includes the segment data points occurring during a particular <NUM>-second segment The segments for the rest of the trip's data points are omitted for brevity:.

The following table shows time-stamped data points occurring between <NUM> and <NUM> seconds after the end of the segment to offer clarity on the computation of the feature accel_after_span:.

The following table shows additional data for each of the above data points.

The following features are computed based on the above data points:.

The parameters and corresponding features of the telematics data are combined to reach a conclusion about whether the segment contains a crash. For example, accel_impact_max had a value of <NUM>/s<NUM> in the above worked example, which is a relatively large value which we would expect to encounter during a real crash. Furthermore, accel_after_span had a value of <NUM>/s^<NUM>, which is a small value which we might expect from a car undergoing relatively little acceleration after a crash, having come to rest. This example in particular corresponds to a segment which can be ruled to have included a crash.

Other parameters and features and corresponding equations may also be effective in inferring that a crash has occurred. For example, the higher-order derivatives of the acceleration signal, the jerk and snap, might offer further evidence of an impact.

After calculating one or more of the above features for each of the segments, the system <NUM> may feed the features into an impact and crash detection model <NUM> (which we sometimes call more simply a "crash detection model"), such as a model generated by the crash detection module <NUM> and stored in the database <NUM>, to determine a probability of a vehicle impact or vehicle crash <NUM> having occurred beginning during a particular segment. The probability of an impact may be expressed as a simple probability that an impact occurred in that segment or that a crash involving more than one impact began in that segment.

The crash detection model <NUM> may include a classification model or other algorithm to determine the probability of a crash <NUM>. For instance, the model may be derived from one or more of a random forest, a linear regression, a binary decision tree, pattern matching techniques, neural networks, Gaussian processes, among others, or combinations of them. In some cases, the crash detection model may be trained using one or more of the above features.

Once the probability of an impact or a crash <NUM> having occurred (or begun to occur) in a segment is determined, the system <NUM> may apply one or more filters <NUM> to determine if a particular segment contains or was the start of a likely vehicle impact or vehicle crash <NUM>. For example, if the probability of a crash <NUM> is above a certain threshold, such as above <NUM>%, above <NUM>%, above <NUM>%, or above <NUM>%, the system may mark the segment as containing or starting a likely vehicle impact or vehicle crash, and may record the time of the vehicle impact or the start of the vehicle crash. The system may also require that a segment's features meet one or more of the following conditions to be considered a likely vehicle impact or vehicle crash:
GPS_speed _change of at least -<NUM>/s and accel_impact_max of at least <NUM>/s<NUM>; the vehicle impact or vehicle crash is not too soon after the beginning of the trip, when there is a high chance of GPS speed artifact incorrectly suggesting an impactor crash; the GPS should converge to zero after the vehicle crash within about <NUM> to <NUM> seconds, including measurement error, so that the duration of the vehicle crash. Should be less than <NUM> seconds; and the time of the maximum acceleration impact should not be too long after the beginning of the segment in which breaking first occurred, for example, -<NUM> < accel_impact_offset_rel < <NUM>.

After detecting one or more likely vehicle impacts or vehicle crashes <NUM> during the segments associated with the trip <NUM>, the system <NUM> may, in some implementations, calculate a vehicle crash period <NUM> for each of the one or more vehicle crashes (step <NUM>). To do so, the system <NUM> may utilize a vehicle crash period algorithm <NUM> included in, for example, the crash period module <NUM> to determine a start time and an end time of a vehicle crash period <NUM> based on the telematics data <NUM>, the features described above, other information, or combinations of them.

In some implementations, the vehicle crash period algorithm may include two methods for determining a vehicle crash period, referred to here as the "out-in method" <NUM> and the "in-out method" <NUM>, and a diagnostic test <NUM>. In some implementations, the algorithm may use the out-in method as a first pass, and may fall back to the in-out method if the diagnostic test determines that the out-in algorithm was not successful. (A variety of other vehicle crash algorithms would also be possible based on other analytical approaches. These alternate vehicle crash algorithms include sliding a window across the data and labelling a region of each window as "crash" or "not crash" based on telematics data exceeding thresholds; providing a collection of labelled crash regions to a machine learning algorithm such as a neural net or support vector machine, which can then classify portions of the data stream as "crash" or "not crash"; modeling the data using an internal state, as with a hidden Markov model or LSTM, in which a function of the hidden state provides a "crash" or "not crash" label.

The out-in method may start with a wide (overly long duration) estimate of the vehicle crash period and may narrow the period until it matches what we call the true duration of the vehicle crash period <NUM>. For instance, the out-in method may begin by partitioning the trip <NUM> into estimated "before-crash," "crash," and "after-crash" segments. Any segment that ends more than <NUM> seconds (for example) prior to a start time of the vehicle crash <NUM> is designated a before-crash segment. Any segment that begins within <NUM> seconds (for example) after the start time of the vehicle crash period is designated a crash segment. Any segment that begins more than <NUM> seconds (for example) after the start time of the vehicle crash is designated as an after-crash segment. The start time of the estimated vehicle crash period may be increased (e.g., moved later in time) until the algorithm encounters a sample of the telematics data <NUM> that satisfies a predetermined threshold, such as acceleration data that differs in any axis by more than <NUM>/s<NUM> (for example) from the average oriented acceleration of the before-crash segment, or until the process reaches the start time of the estimated vehicle crash period <NUM>. In addition, the end time of the estimated vehicle crash period may be decreased (moved earlier in time) until the algorithm encounters a sample of the telematics data <NUM> that satisfies a predetermined threshold, such as acceleration data that differs by more than <NUM>/s<NUM> in any axis from the average oriented acceleration of the after-crash segment, or until the algorithm reaches the end time of the vehicle crash period <NUM>. The start and end times of the resulting vehicle crash segment may define the out-in estimate of the vehicle crash period <NUM> for the particular vehicle crash <NUM>.

The in-out method, according to the invention, starts with a narrow (relatively short duration) estimate of the vehicle crash period placed at the time of the vehicle crash (impact) <NUM> and extends the duration of the period until it matches the true vehicle crash period <NUM>. For instance, the in-out method may begin by partitioning the trip <NUM> into estimated "before-crash" and "after-crash" segments. The before-crash segment may correspond to a segment of the telematics data <NUM> that ends more than <NUM> seconds prior to the start time of the true vehicle crash <NUM>. The after-crash segment may correspond to a segment of the telematics data that begins more than <NUM> seconds after the time of the true vehicle crash. Beginning from the starting time of the true vehicle crash, an estimated start time of the vehicle crash period <NUM> is determined by moving backwards (e.g., earlier) in time until the algorithm encounters a sample of the telematics data <NUM> that satisfies a predetermined threshold, such as a <NUM>-second <NUM> sample of oriented acceleration data that does not differ in any axis by more than <NUM>/s2 (for example) from the average oriented acceleration data of the after-crash segment. An estimated end time of the vehicle crash period <NUM> is determined by moving forward (e.g., later) in time from the start time of the vehicle crash until the algorithm encounters a sample of the telematics data <NUM> that satisfies a predetermined threshold, such as a <NUM>-second sample of oriented acceleration data that does not differ in any axis by more than <NUM>/s2 (for example) from the average oriented acceleration data of the before-crash segment. The resulting start time and end time may define the in-out estimate of the vehicle crash period <NUM> for the vehicle crash <NUM>.

After computing estimates of the start time and finish time of the vehicle crash period <NUM> using the out-in and in-out methods, the crash period algorithm may use a diagnostic test to determine whether the out-in method was successful, for example, by determining whether the out-in estimate of the vehicle crash period has a duration that is less than a predetermined threshold, such as less than <NUM> seconds ( or another appropriate threshold within a range of <NUM> second and <NUM> seconds). If the duration of the out-in estimate satisfies the threshold, the algorithm may determine that the vehicle crash period <NUM> and the start time and end time for the vehicle crash <NUM> is equal to the out-in estimate of the vehicle crash period and its start time and end time. If the duration of the out-in estimate does not satisfy the predetermined threshold, the algorithm may determine that the estimate is an abnormally long duration for a crash that needs to be substantiated by the continuous presence of consistent acceleration throughout the vehicle crash, which is checked for by the inout method. If substantiated on that basis, the algorithm may determine that the vehicle crash period <NUM> and its start time and end time for the vehicle crash <NUM> are equal to the in-out estimate of the vehicle crash period and its start time and end time.

In some implementations, the algorithm may not rely on the results of only one of the in-out method or the out-in method, but may take account of the results of both algorithms, for example, based on the minimum, maximum, or average of the out-in and in-out estimates of the vehicle crash period and its start time and end time.

After determining or otherwise obtaining the vehicle crash period for a vehicle crash, the system <NUM> may determine one or more impact metrics or crash metrics (we sometimes refer to impact metrics or crash metrics more simply as "crash metrics") related to the vehicle crash based on the telematics data associated with the vehicle crash period, among other information (step <NUM>). In some implementations, the system <NUM> may use the crash metrics module <NUM> to calculate the one or more crash metrics, as described below.

In some cases, the system <NUM> may compute one or more of the following crash metrics: the duration of the vehicle crash period, the speeds of the vehicle before, during, and after the vehicle crash period, the difference between the speed at the start time and the speed at the end time of the vehicle crash period, the minimum and maximum lateral, longitudinal, and vertical accelerations of the vehicle before, during, and after the vehicle crash period, the maximum total planar acceleration experienced during the vehicle crash period, the gyroscopic orientation and angular velocity of the vehicle during the vehicle crash period, the GPS coordinates at each moment during the vehicle crash period, whether a vehicle airbag deployed during the vehicle crash period, whether the vehicle rolled over during the vehicle crash period, the time, direction, and description of one or more impacts during the vehicle crash period, driver actions, such as continued driving, braking or accelerating before impact, swerving, evasive maneuvering, or walking after one or more of the impacts, and the severity of the crash, among others, and combinations of two or more of those crash metrics.

In some implementations, the system <NUM> may determine whether one or more vehicle airbags were deployed during the vehicle crash period based on, for example, barometric and GPS measurements included in the telematics data. In general, deployment of an airbag produces an increase in air pressure within the vehicle, which results in a downward spike in barometric altitude <NUM> that is not corroborated by a downward spike in GPS altitude <NUM>, as shown in <FIG>. Accordingly, the system may detect a discrepancy between the rate of change of the barometric altitude <NUM> and the rate of change of the GPS altitude <NUM> to determine if one or more vehicle airbags were deployed. To do so, the system <NUM> may first compute the first derivative of both the barometric and GPS altitude signals contained in the telematics data. The system <NUM> may then smooth the derivative signals using one or more filters, such as windowed-average filters of width <NUM> seconds (or another period within the range of <NUM> second to <NUM> seconds) for the GPS signal and of width <NUM> seconds(or another period within the range of <NUM> seconds to <NUM> seconds) for the barometric signal and a median filter of width <NUM> seconds (or another period within the range of <NUM> seconds to <NUM> seconds) and taking the 5th percentile values of each signal within the centered window. If the system determines that the difference between the values of the two filtered signals averages out to less than, for example, -<NUM>/s over a <NUM>-second interval (although other thresholds could be used), then the incident may be identified and stored as an airbag deployment, with the time of deployment noted as the latest time within the vehicle crash period satisfying this condition.

The system <NUM> may also determine the time or times of one or more impacts during the vehicle crash period and the direction (that is, the relative direction of the impact from the frame of reference of the vehicle) of each impact on the vehicle. As shown in <FIG>, in some implementations, the system <NUM> may first identify one or more times of peak acceleration <NUM> for each of the longitudinal <NUM>, lateral <NUM>, and vertical <NUM> acceleration signals included in the telematics data associated with the vehicle being tracked. To do so, the system may utilize one or more peak-finding algorithms <NUM>, such as an absolute magnitude peak-finding algorithm which calculates the absolute difference between the longitudinal <NUM>, lateral <NUM>, or vertical <NUM> acceleration signal and its mean value and the absolute value of the second differential of the acceleration signal. The system may then record each time the absolute difference exceeds, for example, <NUM>/s<NUM> (or another value in the range of <NUM>/s<NUM> to <NUM>/s<NUM>) and the absolute second differential (similar. ) exceeds, for example, <NUM>/s<NUM>, (or another value in the range <NUM>/s<NUM> to <NUM>/s<NUM>) as a peak <NUM> of the particular acceleration signal. If the absolute magnitude peak-finding algorithm fails to find any peaks <NUM> in any of the longitudinal <NUM>, lateral <NUM>, or vertical <NUM> acceleration signals, the system may employ a largest extrema peak-finding algorithm. In the largest extrema algorithm, the system may begin by taking the acceleration signal for a particular axis and subtracting its mean value. The system may then identify the two largest values in the resulting signal that arise from this operation. If either or both of these values are a local extremum, the system may record the time associated with the value(s) as corresponding to a peak <NUM>.

After determining the one or more times of peak acceleration <NUM> in each axis, the system <NUM> may merge neighboring peaks to produce what is referred to here as an "acceleration event. " For example, the system may merge one or more peaks <NUM>, <NUM> to produce an acceleration event <NUM>, as shown in <FIG>. In some implementations, each acceleration event <NUM> can correspond to at most one peak <NUM> on each axis of acceleration, although some acceleration events may only contain acceleration data from one or two axes. In some implementations, the system may determine one or more acceleration events <NUM> using a dynamic programming algorithm in which the input is the one or more times of peak acceleration peak <NUM> for each of the longitudinal <NUM>, lateral <NUM>, and vertical <NUM> acceleration signals, and the output is the groupings of these time or times into acceleration events <NUM> such that no acceleration event contains more than one peak from each axis and no acceleration event contains peaks having times separated by more than, for example, <NUM> second (or another threshold within a range of <NUM> to <NUM> seconds). Subject to these constraints, the system may seek to minimize, first, the total number of acceleration events produced and, second, the total sum of time differences between neighboring peaks within all events.

As shown in <FIG>, in some implementations, the system <NUM> may process each acceleration event <NUM> to translate the quantitative acceleration experienced in each axis into a qualitative (in some cases a natural language) description <NUM> of the acceleration event <NUM>. To do so, the system may apply one or more thresholds based on, for example, direction and intensity, to the quantitative acceleration experienced in each axis during the acceleration event <NUM>. Using this data, the system may sort the acceleration event <NUM> into one or more bins <NUM> containing the qualitative description <NUM> of the acceleration event, as shown in <FIG>. The qualitative description <NUM> may indicate, for example, whether the acceleration event <NUM> was due to an impact and where on the vehicle the impact occurred, whether the acceleration event was due to throttling or braking the vehicle, and whether the acceleration event was the result of turning, and in what direction the vehicle turned, among others. Note that the qualitative descriptions are also associated with the respective axes of acceleration <NUM>.

For instance, if the system determines that there is a strong, negative longitudinal acceleration during the acceleration event <NUM>, the system may associate the qualitative description <NUM> "front impact" with the acceleration event <NUM>. By synthesizing the qualitative description <NUM> of the acceleration event <NUM> with other crash metrics, such as airbag deployment, the system <NUM> can create an annotated time line <NUM> of the vehicle crash for use in the human-readable documentation of the crash, as discussed below. Events that occur at specific times <NUM> during the vehicle crash period can be associated with different points along such a time line visually, in prose, or in a combination of the two.

As shown in <FIG>, the system <NUM> may compute a probability of impact <NUM> for each of the four sides of the vehicle with respect to an acceleration event. For instance, the system may determine that those axes with the largest acceleration, as reported by the acceleration events <NUM>, their qualitative descriptions <NUM>, or both, have a higher probability of impact <NUM>. In some implementations, the system may compensate for braking by reducing the probability of impact <NUM> for the front of the vehicle, for example. Furthermore, if there are two spikes of lateral acceleration in different directions, the system may determine that the first spike is more likely to be associated with an impact. After determining the probability of impact <NUM> for each of the four sides, the system <NUM> may synthesize the probabilities into a unified best-guess of a direction of impact <NUM>.

In some implementations, the system <NUM> may use telematics data or other information to identify and label driver state or behavior, such as continued driving, braking or accelerating or both before impact, swerving, evasive maneuvering, walking after the vehicle crash period, or another action, or combinations of them. For example, the system may compute whether the driver continued driving by checking whether the vehicle's speed exceeded, for example, <NUM> kph (or another speed threshold within a range of <NUM> kph to <NUM> kph) at any point more than <NUM> seconds (or another time threshold in the range of <NUM> to <NUM> seconds) after the end of the vehicle crash period.

To determine whether the driver accelerated or applied the brakes or both before the vehicle crash period or before one or more of the impacts of that occurred during the vehicle crash period, the system may first compute the main impact time, that is, the time of the impact having the largest magnitude of acceleration during the vehicle crash period. Beginning at the start time of the vehicle crash period, the system can then search for the first block of time (e.g., a time segment) lasting at least, for example, <NUM> second, and during which all longitudinal acceleration samples are less than, for example <NUM>/s<NUM> (or a threshold within the range of <NUM>/s<NUM> to <NUM>/s<NUM>). Next, the system may check all longitudinal acceleration samples between the end time of the contiguous block and, for example, <NUM> seconds (or a threshold within the range of <NUM> seconds to <NUM> seconds) before the main impact time to see whether there are any samples where the longitudinal acceleration is less than a given threshold, such as <NUM>/s<NUM> (or a threshold within the range -<NUM>/s<NUM> to <NUM>/s<NUM>), and greater than a given threshold, such as <NUM>/s<NUM> (or a threshold within the range <NUM>/s<NUM> to <NUM>/s<NUM> ). If there are samples below the first threshold and no samples above the second threshold, then the driver may be deemed to have been braking before the impact. If there are no samples below the first threshold and samples above the second threshold, then the driver may be deemed to have been accelerating before the impact. In a more complex example, an analysis can be done to determine whether the driver both accelerated and decelerated before the impact.

The system may also determine whether the driver swerved left or right or both before the one or more of the impacts by determining whether a leftward lateral acceleration peak (i.e., a peak with a lateral acceleration > <NUM> (or a threshold within the range <NUM>/s<NUM> to <NUM>/s<NUM>)) occurred before the main impact time and whether a rightward lateral acceleration peak (i.e., a peak with a lateral acceleration < <NUM> (or a threshold within the range -<NUM>/s<NUM> to <NUM>/s<NUM>)) occurred after the main impact time, which indicates a leftward swerve. The system may check for a rightward lateral acceleration peak before the main impact time followed by a leftward lateral acceleration peak after the main impact time, which indicates a rightward swerve (or the converse or a sequence of swerving maneuvers). If neither is present, then the system may determine that a swerve did not occur. Similarly, in some implementations the system may determine whether the driver attempted an evasive maneuver during the vehicle crash period by, for example, determining whether the difference between the speed before the main impact time and the speed at the time of the main impact exceeds a predetermined threshold, such as <NUM> kph (or a threshold within the range <NUM> kph to <NUM> kph).

In more complex analyses, other combinations of such maneuvers during a vehicle crash period may be determined, identified as events on the time line, and expressed either as prose, or visually, or both.

In some implementations, the system can determine whether or not the driver walked away from the vehicle after the vehicle crash period or after an impact. To do so, the system may utilize data received from a device carried by the driver, such as the mobile device <NUM>, to detect a characteristic signature of a human gait, such as described in <CIT>, entitled "Telematics Using Personal Mobile Devices. Walking detection can be useful in understanding the severity of a crash or an impact, for example, by indicating that the driver was not completely disabled, but that the crash or impact was severe enough to warrant leaving the vehicle.

In some implementations, the system <NUM> may compute a severity of a vehicle crash (or of an impact). In particular, the system <NUM> may compute a score (such as a severity score) <NUM> of the vehicle crash as shown in <FIG>, and may provide a severity score meter <NUM> including a textual or graphical depiction or both of the severity score <NUM> as part of the human-readable documentation of the vehicle crash. The severity score <NUM> may be determined by calculating the maximum, average, or weighted average of one or more of the following parameters, or a combination of, with a higher score value indicating a higher severity of the vehicle crash or impact:.

A wide variety of other factors and formulas, and combinations of them, can be used to determine a severity value or a non-severity value or other measures of the significance, cost, effect, or other characteristics of a vehicle crash or an impact that may be of interest to a user such as an insurance company or a governmental agency or other party.

Referring again to <FIG>, one or more of the crash metrics described above can be used alone, or with other information, to automatically generate human-readable documentation of an impact or a vehicle crash or both. The documentation can include prose, images, video content, graphical elements, graphs, charts, tables, and other items of content, and combinations of them, assembled and organized in a format that is easily processed and understood by a human user. The system <NUM> may automatically provide a human-readable documentation of the vehicle crash or vehicle impact based on one or more of the crash metrics, among other information (step <NUM>).

In some implementations, the central server <NUM> may use, for example, the crash documentation module <NUM> to generate the human-readable documentation of the vehicle crash or vehicle impact. The central server <NUM> may then use the communication interface <NUM> to provide the documentation to one or more other components of the system <NUM>, such as the database <NUM> for storage, the mobile device <NUM> using the network <NUM>, or the remote computing device <NUM> using the network <NUM>, or other devices, or combinations of them.

The human-readable documentation may be provided in a variety of forms or combinations of them, such as paper, a computer-readable file, a set of computer-readable instructions, an email, a web page, a web service, an application, a mobile application, or a notification, among others. Once received, the mobile device <NUM>, the remote computing device <NUM>, or other receiving device may print or display or both the human-readable documentation of the vehicle crash or vehicle impact. In some implementations, the human-readable documentation may be interactive, and the central server <NUM> may provide the mobile device <NUM> or the remote computing device <NUM> with software, such as an application, or an interface, such as an API or a web service, which allows for display, interaction, and further processing of the documentation. In other implementations, the telematics device <NUM>, the mobile device <NUM>, or the remote computing device <NUM> may generate the human-readable documentation of the vehicle crash and may provide the documentation locally or to the other components in the system <NUM> for display, interaction, and further processing of the documentation.

The system can also provide aggregated documentation of more than one vehicle crash or impact to enable the user to understand statistical features of the crashes or impacts. The aggregation of information can be done according to geography, time of day, month of the year, demographic characteristics of the operators of the involved vehicles, characteristics of the vehicles, and a variety of other characteristics. In this way, the crash documentation module could construct a narrative spanning multiple crashes which gives English-language descriptions of many impacts (e.g., an English language description of a statistical conclusion about crashes) at a time, e.g., "The majority of impacts which happened within <NUM> miles of Boston during the month of April <NUM> were impacts to the right side of the vehicle".

Furthermore, the technology could provide the raw data used in generating the documentation in a structured way to enable other representations and analyses to be made of the data. For example, the documentation could expose the timestamps, directions, and intensities of all impacts as a list of values. Such values could be used, for example, to produce a pie chart showing the relative amount of damage done to each side of the vehicle during a single crash.

In some cases, the documentation of a crash may be provided verbally (audibly). In that situation, the narrative text would be created as in the description above, but an audible voice would be generated using text-to-speech synthesis software. The narration may be useful for claims adjusters and other operators who cannot easily view a computer screen while assessing one or more crashes or impacts.

The narrative of the documentation can be supplemented with additional features reflecting the driving immediately preceding the impact. For example, the system can indicate whether the driver is familiar with the road they are driving on. Familiarity can be determined by examining the past history of drives over a fixed period of time, such as <NUM> days (or a period of time within the range of <NUM> day to <NUM> year) and examining the road segments extracted from a map-matched trajectory; if the road segment or segments involved in the crash has been driven on <NUM> or fewer times within the period of time (or a threshold within the range <NUM> to <NUM>), then the road segment is considered "unfamiliar". GPS positions can be used instead of map-matched road segments if map-matching is unavailable. If the driver appears to be unfamiliar with the road in the sense above, the textual narrative would be appended with a sentence similar to "The driver has only driven once on this road in the past <NUM> days". Alternately, if the road segment is considered "familiar", a sentence could be added such as "The driver is familiar with this road and has driven on it <NUM> times in the past <NUM> days.

Second, information about phone calls preceding the vehicle crash can also be included in the narrative. In particular, by polling the system API on the smart phone for call state information and transmitting this information to a server in the cloud, the system can determine if the user placed a call and may be able to detect if the call was on a hands-free device or on the handset. If a call was placed, sentences can be appended to the textual narrative such as "<NUM> minutes prior to the crash. The call ended <NUM> seconds prior to the crash.

Third, information about whether the driver appeared to be walking immediately after the crash could be included in the narrative. This could be determined by using the acceleration data from the driver's phone or by using walking classifiers native to the phone. From this data, a sentence could be appended to the narrative, of the form "The driver exited their vehicle and walked around after the crash".

The users of the human-readable document can include insurance companies, governmental agencies, private companies, vehicle owners, vehicle manufacturers, road designers, and a variety of other parties.

<FIG> illustrate an example of a human-readable documentation of a vehicle crash <NUM>. In general, the human-readable documentation of the vehicle crash <NUM> may include, for example, a textual description <NUM>, a graphical description <NUM>, any of the other types of content mentioned above, or combinations of them, of the vehicle crash that refers to one or more features and metrics or other information about the vehicle crash. Referring to <FIG>, in some implementations, the documentation <NUM> may include a natural language description of the vehicle crash including, for example, English sentences that refer to one or more of the location of the vehicle crash, the date of the vehicle crash, the time of the vehicle crash or of features during the vehicle crash period, the duration of the vehicle crash, the severity of the vehicle crash, the minimum, maximum, average, or instantaneous speed and acceleration of the vehicle before, during, and after the vehicle crash, the change in speed and acceleration during the vehicle crash, the gyroscopic orientation and angular velocity of the vehicle during the vehicle crash, the number of impacts during the vehicle crash, the direction of impacts during the vehicle crash, the yaw of the vehicle during the vehicle crash, whether a vehicle airbag deployed, whether the vehicle rolled over, whether the driver of the vehicle attempted any evasive maneuvers and what the maneuvers were, whether the vehicle's brakes or throttle were used during the vehicle crash period, whether there was time for the driver to slow down before the impacts of the vehicle crash, whether the vehicle was driven after the vehicle crash period, whether the driver exited the vehicle after the vehicle crash period, the driver's gait after the vehicle crash period, environmental conditions during the vehicle crash period, such as the weather conditions, the temperature, the cloud coverage, the solar position, and whether the sun was facing the driver, raw telematics data for the vehicle crash period, crash metrics data during the vehicle crash period, and other features and combinations of them related to the vehicle crash.

For example, the natural language description <NUM> of the vehicle crash could state the following with respect to a particular vehicle crash: "The driver was traveling down Moody St. , Waltham, United States at <NUM>/h at <NUM>:<NUM>:<NUM> on <NUM>/<NUM>/<NUM>. At <NUM>:<NUM>:<NUM> a collision occurred on the left side of the vehicle while the vehicle was traveling at <NUM>/h. The collision event ended at <NUM>:<NUM>:<NUM> when the vehicle reached a speed of <NUM>/h. The vehicle was yawing during the crash. The driver did not continue on their trip after the crash event.

In some implementations, the human-readable documentation of the vehicle crash <NUM> may include crash reconstruction data <NUM> to provide a textual or graphical summary of crash metrics and other features related to the vehicle crash. The documentation <NUM> may include the severity score meter <NUM> to provide a textual and graphical depicture of the severity score <NUM> for the vehicle crash.

As shown in <FIG>, the human-readable documentation of the vehicle crash <NUM> may include the annotated timeline <NUM> of the vehicle crash. As described above, the annotated timeline <NUM> may include the one or more acceleration events <NUM> or other events, their qualitative descriptions <NUM>, and indicators of other metrics or features related to the vehicle crash. In some implementations, the documentation <NUM> can include the longitudinal <NUM>, lateral <NUM>, and vertical <NUM> acceleration signals with indicators for the one or more acceleration peaks <NUM> as discussed with reference to <FIG>.

The documentation <NUM> can include textual or graphical representations of other telematics data, such as the GPS speed of the vehicle during the vehicle crash period, or the lateral, longitudinal, and/or vertical acceleration experienced by the vehicle during the vehicle crash period. For instance, the documentation <NUM> may include a line segment plot <NUM> for every point of lateral and longitudinal acceleration experienced by the vehicle during the vehicle crash period as shown in <FIG>. In some implementations, the documentation <NUM> can include a visualization of the probability of impact <NUM> for each of the four sides of the vehicle and a unified best-guess of the direction of impact (not shown).

As shown in <FIG>, in some implementations, the documentation <NUM> may include a map <NUM> of the crash. The map <NUM> of the crash can include markers indicating the GPS position of a start <NUM> and an end <NUM> of the vehicle crash or vehicle crash period. The documentation <NUM> may include a street view <NUM> of the vehicle crash containing images of the street and surrounding area where the vehicle crash took place. In some implementations, a user may interact with the street view <NUM> to rotate the view or move the view along the street, such as by using a movable marker <NUM> placed on map <NUM>.

In some implementations, the documentation of a vehicle crash is produced as follows. As mentioned above, from the telematics data the following features of a crash can be derived, among others: the crash extent (e.g., the crash duration), the speed of the vehicle at the beginning and end of the crash period, the timestamps of impacts that occurred during the crash period, whether the driver continued driving to their destination after the crash, whether the vehicle rolled over during the crash, whether the airbag deployed during the crash, and whether the driver took a swerving or braking action prior to the first impact. From these features, the system computes several more features in order to produce the crash narrative.

The system computes the timestamp of the largest impact by comparing all impacts and taking the timestamp of the one with the largest deviation from gravity.

The system computes the latitude and longitude of the crash as the latitude and longitude reading in the telematics data that is closest to the timestamp of the largest impact.

The system computes whether the driver was facing the sun at the time of the impact as whether the solar azimuth and GPS bearing differed by fewer than <NUM> degrees and the sky was sunny or partly cloudy (as determined, for example, by querying a database of weather observations) and the solar altitude was between <NUM> and <NUM> degrees at the time of the largest impact.

The system computes the most-damaged side of the car by taking each impact direction and assigning it a severity and a direction (as described previously in the disclosure) and summing the total weight over each of the directions left, right, front and rear in order to determine the direction of maximum hit intensity.

The system computes whether the car was yawing during the crash as whether the gyroscope's measured oriented yaw rate exceeded <NUM> radians per second at the time of the largest impact.

The system computes the name of the street where the impact occurred by querying a map database to determine the street closest to the latitude and longitude of the impact.

With the above values determined the crash documentation module generates a narrative according to the following procedure:
The module starts with the phrase "The driver was traveling down STREET_LOCATION at CRASH_START_SPEED km/h at CRASH_START_TIME. " STREET_LOCATION is the written-language description of the street, e.g., "Main St, Lincoln, Nebraska". CRASH_START_SPEED is the speed at the beginning of the crash period in kph. CRASH_START_TIME is the written-language time of the beginning of the crash, eg. "<NUM>:<NUM>:<NUM> on <NUM>/<NUM>/<NUM>".

If only a braking maneuver was attempted, the module appends to the narrative the phrase "The driver then began braking prior to the impact. " If only a swerving maneuver was attempted, the module appends to the narrative the phrase "The driver began swerving prior to the impact". If both a braking and a swerving were attempted, the module appends to the narrative the phrase "The driver began braking and swerving prior to the impact.

If the module determined what was the most-damaged side of the car, the module appends to the narrative the phrase "At LARGEST_IMPACT_TIME a collision occurred on the HIT_DIRECTION side of the vehicle while the vehicle was traveling at LARGEST_IMPACT_SPEED km/h. The collision event ended at CRASH_END_TIME when the vehicle reached a speed of CRASH_END_SPEED km/h. " Otherwise, the module appends the phrase "At LARGEST_IMPACT_TIME a collision occurred while the vehicle was traveling at LARGEST_IMPACT_SPEED km/h. The collision event ended at CRASH_END_TIME when the vehicle reached a speed of CRASH_END_SPEED km/h. " LARGEST_IMPACT_TIME is the written-language time of the largest impact, eg. "<NUM>:<NUM>:<NUM> on <NUM>/<NUM>/<NUM>". HIT_DIRECTION is one of "left", "right", "front", or "rear". LARGEST_IMPACT_SPEED is the speed at the time of largest impact in kph. CRASH_END_TIME is the written-language time of the end of the crash period, eg. "<NUM>:<NUM>:<NUM> on <NUM>/<NUM>/<NUM>". CRASH_END_SPEED is the speed at end of the crash period in kph.

If the vehicle was determined to be yawing, the module appends to the narrative the phrase "The vehicle was yawing during the crash.

If the module determined that the airbag deployed, the module appends to the narrative the phrase "The airbag deployed at AIRBAG_DEPLOY_TIME. " AIRBAG_DEPLOY_TIME is the written-language time of airbag deployment, eg. "<NUM>:<NUM>:<NUM> on <NUM>/<NUM>/<NUM>".

If the module determined that the vehicle rolled over, the module appends to the narrative the phrase "The vehicle rolled over as a result of the collision.

If the module determined that the driver was facing the sun, the module appends to the narrative the phrase "The driver was driving into the sun, which may have caused glare and/or poor visibility.

If the module determined that the driver continued driving, the module appends to the narrative the phrase "The driver continued driving after the crash event toward their destination. " Otherwise, we append to the narrative the phrase "The driver did not continue on their trip after the crash event.

A wide variety of other data elements can form the basis of elements of the narrative. And the narrative can be phrased in a broad range of ways for a given set of data elements.

Claim 1:
A method for automatically generating a documentation of a crash involving a vehicle, the method comprising:
receiving telematics data produced by one or more sensors associated with a telematics device at the vehicle, the sensors including an accelerometer, and optionally including at least one of a speedometer, a barometer, a gyroscope, a compass, and a position sensor;
based on the telematics data, determining (<NUM>) a vehicle crash period that begins at a start time of the vehicle crash period that is a beginning of a vehicle crash period and ends at an end time of the vehicle crash period, the vehicle crash period including one or more impacts of the vehicle, wherein
i) the start time of the vehicle crash period is determined by detecting, using the telematics data, a likely vehicle crash (<NUM>) and moving backwards in time from a start time of the detected likely vehicle crash until a sample of oriented acceleration data that does not differ in any axis by more than a first threshold from an average oriented acceleration data of an after-crash segment is encountered, and
ii) the end time of the vehicle crash period is determined by moving forward in time from the start time of the detected likely vehicle crash until a sample of oriented acceleration data that does not differ in any axis by more than a second threshold from the average oriented acceleration data of a before-crash segment is encountered;
determining (<NUM>), based on the telematics data, one or more metrics associated with the vehicle during the vehicle crash period;
automatically generating (<NUM>), based on the one or more metrics, a human-readable documentation of the vehicle crash comprising a narrative description of features of the vehicle crash during the vehicle crash period, wherein the narrative description comprises assembling predetermined prose phrases corresponding to the features of the crash and one or more of the determined metrics; and
communicating the narrative human-readable prose documentation to a device for presentation in an audible form or a readable form to a user.