Patent ID: 12236728

DETAILED DESCRIPTION OF THE EMBODIMENTS

Specific embodiments are now described with reference to the appended figures.

A preferred embodiment of the present invention relates to a mobile telecommunications device for recording events associated with a vehicle, such as a car. In particular, the mobile telecommunications device is loaded with an application—a ‘mobile app’—which is arranged to record and document the events surrounding an incident involving the vehicle (a ‘driving incident’), such as a vehicle collision. The mobile application may be referred to as the ‘Witness’ application in the ensuing description.

FIG.1illustrates a system1within which a vehicle, such as an automobile3configured with a mobile telecommunications device executing the Witness application, may communicate with one or more remotely located devices or systems. Such devices may relate to one or more servers, such as, an emergency services server7, a content provider server9, and a personal computer11. Communication between the automobile3and the one or more remotely located devices5,7,9,11may be effected via a shared communication network13. The shared communication network may relate to a wide area network (WAN) such as the Internet, and may also comprise telecommunications networks. For example, the mobile telecommunications device of the present embodiments may be arranged to communicate with any one of the remotely located devices5,7,9,11via a mobile telecommunications base station15, which is coupled to the shared mobile telecommunications network13.

A main purpose of the mobile telecommunications device, when arranged within a vehicle in motion, such as the illustrated automobile3, is to monitor and process sensor data captured by the mobile device, to derive driving information associated with the moving automobile3. As mentioned previously, this driving information is subsequently used by the mobile device to determine if a driving incident or event has occurred. The term “driving incident” as used within the present context may relate to an event such as a collision, a near-miss, dangerous driving, erratic driving or similar. Similarly, the driving information may be used as a means for monitoring a driver's driving performance, and determining a driving characteristic, or user profile for the subject driver. In other words, the sensor data and/or the driving information may be used to determine for example, if the driver is a cautious driver, or a reckless driver. Further details of this alternative embodiment are described below.

The mobile telecommunication device's functionality is provided by the Witness application, which is installed in the device's local non-volatile storage, and which is executed by the device's native processor. The application may be downloaded to the mobile device via the shared communications network13and telephone base station15, from the content provider server9.

In use (once the application has been configured for execution on the mobile telecommunications device), when a driving incident has been detected by the mobile telecommunications device, data, comprising one or more of mobile device sensor data, derived driving information, and captured image data may be automatically forwarded to one or more of the remotely located devices5,7,9,11. For example, the mobile device may be configured such that when a driving incident is detected, such as a collision involving an automobile3, driving information (including image data captured by the mobile device) derived from the sensor data is automatically forwarded, via the base station15and shared communications network13.

Similarly, once a driving incident has been detected, data may also be automatically forwarded to the emergency services server7. For example, such data may comprise the position information of the automobile3, along with an automated S.O.S. message requesting assistance from the emergency services. Forwarded data may also comprise sensor data, and any relevant derived driving information, such as speed of impact, the g-forces the vehicle was subjected to during the collision, or any other information wherefrom the severity of the collision may be determined, and which may be used to assist the emergency services in coordinating the appropriate level of response.

Optionally, an electronic message may be sent to a personal contact preselected by the user. For example, an automated message, such as an e-mail, may be forwarded to the PC11of the user nominated personal contact, informing the personal contact that the user of the vehicle3has been involved in a driving incident. Similarly, and due to the mobile device's telecommunications functionality, an electronic text message, such as an SMS (Short Message Service) may be forwarded to the telephone of the user selected personal contact, informing the contact of the driving incident. The mobile device may be equally be arranged to communicate with a personal contact via any known electronic messaging service and/or instant messaging service too. For example, via Apple's® iMessage, or RIM's® BlackBerry Messenger (BBM) service.

This functionality of forwarding a message to a nominated contact, may also be of particular benefit for managing large fleets of vehicles. For example, a car hire service. In this way, if any of the vehicles comprised in the fleet are involved in a driving incident, the car hire service may be automatically notified of the identity of the specific vehicle involved in the driving incident. In such embodiments, the nominated personal contact would be preselected by the vehicle fleet manager. The option of providing a second user selected personal contact is also envisaged. In this way, a message may be forwarded to both the car hire service for example, and to the driver's selected personal contact.

In preferred embodiments the mobile telecommunications device relates to a mobile telephone having native processing functionality, and preferably relates to a smartphone. Preferably, the mobile telephone comprises a camera arranged to enable image capture, and preferably arranged to enable a sequence of images to be captured taken in temporal succession. In other words, in preferred embodiments the mobile telephone comprises a camera configured to enable video footage of a sequence of events to be captured.

Alternatively, the mobile telecommunications device may relate to a PDA, a notepad such as an iPad®, or any other mobile device comprising local processing means and means for communicating with a telecommunications network.

FIG.2provides a more detailed view of the automobile3ofFIG.1, and illustrates a preferred arrangement of the mobile telecommunications device17within the automobile3. In the illustrated embodiment, the mobile telecommunications device17relates to a smartphone configured with an image capture device, such as a camera. Preferably, the device17is arranged within the vehicle, such that the camera has a clear line of sight of the road in the principle direction of travel of the automobile3. For example, the device17may be attached to the windshield19of the automobile3in an adapter21. The adapter21may comprise a flexible telescopic arm, configured with a suction cup at one end for affixing the adapter to the windshield19, and a dock arranged at the opposite end for securely holding the telecommunications device17in place. The telescopic arm enables the position of the device17to be varied, such that a clear line of sight in the direction of travel may be obtained.

The details of the adapter21are irrelevant for present purposes, with the exception that it must enable a clear line of sight in the principle direction of travel of the vehicle to be obtained, and it must firmly affix the mobile to the vehicle. Affixing the mobile telecommunications device17to the automobile3, ensures that the mobile device17is able to accurately capture the automobile's state of motion. By principle direction of travel is intended the main direction of travel of the vehicle when operated in a conventional way, indicated by the arrow A inFIG.2. In other words, the forward direction of travel. The skilled reader will appreciate that whilst most vehicles, such as an automobile, may have more than one direction of travel (e.g. travelling backwards when in reverse gear), the majority of vehicles have a primary direction of travel, which is the intended direction of travel for any transit of substantial length and/or distance. Arranging the telecommunications device17relative to the direction of principal travel, ensures that the camera (not shown) of the telecommunications device17is well placed to capture any image data which may be pertinent to a subsequently detected driving incident, such as a collision.

FIG.3is a schematic of the modular components of the mobile telecommunications device17ofFIG.2. Preferably the mobile telecommunications device17comprises: an image capture module21, an accelerometer23, a GPS receiver25, an audio capture module27, a communications module29, a processor module31and a storage module33. The image capture module21, accelerometer23, GPS receiver25, and audio capture module27form a sensor set and are generically referred to as data capture modules in the ensuing description, and are differentiated from the communications module29, processor module31and storage module33, in that they comprise sensors for sampling physical data.

This sampled physical data, which is also referred to as sensor data, is subsequently processed by the processor module31and stored in the storage module33.

The image capture module21may relate to any image capture device, such as an integrated camera commonly found in smartphones or similar mobile devices. As mentioned previously, the image capture module21is preferably configured to capture a plurality of images taken in temporal succession, such as provided by a video camera.

The accelerometer23is arranged to provide information regarding the motion of the automobile3along all three dimensional axes. For example, the accelerometer23provides information regarding the pitch, yaw and roll of the automobile3. Sensor data captured from the accelerometer23may be used to determine the g-forces the automobile has been subjected to. This is particularly useful in determining the severity of a collision. In general, the greater the g-forces experienced in a collision, the greater the risk of serious injury to the passengers of the vehicle. This information may assist the emergency services in forming an initial assessment of the severity of a collision. Furthermore, this type of data may also assist the emergency services and/or insurance provider to obtain a better understanding of the driving incident. For example, in the event of a collision, this data may assist the emergency services to obtain a better understanding of how the collision occurred. This information may subsequently be used for dispute resolution, and/or for determining culpability.

The GPS receiver25is arranged to provide location information, such as positional coordinates, and can also provide velocity information. When combined with the accelerometer sensor data, the GPS receiver data can be used to provide a very accurate model of a driving incident, such as a collision. In particular, the GPS sensor data provides velocity data, such as the velocity of impact. The velocity data enables one to determine if a vehicle was being driven at speeds greater than the legally permitted maximum speeds.

The audio capture module27provides means, such as a microphone, for recording audio data, which might be generated by a driving incident. This includes any sounds generated externally to the vehicle, for example the sound of an impact, or the sound of the crumple zone being crushed. Additionally, sounds generated internally to the vehicle are also recorded. Such audio data may also help to recreate a driving incident, and understanding the causes of the incident.

The communication module29provides the mobile telecommunications device17with functionality for communicating with the remotely located devices5,7,9,11ofFIG.1. The communication module29comprises a wireless telecommunications sub-module31enabling communication over a telecommunications network. An optional wi-fi communication sub-module is also envisaged. Similarly, the presence of wired communication modules are also envisaged, such as a USB port and/or an IEEE 1394 interface (more commonly known as FireWire™) to support wired communication with a remote device, such as a personal computer or similar. Such a connection may be useful for the purposes of side-loading the application to the mobile device.

As mentioned previously, sensor data captured from any one of the aforementioned data capture modules21,23,25,27is processed by the processor module33, to generate driving information. By driving information is intended any data which may be derived from raw sensor data captured by any one of the aforementioned modules21,23,25,27. For example, g-force data is driving information which is derived from the sensor data sampled by the accelerometer23. The skilled reader will be familiar with the plurality of types of driving information that may be derived from sensor data sampled by the aforementioned modules, and accordingly a complete list of the different types of driving information that may be derived from sampled sensor data is omitted for brevity.

The processor module33is also configured to analyse sampled sensor data and generated driving information to determine if a driving incident has occurred (described in more detail below).

Sampled sensor data is stored in storage34, which is operatively coupled to the processor module33, along with any generated driving information. The storage34is preferably configured with a FIFO (First In First Out) storage buffer34a, and a permanent storage component34b. In preferred embodiments, the data capture modules are configured to sample data at periodic intervals. Preferably, these intervals are sufficiently small, of the order of milliseconds, such that for practical purposes the data capture modules may be considered to sample data continuously. The sampled data, along with any derived driving information is preferably stored in the storage buffer34a, unless a driving incident has been identified, in which case the associated sensor data and driving information is stored in the permanent storage component34bto avoid undesirable overwriting.

In preferred embodiments, the FIFO storage buffer34ais provided with a finite amount of storage space. Nonetheless, said storage space may be predefined by the user as will be described below. In any case, once this storage space has been exhausted, the oldest recorded data is overwritten by newly sampled data, and this cycle of overwriting older data with newly sampled data is continuously carried out during operation of the telecommunications device17, unless a driving incident has been detected, in which case, and as mentioned previously, all data related to the driving incident is stored in a long term protected storage34bto safeguard it from being overwritten by newer data.

In preferred embodiments, the mobile telecommunications device17may be configured with a data recording strategy by the user. This might define the frequency with which sensor data is sampled. In other words, how many measurements are made per unit of time. Furthermore, the recording strategy also defines how data is recorded. In preferred embodiments, sampled sensor data is stored in data files in the buffer34a. Each data file represents a plurality of sequentially sampled sensor data, captured over a defined period of time, which will be referred to as a ‘data file period’. This is best illustrated by considering captured image data, such as video footage. A video period may be defined, which period defines the unit of physical time covered by captured video footage comprised in a single video data file—this is the data file period for image data. The video data file is subsequently stored in the buffer34a. For example, a five minute video period setting, instructs the processor33to store all sequentially captured image data captured by the image capture module21over a period of five minutes, in a separate video data file. It is to be understood that whilst the image capture module21is continuously sampling image data (in other words, it is continuously capturing image data), this plurality of captured image data is grouped together for storage in video data files, each data file representing a five minute data file period.

Sampled sensor data and derived driving information is also continuously analysed by the processor module33for the purposes of detecting a driving incident. As soon as a driving incident is identified, the associated sensor data and derived driving information is stored in a data file in accordance with the data recording strategy. Returning to the example described in the preceding paragraph, this entails combining the video footage captured within a temporal window of five minutes leading up to the detected driving incident, in a single video file, and storing this data file in protected storage34b. Whilst each different type of sensor data may be recorded in separate data files, in preferred embodiments all the different types of sensor data sampled by the mobile telecommunications device17are stored together in a single data file in accordance with the user selected data recording strategy. This means that the five minute data file referred to above, preferably also comprises GPS data, accelerometer data, and audio data sampled over the five minute time period.

Data files are stored in the buffer34a, unless they are associated with a driving incident, in which case they are then stored in protected storage34b, which cannot be overwritten. Once the storage space comprised in the buffer34ahas been exhausted, the oldest data file is overwritten by a newer data file.

Data compression methods may also be used in conjunction with the present invention to improve the use of storage. For example, data comprised in data files which have not been associated with a driving incident may be compressed using compression techniques, which techniques will be known to the skilled reader. Similarly, within a data file associated with a driving incident, sensor data captured at time coordinates which are distant from the determined driving incident may be compressed. In this way, the resolution of sensor data which is directly pertinent to a driving incident is maintained, whilst sensor data which may be less relevant to the driving incident is maintained at a lower resolution.

Since the sensors of the mobile telecommunications device17are continuously recording sensor data, even when a driving incident is detected, the device17may be configured to comprise sensor data and/or driving information recorded/derived shortly after the driving incident in the same data file, since this data may also be relevant for understanding a driving incident. Furthermore, this also enables the mobile telecommunications device to record multiple driving incidents. For example, a multiple collision.

The processor module33may be configured with encryption means, enabling stored data files to be encrypted to prevent data tampering. Envisaged encryption means may comprise both software solutions and hardware solutions. For example, the processor module may comprise a cryptoprocessor, or the processor may be configured with code to carry out a cryptographic process.

FIG.4ais a flow chart outlining the method carried out by the mobile telecommunications device17, to determine if a driving incident has occurred, in accordance with a preferred embodiment. An application is downloaded from the content provider server9ofFIG.1, onto the mobile telecommunications device17, at step36, as previously described. This may be done over a telecommunications network. The application provides the mobile telecommunications device17with the previously described functionality, when executed on the device. The mobile telecommunications device is configured within the vehicle, at step38. This may comprise affixing the mobile telecommunications device17to the vehicle via an adapter, as described previously.

The recording strategy is specified at step40. As mentioned previously, this comprises setting the period of time that each recorded data file represents. Furthermore, it may also comprise defining the size of the buffer34aand/or the number of data files that are to be stored within the buffer. Preferably, the recording strategy is specified only once, upon first use of the mobile telecommunications device17. Where the recording strategy has already been defined, method step40is skipped, and the method continues with step42.

The start of a driving period is registered, at step42. The start of the driving period determines when the recording of sampled sensor data begins. The start of a driving period may be manually entered by the user via the application's graphical user interface (GUI). Alternatively, the start of the driving period may be automated. For example, the mobile device17may be configured to initiate the driving period once sensor data above a minimum threshold value is recorded, indicative of the vehicle being in motion. For example, once a velocity greater than 20 kilometres per hour is detected.

Alternatively, the start of the driving period may be initiated once the application determines that the mobile telecommunications device17has been affixed to the vehicle. For example, the adapter21may comprise a registration module (not shown) arranged to register the installation and fixation of the mobile device to the vehicle and/or adapter. The registration module may comprise an NFC device. When the mobile device is brought into close proximity with the registration module, a driving period is initiated.

Once the driving period has been initiated, sensor data is sampled and recorded in storage34, at step44. Additionally, the sampled sensor data is used to generate driving information by the processor module33. The sampled sensor data and the driving information is continuously analysed by the processor module33, at step46.

The processor module33determines if a driving incident has been detected, at step48. This is determined on the basis of the analysis carried out at step46. If a driving incident has been detected, all the sensor data and driving information associated with the data file period, is stored in a data file in protected storage34b, at step50.

There are several ways in which a driving incident may be detected. Preferably, this is an automated process, wherein the processor module33determines a driving incident has occurred on the basis of an observed marked variation in sensor data and/or driving information. The term ‘marked variation’ is to be understood to relate to a significant change in sensor data and/or driving information occurring over a very short time period. In other words, a detected impulse in recorded sensor data and/or driving information, indicative of a significant change in the state of motion of the vehicle occurring over a short time period. For example, a sudden increase in the g-forces the vehicle is subjected to, may be indicative of a collision and if observed, result in a driving incident being determined to have occurred by the processor module33.

Predefined threshold values may also be used to automate the detection of a driving incident. For example, each data type and/or derived driving information (e.g. acceleration, velocity, g-force, pitch, roll, yaw etc.) may be associated with a threshold value. When any one of these threshold values is exceeded, the processor module33may be configured to determine that a driving incident has occurred. Similarly, the automated detection of a driving incident may require that a threshold condition associated with a combination of predefined threshold values, each threshold value being associated with a different data type and/or type of driving information, must be exceeded, in order for the processor module33to automatically determine that a driving incident has occurred. For example, in the event of a collision, it is reasonable to expect to observe a marked variation in g-force, accompanied by a marked variation in velocity. Accordingly, the threshold condition may require that in order for a collision to be automatically detected, both a marked variation in g-force and a marked variation in speed, in excess of predefined threshold values must be observed.

Similarly, the occurrence of a driving incident may also be recorded manually by the user via the application GUI. This may be beneficial for the purposes of documenting data associated with a low velocity collision—colloquially referred to as a ‘fender-bender’—which may not result in any marked variations in sampled sensor data, and therefore may not be automatically detected.

Similarly, once a driving incident has been detected, an S.O.S. message may be automatically forwarded from the mobile telecommunications device17to the emergency services server7, at step54. The S.O.S. message may also comprise sensor data and derived driving information, which may assist the emergency services in coordinating their response.

As mentioned previously, the mobile telecommunications device17will continue to sample and record sensor data even once a driving incident has been detected, unless the mobile telecommunications device17detects that the driving period has terminated, at step56. If the driving period has terminated, the present method is terminated and the device stops recording sensor data. The end of a driving period may be automatically detected by the processor module33, if certain conditions are met. For example, if the measured velocity of the vehicle remains zero for a predetermined period of time, the processor module33may infer that the vehicle is stationary and no longer being driven, and accordingly ceases recording sensor data, and the method is terminated. If instead the processor determines that the driving period has not yet terminated, a new data file period is initiated, at step58, and steps44through62are repeated.

Where a driving incident is not detected at step48, the processor module33will determine if the data file period has expired, at step60. If the data file period has expired, then the sensor data and the derived driving information generated during the data file period is combined and stored in a single data file, for storage in the buffer34b, at step62. The processor3then determines, at step56, if the driving period has terminated. If the driving period has not terminated, a new data file period is initiated, at step58. The mobile telecommunications device17continues to sample sensor data, to derive driving information, and steps44through62are repeated, until the driving period is determined to have terminated.

FIG.4bprovides more detail regarding how a driving incident may be automatically detected (i.e. steps46and48ofFIG.4a) in a preferred embodiment, where the mobile telecommunications device17is configured to sample audio data, accelerometer data, and GPS data only. Each one of these types of data is analysed in turn, at steps64,66and68. The analysis may comprise comparing measured sensor data and/or derived driving information with a data model. The model may comprise defined threshold values for different data types. Sampled sensor data and/or driving information may be compared with the data model to determine if a driving incident has occurred.

For example, the analysis of sampled audio data, at step64, may comprise comparing the recorded audio data with predetermined audio data models representing specific sounds. Such sounds may relate to the sound of tyre squeals, the sound of deforming metal, the sound of breaking glass, passenger screaming and/or shouting, the sound of airbag deployment, and any other sound which may be associated with a driving incident. Effectively, this may be considered audio fingerprinting, which serves to identify specific sounds associated with a driving incident by carrying out a signal profile analyses of the audio signal captured by the audio capture module27. To achieve this, the storage device34may comprise a database of prestored audio sound files. The audio sound files represent sounds associated with a driving incident. The captured audio signal is compared with the sound files comprised in the database, to identify matches between the captured audio signal and the database of sound files. This facilitates the audio fingerprinting of specific sounds, indicative of a driving incident, present within the captured audio data signal captured by the audio capture module27.

Similarly, the analysis of sampled accelerometer data, at step66, may comprise comparing the sampled data with predetermined accelerometer data models. The data models represent specific states of motion of the vehicle. For example, this might comprise defining threshold values for yaw, pitch and roll, which if exceeded, indicate a state of motion of the vehicle indicative of a driving incident. For example, a measured yaw value above a predetermined threshold value may be indicative of the vehicle having lost traction and is fishtailing and/or skidding. Similarly, a roll and/or pitch value above a predetermined threshold value may be indicative of the vehicle having overturned.

Accelerometer sensor data is also used for deriving driving information such as g-forces. Analysis of g-force data is also used to determine if a driving incident has occurred. For example, approximate g-force values for detecting certain driving incidents are as follows:Harsh braking—a deceleration of greater than 2.5 m/s2or forward G-force of greater than 0.7 G for more than 400 msec.Harsh acceleration—from stationary, an acceleration greater than 2.5 m/s2or backward G-force of greater than 0.7 G for more than 800 msec.Harsh swerving—lateral G-forces greater than 0.7 G for more than 400 msec.

The data models are preferably preconfigured and are comprised within the application executed on the mobile telecommunications device. Different data models are used depending on the type of vehicle the telecommunications device is being used with. Different vehicle types will experience different states of motion during regular operation, which must be considered in determining if a driving incident has occurred. For example, a motorcycle will display more roll and/or pitch than an automobile during regular operation. Accordingly, different data models and/or threshold values must be used to automate the identification of a driving incident for different vehicle types. The specific data models used may be selected during an initial configuration of the mobile telecommunications device, by indicating the type of vehicle the device is being used with.

On the basis of the audio data analysis and the accelerometer data analysis, the mobile telecommunications device determines, at step72, if a driving incident has occurred. If it is determined that a driving incident has occurred, then the telecommunications device proceeds with step50ofFIG.4a. If instead a driving incident is not detected, the telecommunications device proceeds with step60ofFIG.4a.

The GPS data analysis, at step68, comprises analysing positional data and velocity data for any anomalous readings. For example, a sudden deceleration followed by a zero-velocity reading lasting a predetermined period of time, may be indicative of a collision. If such a zero-velocity reading is observed at step70, in conjunction with anomalous audio and/or accelerometer sensor at step72, then a driving incident is determined and the mobile telecommunications device proceeds with step50ofFIG.4a. This is a further example of a threshold condition, discussed previously.

The previously described mobile telecommunications device and method may also be used to monitor and generate a driver profile. The driver profile may be indicative of the type of driver a user is. For example, this may comprise determining if a user is a calm and patient driver, or an aggressive driver. Also, this may comprise determining whether a user regularly flouts speed limits, and/or ignores safety distances.

For example, analysis of captured image data, such as video footage, may be used to determine if a user regularly flouts safety distances. In preferred embodiments, the mobile telecommunications device is arranged within the subject vehicle to have a clear line of sight of the road in the direction of principle motion. Accordingly, the number plate of any vehicle preceding the subject vehicle will be captured by the image capture module. Since the physical dimensions of number plates are standardised and known in each country, they may be used as a metric to scale the captured image. When combined with the known optical characteristics of the image capture module, this enables the distance of the image capture module from the image object (i.e. the preceding vehicle) to be determined at the time of image capture. This information may then be used to see if a user adheres to recommended safety distances.

Image analysis can also be used to determine driving conditions and the driving environments. For example, image processing can detect road signs, such as stop signs or traffic lights. Furthermore, driving conditions, as affected by the weather can be determined. For example, if the windscreen wipers are detected to be in motion, it can be inferred to be raining. Once the driving conditions are so determined, an assessment of the driving performance of a user can be made by determining whether the user reacts or behaves appropriately to the driving conditions. For example, if the driver is seen to be jumping red lights, or driving dangerously in rain or snow, then a higher risk profile may be assigned to that driver.

Similarly, accelerometer and g-force data may be used to determine if a user has an erratic driving style. For example, a user that generates sharp variations in g-force data and accelerometer data during regular operation of a vehicle, may be considered to drive erratically, and therefore at greater risk of being involved in an accident,

The mobile telecommunications device may also be configured to interface and communicate directly with a vehicles native telemetry systems. For example, the majority of modem cars have inbuilt electronic control systems or engine management systems, arranged to collect data regarding vehicle system performance. This data may be communicated to the mobile telecommunications device either via a wired physical connection, such as USB (Universal Serial Bus), or via a wireless communication protocol such as Bluetooth®. This data may subsequently be used by the mobile telecommunications device to complement sensor data captured directly by the telecommunications device. In this way a more accurate model of a driving incident, and/or of a user profile may be generated. For example, native vehicle telemetric systems may comprise electronic tyre pressure sensors, and are able to detect if a tyre is under and/or over-inflated. This information may be communicated to the mobile telecommunications device and may help to explain the causes for a driving incident, such as a sudden loss of traction resulting from a burst tyre.

Further features of the Witness application, are set out below.

Benchmarking

It will be appreciated that different sensor types, phones, mounting positions, vehicles, drivers and road conditions may generate differing outputs for driving behaviour that is ‘safe’. To account for this, the Witness application may have the following functionality:

During a ‘training mode’ (e.g. first week of enabling the Witness application) the input from the sensors are used to build up a ‘benchmark’ for a particular driver's typical driving conditions.

Assuming an accident does not occur during this training mode, the benchmark data can be subsequently used to assess the occurrence of driving incident.

There is preferably an option to notify the Witness application of a change in parameters (e.g. different driver, driving off-road etc). Thus a number of ‘profiles’ may be set up. Each profile may require an independent training mode period.

If the Witness application incorrectly detects that a driving incident has occurred, it can receive feedback from the user to modify its sensitivities. E.g. a more aggressive driver is actually driving.

Crash/Collision Management

On detection of a crash (or other driving incident) the Witness application is arranged to take one or more of the following actions:Announce that it has detected an accident (audio/screen prompt)Call the emergency services (with option to cancel)—e.g: Audio/screen prompt: ‘Witness has detected that you have been involved in a (serious) accident and will call the emergency services. If this is not the case, please cancel within 10 seconds.’Provide reassuranceProvide the user with a checklist of things to do:Take photographs of vehicles involved in the incident (inc number plates)Take down name, address, insurance details etc of 3rdparties involved in incidentCommunicate to the insurance company that an incident has been detected (e.g. low bandwidth data or text message)Protect the high quality recorded data so it is not overwrittenIf appropriate—or in response to a request from a communication to the phone from the insurance company—transmit all or selected portions of the recorded data associated with the incident. Data processing for generating a user profile

Further features of the user profile generation embodiment are summarised:

If the Witness application incorrectly detects an event or incident of significance, and receives feedback that the vehicle was not involved in an accident (but was in a near miss)—this could alter the risk profile of the driver. For example—if this happens frequently, but no accident occurs over a given period, this could be an indicator that the driver is good at reacting to potential hazards.

Erratic driving (e.g. jerky steering or braking detected by G-force sensor).

Driving faster than the speed limit allocated to a given road (detected by GPS).

Further details regarding the features and functionality of the Witness application, in particular the graphical user interface of the Witness application, are now described.

Referring toFIG.5, a first page80of the Witness application user manual is shown in which an image of the Main Menu (the top-level menu)82is displayed. The Main Menu is displayed when a user first runs the application on a smart-phone, such as the iPhone® 4. The main menu includes the following user-selectable buttons:Recording Screen84File Management86Settings88Accident Management90Information92(displays up a manual, as shown inFIGS.5to14).

Selecting the Recording Screen button84opens the Recording Screen94—the second image shown inFIG.5. The Recording Screen94contains a video feed96from the camera of the smart-phone, which occupies the majority of the visible screen area. Overlaid on to the video feed is the detected speed of the vehicle98(e.g. 0 mph), heading100(e.g. south by south-west) and the elapsed recorded time102(e.g. 00:00). Displayed in a left column of the Recording Screen are additional user selectable buttons:

Keep current video104(pressing this button will automatically copy the current video—and the previous video segment—to the protected storage34b, and prevents that information from being overwritten. The user is advised to press this button in the event of a driving incident that needs to be recorded).

Take photo106(captures a still photograph).

Exit Recorder108(returns to main screen).

Start/Stop Recording110(starts recording video footage—and other data).

Referring toFIG.6, a second page112of the Witness application user manual is shown, in which an image of the Recording Screen114is shown and described operating in a map-displaying mode rather than a video-feed mode.

During recording, (i.e. when the ‘Start/Stop Recording’ button110is pressed) the Exit Recorder button in the Recording Screen is substituted with a Map Display button116. Pressing it will toggle between the modes showing the video feed and a map of the current location.

Referring toFIG.7, a third page118of the Witness application user manual is shown, in which the File Management Screen120is shown and described. The File Management Screen120can be accessed by pressing the File Management Button86in the Main Menu.

The File Management Screen120displays video and associated data (e.g. telemetry data) that has been previously recorded. The stored data is contained in either a protected area of storage or in a “Recording Stack”. Data files in the protected area are saved and so not overwritten as part of a Recording Strategy. In contrast, data files in the Recording Stack may be overwritten as part of the Recording Strategy.

As illustrated inFIG.7, the bottom section of the list represents the “Kept” data files122, whereas the top section of the list represents the “Recording Stack”124. “Kept” data files122can be edited and deleted from the File Management Screen120, whereas “Recording Stack” data files cannot. Editing “Kept” data files can involve renaming them.

The recorded data files are listed on screen along with the time and date of the recording and the electronic size of the data file. Accordingly, the user is provided with feedback about how big the data files are, and so if the smart-phone is low in storage, the user can elect to delete certain “Kept” data files. “Recording Stack” data files will be automatically overridden by the Recording Strategy.

The controls on the toolbar at the bottom of the recordings screen allow a user to change the selection mode of the video stack and includes:

Video126(when highlighted, if a data file is selected, a video recording playback screen will be displayed).

Map128(when highlighted, if a data file is selected, then a map will be displayed showing the area over which the recording was made).

Export130(when highlighted, a selected data file will be passed to an Export Screen where export options will be provided).

Keep132(when highlighted, if a “Recording Stack” data file is selected, then a user will be prompted to name it, and then it will be stored as a “Kept” data file).

The icons in the File Management Screen change in dependence on the selected mode. For example,FIG.20shows the icons134displayed when the Video selection mode is highlighted;FIG.21shows the icons136displayed when the Map selection mode is highlighted;FIG.22shows the icons138displayed when the Export selection mode is highlighted andFIG.23shows the icons140displayed when the Keep selection mode is highlighted. Advantageously, this provides improved feedback to a user about what selection mode is highlighted and so what action is likely from the selection of a data file.

Referring toFIG.8, a fourth page142of the Witness application user manual is shown, in which an image of a Recording Playback screen144is shown and described. The Recording Playback screen is invoked by highlighting the Video Mode126in the File Management Screen120and selecting an appropriate data file.

In the Recording Playback screen144it is possible to play back a pre-recorded video feed associated with a chosen data file. As well as playing back the recorded video feed, the Recording Playback screen also displays other associated data such as telemetry data146. For example, date, time, speed, forward/backward G-forces, latitude, longitude and heading information is overlaid onto the video feed. As the video is played back, these values will typically change depending on the behaviour of the vehicle, as recorded by the mobile device.

Forward and backward G-forces are those that correspond to the forward and backward movement of the vehicle and are the primary metric for determining an event such as a crash. However, other G-forces (e.g. up/down and side-to-side) may also be measured by the device—even if they are not necessarily displayed on the mobile screen.

As recited inFIG.8, the Recording Playback screen provides further user controls in the form of:

Playback Position Scrub Bar148(horizontal bar at bottom of screen)

Playback Speed Bar150(vertical bar at left side of screen)

Pause152(pauses playback)

Eject154(return to previous screen)

Loop156(plays the data file continuously)

The Playback Position Scrub Bar148allows a user to jump to different positions within the recording without necessarily needing to watch all of the recorded footage. Advantageously, this allows a user to more quickly locate a desired item of footage within a given data file. Also, the Playback Speed Bar150can be used to speed up or slow down the playback of the data file. This allows a desired item of footage to be found more quickly through sped-up playback, and also allows an item of footage to be more carefully analysed through slowed-down playback. Furthermore, it is possible to zoom in and out of a region of the video file using a ‘pinch and zoom’ movement as is standard with most multi-touch touch-screen devices.

Referring toFIG.9, a fifth page158of the Witness application user manual is shown, in which a different type of Recording Playback screen160is shown and described. Specifically, the image represents playback of a pre-recorded data file as can be invoked by highlighting the Map Mode128in the File Management Screen120and selecting an appropriate data file.

In contrast with the Video Mode playback, this Map Mode playback shows a map of the area162where the recording took place. A blue breadcrumb trail164is overlaid on to the map showing the extent of movement of the vehicle during the recorded period. A scrub slider166is shown, which is user-operable to adjust the time within the recorded period. As the slider is adjusted, a pin168moves along the blue breadcrumb164to show the position of the vehicle at the time as specified by the slider166. Tapping the pin168displays associated telemetry data170at that position and time.

Referring toFIG.10, a sixth page172of the Witness application user manual is shown, in which an image of a Video Export Screen174is described. The Video Export Screen174may be invoked by highlighting the Export button130in the File Management Screen120and selecting an appropriate data file.

The Video Export Screen174allows the quality of the video associated with the selected data file to be adjusted, before that video is exported. Advantageously, this can allow the user to control the size of the data to be exported. Telemetry data (i.e. sensor data) is also exported, embedded within the video file. Video data can be exported with or without sound, depending on the permissions of the user. Export typically involves copying data files from the mobile device to a local computer (e.g. via a data cable) or a remote server (e.g. via a wireless connection).

Referring toFIG.11, a seventh page176of the Witness application user manual is shown, in which an image of the Settings Screen178is displayed and described. The Settings Screen178allows the operation of the Recording Strategy to be controlled. Specifically, the Settings Screen allows a user to select the number of video segments that the application should store, and the size of those segments (i.e. a data file period). Accordingly, the user is able to set a limit to the storage usage of the Recording Stack (i.e. the size of the buffer34a) part of the program. The user can also control the storage usage of the Recording Stack through the use of the video quality buttons.

Note that the Witness application may be arranged to calculate the remaining memory available on the smart-phone and suggest the appropriate settings automatically.

The Recording Strategy involves maintaining a user-controlled number of video segments. When a new video recording is initialised—instantiating a new segment—this is written over the oldest video segment. Thus only the most recently recorded videos are maintained in the Recording Stack.

Another setting that can be controlled in the Settings Screen178is the G-force threshold at which the Witness application will assume that a crash has taken place. It is expected that different vehicles and driving styles will need different G-force thresholds to be set to ensure a reasonable sensitivity to crash forces whilst also prevent crash detection false positives. It should be noted that although the manual recites “Raise the sensitivity if you find that crash detection is triggered during normal driving . . . ” it is the sensitivity threshold that is to be raised, and not the sensitivity itself. A slider180allows the sensitivity threshold to be set via the touch-sensitive screen.

The Settings Screen178also has a button182to allow a user to define more settings via a More Settings Screen.

Referring toFIG.12, an eighth page184of the Witness application user manual is shown, in which an image of the More Settings Screen186is displayed and described. Here, it is possible for the user to select speed units188and also select whether the map should be displayed during recording190, and at which speed it should be displayed in favour of the video feed. This is a safety feature of the Witness application that hides the video feed during recording when the vehicle is detected as travelling above a predetermined speed. The video feed is replaced by a map of the location of the vehicle—as is typical with in-vehicle GPS devices. Note that although the on-screen video feed is replaced with a map, video recording continues in the background.

In alternatives, the application may be arranged to detect the vehicle speed, and at a particular speed, switch off the screen entirely. It should be understood that the device will continue to record video, telemetry and other information even when the screen is switched off. Entirely switching off the screen of the device is advantageous as it significantly reduces the drain on the battery of the mobile device.

Note that the Witness application is also arranged to interface with the phone to detect low-light conditions and in response change the brightness of the screen to prevent the user/driver being dazzled. This can also save battery life.

The More Settings Screen186also includes a Personal Details button192which, when pressed invokes a Personal Details Screen.

Referring toFIG.13, a ninth page194of the Witness application user manual is shown, in which an image of the Personal Details Screen196is displayed and described.

Here, the name198, vehicle registration number200, mobile number202and email address204to be used in an emergency, can be specified by the user. In the event of a detected incident, these details, along with information regarding the detected incident (e.g. time of incident, location and optionally telemetry data) will be emailed to the specified email address automatically. This can ensure that the chosen recipient of that email will be informed immediately about the occurrence and nature of the detected incident.

This Personal Details Screen196also allows the user to set whether it is possible for the video stack to be deleted206. A security PIN protects the changing of this option so that if a first user having the PIN (for example, a parent) can review the driving style and behaviour of a second user not having the PIN (for example, their son or daughter) by reviewing the Recording Stack—as the Recording Stack cannot be deleted by that second user. As shown inFIG.24, the PIN can be set for the first time by entering it twice into the appropriate PIN fields208.

Referring toFIG.14, a tenth page210of the Witness application user manual is shown, in which images of an Accident Management Screen212and Witness Details Screen214are displayed and described.

The Accident Management Screen212can be invoked by the user selecting the accident management button90on the Main Menu, or can be automatically switched to after the Witness application has detected that there has been an incident (e.g. via G-forces exceeding a threshold level). Similarly, data can automatically be permanently stored as “Kept” and/or sent if high G-forces are detected.

As can be seen on the Accident Management Screen212, there are the following user selectable buttons:

Accident Advice/What to do214

This option provides guidance as to what to do during an accident—seeFIGS.15and16for the displayed Accident Advice Screens.

Witnesses216

This option invokes functionality to allow the collection of information from or about witnesses to an incident. An image of the Witnesses Detail information summary screen214is shown inFIG.14.FIG.17shows the Witness Detail information collection screen218. Voice notes can also be taken via the microphone of the smart-phone.

Photos220

This option enables the camera to allow a user to capture images associated with an incident.

Drivers222

This option invokes functionality to allow collection about the other drivers involved in an incident. The Drivers Detail information collection screen224is shown inFIG.18.

Your Details226

This option provides a readily available store of the details of the user—as to be provided to other drivers—and can contain name and address. The ‘Your Details’ screen228is shown inFIG.19.

Details recorded via the Accident Management Screen212are electronically authenticated and feature a time-stamp. In alternatives, this data may be encrypted to ensure the integrity of the stored data.

Furthermore, in alternatives, the exchange of the details of the drivers and witnesses can be conducted, at least in part, via Bluetooth® (for example, between mobile devices)—and/or via email. Relevant information associated with the driver is pre-stored on the Witness application in a format that can be sent readily, via a communication channel such as Bluetooth® or email.

Finally, note that in each Screen or Menu, there is a button230provided to go back to the previous menu. Also, when certain actions are performed or selected by a user, the Witness application is arranged to provide an audible feedback signal (for example, a beep). For example, this could be in response to starting a recording, keeping a video segment and/or a stopped recording.

ALTERNATIVES AND EXTENSIONS

The Witness application can be extended to interface with remote users to allow them to control the operation of the application as well as view information logged by the application.

With such a monitoring system, it is unpractical to actively monitor each and every mobile device and receiving and handling the sheer quantity of data from each and every device would also be very difficult.

Consequently, the device monitoring system and the mobile devices are advantageously arranged to automatically and intelligently interface with one another in a way that minimises the computational and bandwidth burden on the device monitoring system.

In particular, each mobile device running the Witness application is arranged to make a determination as to whether there is a need to transfer data to the device monitoring system. For example, the Witness application may be arranged to automatically send data to the device monitoring system that only has high G-force activity associated with it, as this may be indicative of a crash or another driving incident. Alternatively, the Witness application may be arranged to send detailed or high-bandwidth information if a user indicates that an accident has occurred.

In either case, the video segments and associated sensor and/or derived driving information data associated with the relevant event will be sent. However, no other data (for example, from another date) will be sent.

Alternatively, less computationally or bandwidth intensive information may be sent on a periodic basis—for example, every day, week or month. Such general information may be sent to establish a profile of a particular user. For example, many high G-force related activities may indicate that the driver is driving aggressively. A determination about the profile can be made automatically by requesting and automatically analysing further data. For example, the location and speed information can be used to determine whether a vehicle is found to frequently break the speed limit. If this is the case, then the profile of the user can be set accordingly.

Such general and periodically obtained information can also be used to remotely adjust the settings of the application. For example, many high G-force related activities may instead indicate that the set up of the car is such that the mobile device is subjected to high G forces (rather than the driver driving aggressively). If the driving is determined to be safe, but the mobile device is setting off many false positives, then the device monitoring system can automatically adjust the settings of the application. For example, the G-force sensitivity threshold may be increased remotely.

Thus, the initial data that is automatically transmitted from the mobile device to the vehicle monitoring system is limited in bandwidth so as not to overload the vehicle monitoring system as a whole. However, after processing or analysis is performed on the initial data, further information may be requested. For example, further information may be requested automatically by the system or manually by a controller. Such further information may include high resolution video logs and sensor data and derived driving information such as G-force measurements.

It will be understood that the mobile device may also automatically keep certain data at random. Furthermore, the reason for keeping certain data may also be logged (e.g. logged as a result of a manual request by the user, or in response to a high G-force event).

However, so that users do not wrongfully benefit from such incentives, it is necessary to enforce the correct and consistent use of the Witness application.

Accordingly, the application may include measures to guarantee that the application is enabled whenever a given insured vehicle is being driven.

Such measures may involve matching data recorded by the Witness application with that recorded independently by the vehicle. For example, the Witness application records the distance travelled during every journey. To ensure the summed distances of all journeys tracked by the Witness application tally with the total travelled distance of the vehicle, the user may be prompted to enter the odometer mileage periodically.

It will be appreciated that such a tallying exercise depends on the Witness application being used every time one particular vehicle is driven. However, in alternatives, if the Witness application is used with different vehicles, the Witness application may be arranged to register the different vehicles so that their respective odometer readings can be tallied with distance recordings associated with each respective vehicle.

Other measures can be implemented in conjunction with the Witness application to guarantee that the application is enabled whenever a given vehicle is being driven. For example, the mobile device on which the Witness application is installed may comprise an NFC device, such as an RFID tag. The NFC device may be compatible with a complementary device on a so-called ‘smart-holster’ into which the mobile device may be fitted during operation.

The smart-holster interacts with the NFC device on the smart-phone to determine whether or not the smart-phone is inserted into the smart-holster. The smart-holster can then be interfaced with the engine management system of the vehicle so that the vehicle may be activated and driven only when the mobile device is in place within the smart-holster.

It will be appreciated that the mobile device has so far been described in the context of a smart-phone. However, it will be appreciated by a person skilled in the art that other devices may also be suitable for performing the functions described in relation to the Witness application. For example, the Witness application may be adapted to run on a general purpose tablet-style computing device, such as an iPad®.

Furthermore, it will be understood that features, advantages and functionality of the different embodiments described herein may be combined where context allows. In addition, a skilled person will appreciated that the functionality described above may be implemented using the mobile device suitably programmed.

Having described several exemplary embodiments of the present invention and the implementation of different functions of the device in detail, it is to be appreciated that the skilled addressee will readily be able to adapt the basic configuration of the device to carry out described functionality without requiring detailed explanation of how this would be achieved. Therefore, in the present specification several functions of the device have been described in different places without an explanation of the required detailed implementation as this not necessary given the abilities of the skilled addressee to code functionality into the device.