Patent ID: 12258026

DETAILED DESCRIPTION

The advantages and features of the present disclosure and ways to achieve them will be apparent by making reference to embodiments as described below in detail in conjunction with the accompanying drawings. However, it should be construed that the present disclosure is not limited to the embodiments disclosed below but may be implemented in various different forms, and covers all the modifications, equivalents, and substitutions belonging to the spirit and technical scope of the present disclosure. The embodiments disclosed below are provided so that this disclosure will be thorough and complete and will fully convey the scope of the present disclosure to those skilled in the art. Further, in the following description of the present disclosure, a detailed description of known technologies incorporated herein will be omitted when it may make the subject matter of the present disclosure rather unclear.

The terms used in this application is for the purpose of describing particular embodiments only and is not intended to limit the disclosure. As used herein, the singular forms are intended to include the plural forms as well, unless the context clearly indicates otherwise. In the description, it should be understood that the terms “include” or “have” indicate existence of a feature, a number, a step, an operation, a structural element, parts, or a combination thereof, and do not previously exclude the existences or probability of addition of one or more another features, numeral, steps, operations, structural elements, parts, or combinations thereof. Although the terms “ordinal numbers” such as first, second and the like are used to describe various structural elements, the structural elements should not be defined by the terms. The terms are used merely for the purpose to distinguish an element from the other elements.

In this application, the term “unit” may refer to a hardware component such as a processor or circuit, and/or a software component executed by a hardware component such as a processor.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings, and in the description with reference to the accompanying drawings, the same or corresponding components have the same reference numeral, and a duplicate description thereof will be omitted.

FIG.1is an exemplary diagram of a 3D simulator-based fault ratio calculation environment including a fault ratio calculation device, a user terminal, a black box, and a network connecting the same, according to the present embodiment. Referring toFIG.1, a 3D simulator-based fault ratio calculation environment1may include a fault ratio calculation device100, a user terminal200, a black box300, and a network400.

The fault ratio calculation device100may collect images stored in the black box300from the user terminal200, may generate a 3D point cloud map by using a feature point and a depth estimation result, and may generate, based on the 3D point cloud map, a driving route in which a first vehicle travels.

The black box300may be provided in the first vehicle as an accident vehicle, and the black box300may include a mono camera. The vehicle black box300is to record situations before and after a vehicle crash accident so as to perform rapid post-processing of the accident and scientific accident interpretation, and the importance thereof is strongly recognized with a recent rapid increase in vehicle accidents. In particular, in order to effectively prevent a vehicle accident, it is very important to comprehensively analyze all kinds of information, such as a driver, vehicle driving information, and a surrounding environment, so as to accurately identify occurrence of a vehicle accident. Various data recorded in the vehicle black box300may be used as proof data for resolving various civil and criminal disputes, as well as data for analysis of a cause of an accident. The vehicle black box300uses an external storage medium for storing image information, to generate and store image data obtained by image capturing of an external situation and vehicle driving information, such as a vehicle speed and a vehicle position, and a situation when driving of a vehicle may be accurately identified by analyzing the vehicle driving information. In the present embodiment, the fault ratio calculation device100may not be limited to collecting images stored in the black box300through the network400, but may collect images through various paths.

The fault rate calculation device100may generate a 3D point cloud map by using feature points extracted from image frames collected from the mono camera and a depth estimation result acquired via comparison between two adjacent image frames from among the image frames, and may generate, based on the 3D point cloud map, a driving route in which a first vehicle travels. Here, a point cloud, which is a method of expressing 3D data, refers to a point set of a large amount, and a large amount of three-dimensional data may be expressed as a point cloud. A point cloud is a value that is compared to a 2D image, corresponds to a method of expressing one point in three dimensions, and is a vector form that may include both position coordinates and color. For example, a point cloud may be expressed as (x, y, z, R, G, B). A point cloud, which forms a spatial configuration by gathering of countless colors and position data, may become more and more specific data as a density thereof increases. A 3D point cloud may include at least one piece of information among an image capturing position (e.g., a name of a region or a group name), the number of feature points included in an image frame, a type of a captured image, and a method of matching feature points. The fault ratio calculation device100may generate a 3D point cloud map by using the 3D point cloud and information included therein, and may generate, from the 3D point cloud map, a driving route in which a first vehicle travels.

The fault ratio calculation device100may acquire, using the driving route and image frames, a classification result of a traffic accident case and fault-related information including a basic fault ratio of the first vehicle and a second vehicle as an assault vehicle, wherein the basic fault ratio corresponds to the classification result of the traffic accident case.

In the present embodiment, the fault ratio calculation device100may generate a fault-related information prediction result corresponding to the driving route and the image frames by using a deep neural network model trained in advance to predict fault-related information by using the driving route and the image frames.

As an optional embodiment, the fault ratio calculation device100may transmit, to a fault ratio information portal (not shown) of the dispute deliberation committee, the image frames collected from the mono camera of the first vehicle and a fault-related information request signal, and may receive, from the fault ratio information portal (not shown) of the dispute deliberation committee, a fault-related response signal including a classification result of a traffic accident case, a basic fault ratio of the first vehicle and the second vehicle, and a score according to addition/subtraction factors for the first vehicle and the second vehicle.

The fault ratio calculation device100may generate multiple simulation driving scenarios by simulating the described driving route with the first vehicle and the second vehicle, in which ratio adjustments of a first factor representing braking performance and a second factor representing steering performance are applied. The fault ratio calculation device100may apply dynamic characteristics of the first vehicle and the second vehicle when simulating the driving route. In this case, a vehicle model with the dynamic characteristics, detailed vehicle specifications, and the like may be included.

In the present embodiment, the fault ratio calculation device100may generate N×K simulation driving scenarios with respect to a first vehicle by using the first factor including N ratio adjustments and a second factor including K ratio adjustments. In the present embodiment, the fault ratio calculation device100may generate M×J simulation driving scenarios with respect to the second vehicle by using a first factor including M ratio adjustments and a second factor including J ratio adjustments.

In relation to generating the driving scenarios, more specifically, the fault ratio calculation device100may generate a driving trajectory so as to extract longitudinal and lateral control values of the driving route. The longitudinal control value may refer to a control value in a direction parallel to a driving direction on the driving route, and the lateral control value may refer to a control value in a direction perpendicular to the driving direction on the driving route. Multiple driving scenarios may be generated by combining the extracted longitudinal control values and lateral control values of the driving route in various different ways.

The fault ratio calculation device100may compute a fault ratio weight by loading weights assigned to the ratio of the first factor and the ratio of the second factor, with respect to the first vehicle and second vehicle corresponding to each of the multiple simulation driving scenarios. In the present embodiment, in the fault ratio calculation device100, first factor weights corresponding to respective multiple (N and M) ratios for the first factor and second factor weights corresponding to respective multiple (K and J) ratios for the second factor are built as a database, and a weight for each factor built in the database may be loaded when a fault ratio weight is computed.

In response to receiving selection of one or more addition/subtraction factors capable of changing the basic fault ratio, the fault ratio calculation device100may generate modified addition/subtraction scores by modifying addition/subtraction scores preconfigured for the addition/subtraction factors, on the basis of the fault ratio weight, and may calculate a final fault ratio for the first vehicle and the second vehicle by applying the modified addition/subtraction scores to the basic fault ratio.

The user terminal200may be provided with the described service relating to fault ratio calculation, by accessing a fault ratio calculation site and/or a fault ratio calculation application provided by the fault ratio calculation device100.

The user terminal200may include a communication terminal capable of performing a function of a computing device (not shown), and may be, but not limited to, a desktop computer, a smartphone, a tablet PC, a notebook computer, a smart TV, a mobile phone, a personal digital assistant (PDA), a laptop, a media player, a micro server, a global positioning system (GPS) device, an e-book terminal, a digital broadcast terminal, a navigation system, a kiosk, an MP3 player, a digital camera, a home appliance, and other mobile or non-mobile computing devices, which are operated by users. The user terminal200may be a wearable terminal, such as a watch, glasses, a hair band, and a ring, having a communication function and a data processing function. The user terminal200is not limited to the aforementioned contents, and a terminal capable of web browsing may be adopted without limitation.

The network400may serve to connect the fault ratio calculation device100and the user terminal200. The network400may encompass, for example, wired networks, such as local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), and integrated service digital networks (ISDNs), or wireless networks, such as wireless LANs, CDMA, Bluetooth, and satellite communications, but the range of the present disclosure is not limited thereto. The network400may transmit or receive information by using short-range communication and/or telecommunication. In this case, the short-range communication may include Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra-wideband (UWB), ZigBee, and wireless fidelity (Wi-Fi) technologies, and the telecommunication may include code division multiple access (CDMA), frequency division multiple access (FDMA), time division multiple access (TDMA), orthogonal frequency division multiple access (OFDMA), single carrier frequency division multiple access (SC-FDMA) technologies.

The network400may include connections of network elements, such as hubs, bridges, routers, and switches. The network400may include one or more connected networks, e.g., multiple network environments, including a public network, such as the Internet, and a private network, such as a secure enterprise private network. Access to the network400may be provided via one or more wired or wireless access networks. Furthermore, the network400may support an Internet of Things (IoT) network and/or 5G communication, which exchange and process information between distributed components, such as things.

FIG.2is a block diagram provided to schematically illustrate a configuration of the fault ratio calculation device according to the present embodiment. In the following description, descriptions of parts overlapping with those ofFIG.1will be omitted. Referring toFIG.2, the fault ratio calculation device100may include a communication unit110, a storage medium120, a program storage unit130, a database140, an information processing unit150, and a controller160.

The communication unit110may provide a communication interface necessary for providing a transmission/reception signal between the fault ratio calculation device100and the user terminal200in a packet data form in conjunction with the network400. Furthermore, the communication unit110may serve to receive a predetermined information request signal from the user terminal200, and may serve to transmit information processed by the information processing unit150to the user terminal200. A communication network is a medium that serves to connect the fault ratio calculation device100and the user terminal200, and may include a path providing an access path so that the user terminal200is able to transmit or receive information after accessing the fault ratio calculating device100. The communication unit110may be a device including hardware and software necessary for transmitting or receiving a signal, such as a control signal or a data signal, via a wired/wireless connection with another network device.

The storage medium120performs a function of temporarily or permanently storing data processed by the controller160. The storage medium120may include a magnetic storage medium or a flash storage medium, but the scope of the present disclosure is not limited thereto.

The program storage unit130is loaded with control software configured to perform an operation of generating a driving route, in which a first vehicle travels, based on image frames collected from the user terminal200and/or the black box300, an operation of acquiring, using the driving route and the image frames, fault-related information including a classification result of a traffic accident case and a basic fault ratio of the first vehicle and a second vehicle, an operation of generating multiple simulation driving scenarios by simulating the driving route with the first vehicle and the second vehicle, in which ratio adjustments of a first factor and a second factor are applied, an operation of, with respect to the first vehicle and the second vehicle, which correspond to each of the multiple simulation driving scenarios, computing a fault ratio weight by loading weights assigned to a ratio of the first factor and a ratio of the second factor, an operation of, in response to receiving selection of one or more addition/subtraction factors capable of changing the basic fault ratio, generating modified addition/subtraction scores by modifying addition/subtraction scores preconfigured for the addition/subtraction factors, on the basis of the fault ratio weight, an operation of calculating a final fault ratio for the first vehicle and the second vehicle by applying the modified addition/subtraction scores to the basic fault ratio, and the like.

The database140may include a management database configured to store an algorithm and information for calculation of a fault ratio. A simultaneous localization and mapping (SLAM) algorithm for generation of a map for a space, in which a vehicle travels, by using depth estimation information and feature points from collected image frames may be built in the management database. In the management database, a 3D model that enables reflection of dynamic characteristics of a vehicle during a simulation of a driving route may be built. In order to compute a fault ratio weight, first factor weights corresponding to respective N and M ratios of the first factor and second factor weights corresponding to respective K and J ratios of the second factor may be built in the management database.

The database140may include a user database configured to store information of a user who is to be provided with a fault ratio calculation service. The information of the user may include basic information on a subject's name, affiliation, personal information, gender, age, contact information, e-mail, address, image, etc., information on authentication (login) of the subject, such as an ID (or email) and a password, access-related information, such as an access country, an access location, information on a device used for access, and an accessed network environment, etc.

In the present embodiment, the user database or the management database may store information and/or a category history provided to a user accessing a fault ratio calculation application or a fault ratio calculation site, user-configured environment configuration information, information on the amount of resources used by the user, and billing and payment information corresponding to the amount of resource used by the user.

The information processing unit150may generate a driving route, in which a first vehicle travels, based on image frames collected from the user terminal200and/or the black box300.

The information processing unit150may acquire, using the driving route and the image frames, a classification result of a traffic accident case and fault-related information including a basic fault ratio of the first vehicle and a second vehicle.

The information processing unit150may generate multiple simulation driving scenarios by simulating the driving route with the first vehicle and the second vehicle, in which ratio adjustments of a first factor and a second factor are applied.

The information processing unit150may compute a fault ratio weight by loading weights assigned to a ratio of the first factor and a ratio of the second factor, with respect to the first vehicle and second vehicle corresponding to each of the multiple simulation driving scenarios.

The information processing unit150may generate, in response to receiving selection of one or more addition/subtraction factors capable of changing a basic fault ratio, modified addition/subtraction scores by modifying addition/subtraction scores preconfigured for the addition/subtraction factors, on the basis of the fault ratio weight.

The information processing unit150may calculate a final fault ratio for the first vehicle and the second vehicle by applying the modified addition/subtraction scores to the basic fault ratio.

The controller160is a kind of a central processing unit, and may control all operations of the fault ratio calculation device100by driving the control software loaded on the program storage unit130. The controller160may include all kinds of devices capable of processing data, such as a processor. Here, the “processor” may refer to, for example, a data processing device embedded in hardware, the device having a physically structured circuit to perform a function expressed as a code or an instruction included in a program. As an example, the data processing device embedded in hardware may encompass processing devices, such as a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA), but the scope of the present disclosure is not limited thereto.

FIG.3is a block diagram provided to schematically illustrate a configuration of an information processing unit in the fault ratio calculation device ofFIG.2. In the following description, descriptions of parts overlapping with those ofFIG.1andFIG.2will be omitted. Referring toFIG.3, the information processing unit150may include a first generator151, an acquisition unit152, a second generator153, a computation unit154, a third generator155, and a calculator156.

The first generator151may be configured to load an SALM algorithm stored in the database140, extract feature points by applying image frames collected from the mono camera (the black box300) of the first vehicle to the SLAM algorithm, generate a 3D point cloud map by using a depth estimation result acquired via comparison of two adjacent image frames among the image frames, and generate, based on the 3D point cloud map, a driving route in which the first vehicle travels.

FIGS.4A and4Billustrate an example in which a 3D point cloud map is generated using image frames collected by the first generator151, and a driving route may be reconfigured based on the 3D point cloud map. Reference numeral410inFIG.4Bindicates one collected image frame, and reference numeral420inFIG.4Amay indicate a point cloud map. In the point cloud map illustrated in reference numeral420, a blue triangle may indicate a key frame, and a green frame may indicate a current frame. This includes a camera pose (position), an angle (viewpoint), and a speed and may match a waypoint. Accordingly, some of the image frames may be extracted at each predetermined time so as to be used as waypoints, and each image frame may include a time sequence.

The first generator151may generate, as the driving route, a set of waypoints of a first vehicle corresponding to image frames extracted at a preconfigured time period.

The acquisition unit152may acquire, using the driving route and the image frames, a classification result of a traffic accident case and fault-related information including a basic fault ratio of the first vehicle and a second vehicle as an assault vehicle, wherein the basic fault ratio corresponds to the classification result of the traffic accident case.

FIG.5illustrates a classification result of a traffic accident case and fault-related information including a basic fault ratio of a first vehicle and a second vehicle, wherein the classification result and the fault-related information are acquired by the acquisition unit152. The fault-related information including the basic fault ratio may further include an accident situation, an applied fault, and an addition/subtraction factor.

In the present embodiment, the acquisition unit152may generate a fault-related information prediction result corresponding to a driving route and image frames by using a deep neural network model trained in advance to predict fault-related information by using the driving route and the image frames. In the present embodiment, the deep neural network model may be a model trained according to a supervised learning scheme on the basis of training data in which image frames collected from a mono camera of the first vehicle and the driving route of the first vehicle are received as inputs, and the traffic accident case and the basic fault ratio of the first vehicle and the second vehicle are configured as labels.

The acquisition unit152may train the initially configured deep neural network model according to the supervised learning scheme by using labeled training data. Here, the initially configured deep neural network model is an initial model designed to be configured as a model capable of predicting fault-related information, and parameter values are configured to any initial values. The initial model may be completed as a fault prediction model capable of accurately predicting fault-related information as parameter values are optimized while training is performed via the described train data.

As an optional embodiment, the acquisition unit152may transmit, to a fault ratio information portal of the dispute deliberation committee, the image frames collected from the mono camera of the first vehicle and a fault-related information request signal, and may receive, from the fault ratio information portal of the dispute deliberation committee, a fault-related response signal including a classification result of a traffic accident case, a basic fault ratio of the first vehicle and the second vehicle, and a score according to addition/subtraction factors for the first vehicle and the second vehicle.

The second generator153may generate multiple simulation driving scenarios by simulating a driving route with the first vehicle and the second vehicle, in which ratio adjustments of a first factor representing braking performance and a second factor representing steering performance are applied.

The second generator153may apply dynamic characteristics of the first vehicle and the second vehicle when simulating the driving route. In this case, a vehicle model with the dynamic characteristics, detailed vehicle specifications, and the like may be included.

FIG.6AandFIG.6Billustrate a simulation screen620and a vehicle model data set and configuration screen610applied to generate multiple simulation driving scenarios by the second generator153. In order to generate a simulation driving scenario which is similar to reality and has high data reliability, a dynamic model of a vehicle may be essential. By loading the dynamic model of the vehicle from the database140, physical properties, such as weight, may be changed according to situations of the first vehicle and the second vehicle. In the vehicle model data set and configuration screen610, reference numeral611is a part for selecting of a vehicle model having a unique dynamic characteristic, and reference numeral612is a part for selecting of details of a vehicle, wherein a model and details may be selected according to situations of the first vehicle and the second vehicle by using reference numerals611and612. A performance difference and a weight (overloading) depending on a vehicle are applicable to physical properties of a vehicle, and thus a more reliable result is obtainable.

When generating multiple simulation driving scenarios, the second generator153may configure, on a driving route, a start point reflecting ratio adjustments of a first factor (braking performance) and a second factor (steering performance). Here, the start point may include an image frame, in which an object appears and the number of feature points is equal to or greater than a preconfigured number, from among image frames included in the driving route.

FIG.7AtoFIG.7Hillustrate image frames for describing of a start point configuration reflecting ratio adjustments of a first factor (braking performance) and a second factor (steering performance). Reference numerals711,721,731, and741inFIGS.7A,7C,7E, and7Gillustrate image frames included in a driving route, reference numeral712inFIG.7Billustrates a feature point extraction result of an image frame of reference numeral711, reference numeral722illustrates a feature point extraction result of an image frame of reference numeral721, reference numeral732illustrates a feature point extraction result of an image frame of reference numeral731, and reference numeral742illustrates a feature point extraction result of an image frame of reference numeral741.

A scenario start point, which is a point in time when a factor of a vehicle is changed, is based on an image frame in which an object appears from among waypoints, and is based on an image frame from which sufficient feature points (e.g., preconfigured100feature points or more) are extracted. According toFIG.7A to7H, the preconfigured number of feature points or more are not extracted from the image frames of reference numerals711(712),721(722), and731(732), and thus no start point is configurable therefrom, and since the preconfigured number of feature points or more are extracted from the image frame of reference numeral741(742), a start point is configurable in this case.

In the present embodiment, the second generator153may acquire initial braking performances corresponding to driving speeds and initial steering performances corresponding to driving directions of a first vehicle and a second vehicle from a 3D point cloud map corresponding to the start point.

With respect to a first vehicle, the second generator153may simulate, on a driving route, N ratios for a first factor adjusted based on the initial braking performances and K ratios for a second factor adjusted based on the initial steering performances, so as to generate N×K simulation driving scenarios for the first vehicle.

With respect to a second vehicle, the second generator153may simulate, on the driving route, M ratios for the first factor adjusted based on the initial braking performances and J ratios for the second factor adjusted based on the initial steering performances, so as to generate M×J simulation driving scenarios for the second vehicle.

The computation unit154may compute a fault ratio weight by loading weights assigned to the ratio of the first factor and the ratio of the second factor, with respect to the first vehicle and second vehicle corresponding to each of the multiple simulation driving scenarios.

In order to compute the fault ratio weight, the computation unit154may load, from the database140, first factor weights corresponding to respective multiple (N and M) ratios for the first factor and second factor weights corresponding to respective multiple (K and J) ratios of the second factor.

FIG.8AandFIG.8Billustrate a waveform diagram for describing a weight for a first factor. Referring toFIG.8A, reference numeral810illustrates a graph of a first factor (braking performance and braking pressure) according to a pedal depth, and it may be seen that a braking pressure increases rapidly at a small pedal depth, and then increases linearly after about 25 mm. If this is applied to weights, a form as in reference numeral820may appear, which is not a linear weight. This non-linear weight for the first factor may be stored in the database140, and when a fault ratio weight is calculated, weights assigned to ratios (e.g., 20%, 40%, 60%, 80%, etc. compared to a maximum deceleration) of the first factor may be loaded and used.

FIG.9AtoFIG.9Cillustrate a waveform diagram for describing a weight for a second factor. Referring toFIG.9A, reference numeral910shows that a steering ratio becomes smaller when a steering wheel is turned about one full turn (360 degrees) compared to when the steering wheel is turned a little, and it is thus seen that a steering angle becomes larger even when the steering wheel is turned at the same angle. By applying this difference to weights, a non-linear weight as in reference numeral930may be used instead of using a linear weight as in reference numeral920. This non-linear weight for the second factor may be stored in the database140, and when a fault ratio weight is calculated, weights assigned to ratios (e.g., 20%, 40%, 60%, 80%, etc. compared to a maximum steering speed) of the second factor may be loaded and used.

With respect to a first vehicle, the computation unit154may, for respective simulation driving scenarios from a first simulation driving scenario to N×K simulation driving scenarios, compute and sum products of first factor weights and second factor weights so as to compute first vehicle fault ratio weights.

When computing the first vehicle fault ratio weights, the computation unit154may exclude products of first factor weights and second factor weights, which correspond to one or more simulation driving scenarios, in which a traffic accident occurs, from among N×K simulation driving scenarios. If a sum of configured weights is 1, and an accident occurs during a simulation, a corresponding product is excluded from the weights so that the sum of the weights is not 1.

With respect to a second vehicle, the computation unit154may, for respective simulation driving scenarios from a first simulation driving scenario to M×J simulation driving scenarios, compute and sum products of first factor weights and second factor weights so as to compute second vehicle fault ratio weights. When computing the second vehicle fault ratio weights, the computation unit154may exclude products of first factor weights and second factor weights, which correspond to one or more simulation driving scenarios in which a traffic accident occurs from among M×J simulation driving scenarios.

FIG.10illustrates multiple driving scenarios in which a first factor and a second factor are combined. For convenience of description, if it is assumed that a ratio of a first factor (braking performance) is adjusted to 20%, 40%, 60%, and 80% of a maximum deceleration, and a ratio of a second factor (steering performance) is adjusted to 20%, 40%, 60%, and 80% of a maximum steering speed, a total of 16 simulation driving scenarios may be generated. For respective 16 simulation driving scenarios, products of first factor weights and second factor weights may be computed and summed to generate second vehicle fault ratio weights. Here, products of first factor weights and second factor weights, which correspond to one or more simulation driving scenarios, in which a traffic accident occurs, may be excluded.

The third generator155may generate, in response to receiving selection of one or more addition/subtraction factors capable of changing a basic fault ratio, modified addition/subtraction scores by modifying addition/subtraction scores preconfigured for the addition/subtraction factors, on the basis of the fault ratio weight.

The third generator155may receive selection signals for one or more addition/subtraction factors capable of changing the basic fault ratio when generating the modified addition/subtraction scores. Here, an addition/subtraction factors is a parameter configured to change a fault ratio by selection according to fault degrees of a first vehicle and a second vehicle, and may be different for each traffic accident case classification result. For example, if a traffic accident case classification result is a lane change as shown in the above example, the addition/subtraction factor may include a significant fault, a gross fault, etc. of an accident vehicle (first vehicle, A) and may include non-compliance/delay of a course change signal, violation of bus/high-occupancy vehicle lane, a course change prohibited place, a significant fault, a gross fault, etc. of an assault vehicle (second vehicle, B).

The third generator155may determine whether one or more selected and received addition/subtraction factors include one or more of the first factor and the second factor. In the addition/subtraction factors, one or more addition/subtraction factors which are far from the first factor and the second factor exist, and one or more addition/subtraction factors which are not related to the first factor and the second factor may be excluded from generation of modified addition/subtraction scores.

According to a fact that the selected and received one or more addition/subtraction factors include no first and second factors, the third generator155may maintain addition/subtraction scores preconfigured for the addition/subtraction factors, and according to a fact that the selected and received one or more addition/subtraction factors include one or more of the first factor and the second factor, the third generator155may generate modified addition/subtraction scores by modifying the addition/subtraction scores preconfigured to the addition/subtraction factors on the basis of the fault ratio weight.

The calculator156may calculate a final fault ratio for the first vehicle and the second vehicle by applying the modified addition/subtraction scores to the basic fault ratio.

FIG.11is an exemplary diagram illustrating a final fault ratio calculation result. For example, if a weight sum value of a first vehicle is 0.3 and a weight sum value of a second vehicle is 0.8 according toFIG.11, and a basic fault ratio based on the traffic accident case classification result (e.g., lane change) according toFIG.5satisfies A (first vehicle):B (second vehicle)=30:70, since, in addition/subtraction factors, significant fault (addition/subtraction score of 10)×weight of 0.3 for A (first vehicle) is satisfied, and significant fault (addition/subtraction score of 10)×weight of 0.8 for B (second vehicle) is satisfied, a result of A (first vehicle):B (second vehicle)=25:75 may be obtained.

In the aforementioned example, a reason that the addition/subtraction score is 10 (significant fault) rather than 20 (gross fault) in “addition/subtraction score 10×weight of 0.3=3” is that gross fault falls under “drunk driving, drug driving, unlicensed driving, and speeding” rather than a driver's inexperienced driving, so that gross fault has been excluded.

FIG.12AandFIG.12Bshow graphs for comparison between a current addition/subtraction score calculation result and an addition/subtraction score calculation result reflecting weights according to the present embodiment. Referring toFIG.12A, it is not uncommon for addition/subtraction score calculation results according to a discrete scheme as shown in reference numeral1210to receive the same addition/subtraction score even though fault degrees are different. However, as shown in reference numeral1220inFIG.12B, when addition/subtraction score calculation reflecting weights is used, a continuous type of addition/subtraction score may be acquired unlike an existing discrete addition/subtraction score. In this case, a fault degree and an operation difficulty in a simulation driving scenario may be inversely proportional. For example, if an accident is avoided in a scenario having a low operation difficulty, in which only 20% of a maximum braking force is used, this may indicate a high degree of fault.

An existing scheme evaluates, with an addition/subtraction score of 0 points (mistake X) or an addition/subtraction score of 10 points (mistake O), whether a driver has made a mistake in an accident situation, based on a basic fault score (e.g., A30:B70) according to an accident type, and the present embodiment is to subdivide degrees of a driver's mistakes into addition/subtraction scores 0 to 10 points (weights 0 to 1×10 (significant fault addition/subtraction scores)).

FIG.13is a block diagram provided to schematically illustrate a configuration of the fault ratio calculation device according to another embodiment. In the following description, descriptions of parts overlapping with those ofFIG.1toFIG.12will be omitted. Referring toFIG.13, a fault ratio calculation device according to another embodiment may include a processor170and a memory180.

In the present embodiment, the processor170may process functions performed by the communication unit110, storage medium120, program storage unit130, database140, information processing unit150, and controller160ofFIG.2and the first generator151, acquisition unit152, second generator153, computation unit154, third generator155, and calculator156ofFIG.3.

In the present embodiment, the processor170may generate a fault-related information prediction result by using artificial intelligence.

Here, the artificial intelligence (AI), which is an area of computer engineering and information technology for studying methods for enabling computers to mimic thinking, learning, self-development, or the like that can be carried out with human intelligence, may represent enabling computers to mimic human intelligent behavior.

In addition, artificial intelligence does not exist on its own, but is rather directly or indirectly related to a number of other fields in computer science. In particular, in recent years, there have been numerous attempts to introduce an element of AI into various fields of information technology to solve problems in the respective fields.

Machine learning is an area of artificial intelligence that may include the field of study that gives computers the capability to learn without being explicitly programmed. Specifically, machine learning is a technology that studies and builds systems capable of learning, making predictions, and enhancing its own performance on the basis of experiential data, and algorithms for such systems. Algorithms in machine learning may take an approach of building a specific model to make predictions or decisions based on input data, rather than executing strictly set static program instructions.

As the machine learning method of an artificial neural network, both unsupervised learning and supervised learning may be used. A deep learning technique, which is a type of machine learning, enables data-based learning through multiple layers. As the number of layers in deep learning increases, the deep learning network may acquire a collection of machine learning algorithms that extract core data from multiple datasets.

The processor170may control the overall operation of the fault ratio calculation device100. Here, the “processor” may refer to, for example, a data processing device embedded in hardware, the device having a physically structured circuit to perform a function expressed as a code or an instruction included in a program. As an example, the data processing device embedded in hardware may encompass processing devices, such as a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA), but the scope of the present disclosure is not limited thereto.

The memory180may be operatively connected to the processor170and store at least one code in association with an operation performed by the processor170.

The memory180may perform a function of temporarily or permanently storing data processed by the processor170, and may store, for example, data stored in the database140ofFIG.1. Here, the memory180may include a magnetic storage medium or a flash storage medium, but the scope of the present disclosure is not limited thereto. The memory180may include an internal memory and/or an external memory, and may include a volatile memory, such as a DRAM, an SRAM or an SDRAM, a non-volatile memory, such as a one-time programmable ROM (OTPROM), a PROM, an EPROM, an EEPROM, a mask ROM, a flash ROM, a NAND flash memory or a NOR flash memory, a flash drive, such as an SSD, a compact flash (CF) card, an SD card, a Micro-SD card, a Mini-SD card, an XD card, or memory stick, or a storage device such as an HDD.

FIG.14is a flowchart for illustrating a method of calculating a fault ratio based on a 3D simulator according to the present embodiment. In the following description, descriptions of parts overlapping with those ofFIG.1toFIG.12will be omitted.

Referring toFIG.14, in operation S1410, the fault rate calculation device100may generate a 3D point cloud map by using feature points extracted from image frames collected from a mono camera of a first vehicle as an accident vehicle and a depth estimation result acquired via comparison between two adjacent image frames from among the image frames, and may generate, based on the 3D point cloud map, a driving route in which the first vehicle travels.

In the present embodiment, when generating the driving route, the fault ratio calculation device100may generate, as the driving route, a set of waypoints of the first vehicle corresponding to image frames extracted at a preconfigured time period.

In operation S1420, the fault ratio calculation device100may acquire, using the driving route and the image frames, a classification result of a traffic accident case and fault-related information including a basic fault ratio of the first vehicle and a second vehicle as an assault vehicle, wherein the basic fault ratio corresponds to the classification result of the traffic accident case.

In the present embodiment, when the fault-related information is acquired, the fault ratio calculation device100may generate a fault-related information prediction result corresponding to the driving route and the image frames by using a deep neural network model trained in advance to predict fault-related information by using the driving route and the image frames. Here, the deep neural network model may be a model trained according to a supervised learning scheme on the basis of training data in which the image frames collected from the mono camera of the first vehicle and the driving route of the first vehicle are received as inputs, and the traffic accident case and the basic fault ratio of the first vehicle and the second vehicle are configured as labels.

In operation S1430, the fault ratio calculation device100may generate multiple simulation driving scenarios by simulating the driving route with the first vehicle and the second vehicle, in which ratio adjustments of a first factor representing braking performance and a second factor representing steering performance are applied.

In the present embodiment, the fault ratio calculation device100may apply dynamic characteristics of the first vehicle and the second vehicle when simulating the driving route. The fault ratio calculation device100may configure, on the driving route, a start point that reflects the ratio adjustments of the first factor and the second factor. Here, the start point may include an image frame, in which an object appears and the number of feature points is equal to or greater than a preconfigured number, from among image frames included in the driving route. The fault ratio calculation device100may acquire initial braking performances corresponding to driving speeds of the first vehicle and the second vehicle and initial steering performances corresponding to driving directions of the first vehicle and the second vehicle from the 3D point cloud map corresponding to the start point. With respect to the first vehicle, the fault ratio calculation device100may simulate, on the driving route, N ratios for the first factor adjusted based on the initial braking performances and K ratios for the second factor adjusted based on the initial steering performances, so as to generate N×K simulation driving scenarios for the first vehicle. With respect to the second vehicle, the fault ratio calculation device100may simulate, on the driving route, M ratios for the first factor adjusted based on the initial braking performances and J ratios for the second factor adjusted based on the initial steering performances, so as to generate M×J simulation driving scenarios for the first vehicle.

In operation S1440, with respect to the first vehicle and the second vehicle, which correspond to each of the multiple simulation driving scenarios, the fault ratio calculation device100may compute a fault ratio weight by loading weights assigned to a ratio of the first factor and a ratio of the second factor.

In the present embodiment, when calculating the fault ratio weight, the fault ratio calculation device100may load, from a database, first factor weights corresponding to respective multiple (N and M) ratios for the first factor and second factor weights corresponding to respective multiple (K and J) ratios of the second factor. With respect to the first vehicle, the fault ratio calculation device100may, for respective simulation driving scenarios from a first simulation driving scenario to N×K simulation driving scenarios, compute and sum products of first factor weights and second factor weights so as to compute first vehicle fault ratio weights. When computing the first vehicle fault ratio weights, the fault ratio calculation device100may exclude products of first factor weights and second factor weights, which correspond to one or more simulation driving scenarios, in which a traffic accident occurs, from among the N×K simulation driving scenarios. With respect to the second vehicle, the fault ratio calculation device100may, for respective simulation driving scenarios from a first simulation driving scenario to M×J simulation driving scenarios, compute and sum products of first factor weights and second factor weights so as to compute second vehicle fault ratio weights. When computing the second vehicle fault ratio weights, the fault ratio calculation device100may exclude products of first factor weights and second factor weights, which correspond to one or more simulation driving scenarios in which a traffic accident occurs from among the M×J simulation driving scenarios.

In operation S1450, the fault ratio calculation device100may generate, in response to receiving selection of one or more addition/subtraction factors capable of changing the basic fault ratio, modified addition/subtraction scores by modifying addition/subtraction scores preconfigured for the addition/subtraction factors, on the basis of the fault ratio weight.

In the present embodiment, the fault ratio calculation device100may receive selection signals for one or more addition/subtraction factors capable of changing the basic fault ratio when generating the modified addition/subtraction scores. The fault ratio calculation device100may determine whether one or more addition/subtraction factors include one or more of the first factor and the second factor. The fault ratio calculation device100may maintain the addition/subtraction scores preconfigured for the addition/subtraction factors, according to a fact that the one or more addition/subtraction factors do not include the first factor and the second factor. The fault ratio calculation device100may generate modified addition/subtraction scores by modifying the addition/subtraction scores preconfigured to the addition/subtraction factors on the basis of the fault ratio weight, according to a fact that the one or more addition/subtraction factors include one or more of the first factor and the second factor.

In operation S1460, the fault ratio calculation device100may calculate a final fault ratio for the first vehicle and the second vehicle by applying the modified addition/subtraction scores to the basic fault ratio.

The embodiments of the present disclosure as described above may be implemented in the form of a computer program which can be executed through various elements on a computer, and the computer program may be recorded in a computer-readable medium. Examples of the computer-readable medium may include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as a CD-ROM and a DVD, magneto-optical media such as floptical disks, and hardware devices such as a ROM, a RAM, and a flash memory, which are specially configured to store and execute program instructions.

Meanwhile, the computer program may be specifically designed for the present disclosure, and may be known to and used by those skilled in computer software fields. Examples of the computer program may include machine language code generated by a compiler and high-level language code executable by a computer through an interpreter or the like.

In the specification (particularly, in the claims) of the present disclosure, the term “the” and the indication term similar thereto may correspond to both the singular and the plural. When the present disclosure includes a range, the present disclosure includes an disclosure to which an individual value belonging to the range is applied (unless there is description against it), which means that the detailed description of the present disclosure includes the individual value within the range.

Unless there is clear description of the order of steps included in the method according to the present disclosure or unless indicated otherwise, the steps can be conducted in appropriate order. The present disclosure is not necessarily limited to the order of the steps described therein. All examples or example terms (for example, “etc.”) may be simply used to describe the present disclosure in detail but do not limit the scope of the present disclosure unless the scope of the present disclosure is limited by the claims. Further, those skilled in the art can identify that various modifications, combinations, and changes can be configured according to design conditions and factors within the range of appended claims and equivalents thereof.

Accordingly, the spirit and scope of the present disclosure should not be limited or determined by the above-described embodiments, and it should be noted that not only the claims which will be described below but also their equivalents fall within the spirit and scope of the present disclosure.