Patent Description:
In conventional a damage assessment technique, a vehicle insurance company may send a professional claim adjuster to the location of a damaged vehicle to conduct a manual survey and damage assessment. The survey and damage assessment conducted by the adjuster can include determining a repair solution, estimating an indemnity, taking photographs of the vehicle, and archiving the photographs for subsequent assessment of the damage by a damage inspector at the vehicle insurance company. Since the survey and subsequent damage assessment are performed manually, an insurance claim may require a significant number of days to resolve. Such delays in the processing time can lead to poor user experience with the vehicle insurance company. Furthermore, the manual assessments may also incur a large cost (e.g., labor, training, licensing, etc.).

To address this issue, some vehicle insurance companies use image-based artificial intelligence (AI) models (e.g., machine-learning-based techniques) for assessing vehicle damages. Since the AI models may automatically detect the damages on a vehicle based on images, the automated assessment technique can shorten the wait time and reduce labor costs. For example, an AI-based assessment technique can be used for automatic identification of the damage of the vehicle (e.g., the parts of the vehicle). Typically, a user can capture a set of images of the vehicle depicting the damages from the user's location, such as the user's home or work, and send the images to the insurance company (e.g., using an app or a web interface). These images can be used by an AI model to identify the damage on the vehicle. In this way, the automated assessment process may reduce the labor costs for a vehicle insurance company and improve user experience associated the claim processing.

Even though automation has brought many desirable features to a damage assessment system, many problems remain unsolved in universal damage detection (e.g., independent of the damaged parts).

<CIT> in an abstract states that "Disclosed is an apparatus for pedestrian detection. The system comprises: a first box generator for generating candidate boxes from a plurality of pedestrian training images; a training patch generator for generating training part patches from the candidate boxes generated by the first box generator and ground truth boxes; a detector training unit for training part detectors from the training part patches; a detector selecting unit for selecting complementary part detectors from all the trained part detectors; a second box generator for generating candidate boxes from a plurality of pedestrian testing images; a testing patch generator for generating testing part patches from the candidate boxes generated by the second box generator; and a testing unit for generating a detection result from the testing part patches and the selected part detectors. A method and a system for pedestrian detection are also disclosed.

<NPL>, states that "Object localization is an important task in computer vision but requires a large amount of computational power due mainly to an exhaustive multiscale search on the input image. In this paper, we describe a near real-time multiscale search on a deep CNN feature map that does not use region proposals. The proposed approach effectively exploits local semantic information preserved in the feature map of the outermost convolutional layer. A multi-scale search is performed on the feature map by processing all the sub-regions of different sizes using separate expert units of fully connected layers. Each expert unit receives as input local semantic features only from the corresponding sub-regions of a specific geometric shape. Therefore, it contains more nearly optimal parameters tailored to the corresponding shape. This multi-scale and multi-aspect ratio scanning strategy can effectively localize a potential object of an arbitrary size. The proposed approach is fast and able to localize objects of interest with a frame rate of <NUM> fps while providing improved detection performance over the state-of-the art on the PASCAL VOC <NUM> and MSCOCO data sets.

<CIT> in an abstract states that "The invention discloses a method and device for identifying a vehicle damaged components, a server, a client, and a system. The method comprises pre-establishing an identifiable vehicle feature library comprising a plurality of vehicle components and a feature corresponding relation library of the relative position relations of the vehicle components. In a loss assessment image shooting process, auser can manually enclose a damaged position at the client. The server can identify the identifiable vehicle features in the image, and the relative position of the mark enclosed by the user is determined according to the identifiable features. The relative position is further matched in the feature relation library, and the damaged component is determined, the auxiliary positioning of the damaged position can be realized through the manual and simple operation of the user on site, the insurance company can be assisted to position the vehicle damage component matching, and the accuracy and the processing efficiency of the damaged component in the loss assessment are improved.

<CIT> in an abstract states that "An artificial neural network for learning to track a target across a sequence of frames includes a representation network configured to extract a target region representation from a first frame and a search region representation from a subsequent frame. The artificial neural network also includes a cross-correlation layer configured to convolve the extracted target region representation with the extracted search region representation to determine a cross-correlation map. The artificial neural network further includes a loss layer configured to compare the cross-correlation map with a ground truth cross-correlation map to determine a loss value and to back propagate the loss value into the artificial neural network to update filter weights of the artificial neural network.

<NPL>, states that "Pedestrian detection is an essential step in many important applications of Computer Vision. Most detectors require manually annotated ground-truth to train, the collection of which is labor intensive and time-consuming. Generally, this training data is from representative views of pedestrians captured from a variety of scenes. Unsurprisingly, the performance of a detector on a new scene can be improved by tailoring the detector to the specific viewpoint, background and imaging conditions of the scene. Unfortunately, for many applications it is not practical to acquire this scene-specific training data by hand. In this paper, we propose a novel algorithm to automatically adapt and tune a generic pedestrian detector to specific scenes which may possess different data distributions than the original dataset from which the detector was trained. Most state-of-the-art approaches can be inefficient, require manually set number of iterations to converge and some form of human intervention. Our algorithm is a step towards overcoming these problems and although simple to implement, our algorithm exceeds state-of-the-art performance.

<CIT> in an abstract states that "A method, non-transitory computer readable medium, and an image analysis computing device (<NUM>) that retrieves (<NUM>), based on a captured version of an object in a received image, training images which display related versions of the object and items of data related to the related versions of the object of the training images. Keypoints which are invariant to changes in scale and rotation in the captured version of the object in the received image and in the related versions of the object in the training images are determined (<NUM>). Changes to the object in the received image based on any of the determined keypoints in the related version of the object which do not match the determined keypoints in the captured version of the object are identified (<NUM>). The identified changes in the captured version of the object in the received image are provided (<NUM>).

Aspects of the present disclosure are set out in the independent claim(s). Other aspects and features of the present disclosure are set out in the claims and the description below.

Embodiments described herein provide a system for facilitating image sampling for training a target detector. During operation, the system obtains a first image depicting a first target. Here, the continuous part of the first target in the first image is labeled and enclosed in a target bounding box. The system then generates a set of positive image samples from an area of the first image enclosed by the target bounding box. A respective positive image sample includes at least a part of the first target. The system can train the target detector with the set of positive image samples to detect a second target from a second image. The target detector can be an artificial intelligence (AI) model capable of detecting an object.

In a variation on this embodiment, the first and second targets indicate a first and second vehicular damages, respectively. The label of the continuous part indicate a material impacted by the first vehicular damage.

In a further variation, the system detects the second target by detecting the second vehicular damage based on a corresponding material independent of identifying a part of a vehicle impacted by the second vehicular damage.

In a variation on this embodiment, the system generates the set of positive image samples by determining a region proposal in the area of the first image enclosed by the target bounding box and selecting the region proposal as a positive sample if an overlapping parameter of the region proposal is in a threshold range.

In a further variation, the overlapping parameter is a ratio of an overlapping region and a surrounding region of the region proposal. The overlapping region indicates a common region covered by both the region proposal and a set of internal bounding boxes within the target bounding box. A respective internal bounding box can include at least a part of the continuous region. The surrounding region indicates a total region covered by the region proposal and the set of internal bounding boxes.

In a further variation, the system selects the set of internal bounding boxes based on one of an intersection with the region proposal, a distance from the region proposal, and a total number of internal bounding boxes in the target bounding box.

In a further variation, the system generates a negative sample, which excludes any part of the first target, from the first image. To do so, the system can select the region proposal as the negative sample in response to determining that the overlapping parameter of the region proposal is in a low threshold range. The system may also select an area outside of the target bounding box in the first image as the negative sample.

In a further variation, the system determines a set of subsequent region proposals in the area of the first image enclosed by the target bounding box. To do so, the system can apply a movement rule to a previous region proposal and terminate based on a termination condition.

In a variation on this embodiment, the system generates a second set of positive image samples. To do so, the system can select a positive image sample from a region proposal in a second target bounding box in the first image. The system may also change the size or shape of a bounding box of a region proposal of a previous round.

In a variation on this embodiment, the system optimizes the training of the target detector by generating a plurality of bounding boxes for a plurality of image samples in the set of positive image samples and combining the plurality of bounding boxes to generate a combined bounding box and a corresponding label. Here, a respective bounding box identifies the corresponding part of the continuous region.

In the figures, like reference numerals refer to the same figure elements.

The following description is presented to enable any person skilled in the art to make and use the embodiments, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Thus, the embodiments described herein are not limited to the embodiments shown, but are to be accorded the widest scope consistent with the principles and features disclosed herein.

The embodiments described herein solve the problem of efficiently detecting a damage of a vehicle by (i) generating positive and negative image samples from a labeled image for training a target detection system; and (ii) integrating detection results of multiple image samples associated with a damage to increase the efficiency of the target detection system. In this way, an assessment system can use the target detection system to identify damages on a vehicle independent of the damaged parts and generate a repair plan based on the identification.

With existing technologies, an AI-based technique for determining vehicular damages from an image may include determining the damaged parts of a vehicle and the degree of the damages based on the similar images in historical image data. Another technique may involve identifying the area of a damaged part in the center of an input image through an identification method and comparing the area of the part with the historical image data to obtain a similar image. By comparing the obtained image with the input image, the technique may determine the degree of damages. However, these techniques are prone to interference from the additional information of the damaged part in the input image, a reflection of light, contaminants, etc. As a result, these techniques may operate with low accuracy while determining the degree of damages.

For example, for identifying damages on a vehicle using target detection, the technique typically needs to be trained with a certain number of positive samples and negative samples. Here, a certain number of images depicting the damages need to serve as the positive samples, and a certain number of images not depicting the damages need to serve as the negative samples. However, obtaining positive samples in sufficient numbers can be challenging. Furthermore, a negative sample may include at least a segment of a damaged part and cause interference in the training process. As a result, the AI-based model may not be equipped to detect damages on a part of a vehicle, especially if the model has not been trained with similar damages on the part of the vehicle.

To solve this problem, embodiments described herein provide an assessment system that can identify a damaged area of a vehicle (i.e., a target) from one or more images of the vehicle and assess the degree of damage on the identified damaged area. The system can assess damage to a vehicle in two dimensions. The system can identify a part of the vehicle based on object detection in one dimension and determine the damage in another dimension. To determine the damage, the system can identify the damaged area based on the material on which the damage has been inflicted. Hence, the system can execute the damage detection independent of the underlying vehicle part. This allows the system to efficiently detect a damaged area on a vehicle without relying on how that damaged area may appear on a specific part of the vehicle.

To do so, the system can identify damages and the degree of damages on materials, such as the paint surface, plastic components, frosted components, glasses, lights, mirrors, etc., without requiring information of the underlying parts. As a result, the system can also be used for the identification of damages on similar materials in other scenarios (i.e., other than the damages on a vehicle). On the other hand, the system can independently identify one or more parts that may represent the damaged area. In this way, the system can identify the damaged area and the degree of damages, and the parts that construct the damaged area. Based on the damage information, the system then performs a damage assessment, determines a repair plan, and generates a cost estimate. For example, the system can estimate the cost and/or availability of the parts, determine whether a repair or replacement is needed based the degree of damage, determine the deductibles and fees, and schedule a repair operation based on calendar information of a repair shop.

However, the target detector (e.g., a deep-learning network) can operate with high accuracy if the target detector is trained with sufficient number of positive and negative samples. In some embodiments, the system can also generate image samples from labeled images (e.g., images with labeled targets). A labeled image may at least include a target bounding box that can be hand-labeled in advance and a plurality of internal bounding boxes in the target bounding box. The target bounding box is used for surrounding a continuous region of a target (e.g., the largest continuous region of damage), and each of the plurality of internal bounding boxes surrounds a segment of the continuous region of the target.

During operation, the system can obtain the labeled images and determine region proposals for sampling in the target bounding box. The region proposal can be represented based on a pre-determined bounding box (e.g., with predetermined size and shape). This bounding box can be placed in the target bounding box based on a sliding window or an image segmentation algorithm. The system then compares the region proposal with the corresponding internal bounding boxes to determine overlapping parameters.

Based on whether the overlapping parameters are in a threshold range, the system may collect the region proposal as a positive sample for training the target detector. Otherwise, if the overlapping parameters are below a low threshold range, the system may collect the region proposal as a negative sample. In addition, the system can also collect negative samples from outside of the target bounding box to ensure that the negative sample does not include any damage information. In this way, the system can reduce interference and improve the accuracy of the target detector.

<FIG> illustrates exemplary infrastructure and environment facilitating an efficient assessment system, in accordance with an embodiment of the present application. In this example, an infrastructure <NUM> can include an automated assessment environment <NUM>. Environment <NUM> can facilitate automated damage assessment in a distributed environment. Environment <NUM> can serve a client device <NUM> using an assessment server <NUM>. Server <NUM> can communicate with client device <NUM> via a network <NUM> (e.g., a local or a wide area network, such as the Internet). Server <NUM> can include components such as a number of central processing unit (CPU) cores, a system memory (e.g., a dual in-line memory module), a network interface card (NIC), and a number of storage devices/disks. Server <NUM> can run a database system (e.g., a database management system (DBMS)) for maintaining database instances.

Suppose that a user <NUM> needs to file an insurance claim regarding damage <NUM> on a vehicle. If the insurance company deploys an AI-based technique for automatically determining damages from an image, user <NUM> may use client device <NUM> to capture an image <NUM> depicting damage <NUM>. User <NUM> can then send an insurance claim <NUM> comprising image <NUM> as an input image from client device <NUM> via network <NUM>. With existing technologies, the AI-based technique may determine the parts damaged by damage <NUM> and the degree of damage <NUM> based on the similar images in historical image data. Another technique may involve identifying the area of damage <NUM> in the center of input image <NUM> through an identification method and comparing the area of damage <NUM> with the historical image data to obtain a similar image. By comparing the obtained image with input image <NUM>, the technique may determine the degree of damages.

However, these techniques are prone to interference from the additional information in input image <NUM>, such as undamaged segments, a reflection of light, contaminants, etc. As a result, these techniques may operate with low accuracy while determining the degree of damages. Furthermore, these techniques typically need to be trained with a certain number of positive samples and negative samples. However, obtaining positive samples in sufficient numbers can be challenging. Furthermore, a negative sample may include interfering elements. As a result, the AI-based technique may not be equipped to detect damage <NUM>, especially if the technique has not been trained with damages similar to damage <NUM>.

To solve these problems, an automated assessment system <NUM> can efficiently and accurately identify the area and vehicle parts impacted by damage <NUM> (i.e., one or more targets) from image <NUM>, and assess the degree of damage <NUM>. System <NUM> can run on server <NUM> and communicate with client device <NUM> via network <NUM>. In some embodiments, system <NUM> includes a target detector <NUM> that can assess damage <NUM> in two dimensions. Target detector <NUM> can identify a part of the vehicle impacted by damage <NUM> in one dimension and determine damage <NUM> in another dimension. Upon determining the damage, target detector <NUM> can apply geometric calculation and division to determine the degree of damage <NUM> as target <NUM> that can include the location of damage <NUM>, the parts impacted by damage <NUM>, and the degree of damage <NUM>.

Furthermore, target detector <NUM> can identify the area or location of damage <NUM> based on the material on which the damage has been inflicted. As a result, target detector <NUM> can execute the damage detection independent of the underlying vehicle part. This allows target detector <NUM> to efficiently detect the area or location of damage <NUM> without relying on how that damaged area may appear on a specific part of the vehicle. In other words, target detector <NUM> can identify damage <NUM> and the degree of damage <NUM> on the material on which damage <NUM> appears without requiring information of the underlying parts. In addition, target detector <NUM> can independently identify one or more parts that may be impacted by damage <NUM>. In this way, target detector <NUM> can identify the area and the degree of damage <NUM>, and the parts impacted by damage <NUM>.

Based on the damage information generated by target detector <NUM>, system <NUM> then generates a damage assessment <NUM> to determine a repair plan and generate a cost estimate for user <NUM>. System <NUM> can estimate the cost and/or availability of the parts impacted by damage <NUM>, determine whether a repair or replacement is needed based the degree of damage <NUM>, and schedule a repair operation based on calendar information of a repair shop. System <NUM> can then send assessment <NUM> to client device <NUM> via network <NUM>.

Examples of target detector <NUM> include, but are not limited to, Faster Region-Convolutional Neural Network (R-CNN), You Only Look Once (YOLO), Single Shot MultiBox Detector (SSD), R-CNN, Lighthead R-CNN, and RetinaNet. In some embodiments, target detector <NUM> can reside on client device <NUM>. Target detector <NUM> can then use a mobile end target detection technique, such as MobileNet+SSD.

<FIG> illustrates exemplary training and operation of an efficient assessment system, in accordance with an embodiment of the present application. Target detector <NUM> can operate with high accuracy if target detector <NUM> is trained with sufficient number of positive and negative samples. In some embodiments, system <NUM> can also generate image samples <NUM> from a labeled image <NUM>, which can be an image with labeled targets. It should be noted that the image sampling and target detection can be executed on the same or different devices. Labeled image <NUM> may at least include a target bounding box <NUM> that can be hand-labeled in advance and a plurality of internal bounding boxes <NUM> and <NUM> in target bounding box <NUM>. Target bounding box <NUM> is used for surrounding a continuous region of a target (e.g., the largest continuous region of damage), and each of internal bounding boxes <NUM> and <NUM> surrounds a segment of the continuous region of the target.

The labeling of on image <NUM> can indicate a damage definition, which can include the area and the class, associated with a respective continuous damage segment depicted in image <NUM>. The various degrees of damages corresponding to the various materials are defined as the damage classes. For example, if the material is glass, the damage class can include minor scratches, major scratches, glass cracks, etc. The smallest area that includes a continuous segment of the damage is defined as the area of the damage segment. Therefore, for each continuous segment of the damage in image <NUM>, the labeling can indicate the area and the class of the damage segment. Upon determining the damage definition of a respective damage segment, the labeling can indicate the damaged definition of the corresponding damage segment. With such labeling of a damage segment, the damage becomes related only to the material and not to a specific part.

During operation, system <NUM> can obtain labeled image <NUM> and determine region proposals for sampling in target bounding box <NUM>. The region proposal can be represented based on a pre-determined bounding box (e.g., with predetermined size and shape). The bounding box of the region proposal can be placed in target bounding box <NUM> based on a sliding window or an image segmentation algorithm. System <NUM> then compares the region proposal with the corresponding internal bounding boxes to determine overlapping parameters (e.g., an intersection over union (IoU)). System <NUM> may only compare the region proposal with the internal bounding boxes that are within a distance threshold of the region proposal (e.g., <NUM> pixels) or have an intersection with the region proposal.

Based on whether the overlapping parameters are in a threshold range (e.g., greater than <NUM> or falls within <NUM>-<NUM>), system <NUM> may collect the region proposal as a positive sample for training target detector <NUM>. Otherwise, if the overlapping parameters are below a low threshold range (e.g., less than <NUM>), system <NUM> may collect the region proposal as a negative sample. In addition, system <NUM> can also collect negative samples from outside of target bounding box <NUM> to ensure that the negative sample does not include any damage information. In this way, system <NUM> can generate image samples <NUM> that can include accurate positive and negative samples. By training object detector <NUM> using image samples <NUM>, system <NUM> can reduce the interference and improve the accuracy of target detector <NUM>, thereby allowing target detector <NUM> to accurately detect target <NUM> from input image <NUM>.

<FIG> illustrates exemplary bounding boxes for generating image samples for training a target detection system of an efficient assessment system, in accordance with an embodiment of the present application. During operation, system <NUM> can receive an input image <NUM> for image sampling. Image <NUM> may depict a vehicular damage <NUM> (i.e., damage on a vehicle). A user <NUM> may label image <NUM> with one or more bounding boxes. Each of the bounding boxes can correspond to a label that indicates a damage definition (i.e., the area of the damage and the class of damage). The bounding boxes include at least one target bounding box and may include a set of internal bounding boxes located in the target bounding box.

User <NUM> may determine the largest continuous region of damage <NUM> and apply a target bounding box <NUM> on the largest continuous region. Here, a target bounding box surrounds a continuous region of damage. User <NUM> may start from the largest continuous region for determining a target bounding box, and continue with the next largest continuous region for a subsequent target bounding box in the same image. User <NUM> can then select a part of the continuous region in bounding box <NUM> with an internal bounding box <NUM>. In the same way, user <NUM> can select internal bounding boxes <NUM> and <NUM> in bounding box <NUM>.

It should be noted that, even though a bounding box typically takes a square or rectangular shape, the bounding box may take any other shape, such a triangular or oval shape. In this example, shapes and sizes of internal bounding boxes <NUM>, <NUM>, and <NUM> may take the same or different forms. Furthermore, two adjacent bounding boxes may or may not be joined and/or overlapping. In addition, the internal bounding boxes within target bounding box <NUM> may or may not cover the continuous region of damage in its entirety.

During operation, system <NUM> can then determine region proposals for sampling in target bounding box <NUM>. The region proposal can be placed in target bounding box <NUM> based on a sliding window or an image segmentation algorithm. For collecting a positive sample, system <NUM> may determine a region proposal <NUM> in a region covered by a portion of damage <NUM>. System <NUM> compares region proposal <NUM> with corresponding internal bounding boxes <NUM> and <NUM> to determine overlapping parameters. Based on whether the overlapping parameters are in a threshold range, system <NUM> may collect region proposal <NUM> as a positive sample.

Otherwise, if the overlapping parameters are below a low threshold range, system <NUM> may collect region proposal <NUM> as a negative sample. In addition, system <NUM> can also determine a region proposal <NUM> in a region of target bounding box <NUM> that may not include damage <NUM> (e.g., using segmentation). System <NUM> can also determine a region proposal <NUM> outside of target bounding box <NUM> to collect a negative sample. In this way, system <NUM> can use region proposals <NUM> and <NUM> for negative samples, thereby ensuring that the corresponding negative samples do not include any damage information.

<FIG> illustrates an exemplary region proposal generation process for generating image samples, in accordance with an embodiment of the present application. System <NUM> may determine a set of movement rules that determines the placement of a region proposal in target bounding box <NUM>. System <NUM> can also receive the movement rules as input. The movement rules can be defined so that, from a current region proposal, a subsequent region proposal can be determined within the region enclosed by target bounding box <NUM>. Such rules can include an initial position of a region proposal (i.e., the position of a bounding box corresponding to the region proposal), the deviation distance from a previous position for a movement, and a movement direction, and a movement termination condition. The movement termination condition can be based on one or more of: a number of region proposals and/or movements in a target bounding box and the region covered by the region proposals (e.g., a threshold region).

Based on the movement rules, system <NUM> can determine a number of region proposals in target bounding box <NUM>. In some embodiments, the upper left corner of target bounding box <NUM> is selected as the position for of the initial region proposal <NUM>. The next region proposal <NUM> can be selected based on a movement from left to right along the left-to-right width of target bounding box <NUM>. A predetermined step length can dictate how far region proposal <NUM> should be from region proposal <NUM>. In this way, a sample can be generated for each movement.

In some further embodiments, the position of a region proposal in target bounding box <NUM> can be randomly selected. To do so, system <NUM> can randomly determine a reference point of region proposal <NUM> (e.g., a center or corner point of region proposal <NUM>). The position of the reference point can be selected based on a movement range of the reference point (e.g., a certain distance between the reference point and the boundary of target bounding box <NUM> should be maintained). System <NUM> can then place region proposal <NUM> in target bounding box <NUM> based on a predetermined size of a region proposal with respect to the reference point.

<FIG> illustrates an exemplary assessment of a region proposal for generating image samples, in accordance with an embodiment of the present application. Suppose that system <NUM> has determined a region proposal <NUM> in target bounding box <NUM>. To assess region proposal <NUM>, system <NUM> determines the overlapping parameters that indicate the degree and/or proportion of overlap between region proposal <NUM> and the region enclosed by a respective internal bounding box in target bounding box <NUM>. Since parts of the continuous region of damage <NUM> are represented by the internal bounding boxes, a high degree of overlap between region proposal <NUM> and the internal bounding boxes indicates that region proposal <NUM> includes a significant portion of damage <NUM>. Based on this assessment, system <NUM> can select region proposal <NUM> as a positive sample.

When system <NUM> performs image sampling in the region enclosed by target bounding box <NUM>, system <NUM> may compare region proposal <NUM> with a respective internal bounding box of target bounding box <NUM>. This comparison can be executed based on an arrangement order of the internal bounding boxes, or based on the distance to region proposal <NUM> (e.g., from near to far). In some embodiments, system <NUM> may compare region proposal <NUM> only with the internal bounding boxes in the vicinity of region proposal <NUM>. For example, system <NUM> can compare region proposal <NUM> only with the internal bounding boxes that are within a predetermined threshold distance (e.g., within a <NUM>-pixel distance) or, have an intersection with region proposal <NUM> (e.g., internal bounding boxes <NUM>, <NUM>, and <NUM>). In this way, system <NUM> can significantly reduce the volume of data processing.

System <NUM> can determine whether region proposal <NUM> can be an image sample based on the overlapping parameters of region proposal <NUM> and a surrounding region. The surrounding region includes the total region enclosed by region proposal <NUM> and an internal bounding box that has been compared with region proposal <NUM>. For example, if system <NUM> has compared region proposal <NUM> with internal bounding boxes <NUM>, the surrounding region for region proposal <NUM> can be the region enclosed by internal bounding box <NUM> and region proposal <NUM>. System <NUM> can then determine the overlapping parameters of region proposal <NUM> with respect to the internal region, and determine whether region proposal <NUM> can be an image sample. The overlapping parameters can indicate whether there is an overlapping, the overlapping degree, and the overlapping proportion.

<FIG> illustrates an exemplary determination of whether a region proposal can be an image sample, in accordance with an embodiment of the present application. In this example, system <NUM> determines whether a region proposal <NUM> can be selected as an image sample. System <NUM> can determine the surrounding region <NUM> (denoted with a gray grid) covered by internal bounding box <NUM> and region proposal <NUM>. System <NUM> then determines the overlapping region <NUM> (denoted with a dark line) between region proposal <NUM> and internal bounding box <NUM>. System <NUM> can then determine the overlapping parameters for internal bounding box <NUM> and region proposal <NUM> as a ratio of overlapping region <NUM> and surrounding region <NUM>. In some embodiments, system <NUM> may determine the overlapping parameters for internal bounding box <NUM> and region proposal <NUM> as a ratio of overlapping region <NUM> and the region enclosed by internal bounding box <NUM>.

In some embodiments, the ratio is determined based on an intersection over union (IoU) of the regions. If the regions are represented based on pixels, the ratio can be determined as the ratio of corresponding pixels. In the example in <FIG>, if one grid represents one pixel, the ratio can be calculated as pixels in overlapping region <NUM>/ pixels in surrounding region <NUM>=<NUM>/<NUM>. If the ratio is larger than a predetermined threshold (e.g., <NUM>) or falls within a threshold range (e.g., <NUM>-<NUM>), system <NUM> can select the region proposal as a positive sample. If system <NUM> compares a region proposal with a plurality of internal bounding boxes, system <NUM> can generate a set of overlapping parameters. System <NUM> can select the region proposal as a positive sample if the largest value of the set of overlapping parameters is larger than a threshold or falls within a threshold range.

In some embodiments, the size and shape of the bounding box of a region proposal may be adjusted to obtain another bounding box. System <NUM> can then perform another round of sampling using the new bounding box. If there is another target bounding box in the image, system <NUM> can use the same method for image sampling in that target bounding box. In this way, system <NUM> can perform image sampling for each target bounding box in an image. Each target bounding box may allow system <NUM> to determine a plurality of region proposals for sampling. For each region proposal, system <NUM> can determine whether to select the region proposal as a positive sample for training a target detector.

Optionally, system <NUM> does not select a region proposal as a positive sample (i.e., the overlapping parameters have not met the condition), system <NUM> may further screen the region proposal as a potential negative sample. For example, system <NUM> may select the region proposal as a negative sample if the ratio of the region proposal for each corresponding internal bounding box is below a low threshold range (e.g., <NUM>). In other words, if the comparison results of a region proposal and the regions enclosed by all corresponding internal bounding boxes meet the condition for a negative sample, system <NUM> can select the region proposals as a negative sample. Furthermore, system <NUM> can place a region proposal outside of any target bounding box to collect a negative sample. Since each continuous damage region is covered by a corresponding target bounding box, a region proposal outside of any target bounding box can be selected as a negative sample. In this way, system <NUM> can reduce noise interference in a negative sample.

<FIG> illustrates an exemplary integration of detection results of multiple samples, in accordance with an embodiment of the present application. Suppose that system <NUM> has obtained positive samples <NUM>, <NUM>, and <NUM> from an input image <NUM>. System <NUM> can represent samples <NUM>, <NUM>, and <NUM> as inputs <NUM>, <NUM>, and <NUM>, respectively, for target detector <NUM> of system <NUM>. System <NUM> can use target detector <NUM> to generate corresponding outputs <NUM>, <NUM>, and <NUM>. Each of these outputs can include a characteristic description of the corresponding input. The characteristic description can include a feature vector, a label (e.g., based on the damage class), and a corresponding bounding box.

System <NUM> can then construct a splice of outputs <NUM>, <NUM>, and <NUM> in the bounding box dimension. For example, system <NUM> can perform a concatenation <NUM> of the bounding boxes to improve the accuracy of target detector <NUM>. Target detector <NUM> can be further trained and optimized based on a Gradient Boosted Decision Trees (GBDT) model <NUM>. GBDT model <NUM> can optimize the concatenated bounding boxed and generate a corresponding target indicator <NUM>, which can include an optimized bounding box and a corresponding label, for the damage depicted in samples <NUM>, <NUM>, and <NUM>. In this way, the efficiency and accuracy of target detector <NUM> can be further improved.

<FIG> presents a flowchart <NUM> illustrating a method of an assessment system performing a damage assessment, in accordance with an embodiment of the present application. During operation, the system receives an input image indicating damage on a vehicle (operation <NUM>) and performs target detection on the input image (operation <NUM>). The system then determines the damage information (e.g., the degree of the damage and the parts impacted by the damage) based on the target detection (operation <NUM>). The system assesses the damage based on the damage information (e.g., whether the parts can be repaired or would need replacement) (operation <NUM>). Subsequently, the system determines a repair plan and cost estimate based on the damage assessment and insurance information (operation <NUM>).

<FIG> presents a flowchart <NUM> illustrating a method of an assessment system generating image samples for training a target detection system, in accordance with an embodiment of the present application. During operation, the system obtains an image for sampling (operation <NUM>) and retrieves a target bounding box and a set of internal bounding boxes in the target bounding box in the obtained image (operation <NUM>). The system then determines a region proposal in the target bounding box based on a set of movement criteria (operation <NUM>) and determines overlapping parameters (e.g., IoUs) associated with the region proposal and the corresponding internal bounding boxes (operation <NUM>).

Subsequently, the system determines whether the overlapping parameters are in the threshold range (operation <NUM>). If the overlapping parameters are in the threshold range, the system can select the region proposal as a positive sample (operation <NUM>). If the overlapping parameters are not in the threshold range, the system determines whether the overlapping parameters are below a low threshold range (operation <NUM>). If the overlapping parameters are below a low threshold range, the system can select the region proposal as a negative sample (operation <NUM>).

Upon selecting the region proposal as a sample (operation <NUM> or <NUM>), or if the overlapping parameters are not below a low threshold range (operation <NUM>), the system determines whether the target bounding box has been fully sampled (i.e., the set of movement criteria has met a termination condition) (operation <NUM>). If the target bounding box has not been fully sampled, the system determines another region proposal in the target bounding box based on the set of movement criteria (operation <NUM>). If the target bounding box has been fully sampled, the system determines whether the input image has been fully sampled (i.e., all target bounding boxes have been sampled) (operation <NUM>). If the input image has not been fully sampled, the system retrieves another target bounding box and another set of internal bounding boxes in the target bounding box in the obtained image (operation <NUM>).

<FIG> presents a flowchart <NUM> illustrating a method of an assessment system integrating detection results of multiple samples, in accordance with an embodiment of the present application. During operation, the system obtains a training image indicating a damaged area of a vehicle (i.e., a vehicular damage) (operation <NUM>) and a sample associated with the damage in the image (operation <NUM>). The system then generates corresponding output comprising features (e.g., a feature vector), a bounding box, and a corresponding label using an AI model (e.g., the target detector) (operation <NUM>). The system then checks whether all samples are iterated (operation <NUM>).

If all samples have not been iterated, the system continues to determine another sample associated with the damage in the image (operation <NUM>). If all samples have been iterated, the system performs bounding box concatenation based on the generated outputs to generate a characteristic description of the damage (operation <NUM>). The system then trains and optimizes using a GBDT model (operation <NUM>) and obtains a bounding box and a corresponding label representing the damage based on the training and optimization (operation <NUM>).

<FIG> illustrates an exemplary computer system that facilitates an efficient assessment system, in accordance with an embodiment of the present application. Computer system <NUM> includes a processor <NUM>, a memory device <NUM>, and a storage device <NUM>. Memory device <NUM> can include volatile memory (e.g., a dual in-line memory module (DIMM)). Furthermore, computer system <NUM> can be coupled to a display device <NUM>, a keyboard <NUM>, and a pointing device <NUM>. Storage device <NUM> can be a hard disk drive (HDD) or a solid-state drive (SSD). Storage device <NUM> can store an operating system <NUM>, a damage assessment system <NUM>, and data <NUM>. Damage assessment system <NUM> can facilitate the operations of system <NUM>.

Damage assessment system <NUM> can include instructions, which when executed by computer system <NUM> can cause computer system <NUM> to perform methods and/or processes described in this disclosure. Specifically, damage assessment system <NUM> can include instructions for generating region proposals in a target bounding box of an input image (region proposal module <NUM>). Damage assessment system <NUM> can also include instructions for calculating overlapping parameters for the region proposal (parameter module <NUM>). Furthermore, damage assessment system <NUM> includes instructions for determining whether a region proposal can be a positive or a negative sample based on corresponding thresholds (sampling module <NUM>).

Damage assessment system <NUM> can also include instructions for training and optimizing using a GBDT model (response module <NUM>). Moreover, damage assessment system <NUM> includes instructions for assessing a damaged area (i.e., a target) of a vehicle from an input image and generating a repair plan (planning module <NUM>). Damage assessment system <NUM> may further include instructions for sending and receiving messages (communication module <NUM>). Data <NUM> can include any data that can facilitate the operations of damage assessment system <NUM>, such as labeled images and generated samples.

<FIG> illustrates an exemplary apparatus that facilitates an efficient assessment system, in accordance with an embodiment of the present application. Damage assessment apparatus <NUM> can comprise a plurality of units or apparatuses which may communicate with one another via a wired, wireless, quantum light, or electrical communication channel. Apparatus <NUM> may be realized using one or more integrated circuits, and may include fewer or more units or apparatuses than those shown in <FIG>. Further, apparatus <NUM> may be integrated in a computer system, or realized as a separate device that is capable of communicating with other computer systems and/or devices. Specifically, apparatus <NUM> can include units <NUM>-<NUM>, which perform functions or operations similar to modules <NUM>-<NUM> of computer system <NUM> of <FIG>, including: a region proposal unit <NUM>; a parameter unit <NUM>; a sampling unit <NUM>; an optimization unit <NUM>; a planning unit <NUM>; and a communication unit <NUM>.

The data structures and code described in this detailed description are typically stored on a computer-readable storage medium, which may be any device or medium that can store code and/or data for use by a computer system. The computer-readable storage medium includes, but is not limited to, volatile memory, non-volatile memory, magnetic and optical storage devices such as disks, magnetic tape, CDs (compact discs), DVDs (digital versatile discs or digital video discs), or other media capable of storing computer-readable media now known or later developed.

The methods and processes described in the detailed description section can be embodied as code and/or data, which can be stored in a computer-readable storage medium as described above. When a computer system reads and executes the code and/or data stored on the computer-readable storage medium, the computer system performs the methods and processes embodied as data structures and code and stored within the computer-readable storage medium.

Claim 1:
A computer-implemented method for facilitating image sampling for training a target detector to identify damage in a vehicle, comprising:
obtaining a first image depicting a first target which is a continuous region of damage and that is enclosed in a target bounding box, wherein said target bounding box is assigned a label that indicates a damage information including an area of damage and a degree of damage which is related to the material impacted by the damage and wherein said target bounding box comprises internal bounding boxes that each surround a segment of the continuous region;
generating a set of positive image samples from an area of the first image enclosed by the target bounding box comprising determining a region proposal in the area of the first image enclosed by the target bounding box, the region proposal for sampling in the target bounding box; and a region proposal as a positive sample in response to determining that an overlapping parameter of said region proposal with an internal bounding box is in a threshold range; wherein a respective positive image sample includes at least a part of the first target;
and
training the target detector with the set of positive image samples, wherein the target detector is an artificial intelligence (AI) model capable of detecting a damage on a vehicle, the AI model being a deep-learning neural network;
receiving a second image indicating damage on a vehicle;
performing target detection by applying the target detector on the second image to detect a second target and determining the damage information;
assessing the damage based on damage information;
generating a repair plan and cost estimate based on the damage assessment result.