DATA COLLECTION SYSTEM

The purpose of the present invention is to provide a data collection system capable of appropriately setting a range for collecting image data used as training data of a learning device that recognizes an image. A data collection system according to the present invention receives requirement variables that represent the requirement of image data necessary for a learning device to be sufficiently trained and requirement definition data that designates the requirement value, further receives priority data that designates the priority of the requirement variables, presents the requirement variables and the requirement values in the order of the priorities, and presents a requirement value response rate that represents the ratio of the requirement variables for which the requirement value is specified.

TECHNICAL FIELD

The present disclosure relates to a data collection system that collects image data used for learning process performed by a learner that identifies images by machine learning.

BACKGROUND ART

A learner that identifies images uses image data as learning data. In order to appropriately perform learning, it is desirable to collect appropriate learning data as much as possible. This is because amount or quality of image data which is collected as learning data is an important factor for determining quality of the learner. However, it depends on the developer of the learner what learning data should be collected, otherwise the learning data is collected on ad hoc basis in many cases. In addition, the collecting process may be performed in many cases by a person who has small experience of developing learner, which may lead into a consequence that appropriate data is not collected.

Patent Literature 1 below describes a technique for collecting learning data for a learner that identifies images. This literature describes a technical problem as: To suppress, during the collection of learning data used for image recognition, the collection of inappropriate photographic images that are unsuitable for learning; and also describes a solution as: A learning data collection device 3, wherein a processor 51 is configured to acquire a photographic image from a photographic device 2, determine whether the photographic image is suitable for the learning data, and execute a notification process for prompting a photographer to retake photographic images that were determined to be unsuitable for the learning data (refer to Abstract).

CITATION LIST

Patent Literature

Patent Literature 1: JP 2020-008904 A

SUMMARY OF INVENTION

Technical Problem

The technique in Patent Literature 1 determines whether the collected image data is appropriate. However, this literature does not clearly discuss how to define the collected range of the image data. In other words, this literature merely determines whether the collected image data is appropriate after collecting the image data. Thus this literature may not sufficiently consider defining the collected range appropriately.

The present disclosure has been made in view of the problems above, and an objective of the present disclosure is to provide a data collection system that can appropriately define a range in which image data is collected which is used as learning data for a learner that identifies images.

Solution to Problem

A data collection system according to the present disclosure: receives requirement definition data that specifies a requirement variable representing a requirement of image data necessary for a learner to sufficiently perform learning process, and that also specifies a requirement value of the requirement variable; receives priority data that specifies a priority of the requirement variable; presents the requirement variable and the requirement value according to an order of the priority; and present a requirement value answer ratio that represents a ratio of the requirement variable in which the requirement value is already specified.

Advantageous Effects of Invention

According to the data collection system of the present disclosure, it is possible to appropriately define a range in which image data is collected which is used as learning data for a learner that identifies images.

DESCRIPTION OF EMBODIMENTS

FIG.1is a configurational diagram of a data collection system10according to an embodiment 1 of the present disclosure. The data collection system10is a system that collects learning data (hereinafter, also referred to as image data) used for a learner that identifies images. The data collection system10is accessed from a developing site20and a collection site30respectively. The developing site20is a place where developers of the learner work. An operation terminal21used by the developers is installed in the developing site20. The collection site30is a place where a collector who collects image data performs collecting tasks. An operation terminal31is installed in the collection site30. Each site is connected to each other by a network.

An overall operation sequence of the data collection system10will be described below. A developer defines, on the operation terminal21, a requirement of learning data that is necessary for the learner to sufficiently perform the learning process ((1) requirement definition). This requirement definition is also notified to an operation terminal used by a requester who requests to develop the learner. The developer and the requester define a collection requirement by communicating with each other according to a flowchart described later.

The developer configures, on the operation terminal21, a range in which the learning data is collected ((2) collection configuration). The requester may in some cases configure a constraint for collecting data. The developer and the requester configure a collected range by communicating with each other according to a flowchart described later.

The developer instructs, on the operation terminal21, to start collecting learning data ((3) instruct to collect). The data collection system10notifies the contents of instruction to the collector ((3) notify content of instruction). The collector instructs, on the operation terminal31, an imaging device to start collecting data ((4) prepare for collection). A camera device captures image data ((5) capture). The operation terminal31collects the captured image data ((6) collect data).

The operation terminal31sends the collected image data to the data collection system10((7) send data). The data collection system10notifies the operation terminal31and the210of excess or deficiency of image data ((8) notify excess or deficiency). If the image data is not sufficient, the developer or the requester instruct collecting data again. If the image data is sufficient, the operation terminal21acquires the collected image data ((9) acquire data).

FIG.2is a function block diagram of the data collection system10. The data collection system10includes a processor11and a storage unit12. The processor11is a processor that executes software, such as CPU (Central Processing Unit). The storage unit12is a device that stores data, such as hard disk device.

The processor11executes a requirement definition processor111, a collected range processor112, a data collection processor113, and a communicator115. These functional units are configured as software modules executable by the processor11.

The requirement processor111performs a process of when defining a requirements of learning data. The collected range processor112performs a process of when configuring the collected range of the learning data. The data collection processor113performs a process of when collecting image data. The communicator114communicates with other devices via a network. Functional units other than the communicator114communicates with other devices via the communicator114.

Hereinafter, for the sake of description, the functional units included in the data collection system10may be described as actors of operation. However, the processor11actually executes those functional units. In the operation terminals21and31, each of functional units is similarly executed by a processor included in the terminal, respectively.

The storage unit12stores a requirement management database (DB)121, a collected range management DB122, and an image data management DB123. The requirement management DB121stores results of defining requirements of learning data. The collected range management DB122stores results of defining the collected range of learning data. The image data management DB123stores the collected image data.

FIG.3is a function block diagram of the operation terminal21. The operation terminal21includes a processor211. The processor211executes a requirement setter212, a collected range setter213, a collection instruction processor214, a data acquisition processor215, and a communicator216. These functional units are configured as software modules executable by the processor211.

The requirement setter211performs a process for configuring requirements of learning data. The collected range setter213performs a process for configuring the collected range of learning data. The collection instruction processor214instructs to collect learning data. The data acquisition processor215acquires learning data from the data collection system10. The communicator216communicates with other devices via a network. Functional units other than the communicator216communicate with other devices via the communicator216.

FIG.4is a function block diagram of the operation terminal31. The operation terminal31includes a processor311. The processor311executes a collected range processor312, a collected content displayer313, a data sender314, and a communicator315. These functional units are configured as software modules executable by the processor311.

The collected range processor312performs a process regarding the collected range of learning data. The collected content displayer313displays the collected learning data. The data sender314sends the collected learning data to the data collection system10. The communicator315communicates with other devices via a network. Functional units other than the communicator315communicates with other devices via the communicator315.

FIG.5is a flowchart that explains an operation of the data collection system10. Hereinafter, each step inFIG.5will be described.

The developer and the requester define a requirement for collecting learning data by communicating with each other. The developer and the requester can communicate about the collection requirement via the data collection system10. It also applies to steps below. Details of this step will be described inFIG.6later.

The developer and the requester define a range in which the learning data is collected by communicating with each other. The collector can see the defined collected range. After this step, each of persons concerned determines whether the requirement should be corrected. If the requirement should be corrected, the process returns to step 1. If the requirement is not corrected, the process proceeds to step 4. Details of this step will be described inFIG.10later.

(FIG.5: Step 3. Collect and Check Data)

The operation terminal31collects learning data according to the requirement and the collected range configured in the data collection system10. The collector and the developer check the collected learning data on the data collection system10. If the learning data is not sufficient, the process returns to step 2. If the learning data is sufficient, the process proceeds to step 4. Details of this step will be described inFIG.13later.

The collector or the developer registers the collected learning data into the data collection system10. The registration in this step is to decide employing the collected learning data for the learner. If the requirement is not modified, this flowchart is finished. If the requirement is modified, the process returns to one of steps 1-3. It will be described later which step is the target of return. Details of this step will be described inFIG.14later.

FIG.6is a process flow diagram that explains details of step 1. Hereinafter, each step inFIG.6will be described.

The developer firstly decides requirement items among the requirement for collecting learning data (S601). The developer decides requirement variables and priorities thereof for each of the requirement items, and then registers them into the data collection system10(S602). Specific examples of the requirement items and the requirement variables will be described later. S601-S602are performed by the requirement setter212on the operation terminal21and by the requirement definition processor111on the data collection system10. The requirement setter212sends requirement definition data that describes the requirement variables and priority data that describes the priorities. The requirement definition processor111receives the requirement definition data and the priority data, and then stores them into the requirement management DB121.

The developer (or the requester) inputs requirement values into each of the requirement variables. An example of input screen used in this step will be described later. The input screen is provided by the requirement definition processor111or by the requirement setter212. The requirement definition processor111receives the inputted requirement values. If the input screen is provided by a server application such as Web application, the requirement definition processor111provides the input screen. The screens described below are similarly provided by each functional unit included in the data collection system10if such screen is provided as a server application.

The developer and the requester mutationally check the list of requirement variables and the requirement values. An example of confirmation screen used in this step will be described later. The confirmation screen is provided by the requirement definition processor111. The requirement setter212displays the confirmation screen.

FIG.7Ais a diagram that illustrates an example of requirement item. The requirement for the leaner identifying images always changes. The requirement variables are categorized into each of requirement items. The requirement items include such as objective requirement, operational requirement, and develop requirement. Hereinafter, each requirement items will be described.

The objective requirement defines such as: a target collected as learning data; an actor that collects the learning data. In many cases, the objective requirement is fixed for each of developing project for collecting learning data. Thus the objective requirement is defined independently from other requirements. If the objective requirement is modified in the last step inFIG.5, the process returns to step 1.

The operational requirement defines requirements that temporally change. The operational requirement corresponds to requirements that are corrected depending on such change. The operational requirement corresponds to, for example, environmental requirement where the data collection is performed such as date, time, and location. The operational requirement also corresponds to method or procedure for collecting the learning data. For example, configuration parameters of imaging device correspond to such requirement. If the operational requirement is modified in the last step inFIG.5, the process returns to step 2.

The develop requirement defines technical requirements of the learner. The develop requirement corresponds to requirements that are continuously corrected along with growth of the learner. For example, amount of learning data corresponds to such requirement. If the develop requirement is modified in the last step inFIG.5, the process returns to step 3.

FIG.7Bis a diagram that illustrates an example of requirement variable. One or more of the requirement variables are configured for each of requirement items. Each requirement variable has a requirement value. Hereinafter, each of requirement variables and requirement values will be described.

Requirement variables of the objective requirement include such as: type of device that captures image data; type of learner (target of learning, algorithm, etc.); imaging target.

Requirement variables of the operational requirement include such as: external environment where the image data is collected; configuration parameters of imaging device. The external environment variable includes such as: date and time; location; weather; peripheral objects. The configuration parameter includes such as: parameters of imaging device, illumination parameters.

Requirement variables of the develop requirement include such as: number of captured image data; ratio of number of capture. The number of capture and the ratio of number of capture may be configured for each captured target or may be configured for each requirement variable. For example, if the ratio of number of capture is configured as target A:target B=1:2, the number of captured image data for target B is twice of target A.

FIG.8is a diagram that illustrates a screen transition example of an input screen used in S603. This screen prompts the developer or the requester to input requirement values in the form of hierarchical questions. The question form is presented in a representation depending on type of requirement variable (list, time, numerical range, etc.). If the answerer skips a question, the system returns to the not-inputted portion to question again. If type of requirement variable represents a finite range (time, location, etc.), the system may question again for the non-inputted range if there remains such not-inputted range in the finite range. The system may present a ratio of answered items or ratio of answered ranges, thereby prompting to answer all question items.

If there remains not-inputted portion or not-inputted range, the system may prompt sequential reinput from high-priority requirement variable toward low-priority requirement variable, or may prompt reinput conversely from low-priority requirement variable toward high-priority requirement variable. However, if the requirement variables are hierarchically constructed, the answerer may be rather perplexed when starting the reinput from higher requirement variables after already inputting intermediate-priority requirement variables. If the requirement variables have such hierarchical structure, it may be desirable to prompt sequential reinput from high-priority requirement variable toward low-priority requirement variable.

FIG.9is a diagram that illustrates an example of a confirmation screen used in S604. This screen presents a list of pairs of requirement variable and requirement value in the form of tree, for example. When sequentially inputting requirement values from higher priority to lower priority, those requirement variables are in hierarchical relationship. In such case, the screen presents requirement variables such that: as the requirement variable has higher priority, the variable is placed on more upper node. The screen may also present a cover ratio of answered questions for each requirement value of requirement variable (if the answerer skips to answer when inputting requirement, the cover ratio decreases). The screen may present each requirement value by a representation (e.g. graph) depending on type of requirement variable. The developer and the requester can agree the collection requirement by mutually referring to this screen and by checking quantitative numerical values of requirement variables.

FIG.10is a process flow diagram that explains details of step 2. Hereinafter, each step inFIG.10will be described.

If there exists constraint necessary for collecting learning data, the requester notifies the data collection system10of the constraint. The constraint may be notified via the developer. The collected range processor112may acquire data describing the constraint. Instead of formulating the constraint as data, the developer may manually input the collected range in subsequent steps along with considering the constraint, thereby reflecting the constraint. There are various types of actual constraints, and thus it may be difficult in many cases to formulate the constraint. Thus it is practical to manually input the collected range by the developer along with considering the constraint. Hereinafter, such situation is assumed.

The developer inputs the data collection range and its priority according to requirement variables, requirement values, and the constraint. An example of input screen used in this step will be described later. The input screen is provided by the collected range processor112. The collected range setter213sends collected range data specifying the collected range. The collected range processor112receives the collected range data and stores it into the collected range management DB122. This step configures the data collection range and its priority, and the portions which are not inputted as the collected range are configured as uncollectable range.

The requester, the developer, and the collector check the configured data collection range. If the data collection range is modified, the process returns to S1002. An example of confirmation screen used in this step will be described later. The confirmation screen is provided by the collected range processor112.

FIG.11is a diagram that illustrates an example of an input screen used in S1002. The requirement variables may be grouped as shown in the tree structure ofFIG.9.FIG.9shows an example where the requirement variables are aggregated under assumption that the requirement values are different for each of captured targets. The input screen ofFIG.11is configured such that the collected range may be inputted for each of the groups. The method of grouping the requirement variables may be determined in advance in the data collection system10, or may be determined according to the hierarchical relationship ofFIG.9. For example, a requirement variable belonging at or under an uppermost layer which requires inputting requirement values may be grouped under the uppermost variable.

Requirement variables in one group are sorted in the order of priority. In the example ofFIG.11, the number of captured image has a highest priority. Accordingly, it is possible to prompt to preferentially input a requirement variable with higher priority. The form of input may be presented in a representation depending on type of requirement variable (list, time, numerical range, etc.). Regarding requirement variables with constraint, the constrained portion may be configured to be noneditable.

The order of collection may be configured for each group. InFIG.11, the groups are aligned from left column to right column in the order of group which require learning data collection to be prerentially performed. When adding a new group, the new group is placed at the last order at the time of adding.

The developer inputs the collected range according to the constraint which is recognized by the developer in advance. After inputting all items, the developer presses the determine button. In addition to the range specified by the constraint, the portions which are not inputted in this screen are configured as uncollectable range. Other inputted portions are configured as collected range.

FIG.12is a diagram that illustrates an example of a confirmation screen used in S1003. This screen displays the data collection range in the form of list in the order of collection configured in the input screen ofFIG.11. The tree form shown inFIG.12lower diagram may be used simultaneously. The requirement variables in each collection order are displayed in the order of priority.

The collected range processor112aggregates the range of requirement variables which are not inputted in the input screen ofFIG.11, and the recognizes those range as uncollectable range. This screen presents the collected range along with the uncollectable range. It is desirable to present the uncollectable range along with the list and the tree. Accordingly, it is possible to prompt reinput when there exists input omissions, for example.

The collected range processor112calculates a ratio of portions of requirement variables which are not inputted, and then presents the calculated value as an input cover ratio. Regarding portions that cannot be inputted because of constraint, such portions may or may not be included in the calculation. Accordingly, it is possible to prompt reinput when there exists input omission, for example, in addition to presenting the uncollectable range.

The requester, the developer, and the collector define the data collection plan by checking quantitative numerical values of data collection range on this screen and by agreeing the data collection range. When modifying the collected range, the input screen ofFIG.11is presented again to prompt reinput.

FIG.13is a process flow diagram that explains details of step 3. Hereinafter, each step inFIG.13will be described.

The collector collects learning data using imaging devices, for example, according to the determined data collection range. In the embodiment 1, the operation terminal31does not automatically work together with the imaging device. The collector manually collects learning data. The collected learning data is raw data and is not associated with requirement variables. Thus the collector associates the learning data with requirement variables1requirement values, and then registers it into the data collection system10. The register screen for performing registering operation will be described later.

The collector and the developer check the collected learning data and the requirement variables on the screen. If collecting operation is continued, the process returns to S1301. The confirmation screen may be same one as the register screen in S1301.

FIG.14Ais a conceptual diagram of a work performed by a collector in S1301. The collector collects learning data using such as imaging device according to the determined collected range. The collected learning data is image data. Thus the learning data itself is not associated with requirement variables. The collector registers learning data and requirement variables into the data collection system10in association with each other in order to clarify the requirement variable (and requirement value) on which each collected learning data is based. The additional information inFIG.14Alower diagram is a specific example of requirement variable and requirement value associated with each other.

FIG.14Bis a diagram that illustrates an example of a registration screen used in S1301. The collector selects image data on the register screen shown at left side ofFIG.14Band inputs each requirement value. The data collection processor113receives the selected image data and the inputted requirement value, and then stores them into the image data management DB123temporally. The data collection processor113compares the determined data collection range with the registered requirement value, thereby identifying uncollected range. The data collection processor113presents the uncollected range as a confirmation screen.

The uncollected range may be presented in the form of list as inFIG.12, may be presented in the form of tree, or may be presented using both. Regarding requirement variables that can be shown in the form of figure such as graph, such variables may be illustrated as shown at right bottom ofFIG.14B. The uncollected range may be presented in the order of collection as configured inFIG.11.

The data collection processor113may compare the determined data collection range with the registered requirement value, thereby calculating a ratio of uncollected portions in the collected range and presenting the ratio as a cover ratio on the confirmation screen. After the collector finishes collecting learning data across all data collection range, the cover ratio is 100%.

By presenting the confirmation screen such asFIG.14B, the collector is prompted to collect learning data preferentially from the collected range of earlier collection order. By presenting the cover ratio, it is possible to present, to the developer and to the collector, excess or deficiency of collected learning data.

The data collection system10according to the embodiment 1 provides a requirement confirmation screen (FIG.9) used by the requester and the developer to determine the data collection requirement by mutually checking with each other. The data collection system10presents each requirement variable in the order of priority of requirement variable on the requirement confirmation screen. Accordingly, it is possible to prompt determining requirement variables from those with higher priority. The requirement confirmation screen presents a ratio of requirement variables which requirement values are determined. Accordingly, it is possible to prompt inputting requirement values including input omission. In other words, the requester and the developer check data collection requirement via the requirement confirmation screen, thereby determining the data collection range under support of the developer. Thus even if the requester (or collector in the subsequent process) does not have detailed technical knowledge, it is possible to determine the collected range without omission and exhaustively.

The data collection system10according to the embodiment 1 provides a requirement input screen (FIG.8) for the developer to input requirement variable/requirement value/priority. The requirement input screen prompts inputting requirement values in the form of question in the order of requirement variable priority, and prompts reinput for non-inputted portions. Accordingly, the developer can input requirement values without omission and exhaustively.

The data collection system10according to the embodiment 1 provides a collected range input screen (FIG.10) for the developer to input data collection range. The data collection system10further provides a collected range confirmation screen (FIG.12) for the requester/developer/collector to mutually check the inputted data collection range. By determining the data collection range with those three participants checking the collected range, it is possible to configure the collected range without omission and exhaustively. Accordingly, even if any one of those three participants has small technical knowledge, it is possible to prevent missing consideration for data collection content.

The data collection system10according to the embodiment 1 provides a data collection confirmation screen (FIG.14B) that presents a cover ratio of learning data collected by the collector. Accordingly even if the collector has small technical knowledge or small experience, it is possible to collect the learning data appropriately and exhaustively without omission.

In the embodiment 1, the collector manually operates the imaging device, for example, thereby collecting learning data. Alternatively, the operation terminal31may connect to the imaging device to control the imaging device, thereby collecting learning data without manual operation of the collector. In this case, the operation terminal31(e.g. data sender314) connects to the imaging device via an application interface included in the imaging device, for example, and then controls the imaging device via the interface. The behavior of the imaging device may be determined according to the data collection range acquired from the data collection system10.

In this embodiment, it is possible to automatically collect learning data according to the collected range defined by requirement variables and requirement values, thereby it is possible to automatically associate the collected learning data with requirement variables and requirement values. Accordingly, it is not necessary for the collector to associate the learning data with requirement variables/requirement values respectively, thereby improving collection efficiency.

Modification of Present Disclosure

In the embodiments above, the requirement variables and the requirement values are merely examples. Some requirement variables or requirement values other than those embodiments above may be configured. Categories of requirement variable other than those embodiments above may be similarly employed.

REFERENCE SIGNS LIST