Patent Description:
With the advent of the IoT (Internet of Things) era, various types of devices (IoT devices) have come to be connected to the Internet and used in various ways. Traffic session anomaly detection systems and intrusion detection systems (IDS) for IoT devices are being actively researched as security measures for such IoT devices.

One example of such an anomaly detection system uses a probability density estimator obtained by unsupervised learning, such as a VAE (Variational Auto Encoder). When the probability density estimator performs anomaly detection, high-dimensional data for learning, called traffic feature amounts, is generated from actual communication, and the features of normal traffic are learned using such features in order to be able to estimate the probability of occurrence of a normal communication pattern. Thereafter, the trained model is used to calculate the probability of normal communication pattern occurrence for each type of communication session, and a type of communication session that has a small probability of normal communication pattern occurrence is detected as an anomaly. For this reason, it is possible to detect an anomaly without knowing all malignant conditions, and it is also possible to deal with unknown cyber attacks.

[PL <NUM>] <CIT> describes a traffic session anomaly detection and intrusion detection for loT devices. A detection device acquires a piece of information relevant to communication of an IoT device. The detection device calculates output data by inputting a piece of data representing characteristics of the information relevant to the communication into a generation model for generating a piece of output data on the basis of a latent variable that is a random variable following a Gaussian mixture distribution and the input data. The detection device calculates the anomaly score on the basis of the output data, and when the anomaly score exceeds a threshold value, detects an abnormality of the IoT device.

When using an anomaly detection system that employs a probability density estimator in actual operation, the anomaly detection system needs to ascertain what sort of features of communication tend to be considered to be normal. However, the learning-targeted IoT devices in the anomaly detection system perform a wide variety of types of communication, and it is difficult to ascertain the tendencies of such communication.

Specifically, IoT devices perform communication with use of various protocols depending on the type of device, and even if focus is placed on communication by one HTTP protocol, some types of communication such as WebSocket continue for a long time, and other types of communication such as page loading end very quickly, or in other words, even communication by one protocol can have various characteristics. The traffic feature amounts serving as learning data generated from such types of communication are therefore also diverse, and it is also difficult to ascertain tendencies in such traffic feature amounts for learning by merely performing statistical processing such as average value or median value calculation. If the tendencies of traffic feature amounts for learning cannot be ascertained, the anomaly detection system cannot ascertain what kind of features of communication are considered to be normal, and therefore even if the anomaly detection system detects an anomaly, the reason for detecting the anomaly cannot be understood, and interference with operation is conceivable.

The present invention has been made in view of the above, and an object of the present invention is to provide a learning device, a learning method, and a learning program capable of providing data for ascertaining a tendency of traffic feature amounts for learning.

According to the present invention, it is possible to provide data for ascertaining a tendency of traffic feature amounts for learning.

Note that the present invention is not intended to be limited to this embodiment. Also, in the drawings, like portions are denoted by like reference signs. In the following, the denotation "^A" for A, which is a vector, a matrix, or a scalar, is intended to be equivalent to "A with ^ directly above".

[Embodiment] In the present embodiment, information for ascertaining the tendency of traffic feature amounts for learning is provided by extracting representative points of traffic feature amounts for learning by using a kernel method called kernel herding. Also, in the present embodiment, when a generative model such as VAE is used as the probability density estimator, data is generated by the generative model and kernel herding is used to extract representative points of generated data in order to provide information for ascertaining what kind of communication the generative model actually considers to be normal.

[kernel herding] First, kernel herding will be described. Here, kernel herding has been proposed as an algorithm for finding a sample sequence that efficiently approximates a kernel mean mx with a kernel sample mean (<NUM>/T) ΣtΦ (xt) (see NPL <NUM>). In kernel herding, a sample sequence {xt} is sequentially obtained according to the update equations shown in Expressions <NUM> and <NUM> below. <NUM>]<MAT> [Math. <NUM>]<MAT>.

Here, mx is the kernel average of the data set X shown in Expression <NUM>. Also, Φ(•) is a feature map. Moreover, <•,•> is an inner product in the reproducing kernel Hilbert space associated with a positive-definite kernel. <NUM>]<MAT>.

However, in general, the kernel average mx cannot be calculated directly. In view of this, when actually executing the kernel herding algorithm, the kernel average is replaced with a sample kernel average ^m=(<NUM>/N)ΣnΦ(xn) approximated with a sufficiently large sample (Expressions <NUM> and <NUM>). <NUM>]<MAT> [Math. <NUM>]<MAT>.

It is experimentally known that even if this replacement is performed, an efficient sample can be obtained by kernel herding (see NPL <NUM>).

In the present embodiment, kernel herding is used as a technique for extracting representative points from a data set. If the positive-definite kernels used in the calculation are characteristic, the kernel average mx approximated by kernel herding contains complete information about the distribution of the data set X (see Expression <NUM>).

For this reason, a sample sequence that approximates the kernel average mx, which was obtained by kernel herding, with a small number of data points can be regarded as a set of representative points of data set X. <FIG> (cited from NPL <NUM>) is a diagram showing the results of a comparative experiment regarding ordinary random sampling and sampling by kernel herding in a mixed Gaussian distribution. As shown in <FIG>, it can be seen that, in comparison with random sampling, sampling by kernel herding can qualitatively extract representative data points.

In the present embodiment, a method is proposed in which an anomaly detection system ascertains a traffic tendency that is considered to be normal by extracting representative points of traffic feature amounts for learning.

[Configuration of anomaly detection system] The following describes a communication system in the present embodiment. <FIG> is a block diagram showing an example of the configuration of the communication system according to this embodiment. As shown in <FIG>, a communication system <NUM> according to this embodiment has a configuration in which a plurality of network (NW) devices <NUM> and a detection system <NUM> are connected via a network N. The detection system <NUM> communicates with a user terminal <NUM> that is used by the network administrator or the like.

The NW devices <NUM> each sample packets in traffic that is the target of anomaly detection. The NW devices <NUM> transfer the sampled packets to the detection system <NUM> via the network N.

In the detection system <NUM>, the presence or absence of a communication anomaly is detected using a generative model that has been trained with traffic feature amounts by unsupervised learning based on packets received from the NW devices <NUM>, and the detection result is transmitted to the user terminal <NUM> used by a system administrator. The detection system <NUM> includes a learning device <NUM>, a detection device <NUM>, and an evaluation device <NUM>.

Note that the generative model is a probability density estimator such as a VAE. Due to learning traffic feature amounts, the VAE can output an anomaly score when given a traffic feature amount. If noise is input to an intermediate layer, VAE outputs an output distribution that corresponds to the input noise.

The learning device <NUM> trains the generative model with traffic feature amounts by unsupervised learning based on packets received from the NW devices <NUM>. The learning device <NUM> also uses kernel herding to extract representative points of traffic feature amounts for learning, and outputs the extracted representative points to the user terminal <NUM> as data for evaluating the progress of the generative model.

The detection device <NUM> detects the presence or absence of a communication anomaly in traffic subject to anomaly detection, by using the generative model whose model parameters were optimized by the learning device <NUM>.

The evaluation device <NUM> generates pieces of data from the generative model that was trained by the learning device <NUM>, extracts representative points from the pieces of data using kernel herding, and outputs the extracted representative points to the user terminal <NUM> as data for evaluating the degree of progress of the generative model. Specifically, the evaluation device <NUM> inputs noise to an intermediate layer of the VAE, samples data from an output distribution that corresponds to the noise, and acquires the sampled data as data generated by the generative model. The data generated by this generative model corresponds to the data that can be considered to be normal when the generative model is used as a probability density estimator.

[Processing flow of detection system] Next, a processing flow will be described with reference to <FIG> is a diagram illustrating a processing flow of the detection system <NUM> in this embodiment.

As shown in <FIG>, the learning device <NUM> extracts traffic feature amounts for learning based on packets collected via the NW devices that are learning targets (see (<NUM>) in <FIG>), and trains a generative model such as a VAE using the extracted traffic feature amounts (see (<NUM>) in <FIG>). Additionally, the learning device <NUM> extracts representative points of the traffic feature amounts for learning by kernel herding (see (<NUM>) in <FIG>).

It is assumed that the data set of traffic feature amounts for learning basically contains only normal communication. A probability density estimator (generative model) such as a VAE is used in the learning device <NUM>, and it learns traffic feature amounts that the detection system <NUM> considers to be normal, based on the aforementioned data set. Accordingly, the representative points of the traffic feature amounts for learning correspond to traffic feature amounts that the detection system <NUM> considers to be normal. In the learning device <NUM>, representative communication feature amounts can be automatically extracted by using kernel herding, and the NW administrator can ascertain network tendencies based on the extracted feature amounts.

Also, in the detection system <NUM>, the evaluation device <NUM> creates a data set by generating a large number of pieces of data using the trained generative model. The evaluation device <NUM> uses kernel herding to extract representative points (see (<NUM>) in <FIG>) from sampled data from the VAE or the like (see (<NUM>) in <FIG>).

In this way, the evaluation device <NUM> can extract representative communication learned by the VAE. The data generated by the generative model corresponds to data that the detection system <NUM> considers to be normal when the generative model is used as a probability density estimator. By using kernel herding, the evaluation device <NUM> can more directly ascertain traffic feature amounts that the detection system <NUM> considers to be normal.

The NW administrator can ascertain a tendency of the traffic feature amounts for learning based on the representative points extracted by the learning device <NUM>. The kernel herding application method in the learning device <NUM> is useful when there is a desire to ascertain a network tendency via representative points of traffic feature amounts.

Also, the NW administrator can ascertain what kind of features of communication are actually considered by the generative model to be normal based on the representative points extracted by the evaluation device <NUM>. In other words, the NW administrator can ascertain whether the generative model can generate appropriate data. The kernel herding application method in the evaluation device <NUM> is useful when there is a desire to ascertain what traffic feature amounts are considered to be normal for the detection system <NUM> as a whole including the probability density estimator.

The NW administrator then evaluates the progress of the generative model using the difference between the representative points extracted by the learning device <NUM> and the representative points extracted by the evaluation device <NUM>. For example, if the difference between the representative points extracted by the learning device <NUM> and the representative points extracted by the evaluation device <NUM> is less than a predetermined value, it is deemed that the training of the generative model is proceeding appropriately, whereas if the difference is larger than the predetermined value, it is deemed that the training of the generative model is not progressing appropriately. As a result, the NW administrator can ascertain at the feature amount level whether or not the generative model has been properly trained.

[Learning device] Next, the configurations of devices in the detection system <NUM> will be described. First, the learning device <NUM> will be described. <FIG> is a diagram showing an example of the configuration of the learning device <NUM>. As shown in <FIG>, the learning device <NUM> includes a communication unit <NUM>, a storage unit <NUM>, and a control unit <NUM>.

The communication unit <NUM> is a communication interface for transmitting and receiving various types of information to and from other devices connected via a network or the like. The communication unit <NUM> is realized by an NIC (Network Interface Card) or the like, and performs communication with other devices (e.g., the detection device <NUM> and the evaluation device <NUM>) and the control unit <NUM> (described later) via a telecommunication line such as a LAN (Local Area Network) or the Internet. For example, the communication unit <NUM> connects to an external device via a network or the like, and receives an input of traffic packets that are to be used in learning.

The storage unit <NUM> is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk, and stores a processing program for operation of the learning device <NUM>, data used during execution of the processing program, and the like. The storage unit <NUM> includes a VAE model <NUM>.

The VAE model <NUM> is a generative model that learns feature amounts of communication data. The VAE model <NUM> learns traffic feature amounts for learning. The VAE model <NUM> is a probability density estimator and learns features of the probability density of communication data for learning. The VAE model <NUM> receives a certain data point xi and outputs an anomaly score that corresponds to that data. Letting the estimated value of probability density be p(xi), the anomaly score is an approximation of -logp(xi). Accordingly, the higher the anomaly score output by the VAE is, the higher the degree of anomaly of the communication data is.

The control unit <NUM> includes an internal memory for storing required data and programs that define various processing procedures, and executes various types of processing using such programs and data. For example, the control unit <NUM> is an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control unit <NUM> includes an acquisition unit <NUM>, a feature amount extraction unit <NUM>, and a model training unit <NUM>.

The acquisition unit <NUM> acquires a plurality of pieces of communication data for learning. Specifically, the acquisition unit <NUM> acquires a large number of packets for learning via the NW devices <NUM> that are used for learning.

The feature amount extraction unit <NUM> extracts feature amounts of the pieces of communication data acquired by the acquisition unit <NUM>. The feature amount extraction unit <NUM> performs statistical processing on a large number of packets for learning and generates traffic feature amounts, which are high-dimensional data.

The model training unit <NUM> trains the VAE model <NUM> using the traffic feature amounts that were extracted by the feature amount extraction unit <NUM>. The model training unit <NUM> also uses kernel herding to extract representative points of feature amounts of the pieces of communication data used in learning. The model training unit <NUM> includes a training unit <NUM>, a representative point extraction unit <NUM>, and a presentation unit <NUM>.

The training unit <NUM> trains the VAE model <NUM> with the feature amounts of the communication data extracted by the feature amount extraction unit <NUM>. The training unit <NUM> trains the VAE model <NUM> with probability density features of the communication data. The training unit <NUM> optimizes the parameters of the VAE model <NUM> using the traffic feature amounts generated by the feature amount extraction unit <NUM>. The training unit <NUM> outputs the trained VAE model <NUM> to the detection device <NUM> and the evaluation device <NUM> via the communication unit <NUM>.

The representative point extraction unit <NUM> uses kernel herding to extract representative points of feature amounts of the pieces of communication data for learning. The representative point extraction unit <NUM> uses kernel herding to extract representative points from the data set of traffic feature amounts for learning that were generated by the feature amount extraction unit <NUM>.

The presentation unit <NUM> presents the representative points of the feature amounts of the communication data for learning to the NW administrator by outputting the representative points extracted by the representative point extraction unit <NUM> to the user terminal <NUM> via the communication unit <NUM>.

[Detection device] Next, the detection device <NUM> will be described. <FIG> is a diagram showing an example of the configuration of the detection device <NUM>. As shown in <FIG>, the detection device <NUM> includes a communication unit <NUM>, a storage unit <NUM>, and a control unit <NUM>.

The communication unit <NUM> has functions similar to those of the communication unit <NUM> shown in <FIG>, and performs the input and output of information and communication with other devices (e.g., the learning device <NUM>).

The storage unit <NUM> has functions similar to those of the storage unit <NUM> shown in <FIG>. The storage unit <NUM> has the VAE model <NUM>. The VAE model <NUM> is a model that has been trained by the learning device <NUM>.

The control unit <NUM> has functions similar to those of the control unit <NUM> shown in <FIG>, and performs overall control of the detection device <NUM>. The control unit <NUM> functions as various processing units by the execution of various programs. The control unit <NUM> includes an acquisition unit <NUM>, a feature amount extraction unit <NUM>, and a detection unit <NUM>.

The acquisition unit <NUM> acquires communication data for which detection is to be performed. Specifically, the acquisition unit <NUM> acquires detection target packets via the NW devices <NUM> that capture packets of detection target traffic.

The feature amount extraction unit <NUM> has functions similar to those of the feature amount extraction unit <NUM>, and generates traffic feature amounts from the detection target packets that were acquired by the acquisition unit <NUM>.

The detection unit <NUM> uses the VAE model <NUM> to detect the presence or absence of an anomaly in the detection target traffic. The detection unit <NUM> inputs the traffic feature amounts generated by the feature amount extraction unit <NUM> to the VAE model <NUM>, and acquires an output anomaly score. If the anomaly score is higher than a predetermined value, the detection unit <NUM> detects that the detection target communication data is abnormal. Also, if the anomaly score is less than or equal to the predetermined value, the detection unit <NUM> detects that the detection target communication data is normal.

[Evaluation device] Next, the configuration of the evaluation device <NUM> will be described. <FIG> is a diagram showing an example of the configuration of the evaluation device <NUM>. As shown in <FIG>, the evaluation device <NUM> includes a communication unit <NUM>, a storage unit <NUM>, and a control unit <NUM>.

The control unit <NUM> has functions similar to those of the control unit <NUM> shown in <FIG>, and performs overall control of the evaluation device <NUM>. The control unit <NUM> functions as various processing units by the execution of various programs. The control unit <NUM> has a model evaluation unit <NUM>.

The model evaluation unit <NUM> presents, to the NW administrator, data for evaluating what kind of features of communication are actually considered to be normal by the generative model. The model evaluation unit <NUM> has a data generation unit <NUM>, a representative point extraction unit <NUM>, and a presentation unit <NUM>.

The data generation unit <NUM> generates pieces of data from the VAE model <NUM>, which is a generative model. The data generation unit <NUM> inputs noise to an intermediate layer of the VAE model <NUM>, and acquires an output distribution that corresponds to the noise from the output of the VAE model <NUM>.

The representative point extraction unit <NUM> uses kernel herding to extract representative points of the pieces of data generated by the data generation unit <NUM>.

The presentation unit <NUM> presents the representative points of the feature amounts of the data generated by the VAE model <NUM> to the NW administrator by outputting the representative points extracted by the representative point extraction unit <NUM> to the user terminal <NUM> via the communication unit <NUM>.

[Learning processing] Next, a learning method executed by the learning device <NUM> will be described. <FIG> is a flowchart showing a processing procedure of learning processing according to the embodiment.

As shown in <FIG>, the learning device <NUM> acquires a plurality of packets for learning (step S1) and extracts traffic feature amounts of the acquired packets (step S2).

The learning device <NUM> performs learning processing to train the VAE model <NUM> with the traffic feature amounts (step S3), and outputs the trained VAE model <NUM> to the detection device <NUM> and the evaluation device <NUM> (step S4).

Then, the learning device <NUM> uses kernel herding to extract representative points from the data set of the traffic feature amounts for learning (step S5), and outputs the extracted representative points to the user terminal <NUM> in order to present the representative points of the traffic feature amounts for learning to the NW administrator (step S6).

[Evaluation processing] Next, an evaluation method executed by the evaluation device <NUM> will be described. <FIG> is a flowchart showing a processing procedure of evaluation processing executed by the evaluation device <NUM>.

The evaluation device <NUM> generates pieces of data from the VAE model <NUM>, which is a generative model (step S11). The evaluation device <NUM> uses kernel herding to extract representative points of the data generated in step S11 (step S12).

The evaluation device <NUM> presents, to the NW administrator, the representative points of the feature amounts of the data generated by the VAE model <NUM> by outputting the representative points that were extracted in step S12 to the user terminal <NUM> (step S13).

[Working Example] In one example, the detection system <NUM> in the present embodiment can be applied to IoT device anomaly detection. <FIG> is a diagram illustrating an example of application of the detection system <NUM> according to the embodiment. As shown in <FIG>, the detection system <NUM> is provided in a network <NUM> to which a plurality of IoT devices <NUM> are connected. In this case, the detection system <NUM> collects traffic session information transmitted and received by the IoT devices <NUM>, learns the probability densities of normal traffic sessions, and detects an abnormal traffic session.

In the detection system <NUM>, the model training unit <NUM> receives packets for learning, and outputs, to the detection device <NUM> and the evaluation device <NUM>, a learned VAE model that has been trained with traffic feature amounts of the received packets.

[Experiment] Representative points were actually extracted using kernel herding from a data set of traffic feature amounts for learning. Specifically, a data set was created so as to include a mixture of two types of communication (temperature information transmission (<NUM> sessions) by MQTT (Message Queue Telemetry Transport) and video distribution (<NUM> sessions) by RTMP (Real-Time Messaging Protocol)), and kernel herding was used to extract representative points. The results are shown in Table <NUM>.

The first row of Table <NUM> shows the results of extracting representative points of communication by MQTT. Checking the actual data set shows that about <NUM>% of the communication is <NUM> bytes or <NUM> bytes of uplink communication, the number of packets is <NUM>, and the average packet size is <NUM>×<NUM> bytes, which closely matches representative points extracted manually.

The second row of Table <NUM> shows the results of extracting representative points of RTMP communication. When the actual data was checked visually, considerable variation was seen, but the average session duration was about <NUM> seconds, and the average upstream packet size was about <NUM>×<NUM> bytes, which closely matches representative points extracted manually.

In this way, it was confirmed that traffic feature amounts extracted manually (specifically, by an experienced system manager) closely matched traffic feature amounts automatically extracted using kernel herding.

[Effects of embodiment] As described above, the learning device <NUM> according to the present embodiment extracts feature amounts of a plurality of pieces of communication data and trains a generative model with the feature amounts of the communication data.

The learning device <NUM> also uses kernel herding to extract representative points of the feature amounts of the communication data, and outputs the extracted representative points to the user terminal <NUM> so as to provide the NW administrator with data for ascertaining a tendency of the traffic feature amounts for learning.

As a result, based on the representative points of the feature amount of the communication data, the NW administrator can ascertain the feature amounts that the VAE model <NUM> considers to be normal, and furthermore can ascertain a network tendency based on the representative points of the feature amounts of the communication data.

Also, as shown in the above-described experimental results, traffic feature amounts automatically extracted using kernel herding according to the present embodiment closely matched traffic feature amounts extracted manually. Therefore, according to the present embodiment, representative points of traffic feature amounts for learning can be appropriately extracted using kernel herding instead of being extracted manually, thus making it possible to alleviate the burden on the system administrator. Also, according to the present embodiment, representative points of traffic feature amounts for learning are appropriately extracted and output as data. For this reason, such data can be used by anyone to analyze network feature amounts based on the feature amounts, and it is possible to reduce the amount of skilled worker labor.

Also, the evaluation device <NUM> according to the present embodiment generates pieces of data from the VAE model <NUM>, uses kernel herding to extract representative points of the generated data, and outputs the extracted representative points to the user terminal <NUM>.

Based on the representative points extracted by the evaluation device <NUM>, the NW administrator can ascertain what kind of features of communication are actually considered to be normal by the VAE model <NUM>. In other words, the NW administrator can ascertain whether or not the VAE model <NUM> can generate good data.

Therefore, according to the present embodiment, the NW administrator can qualitatively ascertain the traffic feature amounts that are considered to be normal for the detection system <NUM> as a whole, including the VAE model <NUM>.

Then by using the difference between the representative points extracted by the learning device <NUM> and the representative points extracted by the evaluation device <NUM>, the NW administrator can ascertain the progress of the VAE model <NUM> at the feature amount level.

[System configuration, etc.] The constituent elements of the illustrated devices are functional concepts and do not necessarily need to be physically configured as shown in the figures. In other words, the specific manner of distribution/integration of the devices is not limited to the examples shown in the figures, and all or some of the devices can be functionally or physically distributed or integrated into any number of devices according to various load and usage conditions. Also, the processing functions performed by the devices can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware through wired logic.

Further, all or some of the processing described in the present embodiment as being performed automatically can be performed manually, and all or some of the processing described in the present embodiment as being performed manually can be performed automatically using a known method. Also, the processing procedures, control procedures, specific names, and information including the various types of data and parameters shown in the above description and drawings can be changed as desired unless otherwise specified.

[Program] <FIG> is a diagram showing an example of a computer in which the detection system <NUM> is realized by the execution of a program. The computer <NUM> includes a memory <NUM> and a CPU <NUM>, for example. The computer <NUM> also includes a hard disk drive interface <NUM>, a disk drive interface <NUM>, a serial port interface <NUM>, a video adapter <NUM>, and a network interface <NUM>. These parts are connected to each other by a bus <NUM>.

The memory <NUM> includes a ROM (Read Only Memory) <NUM> and a RAM <NUM>. The ROM <NUM> stores a boot program such as a BIOS (Basic Input Output System), for example. The hard disk drive interface <NUM> is connected to a hard disk drive <NUM>. The disk drive interface <NUM> is connected to a disk drive <NUM>. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive <NUM>. The serial port interface <NUM> is connected to a mouse <NUM> and a keyboard <NUM>, for example. The video adapter <NUM> is connected to a display <NUM>, for example.

The hard disk drive <NUM> stores an OS (Operating System) <NUM>, an application program <NUM>, a program module <NUM>, and program data <NUM>, for example. Specifically, a program that defines the processing of the detection system <NUM> is implemented as the program module <NUM> that describes computer-executable code. The program module <NUM> is stored in the hard disk drive <NUM>, for example. For example, the program module <NUM> for executing processing similar to the functional configuration in the detection system <NUM> is stored in the hard disk drive <NUM>. Note that the hard disk drive <NUM> may be replaced with an SSD (Solid State Drive).

Also, setting data used in the processing of the above-described embodiment is stored as the program data <NUM> in the memory <NUM> or the hard disk drive <NUM>, for example. The CPU <NUM> reads the program module <NUM> and the program data <NUM> stored in the memory <NUM> or the hard disk drive <NUM> to the RAM <NUM> and executes the program as needed.

Note that the program module <NUM> and the program data <NUM> are not limited to being stored in the hard disk drive <NUM>, and may be stored in a removable storage medium or the like and read by the CPU <NUM> via the disk drive <NUM> or the like. Alternatively, the program module <NUM> and the program data <NUM> may be stored in another computer connected via a network (LAN or WAN (Wide Area Network), for example). The program module <NUM> and the program data <NUM> may be read by the CPU <NUM> from another computer via the network interface <NUM>.

Claim 1:
A learning device (<NUM>) comprising:
an acquisition unit (<NUM>) configured to acquire a plurality of pieces of communication data for learning;
a feature amount extraction unit (<NUM>) configured to extract feature amounts of the communication data;
a training unit (<NUM>) configured to train a generative model with the feature amounts of the communication data, the generative model for use by a detection device (<NUM>) to detect the presence or absence of an anomaly in a detection target traffic in a network (N);
characterized by
a first representative point extraction unit (<NUM>) configured to extract representative points of the feature amounts of the communication data using kernel herding; and
an output unit (<NUM>) configured to output the representative points extracted by the first representative point extraction unit to a user terminal (<NUM>) as data for evaluating whether the generative model has been properly trained;
a generation unit (<NUM>) configured to generate a plurality of pieces of data from the generative model, and
a second representative point extraction unit (<NUM>) configured to extract representative points of the generated data using kernel herding,
wherein the output unit (<NUM>) outputs the representative points extracted by the second representative point extraction unit, and
wherein, if a difference between the representative points extracted by the first representative point extraction unit and the representative points extracted by the second representative point extraction unit is larger than a predetermined value, it is determined that the training of the generative model is not progressing appropriately and wherein, if a difference between the representative points extracted by the first representative point extraction unit and the representative points extracted by the second representative point extraction unit is smaller than the predetermined value, it is determined that the training of the generative model is progressing appropriately.