Patent Description:
In recent years, in some cases, attackers have used fake antivirus software to hack a user's terminal or steal personal information. Fake antivirus software is a kind of malware disguised as antivirus software that removes malware (generic term of malicious software) from a user's terminal. Conventionally, attackers make a false virus infection alert or a web advertisement that purports to provide speed-up of a terminal be displayed on a web page to psychologically lead a user to install fake antivirus software.

In addition to deceiving users using a false virus infection alert or web advertisement, in some other cases, attackers prepare a web page that presents a false virus removal method to urge a user to install fake antivirus software. Such web page is referred to as "false removal information presentation site". False removal information presentation sites are targeted for users who have already suffered security damage such as infection with malware or access to a malicious site. The false removal information presentation sites present a false method to cope with such security damage to deceive users. The false removal information presentation sites suggest installation of fake antivirus software and the deceived users download and install the fake antivirus software by himself/herself.

As an existing method for detecting a malicious web page for distributing fake antivirus software, for example, there is a method in which a malicious web page is detected via graph-based clustering using information on registration of domain names and information on networks such as IP addresses as features (see, for example, Non-Patent Literature <NUM>). Malicious web pages to be detected by the method include a web page that makes an attack on a vulnerability existing in a user's system and a web page that displays a false infection alert to deceive a user.

Also, methods in which web pages are accessed using a web browser to extract characteristics particular to malicious web pages such as those for technical support frauds or survey frauds and identify such web pages have been known (see Non-Patent Literatures <NUM> and <NUM>). Crawling of the identified malicious web pages through access using a web browser sometimes leads to a malicious web page that displays a false infection alert to distribute fake antivirus software.

Patent Literature <NUM> relates to a system for prevention of phishing attacks and more specifically for a phishing detection system featuring real time retrieval, analysis and assessment of phishing webpages. Patent Literature <NUM> relates to network monitoring and more particularly to techniques for identifying potential malware domain names.

The aforementioned existing techniques are techniques that detect and efficiently collect malicious web pages making an attack on a vulnerability in a system to install fake antivirus software on a user's system or displaying a false infection alert to deceive a user into installing fake antivirus software by himself/herself. However, false removal information presentation sites that do not make an attack on a vulnerability in a system to install fake antivirus software but deceive a user into installation of fake antivirus software via a psychological leading approach.

Also, unlike the conventional methods in which a fake infection alert is displayed to deceive a user, the psychological leading approach is targeted for a user who has actually suffered security damage such as malware infection and presents a solution to such security damage to deceive the user. Therefore, from the perspective of method of attack, false removal information presentation sites are different from malicious web pages that the existing techniques tackle with, and thus, cannot be identified by the existing techniques that capture characteristics particular to the methods of attack by such malicious web pages to detect malicious web pages.

In other words, the conventional methods have the problem of being unable to detect a web page targeted for users who have suffered security damage, the web page presenting a solution to such damage to the users to urge the users to install fake antivirus software via a psychological leading approach.

The present invention has been made in view of the above and an object of the present invention is to detect a false removal information presentation site, using web page information acquired when a web page was accessed using a web browser, the false removal information presentation site being a malicious web page that presents false removal information to a user who has already suffered security damage to deceive the user into installation of fake antivirus software.

The present invention provides the effect of enabling detecting a false removal information presentation site that is a malicious web page urging installation of fake antivirus software.

An embodiment of a learning apparatus, a detection apparatus, a learning method, a detection method, a learning program and a detection program according to the present application will be described in detail below with reference to the drawings. Note that the embodiment is not intended to limit the learning apparatus, the detection apparatus, the learning method, the detection method, the learning program and the detection program according to the present application.

[Embodiment]An embodiment of the present invention will be described. <FIG> is a diagram illustrating an example of a configuration of a detection system according to the embodiment. As illustrated in <FIG>, a detection system <NUM> according to the embodiment includes a learning apparatus <NUM> and a detection apparatus <NUM>. The learning apparatus <NUM> generates a training model for detecting that a web page is a false removal information presentation site. More specifically, the learning apparatus <NUM> receives an input of information relating to a web page (hereinafter referred to as "web page information"), the web page information being acquired when accessing the web page using a web browser.

The learning apparatus <NUM> generates a training model using, as training data, any one feature or a plurality of features from among a word/phrase appearance frequency feature, an image appearance frequency feature, HTML features and a communication log feature, the features being extracted from the web page information.

The detection apparatus <NUM> receives the training model generated by the learning apparatus <NUM>, and detects that a web page is a false removal information presentation site, using the training model. More specifically, the detection apparatus <NUM> receives an input of web page information acquired when a web page was accessed using a web browser. Using any one feature or a plurality of features from among a word/phrase appearance frequency feature, an image appearance frequency feature, HTML features and a communication log feature, the features being extracted from the web page information, as input data, the detection apparatus <NUM> inputs the input data to the training model learned in advance and detects that the web page is a false removal information presentation site, according to an output result of the training model.

[Configurations of learning apparatus and detection apparatus] Next, a configuration of the learning apparatus <NUM> will be described. <FIG> is a diagram illustrating an example of a configuration of the learning apparatus illustrated in <FIG>. The learning apparatus <NUM> includes a web page information input unit <NUM>, a word/phrase appearance frequency feature extraction unit (first feature extraction unit) <NUM>, an image appearance frequency feature extraction unit (second feature extraction unit) <NUM>, an HTML feature extraction unit (third feature extraction unit) <NUM>, a communication log feature extraction unit (fourth feature extraction unit) <NUM>, a learning unit <NUM> and a storage unit <NUM>.

Next, a configuration of the detection apparatus <NUM> will be described. <FIG> is a diagram illustrating an example of a configuration of the detection apparatus illustrated in <FIG>. The detection apparatus <NUM> includes a web page information input unit <NUM>, a word/phrase appearance frequency feature extraction unit <NUM>, an image appearance frequency feature extraction unit <NUM>, an HTML feature extraction unit <NUM>, a communication log feature extraction unit <NUM>, a detection unit <NUM>, an output unit <NUM> and a storage unit <NUM>.

The respective units of the learning apparatus <NUM> will be described below. The web page information input unit <NUM> receives an input of information relating to a web page, whether or not the web page is a false removal information presentation site being known, the false removal information presentation site presenting a false virus removal method. More specifically, the web page information input unit <NUM> accesses a web page using a web browser and receives an input of web page information acquired from the web browser. For example, the web page information input unit <NUM> receives inputs of web page information pieces of a plurality of known false removal information presentation sites and web page information pieces of those other than the plurality of false removal information presentation sites. Here, web page information is information that can be acquired from a web browser when a web page is accessed using the web browser.

Web page information acquired by the web page information input unit <NUM> includes the items illustrated in <FIG> is a diagram illustrating an example of web page information that can be acquired from a web browser when a web page is accessed using the web browser. In <FIG>, examples of items included in web page information are illustrated. Examples of items of web page information include an image, an HTML source code and a communication log of a web page that have been acquired from a web browser when the web page was accessed using the web browser. The web page information can be acquired by managing the access of the web browser using, e.g., a browser extension introduced to the web browser or a debug tool for a developer of the web browser.

An example of a communication log of a web page will be described using the example in <FIG> is a diagram illustrating an example of communication log information, which is a part of web page information. Examples of items of the communication log include a time stamp of a time of occurrence of a communication, a communication destination URL, a communication destination IP address, an HTML referrer representing a communication destination accessed immediately before and an HTML status code representing a content of HTML communication.

The word/phrase appearance frequency feature extraction unit <NUM> extracts communication destination information and text information from the web page information and measures the number of times a word or a phrase included in the communication destination information or the text information appear, as a word/phrase-related feature. In other words, with a view to capture linguistic characteristics particular to false removal information presentation sites, the linguistic characteristics being included in the web page information, the word/phrase appearance frequency feature extraction unit <NUM> measures a frequency of appearance of a word or a phrase as a feature of the web page, the feature being included in the web page information, and generates a feature vector. Examples of targets for measurement are illustrated in <FIG> is a diagram illustrating examples of targets for which a word/phrase appearance frequency is measured.

As illustrated in <FIG>, the word/phrase appearance frequency feature extraction unit <NUM> measures a frequency of appearance of words and phrases, for any one measurement target or each of a plurality of measurement targets from among a title, text, a domain name and a URL path. The word/phrase appearance frequency feature extraction unit <NUM> extracts a title and text displayed on a web page from HTML source codes of the web page. The title can be acquired by extracting a character string in a title tag. The text can be acquired by extracting character strings in respective HTML tags and excluding script tags each representing a JavaScript (registered trademark) source code to be processed by the web browser and character strings in meta tags each representing meta information of the web page.

Also, the word/phrase appearance frequency feature extraction unit <NUM> acquires a communication destination URL from a communication log and acquires a domain name and a URL path from the communication destination URL. Words and phrases that are targets of appearance frequency measurement are set in advance for each of categories each including words and phrases having a same role. <FIG> is a diagram illustrating examples of words and phrases for which a frequency of appearance is measured. In the example in <FIG>, examples of words and phrases and categories of the words and phrases are illustrated. The word/phrase appearance frequency feature extraction unit <NUM> extracts frequently appearing words and phrases, from known false removal information presentation sites, for any one category or each of a plurality of categories from among "method", "removal", "threat" and "device" in advance, and measures frequencies of appearance of the words and phrases for each category.

<FIG> illustrates an example of a feature vector of features extracted by the word/phrase appearance frequency feature extraction unit <NUM>. <FIG> is a diagram illustrating an example of a feature vector of word/phrase appearance frequencies. For each measurement target, the word/phrase appearance frequency feature extraction unit <NUM> generates a feature vector by measuring a frequency of appearance of the words and phrases set for each category and vectorizing numerical values of the frequencies.

The image appearance frequency feature extraction unit <NUM> extracts image information from the web page information and measures the number of times an image included in the image information appears, as an image-related feature. In other words, with a view to capture image characteristics particular to false removal information presentation sites, the image characteristics being included in the web page information, the image appearance frequency feature extraction unit <NUM> measures a frequency of appearance of an image as a feature of the web page, the feature being included in the web page information, and generates a feature vector. The image appearance frequency feature extraction unit <NUM> measures a frequency of appearance of image data included within an image of the web page drawn by the web browser. An example of an image of a web page of a false removal information presentation site is illustrated in <FIG> is a diagram illustrating an example of an image of a web page of a false removal information presentation site.

For the image data, an image that frequently appears in false removal information presentation sites is set in advance, for each of categories. Examples of the categories for the image data are illustrated in <FIG> is a diagram illustrating examples of categories of image data for which a frequency of appearance is measured. The "fake certification logo" indicates a logo image of a security vendor company or an OS vendor company abused by a false removal information presentation site in order to assert safety of the web page.

The "package of fake antivirus software" indicates an image of a package of a fake antivirus software product. The "download button" indicates a download button for urging download of fake antivirus software. The image appearance frequency feature extraction unit <NUM> extracts image regions of HTML elements corresponding to an a tag or an img tag in the HTML source codes from the web page and measures a degree of similarity to image data set in advance. For a method for similarity degree measurement, an image hashing algorithm such as perceptual hash can be used.

In <FIG>, an example of a feature vector of features extracted by the image appearance frequency feature extraction unit <NUM> are illustrated. <FIG> is a diagram illustrating an example of a feature vector of image appearance frequencies. The image appearance frequency feature extraction unit <NUM> generates a feature vector by measuring a frequency of appearance of the relevant image for each of image data categories and vectorizing numerical values of the frequencies.

The HTML feature extraction unit <NUM> extracts HTML source code information from the web page information and measures the number of times a link destination appears and structure information that are included in the HTML information, as HTML source code-related features. In other words, with a view to capture HTML structural characteristics particular to false removal information presentation site, the structural characteristics being included in the web page information, the HTML feature extraction unit <NUM> measures a frequency of appearance of each of an HTML tag and a URL of a link destination as features of the web page, the features being included in the web page information, and generates respective feature vectors. The HTML feature extraction unit <NUM> measures a frequency of appearance of any one of or each of a plurality of HTML tags from among normally-used HTML tags from the HTML source codes.

Also, the HTML feature extraction unit <NUM> measures a frequency of appearance of a URL of a link destination in the web page, the URL being included in an a tag. Link destination URLs of external sites frequently appearing in false removal information presentation sites are set in advance. In <FIG>, an example of a feature vector of features of frequencies of appearance of HTML tags extracted by the HTML feature extraction unit <NUM> are illustrated. <FIG> is a diagram illustrating an example of a feature vector of HTML tag appearance frequencies. Also, in <FIG>, an example of a feature vector of features of frequencies of appearance of link destination URLs extracted by the HTML feature extraction unit <NUM> is illustrated. <FIG> is a diagram illustrating an example of a feature vector of link destination URL appearance frequencies. The HTML feature extraction unit <NUM> generates a feature vector by measuring frequencies of appearance of HTML tags and frequencies of appearance of link destination URLs and vectorizing numerical values of the frequencies.

The communication log feature extraction unit <NUM> extracts communication log information from the web page information and measures the number of times a communication destination included in the communication log information appears, as a communication log-related feature. In other words, with a view to capture communication characteristics particular to false removal information presentation sites, the communication characteristics being included in the web page information, the communication log feature extraction unit <NUM> measures a frequency of appearance of a communication destination URL as a feature of the web page, the feature being included in the web page information, and generates a feature vector. The communication log feature extraction unit measures a frequency of appearance of a communication destination URL, from contents of communications with an external site from among communications occurred when the web page was accessed using the web browser. URLs of external sites frequently included in communications when false removal information presentation sites are accessed are set in advance.

In <FIG>, an example of a feature vector of features of frequencies of appearance of communication destination URLs, the features being extracted by the HTML feature extraction unit. <FIG> is a diagram illustrating an example of a feature vector of communication destination URL appearance frequencies. The communication log feature extraction unit <NUM> generates a feature vector by measuring frequencies of appearance of respective communication destination URLs and vectorizing numerical values of the frequencies.

The learning unit <NUM> generates a training model using, as training data, any one feature or a plurality of features from among the word/phrase-related feature, the image-related feature, the HTML source code-related feature and the communication log-related feature, the feature or the features being included in the information relating to the web page. For example, the learning unit <NUM> generates a training model using, as training data, a feature vector of any one feature or an integration of a plurality of features from among the word/phrase appearance frequency feature, the image appearance frequency feature, the HTML features and the communication log feature, which have been extracted from the web page information.

In <FIG>, an example of training data resulting from integration of the word/phrase appearance frequency feature, the image appearance frequency feature, the HTML features and the communication log feature, which have been extracted from the web page information is illustrated. <FIG> is a diagram illustrating an example of a feature vector resulting from integration of the features. The learning unit <NUM> generates a training model using a supervised machine learning method in which two-class classification is possible, and stores the training model in the storage unit <NUM>. Examples of the supervised machine learning method in which two-class classification is possible include, but are not limited to, a support-vector machine and a random forest. The learning unit <NUM> generates training data by extracting features from known false removal information presentation sites and other web pages, and generates a training model using the supervised machine learning method.

Next, the respective units of the detection apparatus <NUM> will be described below. Note that the web page information input unit <NUM>, the word/phrase appearance frequency feature extraction unit <NUM>, the image appearance frequency feature extraction unit <NUM>, the HTML feature extraction unit <NUM> and the communication log feature extraction unit <NUM> perform processing that is similar to the above-described processing in the web page information input unit <NUM>, the word/phrase appearance frequency feature extraction unit <NUM>, the image appearance frequency feature extraction unit <NUM>, the HTML feature extraction unit <NUM> and the communication log feature extraction unit <NUM>, respectively, and thus, brief description will be provided with overlapping description omitted.

The web page information input unit <NUM> receives an input of information relating to a web page that is a detection target. More specifically, the web page information input unit <NUM> accesses a web page using a web browser and receives an input of web page information acquired from the web browser.

The word/phrase appearance frequency feature extraction unit <NUM> extracts communication destination information and text information from the web page information and measures the number of times a word or a phrase included in the communication destination information or the text information appears, as a word/phrase-related feature. The image appearance frequency feature extraction unit <NUM> extracts image information from the web page information and measures the number of times an image included in the image information appears, as an image-related feature.

The HTML feature extraction unit <NUM> extracts HTML source code information from the web page information and measures the number of times a link destination appears and structure information that are included in the HTML information, as HTML source code-related features. The communication log feature extraction unit <NUM> extracts communication log information from the web page information and measures the number of times a communication destination included in the communication log information appears, as a communication log-related feature.

Using any one feature or a plurality of features from among the word/phrase-related feature, the image-related feature, the HTML source code-related feature and the communication log-related feature, the feature or the features being included in the information relating to the web page, as input data, the detection unit <NUM> inputs the input data to a training model learned in advance, and detects that the detection target web page is a false removal information presentation site, according to an output result of the training model.

More specifically, the detection unit <NUM> reads a training model from the storage unit <NUM>, and as with the learning unit <NUM>, inputs input data to the training model learned in advance, using, as the input data, a feature vector extracted from the web page information, and detects that the web page is a false removal information presentation site, according to an output result of the training model. Note that the detection unit <NUM> not only determines that the detection target web page is a false removal information presentation site, but also may calculate a numerical value indicating a probability of the detection target web page being a false removal information presentation site, according to an output result of the training model.

The output unit <NUM> outputs a result of the detection by the detection unit <NUM>. For example, the output unit <NUM> may output a message indicating that the detection target web page is a false removal information presentation site or may output a message indicating a probability of the detection target web page being a false removal information presentation site. Note that a mode of the output is not limited to messages and may be any of modes such as images and sounds.

[Procedures of learning processing and detection processing] Next, procedures of learning processing and detection processing according to the embodiment will be described with reference to <FIG> and <FIG>. <FIG> is a diagram illustrating a flowchart of training model generation processing. <FIG> is a diagram illustrating a flowchart of detection processing.

As illustrated in <FIG>, the web page information input unit <NUM> of the learning apparatus <NUM> receives an input of web page information of a web page, whether or not the web page is a false removal information presentation site being known (step S101). Then, the word/phrase appearance frequency feature extraction unit <NUM> performs processing for extraction of a word/phrase appearance frequency feature (step S102). More specifically, the word/phrase appearance frequency feature extraction unit <NUM> performs processing for extracting communication destination information and text information from the web page information and measures the number of times a word or a phrase included in the communication destination information or the text information appears, as a word/phrase-related feature.

Subsequently, the image appearance frequency feature extraction unit <NUM> performs processing for extraction of an image appearance frequency feature (step S103). More specifically, the image appearance frequency feature extraction unit <NUM> extracts image information from the web page information and measures the number of times an image included in the image information appears, as an image-related feature. Then, the HTML feature extraction unit <NUM> performs processing for extraction of an HTML feature (step S104). More specifically, the HTML feature extraction unit <NUM> extracts HTML source code information from the web page information and measures the number of times a link destination appears and structure information that are included in the HTML information, as HTML source code-related features.

Subsequently, the communication log feature extraction unit <NUM> performs extraction of a communication log feature (step S105). More specifically, the communication log feature extraction unit <NUM> extracts communication log information from the web page information and measures the number of times a communication destination included in the communication log information appears, as a communication log-related feature. Subsequently, the learning unit <NUM> generates training data by integrating the respective features (step S106). Then, the learning unit <NUM> generates a training model according to a supervised machine learning method (step S107).

Also, as illustrated in <FIG>, the web page information input unit <NUM> of the detection apparatus <NUM> receives an input of web page information of a web page that is a detection target (step S201). Then, the word/phrase appearance frequency feature extraction unit <NUM> performs processing for extraction of a word/phrase appearance frequency feature (step S202). More specifically, the word/phrase appearance frequency feature extraction unit <NUM> performs processing for extracting communication destination information and text information from the web page information and measures the number of times a word or a phrase included in the communication destination information or the text information appears, as a word/phrase-related feature.

Subsequently, the image appearance frequency feature extraction unit <NUM> performs processing for extraction of an image appearance frequency feature (step S203). More specifically, the image appearance frequency feature extraction unit <NUM> extracts image information from the web page information and measures the number of times an image included in the image information appears, as an image-related feature. Then, the HTML feature extraction unit <NUM> performs processing for extraction of an HTML feature (step S204). More specifically, the HTML feature extraction unit <NUM> extracts HTML source code information from the web page information and measures the number of times a link destination appears and structure information that are included in the HTML information, as HTML source code-related features.

Subsequently, the communication log feature extraction unit <NUM> performs extraction of a communication log feature (step S205). More specifically, the communication log feature extraction unit <NUM> extracts communication log information from the web page information and measures the number of times a communication destination included in the communication log information appears, as a communication log-related feature.

Then, the detection unit <NUM> generates input data by integrating the respective features (step S206). Subsequently, the detection unit <NUM> inputs the input data to a learned training model and detects that the web page is a false removal information presentation site (step S207).

[Effects of Embodiment] As above, the learning apparatus <NUM> according to the first embodiment receives an input of information relating to a web page, whether or not the web page is a false removal information presentation site being known, the false removal information presentation site presenting a false virus removal method, and generates a training model using, as training data, any one feature or a plurality of features from among a word/phrase-related feature, an image-related feature, an HTML source code-related feature and a communication log-related feature, the feature or the features being included in the information relating to the web page.

Also, the detection apparatus <NUM> receives an input of information relating to a web page, inputs input data to a training model learned in advance, using, as the input data, any one feature or a plurality of features from among a word/phrase-related feature, an image-related feature, an HTML source code-related feature and a communication log-related feature, the feature or the features being included in the information relating to the web page, and detects that the web page is a false removal information presentation site, according to an output result of the training model.

Therefore, the detection system <NUM> according to the embodiment captures characteristics particular to false removal information presentation sites from web page information acquired from a web browser by analyzing linguistic characteristics, image characteristics, HTML structural characteristics, link destination characteristics and communication destination characteristics, and thus enables highly accurate detection of a false removal information presentation site that cannot be detected by the conventional techniques.

In other words, the detection system <NUM> captures linguistic, image and HTML structure characteristics of a false removal information presentation site, using web page information acquired when a web page was accessed using a web browser, the false removal information presentation site being a malicious web page presenting a false coping method to a user who has already suffered security damage, from the perspective of psychological approach to the user and a system structure accompanying such psychological approach, and provides the effect of enabling detecting a false removal information presentation site from an input arbitrary web page.

[System Configuration, Etc. ] Also, the components of the illustrated apparatuses are those based on functional concepts and do not necessarily need to be physically configured as illustrated in the figures. In other words, specific forms of distribution and integration in each of the apparatuses is not limited to those illustrated in the figures, and the specific forms can be fully or partly configured in such a manner as to be functionally or physically distributed or integrated in arbitrary units according to, e.g., various types of loads and/or use conditions. Furthermore, an entirety or an arbitrary part of each of processing functions executed in each of apparatuses can be implemented by a CPU and a program to be analyzed and executed by the CPU or can be implemented in the form of hardware using wired logic.

Also, from among the processes described in the present embodiment, those that have been described as being automatically performed can be fully or partly manually performed and those that have been described as being manually performed can be fully or partly automatically performed via a known method. In addition, the processing procedures, the control procedures, the specific names and information including various data and parameters included in the above description or the drawings can arbitrarily be changed except as specifically noted otherwise.

[Program] <FIG> is a diagram illustrating a computer that executes a program. <FIG> illustrates an example of a computer in which the learning apparatus <NUM> or the detection apparatus <NUM> is implemented by execution of a program. A computer <NUM> includes, for example, a memory <NUM> and a CPU <NUM>. The computer <NUM> also includes a hard disk drive interface <NUM>, a disk drive interface <NUM>, a serial port interface <NUM>, a video adapter <NUM> and a network interface <NUM>. These components are connected via a bus <NUM>.

The memory <NUM> includes a ROM (read-only memory) <NUM> and a RAM <NUM>. The ROM <NUM> stores, for example, a boot program such as a BIOS (basic input/output system). The hard disk drive interface <NUM> is connected to a hard disk drive <NUM>. The disk drive interface <NUM> is connected to a disk drive <NUM>. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted in the disk drive <NUM>. The serial port interface <NUM> is connected to, for example, a mouse <NUM> and a keyboard <NUM>. The video adapter <NUM> is connected to, for example, a display <NUM>.

The hard disk drive <NUM> stores, for example, an OS <NUM>, an application program <NUM>, a program module <NUM> and program data <NUM>. In other words, programs prescribing the processes in the learning apparatus <NUM> or the detection apparatus <NUM> are implemented in the form of the program module <NUM> in which computer executable codes are written. The program module <NUM> is stored on, for example, the hard disk drive <NUM>. For example, a program module <NUM> for executing processes that are similar to those performed by the functional components in the apparatus is stored on the hard disk drive <NUM>. Note that the hard disk drive <NUM> may be substituted by an SSD (solid state drive).

Also, data used in the processes in the above-described embodiment are stored on, for example, the memory <NUM> or the hard disk drive <NUM> as the program data <NUM>. Then, the CPU <NUM> reads the program module <NUM> or the program data <NUM> stored on the memory <NUM> or the hard disk drive <NUM> onto the RAM <NUM> as necessary and executes the program module <NUM> or the program data <NUM>.

Claim 1:
A detection system (<NUM>) including a detection apparatus (<NUM>) and a learning apparatus (<NUM>),
the learning apparatus (<NUM>) comprising:
a first input unit (<NUM>) configured to receive a first input of first information relating to a web page, whether or not the web page is a malicious site being known, the malicious site presenting a false virus removal method; and
a learning unit (<NUM>) configured to generate a training model using, as training data, any one feature or a plurality of features from among a word/phrase-related feature, an image-related feature, an HTML source code-related feature and a communication log-related feature, the feature or the features being included in the first information relating to the web page, and
the detection apparatus (<NUM>) comprising:
a second input unit (<NUM>) configured to receive a second input of second information relating to a web page; and
a detection unit (<NUM>) configured to input input data to the training model, using, as the input data, any one feature or a plurality of features from among a word/phrase-related feature, an image-related feature, an HTML source code-related feature and a communication log-related feature, the feature or the features being included in the second information relating to the web page, and detect that the web page is a malicious site presenting a false virus removal method, according to an output result of the training model.