Patent ID: 11936673
Assignee: GROUP IB, LTD
Field: Computer technology (Electrical engineering)
Classification: CPC G  H | IPC G  H

Claim 9:
10. A method for detecting harmful content on a network using one or more classifiers, the method comprising training the one or more classifiers based on a respective training set of data by executing the steps of:
receiving a plurality of training HTML documents;
converting each one of the plurality of training HTML documents, to a respective training text of a plurality of training texts, the converting comprising extracting a respective text portion from an HTML body of each one of the plurality of training HTML document;
normalizing a given one of the plurality of training texts, the normalizing comprising transforming each word of the given one of the plurality of training texts to a respective canonic form thereof, thereby generating a respective plurality of training tokens associated with the given one of the plurality of training texts;
determining, for each one of the respective plurality of training tokens, a respective significance parameter,
the respective significance parameter associated with a given one of the respective plurality of training tokens being determined based on a product of multiplication of a local frequency of occurrence and an inverse value of a global frequency of occurrence associated therewith,
the local frequency of occurrence being a frequency of occurrence of the given one of the respective plurality of training tokens within the given one of the plurality training texts associated with a respective one of the plurality of training HTML documents; and
the global frequency of occurrence being a frequency of occurrence of the given one of the respective plurality of training tokens within the plurality of training texts of the plurality of training HTML documents;

aggregating respective pluralities of training tokens of the plurality of training HTML documents into a training token matrix of the respective training set of data, wherein:
a given row of the training token matrix is representative of a respective one of the plurality of training HTML documents;
a given column of the training token matrix is representative of a respective training token; and
a given element of the training token matrix is representative of a respective significance parameter of the respective token in the respective one of the plurality of training HTML documents;

training the one or more classifiers based on the respective training set of data;
using the one or more classifiers to determine a likelihood parameter that a given HTML document contains the harmful content.