# OWGC-HMC: An Online Web Genre Classification Model Based on Hierarchical Multilabel Classification

Authors: Guozhong Dong; Weizhe Zhang; Rahul Yadav; Xin Mu; Zhili Zhou
Journal: Security and Communication Networks
 (2022)

---

## Abstract

Web genre plays an important role in focused crawling, web link analysis, and contextual advertising. In this paper, web genre is defined as the functional purpose and the information type contained in the website. The intelligent classification of web genre can predict the content and functional type of website. However, there are several critical challenges to solve the web genre classification problem: lack of web genre classification dataset and efficient web genre classification mechanism. To improve web genre classification performance, we crawled Chinese websites of different web genres and converted crawled data into a hierarchical multilabel classification dataset. A website knowledge graph is constructed based on the relationship of website and meta tag features. Using entity features extracted from the knowledge graph, we propose an online web genre classification model based on hierarchical multilabel classification (OWGC-HMC) to mine the functional purpose of the corresponding website. Experimental results show that our OWGC-HMC model can mine hierarchical multilabel structure of web genre and outperform other web genre classification methods.


---

## Body

## 1. Introduction

Web classification [1] is the process of assigning websites to one or more classification labels. Web classification plays an important role in content recommendation [2–4] and contextual search [5–8]. According to different types of classification label, web classification can be divided into different classification problems such as topic classification [9, 10] and genre classification [11, 12]. The intelligent recognition of web genre can predict the content and functional type of website, facilitating web retrieval based on keywords. Web genre can be understood as functional categories of website such as online shopping, news media, government organization, resource download, and specialized information search. When a user is searching with keywords and web genres, the user can easily find search results about web genres [13]. Researchers have proposed different definitions of web genre and investigated various genre classification algorithms [14]. In this paper, web genre is defined as the functional purpose and the information type contained in the website. For example, the functional purpose of Taobao (https://www.taobao.com) is to provide users with an interface for searching products, shopping online, and making payments. The purpose of our work is to classify a website into specific web genres.Web genre defined in this paper has a hierarchical structure. The hierarchical structure is determined by its functional types. Each web genre is distinguished by its unique functional purpose. A website may have one or more genres from predefined web genres. Some web genre classifiers were proposed to assign a single genre to a website in web genre classification research. However, multiple functional types can be easily combined in a website, which requires multiple web genre labels. Multilabel classification is more suitable to capture the functional type of website. Multiple web genres of a website may have a hierarchical structure. For example, some “online shopping” websites can share conventions of “online mall.” Besides, some web genres may form a hierarchy relationship. “IT comprehensive” has a hierarchy relationship with “Internet technology.” The above characteristics of the web genre classification can be mapped to the hierarchical multilabel classification (HMC) problem.In order to classify the functional purpose of a website, web pages within the website need to be crawled and analyzed. However, some researchers regarded meta tags of the homepage as website summary and classified websites only by analyzing the homepage of website. A canonical website contains some meta tags, which contain some useful information and can be used to index web page content. Title, description, and keywords are the most well-known and widely used meta tags. In this way, we do not need to analyze all web pages of the website [15]. Besides, there are several critical challenges to solve web genre classification problem: lack of web genre classification dataset and efficient web genre classification mechanism. In this paper, we first construct large-scale Chinese web genre classification dataset that captures both hierarchical and multilabel characteristics. Collected websites are annotated with several of the 119 genres. We removed web genres with fewer than 400 in our collected websites. Final dataset contains 77 web genres. Secondly, we propose a website knowledge graph based on the relationship of website and meta tag features. An online web genre classification model based on hierarchical multilabel classification (OWGC-HMC) is proposed to mine the functional purpose of corresponding website. Finally, we implement an online web genre classification system and verify the validity of the OWGC-HMC model.The remainder of this paper is organized as follows. Section2 presents previous works about web genre classification methods. Section 3 describes the online web genre classification system and OWGC-HMC model. Evaluation of OWGC-HMC is presented in Section 4. Finally, Section 5 concludes our work and proposes future research.


## 2. Related Works

In this section, we present and summarize some recent research about web genre classification methods.Lim et al. [16] proposed multiple feature sets to classify genres for web documents. Multiple features were extracted from URL and HTML tags. Based on a dataset of 15 genres, they found that the main body and anchor text information are most effective to identify web genres. Kennedy and Shepherd [17] proposed a neural network classifier to classify homepages as personal homepages, corporate homepages, or organization homepages. A dataset of 321 web pages was evaluated, and F-measures were from 0.78 to 0.85 when evaluations were conducted on personal and corporate home pages. Elizabeth and Howe [18] proposed a logistic regression classifier to classify genres of web documents. Experiments were performed on a dataset of ten genres, and average correctness of 91.5% was obtained. Gams [19] proposed surface, structural, presentation, and other features, which were utilized to design a genre classifier with several machine learning algorithms. Also, the author indicated that the genre classifier could be a useful addition to search engines. Vidulin et al. [20] used standard ML algorithms and web genre classification to better capture the relations between web genres. Abramson and Aha [21] proposed a new genre classification algorithm based on URL, which combined linear interpolation smoothing for classifying URLs in a Naive Bayes classifier. Jebari [12, 22] proposed a new multilabel centroid-based approach with URL, title, headings, and link features, which calculated the similarity between the new page and each genre centroid. Deri et al. [23] defined 13 genres and extracted the list of all registered .it domains from the domain database. They evaluated how to combine probabilistic classifier and SVM classifier to improve the results. Madjarov et al. [24] constructed a hierarchy of web genres in the web genres labels. The HMC method was used to boost the predictive performance. The experimental results showed that the hierarchy structure of web genres could improve the performance of classifiers. Chaker [25] proposed a segment-based weighting approach for web genre classification, which extracted character n-grams from the URL. Experiments conducted on three web genre datasets showed that the approach could achieve encouraging results. Ebubekir and Banu [15] proposed meta tag features and developed an RNN-based deep learning system. The experimental results showed that the accuracy reaches 85%. Gjorgji et al. [13] treated web genre classification as a predictive task. They investigated the structuring of the output space by constructing hierarchies. The experimental evaluation showed that surface and paragraph features offer the best performance.An overview of web genre classification approaches indicates a lack of large-scale web genre data that capture both hierarchical and multilabel aspects. In this paper, we construct a web genre classification dataset and website knowledge graph that can be used by multilabel web genre classification. More specifically, we provide the OWGC-HMC model to grasp hierarchical and multilabel structures of website based on entity features extracted from the website knowledge graph.


## 3. Web Genre Classification Based on OWGC-HMC

In this section, we present the online web genre classification system and OWGC-HMC model.


### 3.1. Overview of Online Web Genre Classification System

The framework of the online web genre classification system is shown in Figure1, which comprises three functional layers, namely, hierarchical multilabel classifier construction, web genre feature extraction, and web genre prediction.Figure 1
The framework of online web genre classification system.Hierarchical multilabel classifier construction is designed to construct a website knowledge graph and hierarchical multilabel classifier. Each website node in the website knowledge graph has a subdomain relationship and external hyperlink relationship with other website nodes. Each website node has meta entities, which are extracted from the title and description in meta tags. Web genres and meta tags of website in the website ranking list are crawled according to aizhan.com and Chinaz.com. Entities and entity relationships are extracted from the external link crawler and DNS log system. Web genre features extracted from the website knowledge graph and corresponding web genre labels are utilized to train the OWGC-HMC classifier. Web genre feature extraction crawls meta information and extracts entity features based on the website knowledge graph. Web genre prediction utilizes hierarchical multilabel classifier API combined with website features to predict web genres.


### 3.2. Online Web Genre Classification Model Based on Hierarchical Multilabel Classification

LetD=Ei,Nii=1M denote the web genre classification dataset where each Ei is entity set of website, Ni is web genre label set, and M is the number of websites. The hierarchy of web genres is a tree structure, which is defined over web genres set by their parent-child relationships. Collected websites were annotated with several of the 119 genres. The hierarchical structure of web genres is shown in Figure 2. For convenience, τn represents the parent of the web genre n in web genre tree, yin∈−1,1 denotes whether Ei can be classified into web genre n. The problem of hierarchical multilabel web genre classification is to construct a classification model that predicts the target web genre labels of a given website with the smallest possible error.Figure 2
The hierarchical structure of web genres.The hierarchical dependencies between the web genre labels are encoded in a hierarchy used in the learning process. The framework of online web genre classification based on a hierarchical multilabel classification model is composed of four layers.The input layer determines the entity set of website, which is extracted and processed based on the website knowledge graph. For a websitei, let EOi, EHi, and ESi denote its own entity set, external hyperlink entity set, and subdomain entity set. The weight of an entity e∈Ei can be formalized as(1)we=0.5×CeEOi+0.3×CeESi+0.2×CeEHi,where CeEOi, CeESi, and CeEHi represent the number of entities e appearing in the corresponding entity set. Finally, the entities with top K entity weight are selected to Ei.The embedding of input entities will be generated at the embedding layer. Region embedding is used in our OWGC-HMC model, which is a supervised embedding method. The representation of input entities includes the embedding of the entity itself and a weighting matrix to interact with the local entities.The encoder layer input entity featurese1,e2,…,eN and the embedding of input entities. We use a bag of n-grams as additional features to capture partial information about the local entity order. We maintain a fast and efficient memory mapping of the n-grams by using the hashing trick with the same hashing function as in [26]. The features are embedded and averaged to form the hidden variable. The entity representations are then averaged into a text representation, which is fed to a linear classifier. The text representation is a hidden variable that can be potentially reused. This architecture is similar to the model of Mikolov et al. [27], where the middle word is replaced by a label.The output layer determines hierarchical web genre classification tasks. For our classification task, we use BCEWithLogitsLoss for hierarchical classification and add a recursive regularization. Using this regularization framework, we incorporate the hierarchical dependencies between web genre labels into the regularization structure of the parameters. In the problem of hierarchical multilabel classification, the prediction function is formalized as a set of parameters:(2)W=wn,n∈N.Each web genren in the hierarchy is associated with a parameter vector wn. The prediction web genre labels are parameterized by a set of parameters w which are then estimated in the learning process.(3)w^=argminλw+α×Remp,where Remp presents the empirical risk or loss on the web genre training dataset, λw represents the regularization term, and α is a parameter that controls the trade-off between fitting to the given training instances and the complexity of the hierarchical multilabel classification model.The empirical risk in our OWGC-HMC model is defined as the loss incurred by the instances at the leaf nodes of the hierarchy:(4)Remp=∑n∈T∑i=1MLyin,Ei,wn,where L can be any convex loss function. BCEWithLogitsLoss function is used in our OWGC-HMC model.We use the hierarchy in the learning process by incorporating a recursive structure into the regularization term forW.(5)λW=∑n∈T12wn−wτn2.This recursive form of regularization enforces the parameters of web genre node to be similar to the parameters of its parent web genre node under the Euclidean norm. The hierarchical dependencies are considered, which can encourage parameters nearby in the hierarchy to be similar. This can leverage information from nearby web genre labels while estimating model parameters.


## 3.1. Overview of Online Web Genre Classification System

The framework of the online web genre classification system is shown in Figure1, which comprises three functional layers, namely, hierarchical multilabel classifier construction, web genre feature extraction, and web genre prediction.Figure 1
The framework of online web genre classification system.Hierarchical multilabel classifier construction is designed to construct a website knowledge graph and hierarchical multilabel classifier. Each website node in the website knowledge graph has a subdomain relationship and external hyperlink relationship with other website nodes. Each website node has meta entities, which are extracted from the title and description in meta tags. Web genres and meta tags of website in the website ranking list are crawled according to aizhan.com and Chinaz.com. Entities and entity relationships are extracted from the external link crawler and DNS log system. Web genre features extracted from the website knowledge graph and corresponding web genre labels are utilized to train the OWGC-HMC classifier. Web genre feature extraction crawls meta information and extracts entity features based on the website knowledge graph. Web genre prediction utilizes hierarchical multilabel classifier API combined with website features to predict web genres.


## 3.2. Online Web Genre Classification Model Based on Hierarchical Multilabel Classification

LetD=Ei,Nii=1M denote the web genre classification dataset where each Ei is entity set of website, Ni is web genre label set, and M is the number of websites. The hierarchy of web genres is a tree structure, which is defined over web genres set by their parent-child relationships. Collected websites were annotated with several of the 119 genres. The hierarchical structure of web genres is shown in Figure 2. For convenience, τn represents the parent of the web genre n in web genre tree, yin∈−1,1 denotes whether Ei can be classified into web genre n. The problem of hierarchical multilabel web genre classification is to construct a classification model that predicts the target web genre labels of a given website with the smallest possible error.Figure 2
The hierarchical structure of web genres.The hierarchical dependencies between the web genre labels are encoded in a hierarchy used in the learning process. The framework of online web genre classification based on a hierarchical multilabel classification model is composed of four layers.The input layer determines the entity set of website, which is extracted and processed based on the website knowledge graph. For a websitei, let EOi, EHi, and ESi denote its own entity set, external hyperlink entity set, and subdomain entity set. The weight of an entity e∈Ei can be formalized as(1)we=0.5×CeEOi+0.3×CeESi+0.2×CeEHi,where CeEOi, CeESi, and CeEHi represent the number of entities e appearing in the corresponding entity set. Finally, the entities with top K entity weight are selected to Ei.The embedding of input entities will be generated at the embedding layer. Region embedding is used in our OWGC-HMC model, which is a supervised embedding method. The representation of input entities includes the embedding of the entity itself and a weighting matrix to interact with the local entities.The encoder layer input entity featurese1,e2,…,eN and the embedding of input entities. We use a bag of n-grams as additional features to capture partial information about the local entity order. We maintain a fast and efficient memory mapping of the n-grams by using the hashing trick with the same hashing function as in [26]. The features are embedded and averaged to form the hidden variable. The entity representations are then averaged into a text representation, which is fed to a linear classifier. The text representation is a hidden variable that can be potentially reused. This architecture is similar to the model of Mikolov et al. [27], where the middle word is replaced by a label.The output layer determines hierarchical web genre classification tasks. For our classification task, we use BCEWithLogitsLoss for hierarchical classification and add a recursive regularization. Using this regularization framework, we incorporate the hierarchical dependencies between web genre labels into the regularization structure of the parameters. In the problem of hierarchical multilabel classification, the prediction function is formalized as a set of parameters:(2)W=wn,n∈N.Each web genren in the hierarchy is associated with a parameter vector wn. The prediction web genre labels are parameterized by a set of parameters w which are then estimated in the learning process.(3)w^=argminλw+α×Remp,where Remp presents the empirical risk or loss on the web genre training dataset, λw represents the regularization term, and α is a parameter that controls the trade-off between fitting to the given training instances and the complexity of the hierarchical multilabel classification model.The empirical risk in our OWGC-HMC model is defined as the loss incurred by the instances at the leaf nodes of the hierarchy:(4)Remp=∑n∈T∑i=1MLyin,Ei,wn,where L can be any convex loss function. BCEWithLogitsLoss function is used in our OWGC-HMC model.We use the hierarchy in the learning process by incorporating a recursive structure into the regularization term forW.(5)λW=∑n∈T12wn−wτn2.This recursive form of regularization enforces the parameters of web genre node to be similar to the parameters of its parent web genre node under the Euclidean norm. The hierarchical dependencies are considered, which can encourage parameters nearby in the hierarchy to be similar. This can leverage information from nearby web genre labels while estimating model parameters.


## 4. Experimental Results and Analysis

This section presents some experiments to evaluate our OWGC-HMC model and compares it with other web genre classification methods. This section is organized into three subsections. The first section presents the dataset used in our experiments. The second section describes our performance measures. Finally, the third section discusses the experimental results.


### 4.1. Dataset

We construct the hierarchical multilabel dataset that contains 116,350 websites. Selected websites are of high interest for users, which are collected from a highly ranked list of aizhan.com. Collected websites are annotated with several of the 119 genres. We removed web genres with fewer than 400 in our collected websites. Final dataset contains 77 web genres. Web genres and corresponding numbers in the final dataset are listed in Table1. For our final dataset, the web genre cardinality is 2.21, which is the average web genre number for each website in the dataset. The genre density is 0.029, which is the average web genre number for each web divided by the total number of web genre.Table 1
Description of hierarchical multilabel web genre dataset.
Web genresNumberDomestic services35148Education culture17367Leisure and entertainment16103Internet technology14482Local portal13822Industry enterprise11887News media10393School website8267Government organization7135Online shopping6704Government portal6594Online store6379Forum community6200Educational examination5409Job recruitment4467Travel4462Game website4283Video movie3825Software download3772Category message3457News portal3374Real estate website3261Medical health3049Novel reading2754Financial investment2538Training institution2142Design material2141Technical programming1980IT comprehensive1887Auto website1876Physical fitness1832Travel tips1682Travel agency1680Photography picture1661Music website1645Newspapers and magazines1575Diet1540Home building1449Star fashion1278Association organization1183Literary papers1169Web navigation1125Common query1120Webmaster resources1049Hospital clinic1032Mobile digital1027Bancassurance1025Animated cartoons924Early childhood education899Dating878Group-buying website839Search engine819Health care793Broadcasting770Rebate discount767Ticket booking763Foreign language synthesis752Technology information749Government functions633Data analysis623Lease loan621Advertising alliance593Entertainment gossip583Domain host549Hotel521Cyber security509Chat communication499Pharmaceutical medicine488Weather forecast473Sports synthesis456Hobby451Medical health organization451Sexual health443Travel agency431Online education431Traffic map403Sale mall401


### 4.2. Performance Measures

To evaluate our multilabel classification model, we use precision, micro-F1, macro-F1, and Hamming loss evaluation measures. In our experiments, we follow the 10-fold cross-validation procedure. The evaluation result is the average of 10 individual performances. The ratio of the final training set, verification set, and test set is 8 : 1:1. After training, the test set is used to test the web genre classification model.Micro-F1 is a conventional evaluation metric used to evaluate classification performance. The micro-averaged Micro-F1 is given by(6)P=∑l∈LTPl∑l∈LTPl+FPl,R=∑l∈LTPl∑l∈LTPl+FNl,micro−F1=2PRP+R,where TPl, FPl, and FNl represent the true positives, false positives, and false-negatives for web genre label l∈L.Macro-F1 gives equal weight to each class label, which is given by(7)P′=TPlTPl+FPl,R′=TPlTPl+FNl,macro−F1=1L∑l∈L2P′R′P′+R′.


### 4.3. Results and Discussion

In this section, we conduct three experiments. The aim of the first experiment is to compare the accuracy of different encoders [28–30] using word entity and character features in the encoder layer. Experiment results using different encoders are presented in Figure 3, where OWGC-HMC got the best accuracy. We can conclude that the best results are achieved using word entity rather than character. This can be explained by the fact that word entities in title and description contain more web genre specific word entities than characters. The second experiment aims to compare Hamming loss and training time obtained using word entity and character features. Experiment results using different features are presented in Figures 4 and 5, where OWGC-HMC has the lowest loss and training time. Overall, OWGC-HMC outperforms other classifications using word entity and character features concerning all evaluation metrics.Figure 3
Comparison experiment results of accuracy.Figure 4
Comparison experiment results of Hamming loss.Figure 5
Comparison experiment results of training time.In order to evaluate the classification performance of the OWGC-HMC model for different web genres, we construct the confusion matrix on the test dataset and compute it for the classification accuracy of different web genres. Classification accuracy of different web genres is shown in Table2. As shown in Table 2, The F1 score of most web genres in the test set exceeds 0.85. We analyze these web genres of low F1 score and find that their meta tag descriptions are most ambiguous and irregular, such as local portal and entertainment gossip. Most websites have professional SEO managers and meta tag features, which can be applied to web genre classification tasks.Table 2
Classification accuracy of different web genres.
Web genrePrecisionRecallF1 scoreSchool website0.970.960.97Real estate website0.980.960.97Travel0.960.950.96Novel reading0.980.930.95Travel agency0.950.960.95Job recruitment0.970.940.95Hotel0.980.930.95Group-buying website0.990.910.95Education culture0.950.940.95Early childhood education0.970.920.94Home building0.960.920.94Diet0.960.920.94Travel agency0.990.900.94Video movie0.950.930.94Medical health0.950.920.93Auto website0.950.920.93Game website0.940.930.93Music website0.960.900.93Travel tips0.950.910.93Government portal0.930.920.93Animated cartoons0.990.870.93Weather forecast0.910.940.93Photography picture0.960.890.93Hospital clinic0.940.910.92Ticket booking0.940.910.92Financial investment0.940.910.92Bancassurance0.940.900.92Government organization0.950.900.92Leisure and entertainment0.940.900.92Newspapers and magazines0.980.860.92Traffic map0.880.960.92Web navigation0.960.880.91Foreign language synthesis0.980.860.91Domestic services0.930.900.91Design material0.940.880.91Internet technology0.920.890.91Online store0.910.910.91Mobile digital0.920.880.90Dating0.910.880.90Industry enterprise0.920.870.89Software download0.920.870.89Forum community0.920.870.89Online shopping0.900.880.89Star fashion0.940.840.89Hobby0.910.860.89Health care0.950.830.88Broadcasting0.940.830.88Association organization0.950.820.88Educational examination0.910.850.88News portal0.890.860.88Chat communication0.960.810.88News media0.900.850.87Technical programming0.890.850.87Sexual health0.890.850.87Advertising alliance0.910.830.87Data analysis0.910.820.86Webmaster resources0.930.790.85Domain host0.920.800.85Pharmaceutical medicine0.870.840.85Training institution0.920.800.85Lease loan0.880.820.85Search engine0.930.780.85Category message0.910.790.85Common query0.860.840.85Literary papers0.900.790.84Rebate discount0.930.770.84IT comprehensive0.900.780.84Physical fitness0.910.760.83Technology information0.880.780.83Sports synthesis0.830.820.83Local portal0.840.800.82Sale mall0.950.720.82Government functions0.840.780.81Entertainment gossip0.870.730.80Medical health organization0.890.720.79Cyber security0.850.720.78Online education0.980.600.75In the third experiment, we compare our OWGC-HMC model with WGC-SOP [13]. WGC-SOP proposes ten different feature representations of the websites. Predictive clustering trees are used to assess the influence of the different information sources. The hierarchy construction method in WGC-SOP is manual (MAN) for the comparison experiment. Figure 6 shows the comparison results of accuracy. It can be seen from Figure 6 that our method has higher precision, recall, and F1 score compared with WGC-SOP. Compared with WGC-SOP, the improvement of the OWGC-HMC model is obvious. This proves that the OWGC-HMC model based on meta tag features has better classification performance and only fewer features are extracted, which can be used in the online environment.Figure 6
Comparison results with WGC-SOP.


## 4.1. Dataset

We construct the hierarchical multilabel dataset that contains 116,350 websites. Selected websites are of high interest for users, which are collected from a highly ranked list of aizhan.com. Collected websites are annotated with several of the 119 genres. We removed web genres with fewer than 400 in our collected websites. Final dataset contains 77 web genres. Web genres and corresponding numbers in the final dataset are listed in Table1. For our final dataset, the web genre cardinality is 2.21, which is the average web genre number for each website in the dataset. The genre density is 0.029, which is the average web genre number for each web divided by the total number of web genre.Table 1
Description of hierarchical multilabel web genre dataset.
Web genresNumberDomestic services35148Education culture17367Leisure and entertainment16103Internet technology14482Local portal13822Industry enterprise11887News media10393School website8267Government organization7135Online shopping6704Government portal6594Online store6379Forum community6200Educational examination5409Job recruitment4467Travel4462Game website4283Video movie3825Software download3772Category message3457News portal3374Real estate website3261Medical health3049Novel reading2754Financial investment2538Training institution2142Design material2141Technical programming1980IT comprehensive1887Auto website1876Physical fitness1832Travel tips1682Travel agency1680Photography picture1661Music website1645Newspapers and magazines1575Diet1540Home building1449Star fashion1278Association organization1183Literary papers1169Web navigation1125Common query1120Webmaster resources1049Hospital clinic1032Mobile digital1027Bancassurance1025Animated cartoons924Early childhood education899Dating878Group-buying website839Search engine819Health care793Broadcasting770Rebate discount767Ticket booking763Foreign language synthesis752Technology information749Government functions633Data analysis623Lease loan621Advertising alliance593Entertainment gossip583Domain host549Hotel521Cyber security509Chat communication499Pharmaceutical medicine488Weather forecast473Sports synthesis456Hobby451Medical health organization451Sexual health443Travel agency431Online education431Traffic map403Sale mall401


## 4.2. Performance Measures

To evaluate our multilabel classification model, we use precision, micro-F1, macro-F1, and Hamming loss evaluation measures. In our experiments, we follow the 10-fold cross-validation procedure. The evaluation result is the average of 10 individual performances. The ratio of the final training set, verification set, and test set is 8 : 1:1. After training, the test set is used to test the web genre classification model.Micro-F1 is a conventional evaluation metric used to evaluate classification performance. The micro-averaged Micro-F1 is given by(6)P=∑l∈LTPl∑l∈LTPl+FPl,R=∑l∈LTPl∑l∈LTPl+FNl,micro−F1=2PRP+R,where TPl, FPl, and FNl represent the true positives, false positives, and false-negatives for web genre label l∈L.Macro-F1 gives equal weight to each class label, which is given by(7)P′=TPlTPl+FPl,R′=TPlTPl+FNl,macro−F1=1L∑l∈L2P′R′P′+R′.


## 4.3. Results and Discussion

In this section, we conduct three experiments. The aim of the first experiment is to compare the accuracy of different encoders [28–30] using word entity and character features in the encoder layer. Experiment results using different encoders are presented in Figure 3, where OWGC-HMC got the best accuracy. We can conclude that the best results are achieved using word entity rather than character. This can be explained by the fact that word entities in title and description contain more web genre specific word entities than characters. The second experiment aims to compare Hamming loss and training time obtained using word entity and character features. Experiment results using different features are presented in Figures 4 and 5, where OWGC-HMC has the lowest loss and training time. Overall, OWGC-HMC outperforms other classifications using word entity and character features concerning all evaluation metrics.Figure 3
Comparison experiment results of accuracy.Figure 4
Comparison experiment results of Hamming loss.Figure 5
Comparison experiment results of training time.In order to evaluate the classification performance of the OWGC-HMC model for different web genres, we construct the confusion matrix on the test dataset and compute it for the classification accuracy of different web genres. Classification accuracy of different web genres is shown in Table2. As shown in Table 2, The F1 score of most web genres in the test set exceeds 0.85. We analyze these web genres of low F1 score and find that their meta tag descriptions are most ambiguous and irregular, such as local portal and entertainment gossip. Most websites have professional SEO managers and meta tag features, which can be applied to web genre classification tasks.Table 2
Classification accuracy of different web genres.
Web genrePrecisionRecallF1 scoreSchool website0.970.960.97Real estate website0.980.960.97Travel0.960.950.96Novel reading0.980.930.95Travel agency0.950.960.95Job recruitment0.970.940.95Hotel0.980.930.95Group-buying website0.990.910.95Education culture0.950.940.95Early childhood education0.970.920.94Home building0.960.920.94Diet0.960.920.94Travel agency0.990.900.94Video movie0.950.930.94Medical health0.950.920.93Auto website0.950.920.93Game website0.940.930.93Music website0.960.900.93Travel tips0.950.910.93Government portal0.930.920.93Animated cartoons0.990.870.93Weather forecast0.910.940.93Photography picture0.960.890.93Hospital clinic0.940.910.92Ticket booking0.940.910.92Financial investment0.940.910.92Bancassurance0.940.900.92Government organization0.950.900.92Leisure and entertainment0.940.900.92Newspapers and magazines0.980.860.92Traffic map0.880.960.92Web navigation0.960.880.91Foreign language synthesis0.980.860.91Domestic services0.930.900.91Design material0.940.880.91Internet technology0.920.890.91Online store0.910.910.91Mobile digital0.920.880.90Dating0.910.880.90Industry enterprise0.920.870.89Software download0.920.870.89Forum community0.920.870.89Online shopping0.900.880.89Star fashion0.940.840.89Hobby0.910.860.89Health care0.950.830.88Broadcasting0.940.830.88Association organization0.950.820.88Educational examination0.910.850.88News portal0.890.860.88Chat communication0.960.810.88News media0.900.850.87Technical programming0.890.850.87Sexual health0.890.850.87Advertising alliance0.910.830.87Data analysis0.910.820.86Webmaster resources0.930.790.85Domain host0.920.800.85Pharmaceutical medicine0.870.840.85Training institution0.920.800.85Lease loan0.880.820.85Search engine0.930.780.85Category message0.910.790.85Common query0.860.840.85Literary papers0.900.790.84Rebate discount0.930.770.84IT comprehensive0.900.780.84Physical fitness0.910.760.83Technology information0.880.780.83Sports synthesis0.830.820.83Local portal0.840.800.82Sale mall0.950.720.82Government functions0.840.780.81Entertainment gossip0.870.730.80Medical health organization0.890.720.79Cyber security0.850.720.78Online education0.980.600.75In the third experiment, we compare our OWGC-HMC model with WGC-SOP [13]. WGC-SOP proposes ten different feature representations of the websites. Predictive clustering trees are used to assess the influence of the different information sources. The hierarchy construction method in WGC-SOP is manual (MAN) for the comparison experiment. Figure 6 shows the comparison results of accuracy. It can be seen from Figure 6 that our method has higher precision, recall, and F1 score compared with WGC-SOP. Compared with WGC-SOP, the improvement of the OWGC-HMC model is obvious. This proves that the OWGC-HMC model based on meta tag features has better classification performance and only fewer features are extracted, which can be used in the online environment.Figure 6
Comparison results with WGC-SOP.


## 5. Conclusion

In this paper, we constructed a Chinese web genre classification dataset and converted it into an HMC dataset. Based on the Chinese web genre classification dataset, we proposed the OWGC-HMC model to mine hierarchical and multilabel structures of web genre data. Besides, we developed an online web genre classification system based on hierarchical multilabel classification. In our future research work, we intend to evaluate our model on more hierarchical multilabel classification datasets and provide visualization techniques for hierarchical multilabel classification.



---

*Source: 7549880-2022-03-29.xml*