# Deep Learning-Driven Public Opinion Analysis on the Weibo Topic about AI Art
## 2 Materials and Methods
### Clustering Analysis
#### 2.2.2 Clustering Analysis and Keyword Extraction  
This paper selected K-means for the clustering analysis. Compared to DBSCAN and spectral clustering, K-means benefits from its simple structure and high calculation efficiency, which is important for carrying out public opinion analysis in a timely fashion. Considering that the discussions about AI Art are unevenly distributed over time, K-means++ was utilized to optimize the original K-means clustering algorithm.  
The K-means clustering algorithm partitions text data into \(k\) clusters, where each cluster consists of text sharing similar semantic features, while the semantics expressed in different clusters vary. In order to acquire the detailed opinions expressed by users related to different sentiments, a keyword extraction algorithm is necessary to represent the opinions with a list of keywords for each cluster, thereby enabling the visual presentation of in-depth discussion content.  
For the selection of number of clusters \(k\), the elbow method is utilized. In the process of determining \(k\), with an increase in the number of clusters, the partition of data becomes more refined, leading to higher cohesion within each cluster and a gradual decrease in the sum of squared errors (SSE). The calculation of the SSE is as follows:  
\[\text{SSE}= \sum_{i=1}^{k}\sum_{p\in C_{i}}\left|p-m_{i}\right|^{2} \tag{22}\]  
In Equation (22), \(C_{i}\) is one of the clusters, \(p\) is one point in \(C_{i}\), \(m_{i}\) is the centroid of \(C_{i}\). When \(k\) is relatively low, increasing it will enhance the cohesion of the clusters, consequently causing a swift reduction in the SSE; however, when \(k\) is extensive, additional increments will not yield a significant reduction in the SSE; rather, the result will be clustering overlapping, which can have a negative effect on the accurate identification of detailed topics. By identifying the knee point on the curve of SSE, it is possible to determine an appropriate \(k\) that balances the accuracy of clustering and the complexity of the model.  
The TF-IDF algorithm is a frequently used method for keyword extraction. It operates by computing the product of the word's frequency and its inverse document frequency to assess the significance of the word within the text. For each cluster, the TF-IDF algorithm can be applied independently to extract keywords. However, this process assumes homogeneity in the word distribution characteristics within each cluster, overlooking the varying significance of words within clusters.  
To solve this, the C-TF-IDF keyword extraction algorithm [22] employs the calculation method of TF-IDF, while considering the varying importance of words in different clusters. The calculation process of the C-TF-IDF algorithm is described as follows:  
Figure 8: The structure of the autoencoder.  
\[W_{x,c}=\|\text{TF}_{x,c}\|\times log(1+\frac{\text{A}}{f_{x}}), \tag{23}\]  
In Equation (23), \(x\) is any word in the whole corpus, \(c\) stands for the specific cluster generated from K-means. \(\text{TF}_{x,c}\) is the frequency of word \(x\) in cluster \(c\), A is the length of words for all clusters, and \(f_{x}\) stands for the number of documents that consist of word \(x\). Compared to the individual calculation of each cluster, the C-TF-IDF algorithm assesses the differences in word distribution characteristics between clusters, enabling the better extraction of keywords representing the semantic information of clusters.