Semantic Analysis Experiments for F rench Citizens’ Contribution : Combinations of Language Models and Community Detection Algorithms (2025)

Abstract: Following the Yellow Vest crisis that occurred in France in 2018, the French government launched the Grand Débat National, which gathered citizens’ contributions.This paper presents a semantic analysis of these contributions by segmenting them into sentences and identifying the topics addressed using clustering techniques. The study tests several combinations of French language models and community detection algorithms, aiming to identify the most effective pairing for grouping sentences based on thematic similarity. Performance is evaluated using the number of clusters generated and standard clustering metrics.Principal Component Analysis (PCA) is employed to assess the impact of dimensionality reduction on sentence embeddings and clustering quality. Cluster merging methods are also developed to reduce redundancy and improve the relevance of the identified topics.Finally, the results help refine semantic analysis and shed light on the main concerns expressed by citizens.

## introduction as an answer to the yellow vest crisis, in january 2019, the french government launched the grand débat national 1 [in english, large national debate] (gdn) offering both a dematerialized digital platform and physical supports, called cahiers citoyens @xcite , leaved in various public places (town halls, roundabouts, hospitals, prisons, etc.). these notebooks enabled citizens to freely express their views on topics of their choice, 1 https://granddebat.fr/ choosing the format (letters, paragraphs, emails, bullet lists, petitions) and length (ranging from a few words to several pages) that suited them. at the gdn close, in mid-march 2019, cahiers citoyens gathered 225,224 contributions located to the place where each one had been written or deposited. among the 34,970 municipalities in france in 2019, 17,014 proposed at least one notebook. a team has been formed to conduct a semantic analysis of the content of cahiers citoyens, and this paper is part of the project's framework. the analysis is based on both the text of the contributions and their location. 2 due to the volume of contributions, the adopted approach consisted in identifying the topics they addressed @xcite . to achieve this, clustering was applied to the texts of the contributions using community detection algorithms. however, the first clustering results varied widely in number of clusters and of unclassified citizens' contributions, making necessary to explore various combinations of parameters (algorithms, hyperparameters, language models, etc.) and posttreatments. after a presentation of related works in section 2 and the corpus of cahiers citoyens in section 3, this article describes the end-to-end process implemented to identify the contributions' semantic organization, combining text representation models and community detection algorithms. section 4 presents the tested combinations and section 5 evaluates them through different indexes. section 6 introduces post-treatments intended to enhance clustering performance, while section 7 provides a detailed assessment of these methods along with the final results. finally, section 8 presents the main conclusions of this study and discusses the perspectives opened by this work. ## related works community detection in graphs constructed from textual data has emerged as a widely adopted technique in text mining, enabling the unsupervised discovery of latent thematic structures. among the most commonly used algorithms, louvain @xcite and the label propagation algorithm (lpa) @xcite are particularly prominent for their ability to identify coherent clusters within semantic graphs. several studies have employed the louvain algorithm specifically for topic modeling across large textual corpora. for example, @xcite used louvain to improve the semi-supervised clustering of customer reviews in the domain of customer services. @xcite combined doc2vec representations with louvain, kmeans, and spectral clustering to enhance topic classification across a broad document collection. @xcite reformulated the topic modeling task as a community detection problem in a word co-occurrence graph generated from a text corpus. similarly, @xcite applied louvain to cluster covid-19-related articles by thematic similarity, following an automatic summarization process. in all these cases, louvain demonstrated strong capabilities in uncovering semantically meaningful clusters from unstructured textual data. @xcite introduced an innovative method, iecons (implicit and explicit consensus), which combines multiple textual representations -including tf-idf, word2vec, and bert embeddings-to improve the robustness of clustering. their approach uses a dual consensus strategy: explicit consensus through the aggregation of clustering results obtained from each representation independently, and implicit consensus through the fusion of similarity matrices into a unified similarity tensor. for the final clustering step, several algorithms are evaluated, including louvain, which is particularly effective in extracting dense communities from the resulting weighted graph. in a different application, @xcite leveraged the louvain algorithm to detect political communities from a user graph built from french tweets, where edges are defined by retweet behavior. although the graph structure is based on user interaction rather than content similarity, this work reflects a growing interest in combining community detection techniques with language models such as roberta and camembert, which the authors use for offensive language detection. in parallel, the label propagation algorithm (lpa) has also received attention due to its simplicity and computational efficiency on large graphs. @xcite proposed a classification framework for scientific and technical documents (e.g., patents and academic papers) using word2vec embeddings and a consensus clustering approach based on lpa. @xcite developed an lpa-based method for weakly supervised text classification, where documents are modeled as nodes in a similarity graph, and labels are propagated through the network. @xcite focused on improving lpa itself, introducing a modified version, lpaf, that enhances the quality of detected communities in large-scale networks. these contributions illustrate lpa's suitability for fast, scalable classification and clustering tasks over vast document sets. beyond louvain and lpa, other methods have been proposed that integrate semantic information directly into the graph structure. for instance, a community detection method was developed by @xcite , incorporating both network connectivity and tf-idf scores of textual content, demonstrating improved thematic coherence in the resulting communities. similarly, @xcite proposed a sentiment-aware community detection framework, where tf-idf vectors and sentiment scores are jointly used to construct a weighted graph reflecting both topical and emotional affinities between users in social networks. this approach enhances the identification of semantically and emotionally coherent communities. to facilitate the application of community detection algorithms, several studies have introduced dimensionality reduction techniques, particularly when working with high-dimensional vector representations of textual data. these methods aim to project the original graph or embedding space into a lower-dimensional representation while preserving the essential topological or semantic properties of the data. for instance, @xcite employ structural embedding methods such as deepwalk and node2vec to learn low-dimensional node embeddings, enabling more efficient community detection. unlike semantic-based approaches, these methods focus exclusively on the structural properties of the network. while most studies focus on general-purpose corpora or domains such as customer reviews or social media, few works have addressed the analysis of deliberative citizen-generated content. yet, this type of corpus -as exemplified by the cahiers citoyens or participatory platforms-raises important challenges due to its thematic diversity, variability in writing quality, and lack of structure. a government-commissioned report by the roland berger firm @xcite , in collaboration with the agency cognito consulting 3 , served as a starting point for the analysis of the cahiers citoyens. the approach relied on semantic mapping to cluster textual contributions into eight major themes: democracy and citizenship (144,071 ideas), ecological transition (89,103), taxation and public spending (138,667), state organization and public services (62,597), economy and employment (26,686), education and training (9,638), purchasing power (75,652), and health, solidarity, and integration (63,574). however, the methodology has been criticized for its lack of transparency, particularly regarding the definition of what constitutes an "expressed idea", the algorithmic procedures used, and the rapidity with which the results were delivered -all of which raise questions about the robustness and interpretability of the findings. @xcite explored topic extraction methods such as bertopic @xcite and latent dirichlet allocation (lda) @xcite to analyze the contributions from the cahiers citoyens corpus. the objective was to compare the extracted thematic structures with those identified in the roland berger firm report. this approach provides a renewed perspective on the thematic diversity present in citizen contributions. the most salient clusters uncovered by the analysis relate to issues such as pension increases, rural life, education, ecology and agriculture, electoral processes including recognition of blank votes, speed limitations on highways, and taxation. @xcite conducted an in-depth study on the theme of wind power based on the cahiers citoyens corpus. her work adopts a crossdisciplinary perspective in the social sciences, combining linguistic and geographical approaches. the analysis focused on three departments where windrelated concerns were particularly salient, allowwhile previous research has explored a wide range of textual representations -from traditional models such as tf-idf and word2vec to more advanced embeddings from bert-none, to the best of our knowledge, have leveraged pretrained transformer-based language models specifically designed for french (such as camembert) to represent texts, followed by dimensionality reduction of these vectors to construct an optimized semantic graph, on which community detection algorithms are applied. this is the approach proposed here in order to reveal latent topics in large-scale citizengenerated content from the cahiers citoyens. ## corpus of cahiers citoyens the notebooks of cahiers citoyens were made up of handwritten and/or typed texts, emails sent directly to the city councils, files sometimes including attachments, as well as collective petitions, and reflect a diversity of citizen concerns. a textometric and spatialized analysis of the corpus is presented in detail in @xcite . ## combinations of language models and community detection algorithms as stated in section 1, the proposed method consists in representing sentences as embeddings (vectors) using language models, then applying clustering algorithms to these embeddings to obtain clusters. the objective is to compare different combinations of language models and clustering algorithms in order to identify the one that provides the best clustering of sentences and, consequently, the most accurate identification of topics. in addition, each combination is tested both with and without dimensionality reduction using principal component analysis (pca) @xcite . the purpose of applying pca is to evaluate the impact of dimensionality reduction on sentence embeddings and clustering quality. for sentence vector representations, three language models were selected due to their high performance in french: camembertlarge @xcite , solon-large foot_3 , and distil-camembert (delestre and amar, 2022). regarding clustering, since the exact number of topics (clusters) discussed in cahiers citoyens is unknown, we opted for community detection algorithms, which are designed to uncover structure in graphs without requiring a predefined number of clusters. to do this, a graph has been constructed where each sentence embedding represents a node, and an edge is established between two nodes if their cosine similarity exceeds the threshold of 0.68 (this threshold was chosen based on a comparative analysis conducted by @xcite , which examined various pairs of sentences). 7 community detection algorithms are then applied to the graph to obtain clusters. the algorithms have been selected from related works (in section 2): lpa (label propagation algorithm) and the louvain algorithm. with three models, mixed or not with dimension-ality reduction, and two algorithms, we obtain 12 combinations to compare. algorithm 1, presented below, describes the process of applying these combinations, while ## evaluation and interpretations of the combinations evaluation: the performance of each combination (model, algorithm) is measured through the quality of the clustering. several metrics adapted to unsupervised clustering and community detection have been selected: the calinski-harabasz index (chi) @xcite , the davies-bouldin index (dbi) @xcite , and modularity @xcite . these metrics assess the internal cohesion of groups, the separation between clusters, and the structure of communities within the resulting graph. the chi and the dbi assess clustering quality by measuring the separation and compactness of clusters, with a high value being desirable for the former and a low value for the latter. modularity, on the other hand, measures the density of connections within communities in a graph, with a high value indicating well-defined communities. interpretations: the chi shows that camembert-large delivers the best performance among the tested modregarding community detection algorithms, louvain outperforms lpa across all three language models tested. louvain generates better-separated and more homogeneous clusters, confirming its ability to identify distinct communities by minimizing cuts between them and producing more coherent clusters. the impact of pca on clustering quality is also significant. all combinations with pca show higher chi than those without dimension reduction. pca reduces dimensionality while enhancing cluster separation and compactness, although it can sometimes decrease the similarity between embeddings, preventing their clustering. as mentioned before, a low dbi value indicates better clustering with more compact clusters, where cluster points are closer to their centroid. the obtained values show that the solon-large model produces more compact clusters than camembertlarge and distil-camembert, at the cost of a smaller number of classified sentences, which reduces intra-cluster dispersion. concerning the algorithms,the lpa generates clusters with a lower dbi than louvain for all models, indicating more cohesive and less dispersed groups. for all combinations, the use of pca systematically reduces dbi values, confirming that dimensionality reduction improves cluster cohesion by limiting their internal dispersion. in terms of modularity, the results show that solon-large achieves slightly higher modularity because it classifies fewer sentences, reducing the density of the graph. clustering with louvain en-sures better modularity than lpa, meaning that the detected communities are better defined. pca has a very limited impact on the modularity of the camembert-large and distil-camembert models. indeed, although applying pca to the embeddings of these two models reduces the number of nodes in the graph, it nevertheless remains dense. in summary, the most suitable combination for our corpus appears to be camembert-large paired with the louvain algorithm. this configuration maximizes similarity between embeddings, groups more sentences into fewer clusters, and has the best chi value. although its dbi is not the lowest, it remains close to the values obtained with other combinations. similarly, while some configurations show slightly better modularity, the difference remains marginal. finally, pca improves chi, dbi, and modularity scores. however, it reduces the number of classified sentences, as it decreases the model's ability to capture similarities between sentence embeddings, thereby limiting the formation of clusters grouping semantically close sentences. ## post-treatments to merge redundant clusters in section 5, we identified the most effective combination for clustering, namely the use of camembert-large for sentence embeddings and the louvain algorithm for community detection. this configuration enables classifying the largest number of sentences while ensuring good cluster cohesion. however, the number of clusters obtained remains very high (2,394 without pca and 2,446 with pca). yet, some clusters could be redundant in the sense they might address similar topics. so, optimizing clustering results could mean reducing the number of clusters and obtaining more populated clusters while improving performance according to chi, dbi, and modularity. to achieve this, we developed three approaches designed to merge redundant clusters: the evaluation of these approaches, as well as the presentation of the results of the best-performing approach, are detailed in section 7. ## semantic analysis of cahiers citoyens the chi, the dbi, and modularity must be recalculated after applying merging methods. the merging of clusters sharing an identical dbi also results in a modest reduction in the number of clusters, with 18 clusters without pca and 14 with pca. although this approach slightly decreases the number of clusters, the quality of cluster separation deteriorates, as evidenced by the decline in the chi in both cases (with and without pca). furthermore, the values of the dbi and modularity remain unchanged, indicating that this method does not significantly improve cluster compactness or modularity. the best-performing approach is to merge clusters using hc, which reduces the number of clusters by approximately 38 when dimensionality is not reduced, and by 37 when pca is applied. this reduction improves the chi, reflecting better cluster separation. additionally, the merging leads to a decrease in the dbi, indicating that the clusters are now more compact, with points closer to their centroid and reduced intra-cluster dispersion. although odularity remains largely unchanged, likely due to the limited reduction in cluster count, suggesting a stable overall graph structure. in summary, this approach enhances cluster cohesion while maintaining adequate separation. ## conclusions and prospects this study presented a semantic analysis of a realworld corpus collected during a period of social unrest, aiming to understand citizens' concerns through the comparison of several combinations of language models with community detection algorithms. we found that the most effective combination for this purpose was camembert-large for sentence representation paired with the louvain algorithm for community detection. pca played a beneficial role by enhancing cluster separation and reducing intra-cluster dispersion, as shown by the decrease in the dbi and the increase in the chi. given the large number of redundant clusters, a merging strategy was attempted: hc proved to be the most effective, grouping clusters on similar themes while improving compactness and homogeneity, thus strengthening clustering quality. the cluster analysis revealed that citizen concerns focus mainly on economic, fiscal, and political issues. recurring topics include pension reform (pension revaluation, removal of the csg tax), taxation (restoration of the isf, tax reduction), and institutional reforms (reduction in the number of deputies, removal of political privileges). citizen participation, notably through citizens' initiative referendums (ric) and compulsory voting, is also a major concern. wage increases constitute another point of interest, though more diverse. looking forward, a key perspective is to extend this analysis to the full cc corpus using supercomputers to overcome computational limitations. this would provide a more precise overview of french citizens' concerns during the yellow vest crisis and just before the covid lockdown. moreover, a detailed analysis of raw texts within each cluster would refine semantic interpretations and improve understanding of the underlying themes.