puoi aiutarmi a scrivere una relazione sul lavoro che ho svolto basandoci sulle informazioni che ti dirò e che hai già in memoria? l'indice della mia tesi, dal titolo "Concept Based XAI via Deep Learning and Large Language Model for job classification", è: - Abstract (lo facciamo alla fine)

Introduzione: cosa facciamo, a cosa serve, qualcuno lo ha già fatto (in generale)
Stato dell’arte: dividiamola in: 1. concept based embedding e XAI 2. XAI for job classification
Metodologie: che metodi abbiamo utilizzato? 1. Spiegare bene metodi di concept creation SOLO QUELLI USATI e 2. Il processo di XAI delle review partendo da preprocessing etc. 3. LLM che usi per dare il nome ai topic
Esperimenti:  spiegare anche dati (quanti, quali occupation…)  poi esperimenti su creazione topic, inclusi parametri dei metodi (quanti topic, quante parole per topic, etc…). Negli esperimenti abbiamo i seguenti risultati: - Quanti topic per annuncio in media? - Classificazione usando topic. Funziona? - grafici separazione topic (umap, pce) I risultati devono essere presentati come confronto dei 4 metodi.
XAI:

word cloud topic per ogni occupation
lime locale/shap global non usando le parole ma i topic. Se viene bene poi confondiamo con LIME/SHAP usando le parole.
Conclusioni
i principali paper su cui mi sono basato sono quelli di ConceptSHAP, BERTopic e BGE-M3
vorrei che iniziassimo con l'introduzione.
l'idea da cui sono partito è quella di trovare dei metodi che riuscissero a estrarre i concetti presenti in un testo, nel mio caso specifico degli annunci di lavoro, per poter essere usati poi come mezzo di spiegazione del modello sottostante, nel mio caso l'encoder che trasforma in embedding gli annunci che poi verranno usato come feature per la classificazione degli annunci di lavoro. ad esempio, con bertopic l'idea era quella di applicarlo al testo, estraendo i topic e usarli come feature alternativa al testo, così da semplificare il lavoro (questo esempio non voglio che sia presente nell'introduzione).

Understanding how to manage categorical variables in predictive modelling has always been challenging, as most machine-learning algorithms only accept numerical input. Dropping categorical variables is rarely a convenient solution due to the massive loss of data and the high number of qualitative variables in tabular data across various fields of study.

A common solution to this problem is the widely-used \textit{one-hot encoding} method. Despite its simplicity, this method can lead to different issues and may not always be the most suitable choice.

For instance, we need to predict the salary of a person given the education level, represented with the categories \textit{High school diploma}, \textit{Bachelor's degree} and \textit{Master's degree}. By applying the one-hot encoding, the levels are encoded as follows
\begin{equation*}
high\_school=\begin{bmatrix}
                 1\\
                 0\\
                 0
              \end{bmatrix}
bachelor=\begin{bmatrix}
                 0\\
                 1\\
                 0
              \end{bmatrix}
master=\begin{bmatrix}
                 0\\
                 0\\
                 1
              \end{bmatrix}
\end{equation*}
and can be graphically represented as

\tdplotsetmaincoords{54.74}{135}
\begin{center}
\begin{tikzpicture}[tdplot_main_coords]

    % Axes
    \draw[thick,->] (0,0,0) -- (2.5,0,0) node[anchor=north east]{$high\_school$};
    \draw[thick,->] (0,0,0) -- (0,2.5,0) node[anchor=north west]{$bachelor$};
    \draw[thick,->] (0,0,0) -- (0,0,2.5) node[anchor=south]{$master$};
    
    % Points
    \filldraw[black] (2,0,0) circle (2pt) node[anchor=south east] {$(1,0,0)$};
    \filldraw[black] (0,2,0) circle (2pt) node[anchor=south west] {$(0,1,0)$};
    \filldraw[black] (0,0,2) circle (2pt) node[anchor=east] {$(0,0,1)$};

\end{tikzpicture}
\end{center}
It is possible to see that each level can have only one feature at a time, without sharing attributes with other categories and introducing orthogonality. In this example, having a Master's degree implies having a Bachelor's degree and a High school diploma, but one-hot encoding does not convey this hierarchical information. For this reason, the original order and the distance between levels are not preserved and each category is treated with equal importance. This results in a loss of information, not representing each level faithfully.

Furthermore, this method also leads to a dimensionality increase, especially with high-cardinality categorical variables, as one-hot encoding codes each level as a new vector, introducing a sparse matrix, i.e. a matrix with almost every value equal to zero. This behaviour results in a significant memory occupation and higher computational costs.
In response to these challenges, various encoding methods have been proposed as a one-hot encoding alternative. This paper aims to compare different encoding strategies for categorical features in tabular data through their application in supervised machine-learning problems, highlighting their benefits and drawbacks. Firstly, a deep literature review is provided to identify existing work and areas requiring further investigation.
Then each encoding method is described in detail and a taxonomy is presented, followed by the experiment setup explanation and the analysis of results, providing a ranking of methods based on their performance for each machine-learning family. Moreover, an analysis of the encoding computational time and the dimensionality increase is conducted to support the encoder choice by finding the best trade-off between performance and computational costs. Finally, these results are summarized in a guideline for choosing the proper encoder based on the task type and algorithms.

\subsection*{Research contribution}
The proposed work aims to provide a comprehensive overview of encoding methods compared to previous studies. In particular, it focuses on the following key improvements:
\begin{itemize}
  \item \textbf{Number of evaluated encoders}: the scope of the application is broadened by evaluating a larger number of encoding methods than in previous studies.
  \item \textbf{Taxonomy integration}: this work provides a structured taxonomy that categorizes the evaluated encoding methods based on their underlying processes and theoretical foundations. This classification enhances the comprehension of encoders' behaviour.
  \item \textbf{Comprehensive coverage of machine-learning algorithms}: this study encompasses a wide range of machine-learning algorithms by testing each main family of models.
  \item \textbf{Comprehensive coverage of machine-learning tasks}: the analysis comprehends the principal supervised machine-learning tasks, testing the effectiveness of encoders in different conditions
\end{itemize}
Each aspect will further be analyzed in detail in the following sections.

The source code and the experimental results are available at\\
\url{https://github.com/F3z11/CategoryEncodersEvaluation}.
This repository contains all relevant scripts, including data preprocessing, experiment pipeline, analysis, and other tools used in this experiment.

\newpage
\section{Related work}\label{section:review}
Recent studies have compared various encoding methods, although many focus only on certain aspects, lacking completeness. Some researches consider only some predictive tasks, while others evaluate only a limited amount of encoders by including only the most well-known or not covering all the principal supervised machine-learning algorithms.

Cerda and Varoquaux \cite{cerda2020encoding} principally focused on introducing an exhaustive formalization of two Semantic encoders (MinHash and Gamma-Poisson) and comparing them with two alternative encoders, lacking completeness in the number of evaluated encoding methods. Moreover, many other word embedding methods (e.g. Word2vec, BERT) are compared to the formalized encoders, but this type of embedding method is not treated in this study. Every encoder has been evaluated on various tasks and with almost every family of models. No time and computational evaluation has been conducted.

Seca and Moreira \cite{seca2021benchmark} evaluated a broader number of encoders together with a consistent amount of algorithms. However, this work covered only the regression task, not providing any insight into the performance of these encoders in classification tasks. The experiment ends with a computational and time evaluation.

Valdez-Venezuela et al. \cite{valdez2021measuring} conducted an experiment on both real and synthetic data, evaluating 10 encoders together with 5 machine-learning models for all tasks, covering most of the principal aspects aforementioned. The evaluation of classification datasets has been made on Accuracy, which is well known for not being the best metric with unbalanced data. No proper time and computational analysis have been conducted in this study.

Pargent et al. \cite{pargent2022regularized} evaluated 8 encoders together with 5 algorithms, but only on binary and multi-class classification tasks. A complete time and computational analysis have been conducted. However, this experiment evaluates most of the existing encoders applied only to the regression task, which remains the main focus of the experiment.

Matteucci et al. \cite{matteucci2024benchmark} conducted the most well-structured experiment, performing hyperparameters tuning for encoders and models and providing different evaluation methods to do a complete and robust comparison. A computational and time evaluation is provided. On the other hand, the work focused only on binary classification tasks, and not all encoders evaluated in this work have been analyzed.

Zhu et al. \cite{zhu2024comparative} work stands out as the most exhaustive evaluation in terms of the number of encoders and predictive tasks. However, many encoders have been excluded by the authors due to their behaviour or because they are not suitable for all predictive tasks, reducing the experiment coverage in terms of encoders. A computational cost evaluation is provided together with a guideline to choose the right encoder.

Reusser \cite{reusser2024tabular} conducted the most recent study, but only 6 encoders and 2 algorithms on binary classification problems have been evaluated, immediately becoming insufficient compared to the recent literature. Moreover, all numerical features have been discretized and treated as categorical variables, leading to a possible loss of information.

These last two works have not been officially published, but they include methodologies and results that significantly advance our understanding of the topic. Including these contributions is important for a comprehensive overview of the most recent and influential research developments.

\begin{figure}[!b]
\centering
\includegraphics[width=0.6\textwidth]{images/prev_work.png}
\caption{Related work connections graph based on citations}
\label{fig:literature_rev}
\end{figure}

Moreover, a fairly comprehensive survey on categorical data for neural networks has been conducted by Hancock and Khoshgoftaar \cite{hancock2020survey}, and provides insights into the theoretical foundation of some of the encoders used in this study. Figure \ref{fig:literature_rev} shows the relations among the studied papers, highlighting the importance of the first paper of Micci-Barreca \cite{micci2001preprocessing}, which first introduced the Target encoder, beginning the development of Supervised encoders.
Various encoding methods have previously been excluded from recent works \cite{zhu2024comparative} because they are not suitable for every type of data, rely on the same principle with only small changes or cannot be strictly considered as encoders. The proposed work aims to measure the performance of 24 encoding methods, including those previously excluded together with two other encoders, providing a broader overview of alternative methods compared to the existing literature.
Additionally, the selected machine-learning algorithms encompass types of models that have not yet been evaluated in the previous studies (see Section \ref{subsec:algorithms}).\\

il testo che ti ho allegato sopra contiene l'esempio di un introduzione scritta da un mio collega in inglese con latex.

vorrei che l'introduzione avesse un carattere simile a quella che ti ho dato in esempio, ma in italiano. dimmi se hai bisogno di altre informazioni

BGE-M3 è un modello di embedding testuale all'avanguardia che si distingue per la sua versatilità in tre dimensioni chiave:

Multi-lingualità: Supporta oltre 100 lingue, consentendo retrieval multilingue e cross-linguale.
Multi-funzionalità: Unifica tre funzionalità di retrieval comuni:
Retrieval denso: Genera un embedding vettoriale singolo per rappresentare il testo.
Retrieval sparso/lessicale: Produce pesi per i singoli token, simile a BM25.
Retrieval multi-vettore: Crea embeddings multipli per rappresentare parti diverse del testo.
Multi-granularità: Può processare input di diversa lunghezza, da frasi brevi a documenti lunghi fino a 8192 token.
Caratteristiche principali:

Basato su un'architettura transformer pre-addestrata (XLM-RoBERTa esteso a 8192 token)
Utilizza una strategia di "self-knowledge distillation" durante il fine-tuning
Implementa tecniche di batching efficienti per gestire sequenze lunghe
Addestrato su un dataset curato di 1.2 miliardi di coppie di testi in 194 lingue
Funzionamento:

Per il retrieval denso, utilizza l'embedding del token [CLS]
Per il retrieval sparso, applica una proiezione lineare sugli stati nascosti per ottenere pesi lessicali
Per il retrieval multi-vettore, proietta gli stati nascosti in uno spazio a bassa dimensionalità
Prestazioni:

Stato dell'arte su benchmark multilingue come MIRACL e MKQA
Eccellenti risultati su retrieval di documenti lunghi (dataset MLDR e NarrativeQA)
Mantiene buone performance su tutte le lunghezze di input fino a 8192 token
Vantaggi:

Un unico modello per molteplici task di retrieval in molte lingue
Efficace sia per ricerche monolingua che cross-linguali
Può essere utilizzato per migliorare sistemi RAG esistenti
Offre flessibilità nella scelta del metodo di retrieval più adatto per ogni caso d'uso
Il modello è open source e disponibile pubblicamente, rendendo possibile il suo utilizzo e fine-tuning per applicazioni specifiche.