# Hyperparameter Black-Box Optimization to Improve the Automatic Classification of Support Tickets
## 2 Feature Extraction from Textual Tickets  
Each ticket is essentially a small text describing some problem(s) and/or asking some question(s) in natural language. The level may go from very elementary sentences to complex and involved periods. The mentioned semantic categorization of the tickets is needed to assign the tickets to the employees, so that each one is answered by an expert on the specific subject. This would pursue the advantages already described in Section 1. Moreover, this categorization helps in the design and the improvement of the process (e.g.,the survey), since an abnormal frequency of tickets on a specific aspect may highlight the presence of problems (e.g., unclear questions, defective prompts, etc.). In many cases, for example, requests received by telephone and recorded, there is no metadata supporting the semantic categorization. In other cases, such as emails, the metadata (e.g., the subject of the email) may be present, but they are often too generic, not reliable enough, or not concerning the desired categorization. In all the above cases tickets need to be categorized homogeneously and preferably automatically. Evidently, basic features of the tickets, like the length of the message or similar measures, are not meaningful for semantic categorization. Even the basic presence or absence of predetermined words is not enough, because in many cases similar words (answer, form, compile, fill, etc.) may describe completely different types of problems. Therefore, a more in-depth analysis is needed to extract the significance of a ticket from its text in natural language. A scheme of the overall procedure described in this work is reported in the subsequent Figure 1, while the sequence of the steps is reported in Algorithm 1. ```
1. Extraction of Relevant Terms from the Corpus of 15,000 Tickets
1.a Tokenization
1.b Stop-words Elimination
1.c Lemmatization and Part-Of-Speech recognition
2. Preparation of the Records (=Standardized Description) of the Tickets
2.a Word Embedding to associate Relevant terms with vectors
2.b Selection within the 31,000 Relevant Terms

of the 424 Most Relevant Terms
2.c Projection of the Tickets in the space of the Most Relevant Terms

to obtain the Records
3. Labeling of 8076 Records by Human Experts
4. Training of the Classifier with the 8076 Labeled Records
4.a Splitting of training and test sets
4.b Optimization of the Hyperparameters: For every

combination of Hyperparameters V' do the following:

4.b.i Training with Hyperparameters V'

4.b.ii Testing and evaluation of the performance with Hyperp.V'

4.c Final training with the optimal Hyperparameters
5. Categorization of all the Tickets (193,419 Records) using

the optimal Hyperparameters
``` ```  
**Algorithm 1**Categorization of Tickets. After an initial Tokenization (individuation of the words within the sequence of characters) and Stop-words Elimination (elimination of useless parts, like articles, etc. ), we perform Lemmatization with Part-Of-Speech recognition. This means that, for each word, we remove the inflectional ending to identify its basic lemma. This allows us to recognize together the different inflected forms of a word (e.g., plurals of nouns, tenses of the verbs, etc.). Moreover, we still keep track of which part of speech each word is (e.g., noun, verb, adjective, etc.). This Natural Language Processing (NLP) is done by using the Gensim python library from scikit learn [25]. Since the tickets are in the Italian language, we perform the above operations by using an Italian dictionary. However, the language can easily be changed by simply switching the underlying dictionary. After this, we need to convert each ticket into a data record constituting a \"standardized description\" of the ticket. This feature extraction is done by using the word embedding algorithm Word2 vec [20], still in the Gensim python library. This algorithm uses a shallow neural network to learn word associations in a large corpus of text, all the tickets in our case. Hence, it can detect synonymous words.