Automatic Technical Domain Identification (2020)

Abstract: In this paper we present two Machine Learning algorithms namely Stochastic Gradient Descent and Multi Layer Perceptron to Identify the technical domain of given text as such text provides information about the specific domain. We performed our experiments on Coarse-grained technical domains like Computer Science, Physics, Law, etc for English, Bengali, Gujarati, Hindi, Malayalam, Marathi, Tamil, and Telugu languages, and on fine-grained sub domains for Computer Science like Operating System, Computer Network, Database etc for only English language. Using TFIDF as a feature extraction method we show how both the machine learning models perform on the mentioned languages.

## introduction we can frame automatic domain identification of given text as a text classification problem where one needs to assign predefined categories to given texts. text classification is a classic topic for natural language processing (nlp), the range of text classification research goes from designing the best features to choosing the best possible machine learning classifiers. therefore we use term frequency & inverse document frequency (tfidf) to represent our text in terms of vectors, so that the machine learning algorithms will find the relationships between them and classifies the given text. many machine learning algorithms showed the best performances on text classification , but very limited number studies have explored technical domains like computer science, chemistry, management, etc that too on indian languages @xcite . there are numerous applications of text classification in natural language processing tasks like machine translation,etc. for these tasks technical domain identification would be the first process. it determines the domain for a given input text, subsequently machine translation can choose its resources as per the identified domain. the task can also be viewed at the coarse-grained or fine-grained level based on the requirement. we did our experiments on data provided by icon techdofication-2020 shared task, for english, bengali, gujarati, hindi , malayalam , marathi, tamil and telugu languages for coarse grained domain classification . for fin-grained classification we have computer science domain in english. ## related work we treat automatic technical domain identification as a text classification task where we assign predefined categories like chem for chemistry, cs for computer science for the given text. text classification is a fundamental task in nlp applications and it is a crucial technology in many applications, such as web search, ads matching, and sentiment analysis. many researchers found variety of algorithms to solve the text classification problem. the algorithms will vary based on the langauge of text and domain of the text as well. @xcite compared the theory and practice of two different first-order probabilistic classifiers, both of which make the naive bayes assumption. the multinomial model is found to be almost uniformly better than the multi-variate bernoulli model. @xcite introduced transductive support vector machine for text classification. while general support vector machines (svms) try to produce a general decision function for a learning task, transductive support vector machines take a particular test set into account and try to minimize misclassification of just those particular samples. @xcite used maximum entropy for text classification by computing the conditional distribution of the class given the text, and compared accuracy to naive bayes and showed that maximum entropy is sometimes significantly better, but also sometimes worse. @xcite proposed a novel approach for categorizing text documents based on the use of a special kernel called string subsequence kernel. machine learning for text classification is the foundation of document categorization, news filtering, document routing, and personalization. in text domains, effective feature selection is crucial to make the learning task efficient and more accurate, based on this point @xcite presented an extensive comparative study of twelve feature selection metrics like document frequency, etc for the high-dimensional domain of text classification, focusing on support vector machines and 2-class problems, typically with high class skew. in social media such as twitter, facebook the users may become overwhelmed by the raw data. one solution to this problem is the classification of short text, in @xcite they did the same, they proposed an approach to use a small set of domainspecific features extracted from the author's profile and text to classify the text to a predefined set of generic classes such as news, events, opinions, deals, and private messages. apart from machine learning algorithms there are some deep learning techniques as well for text classification. in contrast to traditional methods, @xcite introduced a recurrent convolutional neural network for text classification without human designed features. in their model, they apply a recurrent structure to capture contextual information as far as possible when learning word representations, which may introduce considerably less noise compared to traditional window-based neural networks. @xcite presented a new architecture (vd-cnn) for text processing which operates directly at the character level and uses only small convolutions and pooling operations. @xcite used fasttext for word features and then averaged to get a sentence representation for text classification. @xcite proposed a novel approach for text clasiification termed as graph convolutional networks termed as text-gcn, it can capture global co-occurence information and uses limited labelled texts/documents well. though there exists a lot of work on text classification, very few works are done for technical domains and on indian languages like ours. therefore we present our approach on the provided indian languages along with technical domains. ## approach we evaluate our two models namely, stochastic gradient decent and multi layer perceptron on technical domains(chemistry,communication technology, computer science, law , math and physics,bio-chemistry, management) for coarse grained technical domain classification for all above mentioned languages(though the number of domains may differ from language to language). for fine grained technical domain classification we have only computer science in which sub-domains include ai, algorithm , computer architecture, computer networks , database management system , programming and software engineering for english. we used tfidf for all experiments. ## experiments & results we evaluate two machine learning algorithms on data provided by icon techdofication-2020 shared task. the data statistics in terms of number of sentences for all languages is mentioned in as mentioned in section 3.1 we used tfidf for all experiments in this paper. for sgd classifier we used hinge loss, and we took alpha as 0.00001, it is a constant that multiplies the regularization term. the higher the value, the stronger the regularization. maximum number of iterations taken for this algorithm is 15. in mlpclassifier we used relu activation function, solver as sgd which used to find the gradients and optimize the loss function. we adopted the same alpha as sgd classifier. as mlp is neural network based classifier, there is a need to give hidden layer sizes, we used @xcite for two hidden layers apart from input and output layer.