LFTK : Handcrafted Features in Computational Linguistics (2023)

Abstract: Past research has identified a rich set of handcrafted linguistic features that can potentially assist various tasks. However, their extensive number makes it difficult to effectively select and utilize existing handcrafted features. Coupled with the problem of inconsistent implementation across research works, there has been no categorization scheme or generally-accepted feature names. This creates unwanted confusion. Also, no actively-maintained open-source library extracts a wide variety of handcrafted features. The current handcrafted feature extraction practices have several inefficiencies, and a researcher often has to build such an extraction system from the ground up. We collect and categorize more than 220 popular handcrafted features grounded on past literature. Then, we conduct a correlation analysis study on several task-specific datasets and report the potential use cases of each feature. Lastly, we devise a multilingual handcrafted linguistic feature extraction system in a systematically expandable manner. We open-source our system to give the community a rich set of pre-implemented handcrafted features.

## introduction handcrafted linguistic features have long been inseparable from natural language processing (nlp) research. even though automatically-generated features (e.g., word2vec, bert embeddings) have recently been mainstream focus due to fewer manual efforts required, handcrafted features (e.g., type-token ratio) are still actively found in currently literature trend @xcite @xcite @xcite . therefore, it is evident that there is a constant demand for both 3 core contributor the identification of new handcrafted features and utilization of existing handcrafted features. after reviewing the recent research, we observed that most research on automatically-generated features tends to focus on creating deeper semantic representations of natural language. on the other hand, researchers use handcrafted features to create wider numerical representations, encompassing syntax, discourse, and others. an interesting new trend is that these handcrafted features are often used to assist auto-generated features in creating wide and deep representations for applications like english readability assessment @xcite and automatic essay scoring @xcite . the trend was observed across various tasks and languages. for example, there are arabic speech synthesis @xcite , burmese translation @xcite , english-french term alignment @xcite , german readability assessment @xcite , italian pre-trained language model analysis @xcite , korean news quality prediction @xcite , and spanish hate-speech detection @xcite systems. though using handcrafted features seems to benefit multiple research fields, current feature extraction practices suffer from critical weaknesses. one is the inconsistent implementations of the same handcrafted feature across research works. for example, the exact implementation of the average words per sentence feature can be different in @xcite and @xcite even though both works deal with text readability. also, there have been no standards for categorizing these handcrafted features, which furthers the confusion. in addition, no open-source feature extraction system works multilingual, though handcrafted features are increasingly used in non-english applications. the handcrafted linguistic features can be critical resources for understudied or lowresource languages because they often lack highperformance textual encoding models like bert. in such cases, handcrafted features can be useful in creating text embeddings for machine learning studies @xcite @xcite . in this paper, we make two contributions to address the shortcomings in the current handcrafted feature extraction practices. in a way, we aim to address the recent concern about the lack of ready-to-use code artifacts for handcrafted features @xcite . through this work, we hope to improve the general efficiency of identifying and implementing handcrafted features for researchers in related fields. ## conclusion in this paper, we have reported our open-source, large-scale handcrafted feature extraction system. though our extraction system covers a large set of pre-implemented features, newer, task-specific features are constantly developed. for example, urls count is used for twitter bot detection @xcite and grammatical error count is used for automated essay scoring @xcite . these features, too, fall under our definition (figure 2 ) of handcrafted linguistic features. our open-source script is easily expandable, making creating a modified, research-specific version of our extraction program more convenient. with various foundation features to build from, our extraction program will be a good starting point. another potential user group of our extraction library is those looking to improve a neural or nonneural model's performance by incorporating more features. performance-wise, the breadth of linguistic coverage is often as important as selection @xcite @xcite . our current work has various implemented features, and we believe the extraction system can be a good starting point for many research works. compared to other historically important code artifacts like the coh-metrix @xcite and l2 syntactic complexity analyzer @xcite , our extraction system is comparable or larger in size. to the best of our knowledge, this research is the first attempt to create a "general-purpose" handcrafted feature extraction system. that is, we wanted to build a system that can be widely used across nlp tasks. to do so, we have considered expandability and multilingualism from architecture design. and such consideration is grounded in the systematic categorization of popular handcrafted linguistic features into the attributes like domain and family. with the open-source release of our system, we hope that the current problems in feature extraction practices (section 1) can be alleviated.