# A Novel Framework for Identifying Customers' Unmet Needs on Online Social Media Using Context Tree
## 3 Proposed Method
### Gathering and Pre-Processing the Online Social Media Data  
The first step is to determine the target online social media. Social media has highly diverse categories with numerous topic types. Social network services (SNSs), such as Twitter, Facebook, and LinkedIn, can enable gathering opinions from the unspecific majority, or comments by readers on various news sites and writing on community sites, such as Reddit, where various topics are discussed, can be good analysis targets. Although this study focuses on texts, services, such as Instagram, can also be considered if image data and corresponding hashtags are analyzed together. In addition, if customer's needs to specific domains are required, companies can target comments on movie review sites or online shopping sites. Once the target data is determined, it is necessary to search for a method to collect them. For online social media data collection, it is preferable to use application programming interfaces (APIs) of corresponding sites [37]. Note that APIs may not provide the data fields preferred by users or limited data quantities and collection time while using APIs. A strategic partnership may be necessary to utilize APIs or to share costs. In addition, companies may utilize dumped data disclosed for the purpose of research. Sites, such as Wikipedia, where various users participate, often provide updated dumped data, and Amazon.com discloses product review data for the purpose of research. A proactive method of gathering data is web page information through the web scrapping technique and extracting values that correspond to required data fields using HTML parsers. After data gathering, preprocessing should be conducted. For text data, Natural Language Processing (NLP) tools in accordance with a language type should be used [38]. The important things in the proposed framework particularly are word extraction, word normalization, and out-of-vocabulary problem-solving. Word extraction extracts words used by the majority from free-formed texts. The word 'normalization' refers to a conversion from changed terms into basic type words, which is significantly conducive to unifying information inside texts [39]. Tokenizing, part-of-speech tagging, stemming, and lemmatizing are necessary for word extraction and normalization, as a general procedure in NLP [40]. * Tokenizing: It is the extraction of linguistic units (e.g., words, phrases, or specific sequences of characters) used by people from unstructured texts. A white space and punctuations that determine the boundary of such units are often used in tokenizing, and variations, such as a change in character types, collocations, and idioms, are also considered in the tokenizing process. * Part-of-speech (POS) tagging: POS tagging is the process of assigning part-of-speech tags, including noun, verb, adjective, adverb, etc., to words in a sentence or corpus. A POS tagger determines a POS tag for each word, considering a sentence structure and surrounding context. For English, the Penn Treebank tag set has been widely used. In this framework, it is necessary to derive nouns that refer to objects, adjectives and adverbs that reveal emotion, and verbs that express actions through POS tagging. * Stemming: This is the process of transforming several words into a base root form. For example, 'computer', 'computing', 'computed', and 'computation' are transformed into a root form of 'comput'. Stemming enables information collection from within documents containing the four existing words in an integrated manner. * Lemmatizing: Lemmatizing also refers to the summarization of words utilized in various forms into a single representative lemma. Stemming considers only processing target words, whereas lemmatizing considers elements to identify the surrounding contexts, for example, part-of-speech (POS) tags. Thus, lemmatizing takes more time than stemming. The most difficult aspect of analyzing social media data is processing slang or jargon [41].