Sosyal medya üzerinden metin madenciliği ve duygu analizi ile pazar değerlendirme
Market analysis based over social networks using text mining and cluster analysis
Yazar: HATİME DİLEK BEYHAN
Danışman: YRD. DOÇ. DR. BAŞAR ÖZTAYŞİ
İstanbul Teknik Üniversitesi / Fen Bilimleri Enstitüsü / Endüstri Mühendisliği Ana Bilim Dalı / Mühendislik Yönetimi Bilim Dalı
Derece: Yüksek Lisans, 2014
Konu: Endüstri ve Endüstri Mühendisliği = Industrial and Industrial Engineering ; Mühendislik Bilimleri = Engineering Sciences

Özet:
Günümüzde firmalar için pazarı yakından takip edebilmek, müşterinin nabzını tutabilmek bir avantaj değil bir mecburiyettir. Müşterinin nabzını tutabilmek, reklam kampanyalarının, yeni ürünlerinin piyasada ne tepkiler çektiğini öğrenmek için uygulanan geleneksel bir yöntem olan müşteri anketleri hem çokça vakit almakta hem de büyük bir maaliyet oluşturmaktadır. Bunun yanında müşterileri zaman zaman sıkmakta ve çabuk, baştan savma ve esas izlenimi doğru ifade edemeyen bilgier ile doldurulmaktadır. Bu yüzden de müşterilerin yorumlarını kendi rızaları ile bıraktıkları hem de kullanıcılara ulaşmak için zaman/mekan kısıtı olmayan, insanlara istedikleri konu hakkında anket soruları ile sınırlandırmadan istedikleri yorumu yazma fırsatı sunan sosyal paylaşım ağları'nın kullanımı için yeni yöntemlerin keşfine ihtiyaç duyulmaktadır. Bu ihtiyaca cevap vermek için okuyacağınız çalışmada duygu analizi ve kümeleme yöntemlerinin birleştirildiği bir model oluşturulmuştur. 
Uygulamaya zemin hazırlamak için öncelikle sosyal paylaşım ağlarının nasıl ortaya çıktığı, hayatlarımıza nasıl dahil olduğu, nasıl sınıflara ayrıldığı ile ilgili bilgiye yer verilmiştir. Ardından literatürde yer alan geçmiş çalışmalar incelenmiştir. Son iki yılda kaleme alınan akademik çalışmalarda yükselen bir trende sahip olan sosyal paylaşım ağları ile ilgili olarak duygu analizinin, metin madenciliğinin kullanıldığı pek çok yeni çalışma ve uygulama bulunmasına rağmen duygu analizi ve kümeleme analizinin bir arada kullanıldığı çalışmalara rastlanmamıştır. Literatürün incelenmesinin ardından çalışma boyunca incelenecek olan GSM sektörünün Türkiye'deki durumu ve firmaların pazar içindeki konumları araştırılmıştır.
Uygulamanın ilk aşamasına gelindiğinde verilerin çekilmesi ve duygu analizinin yapılması için Botego firması dış kaynak olarak kullanılmıştır. Firmanın desteği ile ilk olarak belirlenen tarih aralığında GSM sektöründe yer alan üç firmanın Twitter'da adının geçtiği bütün tweetler, duygu analizleri ile birlikte alınmıştır. Ardından aynı tweetler kümeleme analizine alınarak anlamlı sonuçlar ortaya çıkana kadar farklı modeller ile kümelenmeye çalışılmıştır. Anlamlı ve değerlendirilebilir verilere ulaşıldığında kümeleme tekniği sonlandırılmış ve kümele yönteminin sonucu ile duygu analizi sonuçları bir araya getirilmiştir. Elde edilen kümelerde gerçekleştirilen her bir firmaya ait paylaşımların duygu analizleri, modelin ve uygulamanın çıktısını oluşturmuştur. Bu çıktının anlamlılığı, firmaların satış verileri ve yürüttükleri reklam ve pazarlama stratejileri göz önünde bulundurularak değerlendirilmiştir. Son olarak çalışmanın geliştirilmeye açık yönleri belirtilmiş ve gelecek çalışmalar için önerilerde bulunulmuştur.

Abstract:
When we look at the last decade, Social Network usage is the biggest changes in daily life. Internet is not only a business or an academic tool anymore, but also it is a part of the social life. Increasing of Social Network Sites usage also increases the number of users and this situation brings wider usage area for Social Network Sites. Users started to leave comments about their purchasing and they showed Social Networking Sites' commercial impact. In fact, this commercial impact attracts academics' attention to this area. Some of academic studies focused on different usage of Social Networking Sites; for example, they classified users by their usage frequency or personality, they tried to understand human attitude in social network sites, but these types of studies are related with social science. Social sciences show a way to use this information in management science to discover new areas in marketing and other managerial businesses; for instance, competition pattern, position or strategy can be defined by using comments about consumable goods and services.
Getting user comments with traditional methods needs direct contact with customer and takes a long time and too much manpower. As a result of this, making face o face surveys creates high costs for companies. On the other hand, investigating and analysis of tweets do not have any cost except for manpower. This manpower is really low that cannot be compared with making surveys. Because the only effort is spent at the beginning to develop algorithm and this algorithm can be used in a long period. 
First of all, terms and definitions must be known to understand the study. Social Networks are web-based services that ensure people to create profiles which can be seen by public, create permission list and see other users' lists (Boyd and Ellison, 2007). The most important issue is network, which connects people via lists or sharings etc. The first example of social networks is dating sites. Dating sites offer their users to create their own profile page on the website and communicate with others. 
The study that you read is focused on Twitter sharings. Twitter is a microblogging site which ensures its users to express themselves with 140 characters and ability to see others expressions (URL-3). All followers can see instant sharing in the same time. This study focused on usage of this website in competition by evaluating tweets, which includes company names. These companies are chosen from GSM providers in Turkey. Only players of this market are Turkcell (50,77% subscriber, 49,21% market share), Vodafone (28,75% subscriber, 30,56% market share) and Avea (20,48% subscriber, 20,24% market share) (URL-21). The study collects tweets with these brand names, evaluate them as positive, negative or neutral and classify them with their similarities. 
Two methodologies are applied in this study. First, sentiment analysis is a method that tries to find how emotions are expressed in texts and extracts sentiment and emotions from texts (Nasukawa and Yi, 2003). It is possible to reach customer comments on product or services by sentiment analysis instead of customer surveys. In this study, sentiment analysis step is done by Botego, which is a professional company having their own sentiment analysis software. In addition, they supported us to transfer data from Twitter.
Second applied method is text mining, which is the semi-automated process of extracting pattern from large amounts of unstructured data sources (Turban et al. 2011). Text mining has wide usage in some areas; such as law, academic research, finance, medicine, biology, technology and marketing.  
Clustering is a process that divides a set of data or observation into subsets. Each subset has similar features in itself and each subset has different features from each other. Cluster analysis is used in wide area such as business intelligence, image detection, web search, biology etc. 
First step of the application is transferring data from Twitter and make sentiment analysis on them. The software (developed by Botego) is used for this purpose and tweets are classified as positive, negative and neutral. Data from database and domain are taught to software manually in the beginning. This process is done approximately with 2000 different words. After this process, application is ready to classify words automatically. In this study, "Avea", "Turkcell" and "Vodafone" keywords are used for analysis and analysis run between 04.02.2014 and 11.03.2014 dates.
Next step is data cleansing. In order to provide precision to analysis, data must be simplified. First, every single tweets are deleted which have 2 different brand names in themselves; because carrying 2 different brand names confuses sentiment analysis results (the question is "positive for which one?"). Second, commercial tweets are deleted which are posted by brands official accounts for advertising reasons. Third, location sharing (from mobile applications) tweets are deleted; because they are not related with GSM provider usage or activities.
Third step of the study is cluster analysis. Rapidminer 5.3 software is used for building model and creating clusters. The most important process in this model is "process document from data". In this process, data is transferred a new form which can be evaluated by cluster analysis correctly. Example processes are tokenize (disassemble tweets to words), switching all cases to lower case, filtering unnecessary words (prepositions, conjunctions) etc. Unfortuantely, Rapidminer 5.3 software cannot support Turkish language as much as it supports English language. Therefore filtering step is done manually. After this process, data is classified with different number of clusters. The most suitable and interpreted cluster quantity is chosen for the study. Given names of clusters are; Sport (Spor), 	Romantic (Romantik), Internet, Purchasing (Alış-Veriş), Commercial (Reklam) and Others (Diğerleri). Names are given by investigating TF-IDF scores of words and observing tweets which are included by clusters. 
Following step is determination of sentiments according to clusters. After defining clusters with their features, sentiment analysis results and clusters are matched. Sentiment scores of each GSM provider in each cluster are defined. Results show that actual market share and income are parallel with tweet numbers. It can be easily seen that if the brand has the biggest share in the market, it has biggest share in Twitter via tweets as well. However, there is no relationship between positive tweet numbers and actual subscriber quantities. In general view, most of the tweets have negative sentiment scores. It can be said that users tend to share their bad experiences in social media more than their good experiences or pleasures. 
When cluster concepts, tweets and sentiment scores are investigated, every cluster has their own features in market. One of the most noticeable results is either viral or classical commercials are mentioned too much in Twitter. So that, feedbacks, about commercials, can be taken from Twitter or other Social Medium. Another remarkable result is sponsorships are very important to attract attention from private community such as sport fans. Negative ranks are very high in Romantics; because of that, they are very uncomfortable to get messages from their GSM operator instead of their lover. Customers observe purchasing activities and reflect their emotions about these activities into Twitter. Firms can follow these comments to uınderstand reaction of customers about their financial activities. By analyzing social media, firms are able to evaluate themselves among their competitors in market and they can access valuable informations about competitors.
To sum up, one of the wide usage area of social medium is market analysis. Todays, many of customers are also social media users/followers and they post their comments about their purchasing activities to social media sites. Other social media users can see these posts and reinterpret these data and perhaps they can change their consuming activities. Therefore, companies should consider these data to track the condition of the market. To handle with this huge data, companies can divide them into clusters and evaluate each of them separately with respect to their sentiment scors. This is the methodolgy that this study presents. Future studies can focus on other markets and use other methods to evaluate sentiment scores in management sciences.