## 3 Fundamental Concepts
### Data Mining from Twitter  
_Twitter_ is one of the most popular \"microblogging\" tools [45]. In essence, _Twitter_ allows the publication of 280-characters limited messages denominated _tweets_ sharing any kind of information such as ideas, opinions, impressions or facts about the users' everyday life. These _tweets_ are instantly shared to the user network of followers and, unless previously deactivated, all _tweets_ are publicly available. Being a follower of someone on _Twitter_ means that the user receives all the messages published by the followed person in his _timeline_ and this relationship of following and being followed does not require a reciprocate [46; 47]. The reduced size and real-time nature of _Twitter_ messages, associated with the asymmetric social relationship of its users, characterizes this social media as a rich source of fresh information rather than other social networking sites [46; 48]. Over the last years, _Twitter_ has been explored as a distributed social sensor system, allowing real-time detection and tracking of earthquakes [47], detecting patterns on public health [38] or even predicting the stock market [49]. The core of strategies for mining information from the web involves various methods and techniques that originate from \"Information Retrieval\", \"Artificial Intelligence\" and \"Natural Language Processing\" areas [48]. This sum of different techniques is explained by the complexity and challenges imposed due to the velocity, variety and high volume of the data [50]. In _Twitter_ data streams, the challenges are extended due to the short length of _tweets_, frequent and evolving use of informal, irregular and abbreviated words, improper sentence structures and mixed languages. Such characteristics of _Twitter_ data makes the traditional text analysis less suitable for _tweets_[51]. The first step of any data mining pipeline is the data collection and consists in collecting the _tweet_ corpus that will be mined [50]. In _Twitter_, this task can be done by using their public Application Programming Interface (API). By default, a _tweet_\(w\) is composed by several attributes or _features_ attached to itself, including the text \(w.D\), the spatial information \(w.A\), which is composed by a pair of geographic coordinates (longitude and latitude, \(w.A=(w.A_{lon},w.A_{lgt})\)) describing the local from where it was sent, the timestamp \(w.t\) in which the _tweet_ was sent and its corresponding unique id \(w.id.\) Other properties, such as author username, language and a list of used hashtags are also present in the _tweet_ object, which is organized in the JSON format [52]. The raw data acquired from _Twitter_ are noisy and contains lots of uninteresting, missing or inconsistent information. For example, a _tweet_ is not always geotagged and the message may contain typos or abbreviated words. Low-quality data leads to low-quality results, thus the data must be preprocessed in order to improve its quality and, consequently, refining and facilitating the mining process [53]. In general, the data preprocessing phase in _Twitter_ data mining applications contains the following steps:  
* **Feature Extraction:** The raw data extracted from _Twitter_ contains features which may not be important to a given mining application (e.g., user profile image url). This cycle consists in choosing which features objects are relevant and will compose the vector of features \(w.f=[f_{1},f_{2},f_{3},...,f_{n}]\)[54]. For example, in most spatial _Twitter_ applications [55; 56; 57; 58], \(w.f=[w.id,w.D,w.t,w.A]\). The other features are stored and, if necessary, they can be retrieved in any step by using \(w.id\) as identifier. * **Data Cleaning:** Real-world data tend to be incomplete, noisy and inconsistent. To solve this issue, some entries need to be dropped or some of its entries must be estimated in order to remove major inconsistencies [53; 54]. When dealing with _Twitter_ data with the previously mentioned features, two major dissonances can be identified.