Patent Abstract:
Systems and techniques for detecting and verifying social media events are disclosed. The system and techniques allow for processing of social media data to extract potentially valuable information in a timely manner and determine the veracity of the detected information. One implementation of the disclosure relates to event detection. Event detection involves ingestion and processing of social media data. Another implementation of the disclosure relates to verification of a detected event and generating a verification score.

Full Description:
CROSS-REFERENCE TO RELATED APPLICATION 
       [0001]    This application claims priority to U.S. Provisional Application No. 62/158,609, filed May 8, 2015, entitled “DEBUNKING RUMORS IN TWITTER BEFORE NEWS ORGANIZATIONS” and U.S. Provisional Application No. 62/186,419, filed Jun. 30, 2015, entitled “SYSTEM AND METHOD FOR AUTOMATICALLY DETECTING AND VERIFYING SOCIAL MEDIA EVENTS”. Each of the applications referred to in this paragraph is incorporated herein by reference in its entirety. 
     
    
     COPYRIGHT NOTICE 
       [0002]    A portion of the disclosure of this patent document contains material, which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyright rights whatsoever. The following notice applies to this document: Copyright @ 2015 Thomson Reuters. 
       TECHNICAL FIELD 
       [0003]    This disclosure relates to event detection and verification, and more particularly methods and systems for detecting and verifying an event from social media data. 
       BACKGROUND 
       [0004]    Social media platforms like Twitter® or Facebook®, have influenced news gathering. Every minute, people around the world are posting pictures, videos, tweeting and otherwise communicating about all sorts of events and happenings. For example, a person may comment on what they see at a scene of an accident in real-time. Since people geographically close to an event are a valuable source of breaking news, the information generated by them is potentially very valuable. However, leveraging such information is very difficult. 
         [0005]    According to statistics on the Twitter® website, there are approximately 320 million twitter users, of which, 65 million are in the United States and 254 million internationally (Twitter Q4 2015 Earnings Report, pp. 4). There are also approximately 350,000 tweets per minute. The percentage of valuable information is very small compared to the entire social media data available at a time. It has been noted that social media data primarily includes rumors, noise, spam, and mostly information useless to a professional consumer. As a result, potentially useful information is very hard to discover. Furthermore, discovery of useful information does not assure accuracy of the claimed event. 
         [0006]    Currently, the tools in the marketplace take a bottom-up approach to tackling extraction of information from social media. Users interested in niche information may search by keywords or maintain broad databases of people to follow in hope to capture useful information from social media data. This bottom-up approach of information extraction requires guess work and constant maintenance of lists and keywords. 
         [0007]    Accordingly, improved systems and techniques are needed that detect emerging trends at the social media data level and verify the authenticity of the emerging trends. 
       SUMMARY 
       [0008]    Systems and techniques for detecting and verifying social media events are disclosed. The system and techniques allow for processing of social media data to extract potentially valuable information in a timely manner and determine the veracity of the detected information. 
         [0009]    One aspect of the disclosure relates to event detection. Event detection involves ingestion and processing of social media data. For example, according to one aspect, a method includes receiving, by an event detecting server, social media data from at least one data source and applying, by the event detecting server, a set of filters to the social media data to generate a data store (i.e. a database or hashmap), the data store comprising a set of identified concepts and corresponding attributes of the social media data. The method also includes selecting, by the event detecting server, one of the set of identified concepts from the database using a corresponding threshold value associated with the attributes of the social media data and generating, by the event detecting server, an event cluster using the selected identified concept. The method may further include deleting by the event detecting server, the selected identified concept from the database. 
         [0010]    In one implementation, the method also includes detecting language of the social media data and removing the social media data that is not in English. In another implementation, the method also includes detecting profanity used in the social media data and removes the social media data containing the detected profanity. In yet another implementation, the method may include detecting at least one of spam, chat and advertisement in the social media data and removing the social media data that contains the at least one detected spam, chat and advertisement. 
         [0011]    In a further implementation, the method includes applying Parts-Of-Speech tagging of the social media data. In an alternative implementation, the method may include analyzing semantic and syntactic structures in the social media data to determine identified concepts in the social media data. 
         [0012]    A threshold value may be used for selection of one of the set of identified concepts from the database and may be associated with a selectable number of distinct attributes (i.e., three distinct attributes) of the social media data related to the identified concept. In one implementation, one of the attributes of the social media data is an authorship value (i.e., the user) and the corresponding threshold value represents a predetermined number (i.e., three) of similar identified concepts associated with different authorship values (i.e., different users). 
         [0013]    In yet a further implementation, the method includes but is not limited to generating a newsworthy score, a topic classification, a summary, and a credibility score for each cluster and its corresponding data. 
         [0014]    In one implementation, for example, the method further includes generating a verification score for each cluster and its corresponding data, the verification score is indicative of the veracity or accuracy of each assertion in the cluster. The veracity score and event clusters may be provided to the user on a graphical user interface. 
         [0015]    In one implementation, the veracity score is determined by analyzing user category, social media level and event features. 
         [0016]    The user category comprises, but is not limited to, determining at least one of name of author, description of author, URL of author, location of author, location of the author matching the location of the event, author being a witness to the event, protection level of the author&#39;s account, and verification of the author, associated with each item of the social media data. 
         [0017]    The social media level comprises, but is not limited to, determining at least one of multimedia, url, elongated word, url from news source, and word sentiment associated with the social medial data. 
         [0018]    The event features comprises, but is not limited to, determining at least one of topic of the event and portion of the social media that deny, believe or question the event associated with each item of the social media data. 
         [0019]    In a further implementation, wherein the social media data is twitter data, the event features further comprises determining at least one of a count of the most retweeted tweets, a frequency of retweeted tweets and a frequency of hashtags associated with each item of the social media data. 
         [0020]    Systems, devices, as well as articles that include a machine-readable medium storing machine-readable instructions for implementing the various techniques, are disclosed. Details of various implementations are discussed in greater detail below. 
         [0021]    One advantage relates to accuracy and speed. For example, in one implementation, using the above systems and techniques, collective users may be able to predict the veracity of an event with approximately 85% accuracy and faster than mainstream media can confirm the same information. 
         [0022]    Additional features and advantages will be readily apparent from the following detailed description, the accompanying drawings and the claims. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0023]      FIG. 1  is an exemplary architectural diagram of the system; 
           [0024]      FIG. 2  is an exemplary event processing server; 
           [0025]      FIG. 3 a    is an exemplary flow chart of one implementation of the disclosure; 
           [0026]      FIG. 3 b    is an exemplary flow chart of another implementation of the disclosure; 
           [0027]      FIG. 4 a    illustrates exemplary elements in a veracity calculation; 
           [0028]      FIG. 4 b    illustrates exemplary elements in an alternative verification calculation; 
           [0029]      FIG. 5 a    illustrates an exemplary processing of an item of social media data; 
           [0030]      FIG. 5 b    illustrates an example table representation of mapping key concepts to the respective social media data; 
           [0031]      FIG. 5 c    illustrates an example database representation in relation to the exemplary social media data of  FIG. 5   a;    
           [0032]      FIG. 5 d    illustrates an example unit cluster; 
           [0033]      FIG. 5 e    illustrates an exemplary ingested data; 
           [0034]      FIGS. 5 f -5 k    is an exemplary metadata of ingested data in  FIG. 5   e;    
           [0035]      FIGS. 5 l -5 n    is an exemplary metadata of an event detected cluster with ingested data of  FIG. 5 e    as one of the related unit data; 
           [0036]      FIG. 6 a    illustrate default event detected clusters viewable through an exemplary graphical user interface (GUI); 
           [0037]      FIG. 6 b    illustrate exemplary event detected clusters viewable through an exemplary graphical user interface (GUI); 
           [0038]      FIG. 6 c    illustrate a selected event detected cluster viewable through an exemplary graphical user interface (GUI); and 
           [0039]      FIG. 7 a -7 e    illustrate additional filters on event detected clusters available through an exemplary graphical user interface (GUI). 
       
    
    
     DETAILED DESCRIPTION 
       [0040]    In the following description, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration specific implementations in which the disclosure may be practiced. It is to be understood that other implementations may be utilized and structural changes may be made without departing from the scope of the present disclosure. 
         [0041]      FIG. 1  shows an exemplary system  100  for detecting and verifying an event from social media data. As shown in  FIG. 1 , in one implementation, the system  100  is configured to include an event detection server  110  that is in communication with a social media platform  180  over a network  160 . The system  100  further comprises an access device  170  that is in communication with an event processing server  210  over the network  160 . Further details of an exemplary event processing server  210  are illustrated in  FIG. 2 . The event detection server  110  is in communication with the event processing server  210  over the network  160 . Access device  170  can include a personal computer, laptop computer, or other type of electronic device, such as a mobile phone, smart phone, tablet, PDA or PDA phone. In one implementation, for example, the access device  170  is coupled to I/O devices (not shown) that include a keyboard in combination with a point device such as a mouse for sending an event request to the event processing server  210 . Preferably, the access device  170  is configured to include a browser  172  that is used to request and receive information from the event processing server  210 . Communication between the browser  172  of the access device  170  and event processing server  210  may utilize one or more networking protocols, which may include HTTP, HTTPS, RTSP, or RTMP. Although one access device  170  is shown in  FIG. 1 , the system  100  can support one or multiple access devices. 
         [0042]    The network  160  can include various devices such as routers, servers, and switching elements connected in an Intranet, Extranet or Internet configuration. In some implementations, the network  160  uses wired communications to transfer information between the access device  170  and the event processing server  210 , the social media platform  180  and the event detection server  110 . In another implementation, the network  160  employs wireless communication protocols. In yet other implementations, the network  160  employs a combination of wired and wireless technologies. 
         [0043]    As shown in  FIG. 1 , in one implementation, the event detection server  110 , may be a special purpose server, and preferably includes a processor  112 , such as a central processing unit (‘CPU’), random access memory (‘RAM’)  114 , input-output devices  116 , such as a display device (not shown), and non-volatile memory  120 , all of which are interconnect via a common bus  111  and controlled by the processor  112 . 
         [0044]    In one implementation, as shown in the  FIG. 1  example, the non-volatile memory  120  is configured to include an ingestion module  122  for receiving social media data from the social media platform  180 . Exemplary social media platforms are, but not limited to, Twitter®, Reddit®, Facebook®, Instagram® or LinkedIn®. As used herein, the phase “ingested data” refers to received social media data, which may be but is not limited to, tweets and/or online messages, from the social media platform  180 . 
         [0045]    The non-volatile memory  120  also includes a filtering module  124  for processing ingested data. In one implementation, processing of the ingested data may comprise but is not limited to, detecting language of the ingested data and filtering out ingested data that contains profanity, spam, chat and advertisements. 
         [0046]    The non-volatile memory  120  is also configured to include an organization module  126  for analyzing semantic and syntactic structures in the ingested data. In one implementation, the organization module  126  may apply part-of-speech tagging of the ingested data. In another implementation, the organization module  126  detects key concepts included in the ingested data. 
         [0047]    As shown in the  FIG. 1  example, the non-volatile memory  120  may also be configured to include a clustering module  128  for storing key concepts identified by the organization module  126  into a database, an example of which may be but is not limited to a hashmap, and generating an event detected cluster upon reaching a threshold of distinct ingested data containing common key concepts. 
         [0048]    The non-volatile memory  120  is also further configured to include a topic categorization module  131  for classifying the event detected cluster by topics; a summarization module  132  for selecting a representative description for the event detected cluster; and a newsworthiness module  133  for determining a newsworthy score to indicate the importance of the event detected cluster. 
         [0049]    The non-volatile memory  120  is also configured to include an opinion module  134  for detecting if the each ingested data in the event detected cluster contains an opinion of a particular person or is factual (e.g., non-opinionated tone), and a credibility module  135 , for determining the credibility score of the ingested data. In one implementation, the credibility score is associated with three components: user/source credibility: who is providing the information, cluster credibility: what is the information, and tweet credibility: how is the information related to other information. 
         [0050]    The non-volatile memory  120  is further configured to include verification module  150  for determining the accuracy of the event detected cluster. In one implementation, verification may be done by a veracity algorithm which generates a veracity score. In another implementation, the verification module  150  may generate a probability score for an assertion being true based on evidences collected from ingested data. 
         [0051]    The non-volatile memory  120  is further configured to include a knowledge base module  152  for developing a database of information pertaining to credible sources and stores the information in a knowledge base data store  248  ( FIG. 2 ). 
         [0052]    As shown in the exemplary  FIG. 1 , a data store  140  is provided that is utilized by one or more of the software modules  124 ,  126 ,  128 ,  131 ,  132 ,  133 ,  134 ,  135 ,  150 ,  152  to access and store information relating to the ingested data. In one implementation, the data store  140  is a relational database. In another implementation, the data store  140  is a file server. In yet other implementations, the data store  140  is a configured area in the non-volatile memory  120  of the event detection server  110 . Although the data store  140  shown in  FIG. 1  is part of the event detection server  110 , it will be appreciated by one skilled in the art that the data store  140  can be distributed across various servers and be accessible to the server  110  over the network  160 . As shown in  FIG. 1 , in one implementation, the data store  140  is configured to include a filtered data store  141 , an organization data store  142 , a cluster data store  143 , a topic categorization data store  144 , a summarization data store  145 , a newsworthiness data store  146 , an opinion fact data store  147 , a credibility data store  148  and a veracity data store  154 . 
         [0053]    The filtered data store  141  includes ingested data that has been processed by the filtering module  124 . For example, in one implementation, the ingested data processed by filtering module  124  may be English language tweets that do not contain profanity, advertisements, spam, chat or advertisement. 
         [0054]    The organization data store  142  includes ingested data that has been processed by the organization module  126 . In one implementation, the ingested data in organization data store  142  may include parts-of-speech tagging notations or identified key concepts, which are stored as a part of ingested data metadata. 
         [0055]    The cluster data store  143  includes ingested data that has been processed by filtering module  124  and organization module  126  and is queued to be formed into a cluster. In a further implementation, the cluster data store  143  may also contain a data store or database of key concepts (e.g. hashmap) identified by the organization module  126  matched to corresponding ingested data. As used herein with relation to the database of key concepts, ingested data (e.g., tweets and/or online messages) may also be referred to as unit data. 
         [0056]    The topic categorization data store  144  includes the classification of the event detected cluster determined by the topic categorization module  131 . Exemplary topics may include but are not limited to business/finance, technology/science, politics, sports, entertainment, health/medical, crisis(war/disaster), weather, law/crime, life/society, and other. 
         [0057]    The summarization data store  145  includes a selected unit data that is representative of the event detected cluster as determined by the summarization module  132 . 
         [0058]    The newsworthiness data store  146  includes the newsworthy score computed by newsworthiness module  133 . For example, a higher score would imply that the event detected cluster is likely to be important from a journalistic standard. 
         [0059]    The opinion data store  147  includes information pertaining to the determination by the opinion module  134  of whether a given unit data comprises an opinion of a particular person or an assertion of a fact. 
         [0060]    The credibility data store  148  includes a credibility or confidence score as determined by the credibility module  135 . 
         [0061]    The veracity data store  154  includes metrics generated by the verification module  150  regarding the level of accuracy of the event detected cluster. In one implementation, it may be the veracity score determined through a veracity algorithm. In another implementation, it may be a verification score indicating the probability of accuracy based on all the evidences collected from social media. 
         [0062]    In a further implementation, as shown in  FIG. 1 , the Event Processing Server  210  includes a processor (not shown), random access memory (not shown) and non-volatile memory (not shown) which are interconnected via a common bus and controlled by the processor. In one implementation, the Event Processing Server  210  is responsible for storing processed information generated or to be used by the Event Detection Server  110 . In another implementation, the Event Processing Server  210  also communicates directly with the user. The Event Processing Server  210  is further illustrated in relation to  FIG. 2 . 
         [0063]    It should be noted that the system  100  shown in  FIG. 1  is one implementation of the disclosure. Other system implementations of the disclosure may include additional structures that are not shown, such as secondary storage and additional computational devices. In addition, various other implementations of the disclosure include fewer structures than those shown in  FIG. 1 . 
         [0064]    Turning now to  FIG. 2 , the Event Processing Server  210  in one implementation contains a web server  220  with a non-volatile memory  230  and a UI (user interface) module  232 . 
         [0065]    The UI module  232  communicates with the access device  170  over the network  160  via a browser  172 . The UI module  232  may present to a user through the browser  172  detected events clusters and their associated metadata. Exemplary associated metadata may be but are not limited to the topic, newsworthiness indication and verification score associated with one or more event detected clusters. 
         [0066]    The event processing server  210  may further comprise a data store  240  to host an ingested data store  242 , a generated cluster data store  244 , an emitted data store  246  and the knowledge base data store  248 . 
         [0067]    The ingested data store  242  includes ingested data received from social platform  180  and processed by ingestion module  122 . 
         [0068]    The generated cluster datastore  244  includes the event detected clusters that have been processed by modules  122 ,  124 ,  126 ,  128 ,  131 ,  132 ,  133 ,  134 ,  135  and  150 . 
         [0069]    The emitted data store  246  includes key concepts and corresponding ingested data that were discarded by the clustering module  128 , as explained in relation to steps  330 - 332  of  FIG. 3 a   . In an alternative implementation, the emitted data store may be located in the event detection server  110 . 
         [0070]    The knowledge base data store  248  includes a list of credible sources as determined by knowledge base module  152 . 
         [0071]    In one implementation, the Event Processing Server  210  communicates with the Event Detection Server  110  over the network  160 . In another implementation, the Event Processing Server  210  is included in the nonvolatile memory  120  of Event Detection Server  110 . In yet another implementation, the Event Processing Server  210  is configured to communicate directly with the Event Detection Server  110 . An exemplary event processing server  210  may be but is not limited to MongoDB® or ElasticSearch®. 
         [0072]    Referring now to  FIG. 3 , an exemplary method  300  of detecting and verifying social media events is disclosed. As shown in the  FIG. 3 , at step  302 , information from social media platform  180  is retrieved by the ingestion module  122  of event detection server  110 . In one implementation, the ingestion module  122  may include scripts or code that interface with the social media platform  180  application API. The scripts or code are also able to request and pull information from the APIs. In another implementation, the ingestion module  122  may determine the location of the ingested data and the user and append location information as metadata to the ingested data. 
         [0073]    Next at step  304 , upon receiving the ingested data, the ingestion module  122  stores the ingested data into the ingested data store  242  of event processing server  210 . In a further implementation, metadata may also be generated by the ingestion module  122  and appended to the ingested data prior to storage in the ingested data store  242 . 
         [0074]    In an alternative implementation, the knowledge base module  152  may compile the list of credible sources using information gathered from the ingested data. The knowledge base module  152  stores the list of credible sources in the knowledge base data store  248 . In one implementation, the knowledge base module  152  may analyze user profiles from the ingested data to capture information such as user affiliations or geography to be used for compilation of the list of credible sources. In a further implementation, the knowledge base module  152  takes established credible users and reviews lists generated by the user for relevant information that may be used to generate the list of credible sources. For example, if a credible user has a tech list containing a list of tech users, user IDs and related information (e.g., a related tech list associated with the user ID) associated with the tech users are also mined for information. The knowledge base module  152  continually updates knowledge base data store  248  as further social media data are ingested and may be evaluated at a predetermined frequency to ensure the information is current. 
         [0075]    Continuing onto step  306 , the filtering module  124  retrieves the ingested data from ingested data store  242  and processes the ingested data. Exemplary processing by the filtering module  124  may include language detection and profanity detection. In one implementation, the filtering module  124  determines the language of the ingested data and eliminates ingested data that are not in English. In an alternative implementation, elimination of ingested data can be for other languages. 
         [0076]    The filtering module  124  may also detect profane terms in the ingested data and flag the ingested data that contains profanity. Ingested data containing profanity is then eliminated by the filtering module  124 . In one implementation, the detection of profanity is based on querying a dictionary set of profane terms. 
         [0077]    In a further implementation, the filtering module  124  may utilize a classification algorithm that removes ingested data that is recognized to be spam, chat or advertisements. Exemplary indication of spam would be ingested data saying “follow me @xyz”. Exemplary chat in ingested data may be general chatter about daily lives like “good morning”. Exemplary advertisements in ingested data may contain language such as “click here to buy this superb T-shirt for $10.” In one implementation, the classification algorithm is based on a machine learning model that has been trained on a number of features based on language (i.e., terms used in constructing the data), message quality (i.e., presence of capitalization, emoticons), user features (i.e., average registration age). Exemplary machine learning models include, but are not limited to, Support Vector Machines, Random Forests, and Regression Models. The filtered ingested data is then stored in filtered data store  141 . 
         [0078]    Once filtering has been completed by the filtering module  124 , at step  308 , the organization module  126  retrieves the now filtered ingested data from filtered data store  141  and detects key concepts in the ingested data. In one implementation, the organization module  126  detects semantic and syntactic structures in the ingested data. 
         [0079]    In another implementation, the organization module  126  may apply part-of-speech tagging, through a Part-Of-Speech tagger, on the ingested data. For example, the organization module  126  recognizes verbs, adverbs, proper nouns, and adjectives in the ingested data. In a further implementation, there may be a predefined list of terms used for recognition by the organization module  126  that includes, but are not limited to, crisis terms like “fire,” “tornado”, or “blast”. The predefined list of terms may also be further customized based on concepts that are not proper nouns but are a good proxy for the main context of the ingested data. 
         [0080]    Part-of-speech tagging notations or identified key concepts may then be stored into the organization data store  142 . In one implementation, the Part-of-speech tagging notations or identified key concepts may be appended to the ingested data metadata and stored into the organization data store  142 . 
         [0081]    All key concepts, proper nouns, hashtags, and any list terms found in the ingested data are designated as a ‘markable’. In a further implementation, the markable may be further concatenated to produce markables that are more meaningful. For example, if “New” followed by “York” has been identified as a markable, then the terms are concatenated to indicate the revised markable as “New_York” and removing individual “New” and “York”. 
         [0082]    Once the key concepts are identified by the organization module  126 , the clustering module  128  at step  310 , obtains organized ingested data from organization data store  142  and creates a database of key concepts with a reference to the corresponding ingested data. In one implementation, the referenced corresponding ingested data maybe in the form of a unit data. This database is then stored in cluster data store  143 . 
         [0083]    At step  312 , each key concept has a predefined time frame to grow to a minimum count of unit data required to be considered an unit cluster or else it is discarded. An exemplary threshold count, may be but is not limited to, three (3) unit data for a key concept. To illustrate, if collective users (i.e., authorship value) are mentioning similar key concepts in their social media data, there maybe a likelihood of an emerging event. 
         [0084]    Once a threshold number of unit data containing common markables have been met, in step  314 , the clustering module  128  generates a unit cluster. In a further implementation, the unit data corresponding to the markable are generated as the unit cluster in step  314  and are removed from the database in step  316 . 
         [0085]    However, if the threshold has not been met, at step  330 , the markables in the database may be reviewed. For markables that have not exceeded a predefined time window, (i.e. 2 hours), the process starts again from step  302  with newly ingested data. To illustrate, this may be social media information that is so fresh that other collective users did not get to mention it yet. 
         [0086]    However, markables that never grow to the minimum threshold of unit data after a predefined time window (i.e., 2 hours) are removed from the database at step  332 . The discarded markables and unit data may be sent to the emitted data store  246  along with other metadata about it. To illustrate, social media information that no other users are mentioning might not be an event of importance to a professional consumer. 
         [0087]    Returning to step  314 , once the unit cluster is generated, its corresponding markables and unit data are removed from the database in step  316 . The newly generated unit cluster is checked against a set of previously generated event detected clusters, at step  318 . The set of previously generated event detected clusters may be located in the cluster data store  143 . In an alternative implementation, generated clusters may be located in the generated cluster data store  244  of the event processing server  210 . 
         [0088]    If there is not a match to the set of previously generated event detected clusters, continuing onto step  324 , the unit cluster is determined to be a new event detected cluster by the clustering module  128  and is stored into cluster data store  143 . 
         [0089]    However, if there is a match to existing generated event detected clusters, based on a set of predefined rules, at step  320 , a decision to either merge two similar clusters or keep them as two separate clusters is made. In one implementation, the decision to merge may be based on the same underlying concepts. 
         [0090]    If the decision is to merge two similar clusters, continuing onto step  322 , the cluster module  128  merges the clusters and stores the merged event detected cluster is stored into cluster data store  143 . For example, if social media information is the same as a previously detected event, the social media information is then merged with the previously detected event. 
         [0091]    However, if the clusters are to remain distinct, continuing onto step  324 , the unit cluster is determined to be a new event detected cluster and is stored into cluster data store  143 . For example, social media information that is distinct from the previously detected events maybe an event of importance to a professional consumer and should be noted as such, therefore the unit cluster is considered by the clustering module  128  as an event detected cluster. 
         [0092]    Turning now to  FIG. 3 b   , in a further implementation, upon storing the event detected cluster, at step  342 , enrichments may be applied to the event detected cluster. Exemplary enrichments are, but not limited to, topic categorization, summarization, newsworthiness, opinion and credibility. 
         [0093]    As mentioned previously, the topic categorization module  131  may determine one or more classification for the event detected cluster. The classification may be a taxonomy of predefined categories (i.e., politics, entertainment). The classification is added to the metadata for the event detected cluster. 
         [0094]    The summarization module  132  may select a unit data in the event detected cluster that best describes the cluster. The selected unit data is used as a summary for the event detected cluster. In a further implementation, the summarization module  132  may also utilize metrics such as the earliest unit data or a popular unit data in the generation of the summary for the event detected cluster. The summary is added to the metadata for the event detected cluster. 
         [0095]    The newsworthiness module  133  uses a newsworthiness algorithm to calculate a newsworthy score. The newsworthy score is an indication of the importance of the event detected cluster from a journalistic standard. For example, an event detected cluster concerning an airplane crash for a breaking news event is considered more important than a cluster around a viral celebrity picture. In one implementation, the newsworthiness algorithm is a supervised Machine Learning algorithm that has been trained on a newsworthy set of ingested data and predicts a newsworthy score for any ingested data that is passed through it. The newsworthy score is added to the metadata for the event detected cluster. 
         [0096]    The opinion module  134  determines if the each unit data in the event detected cluster contains an opinion of a particular person or an assertion of a fact. In one implementation, for unit data that are an assertion of fact, a score indicative of an assertion as a fact is also assigned to the unit data and likewise for an opinion. In a further implementation, the opinion module  134  executes in a two stage process. In the first stage, a rule based classifier is applied that uses simple rules based on presence/absence of certain types of opinion/sentiment words, and/or usage of personal pronouns to identify opinions. In the second stage, all unit data that are indicated to be non-opinions are passed through a bag-of-words classifier that has been trained specifically to recognize fact assertions. The determination of fact or opinion is then stored as a part of the event detected cluster metadata. 
         [0097]    The credibility module  135  determines the confidence score of each unit data in the event detected cluster. In one implementation, the confidence score is associated with three components: source credibility, cluster credibility, and tweet credibility. The score and information generated by the components are then stored as a part of the event detected cluster metadata. 
         [0098]    Source credibility relates to the source of the unit data. If the source is a credible source, for example, an authority such as the White House stating an event is more credible than a random unknown user. In one implementation, source credibility is measured by an algorithm that uses features like, but not limited to, age of the user, description, and presence of a profile image of the social media account. 
         [0099]    Cluster credibility relates to what the information is. Typically, detected events clusters containing genuine events may have different growth patterns from fake detected events clusters, such as a fake event might be driven by negative motivations like purposely spreading rumors. A supervised learning model is used based on historical data that identifies likelihood of the event detected cluster being true or false based on growth patterns. 
         [0100]    Tweet credibility relates to the content of the individual tweets in the unit data and the language being mentioned therein. In one implementation, the unit data is evaluated against a set of textual words trained on credible and noncredible unit data. 
         [0101]    Next, at step  344 , the verification module  150  analyzes the enrichments applied to the event detected cluster and its related unit data to determine the level of accuracy of the event detected cluster. In one implementation, the verification module  150  may generate a veracity calculation based on three categories: user, tweet-level or social media data level and event, from the unit data. In another implementation, the verification module  150  may compute a probability of the propagating rumor being true using extracted language, user and other metadata features from event detected cluster and its related unit data. Verification is explained in greater detail in relation to  FIGS. 4 a    and  4   b.    
         [0102]    Finally, at step  346 , the enriched event detected cluster is then stored in generated cluster data store  244  of the event processing server  210 . 
         [0103]      FIG. 4 a    illustrates an exemplary description of categories used in a veracity calculation. The first category for consideration pertains to a user category. In one implementation, the user features  402   a  are boolean and may include, but are not limited to: name, description, url, location, matches cluster location, witness, protected (i.e., private or not), verified, as illustrated in  FIG. 4 a   . The user category captures user specific information gathered from their social media profile. Exemplary features like location or url can weigh into the credibility of the user. For example, if the user is anonymous for their location, it is hard to determine the accuracy of what they are saying. However, if their location matches the location of the event detected cluster, the incident as gathered from the ingested data might be viewed in a more favorable way as being accurate. 
         [0104]    The secondary category for consideration is on the social media level. In one implementation, the social media features  402   b  of boolean type, may include, but are not limited to: multimedia, elongated word, url and news url, as illustrated in  FIG. 4 a   . The social medial category may further include numerical type: number sentiment positive words, number sentiment negative words, and sentiment score, which is of numerical type. For example, if a user is attaching a picture or multimedia to the reported incident, that can be a clear indication of the accuracy of the reporting on the social media data. In another example, the type of words used by the user, especially elongated words, i.e. “OMMMMMMGGG!!” might convey the user&#39;s shock related to the event and lend itself to a more credible event. However, if the user uses a url in the social media data, the user might be sharing by reiteration. In a further implementation, the sentiment of the ingested data is also examined. The ingested data may be checked against a set of positive and negative words for an indication of the sentiment. As an example, if the event detected cluster pertains to a disaster, the general tone of the ingested data should be negative. 
         [0105]    The third category for consideration is event features. In one implementation, the event features  402   c  may include: event topic, which may be categorical type, and highest retweet count, retweet sum, hashtag sum, negation fraction, support fraction, question fraction, which may be of numerical type, as illustrated in  FIG. 4 a   . In one implementation, if the ingested data are twitter tweets, the retweeting count and sum are valued, with the assumption that the count correlates to the popularity of the event which weighs more in favor of being accurate. In another implementation, hashtags may also be an indicator of the event. For example, sports related ingested data may contain many hashtags, while a disaster related ingested data may not have many hashtags, as there might not be time to list so many hashtags when a disaster is unfolding at the location of the user. In yet another implementation, the algorithm also takes into consideration the fraction of ingested data that deny, believe or question the event. 
         [0106]    The verification module  150  generates a matrix that is aggregated based on the three categories to generate a veracity score between −1 to 1, ranging from a false rumor to a true story. In one implementation, as illustrated in  FIG. 5 n   , the veracity score  550  may be added to the metadata of the event detected cluster. In a further implementation, as illustrated in  FIG. 6 b   , the veracity score  614  may be presented to the user in the form of circle representations. 
         [0107]      FIG. 4 b    illustrates the determination by the verification module  150  a probability score for the event detected cluster being true based on information collected from social media. In the  FIG. 4 b    example, Twitter is used as an exemplary social media platform. In one implementation, the verification module  150  first determines if the unit data of the event detected cluster is an expert type assertion or a witness type assertion. 
         [0108]    Expert type assertions are assertions that likely to be made only by people or organizations that are considered authoritative for that assertion. An exemplary expert type assertion may be the company Apple® asserting that they will be releasing a new iPhone®. The verification module  150  may invoke the knowledge base module  152  to determine if the identified user of the unit data (i.e., Apple®) is a credible source and awards a higher score if the unit data is originating from a credible source. 
         [0109]    In a further implementation, if the user of the unit data is from the list of credible sources determined by the knowledge base module  152  as authoritative on that topic, then a higher score is given. If the user of the ingested data is not authoritative, then other experts and their recent tweets are considered by the knowledge base module  152  to collect or negate the user assertion. 
         [0110]    Witness type assertions are assertions any random user may potentially make. These include crises type of events (for example, User  123  assets that an explosion took place in a particular area.) In one implementation, the verification module  150  compares either the topic or the geography of the unit data against other unit data from the same geographic area. If other users are not mentioning the same assertion during the same time period, then a lower score may be assigned. 
         [0111]    In yet a further implementation, a knowledge base of organizations as determined by the knowledge base module  152  may also be considered. Social media data from the collective knowledge base of organizations may also be processed by the Event Detection Server  110  to determine if they are discussing about a similar assertion and are used to compare with the current unit data to determine level of authenticity. 
         [0112]    The verification module  150  may then assign a probability that indicates its likeliness to be true or false. In one implementation, the verification module may algorithmically compute a score between −1 and 1, where 0 is neutral depicting our lack of information in the matter, 1 depicts highest level of confidence in the assertion being true and −1 being the highest level of confidence in it being false. For example, if information from very credible sources have confirmed that an assertion is true, then its score is likely 1. However for cases that we cannot find concrete evidences for near accuracy of its authenticity or truthfulness, the score will then fall between −1 and 1 depending on the type of evidences collected. The confidence may be re-evaluated when new evidences are included in its assessment. 
         [0113]    Referring now to  FIG. 5 a   , an exemplary ingested data is illustrated. In one implementation, the ingested data may be but is not limited to a tweet. The organization module  126  analyzes semantic and syntactic structures in the ingested data to identify key concepts. In this example, terms  502   a - 502   d , such as “confederate flag” “rally” “Linn Park” “Birmingham” are identified key concepts by organization module  126 . Although four key concepts are identified in this example, there may be n number of terms identified by the organization module  126 . In one implementation, the key concepts are stored in a database  500 , with the key concepts designated as a “markable” and the corresponding originating ingested data as a “unit data”, as illustrated in  FIG. 5 b   . As shown in  FIG. 5 b   , there may be a column  504  for n number of markables, each with corresponding column  506  pertaining to n number of unit datas. In one implementation, the database may be a hash table or a hashmap. 
         [0114]    Turning to  FIG. 5 c   , an example of the database using information from  FIG. 5 a    is disclosed. In this example, the ingested data in  FIG. 5 a    is represented as Unit data  1 . The identified key concepts  502   a - 502   d  are listed as markable  508   a - 508   d  in the markable column  504 , and the originating ingested data as Unit data  1  is also noted in the corresponding column  506 . As additional ingested data are processed in accordance with steps  302 - 310  of  FIG. 3 a   , each xth ingested data is represented as “Unit data x”. For example, the second ingested data may be represented as “Unit data  2 ”. If “Unit data  2 ” also contains the markable “Linn Park”, it may be added to the row for Linn Park in the database  500  and “Unit data  2 ” will be noted along with “Unit data  1 ” in the corresponding column  506 . Once the unit data for a markable grows and reaches a predefined threshold, it is then emitted as an event detected cluster. To put it a different way, this is an indication that multiple users are reporting similar events and therefore, may be an emerging event. 
         [0115]    Turning to  FIG. 5 d   , an exemplary unit cluster is illustrated. In one implementation, the unit cluster becomes the event detected cluster if the clustering module  128  determines that there is not already an existing cluster, or if there is an existing cluster but based on predetermined rules, the clustering module  128  determines not to merge with an existing cluster. The unit cluster comprises a threshold number n of n unit data (e.g., 3 unit clusters). 
         [0116]      FIG. 5 e    is another exemplary ingested data in the form of a tweet. This ingested data is one of the many unit data from an exemplary event detected cluster pertaining to “Mugabe: Foreign firms ‘stole diamonds’: Zimbabwean President Robert Mugabe accuse foreign mining companies of . . . ”. This ingested data was also selected by the summarization module  132  as a representative summary of the event detected cluster. 
         [0117]      FIGS. 5 f -5 k    are exemplary metadata of ingested data in  FIG. 5 e   . The ingested data comprises default metadata generated by the social media platform (i.e, twitter metadata) as illustrated in  FIGS. 5 f -5 h  and 5 k   . The Event Detection Server generates additional metadata and is appended to metadata of ingested data described above, and is illustrated in  FIGS. 5 i    and  5   j.    
         [0118]    Referring now to  FIG. 5 i   , the added metadata includes, but is not limited to, the credibility score  535  as determined by the credibility module  135 ; the opinion score  534  as determined by the opinion module  134 ; the profanity indicator  524  as determined by filtering module  124  and the markables  526  as determined by organization module  126 . 
         [0119]      FIGS. 5 l -5 n    are an exemplary metadata of an event detected cluster with ingested data of  FIG. 5 e    as one of the related unit data. 
         [0120]    In  FIG. 5 l -5 m   , the cluster metadata includes, but is not limited to, the newsworthiness score  533  as determined by newsworthiness module  133 ; the topic  531  as determined by topic categorization module  131 ; the summary  532  as determined by summarization module  132  and markables  504   a  as identified in the unit data by the organization module  126  and selected to form the event detected cluster. Each markables  504   a  may also include the respective unit data  506   a  information. 
         [0121]    Continuing on to  FIG. 5 n   , the cluster metadata includes, but is not limited to, unit data  506   b  forming the event detected cluster and the veracity score  550  as computed by verification module  150 . 
         [0122]    Now turning to  FIG. 6 a   , an exemplary graphical user interface (GUI) available through a browser  172  of access device  170  is disclosed. In one implementation, the browser  172  includes an application interface  600  that includes a plurality of columns for viewing of a list of event detected clusters pertaining to channels  602 . Within each channel are the event detected clusters relating to the topic of the channel. 
         [0123]    In one implementation, in the  FIG. 6 a    example, there may be channel  602   a  for “newest” and another channel  602   b  for “trending”. However, although only two channels are presented on the application interface  600  to the user in this example, there may be n number of channels displayed on the application interface  600 . The default channels provided by the application interface  600  allow the user to be notified of events that might be new or trending without having to search by key terms. 
         [0124]    In another implementation, continuing onto  FIG. 6 b   , a user through the browser  172  of access device  170  may enter a search term in search field  601  to tailor the application interface  600  to their needs. The UI module  232  of Event Processing Server  210  will then retrieve any event detected clusters matching the user&#39;s search term from the generated cluster datastore  244 . The results are rendered by the UI module  232  and presented to the user through browser  172  under channel  602   a  of program interface  600 , with the channel representing the search term. As shown in  FIG. 6 b    example, channel  602   c  representing the search term “GOP” and channel  602   d  for “Democrats” may be presented for viewing. 
         [0125]    In one implementation, the indication  604  provided before the text of the event detected cluster depicts the number of unit data in the event detected cluster. In a further implementation, there may be additional designation  605  indicating the event detected cluster importance based on the topic to a professional consumer (e.g. topic relating to crises, conflicts (political or geopolitical) or criminal activity). 
         [0126]    In a further implementation, the event detected cluster may also be presented with the topic  606  as determined by topic categorization module  131 ; categories  608  which may be customized terms; summary  616  as determined by summarization module  132 . The event detected cluster may also contain concepts  610 , which are the markables from the unit data that formed the event detected cluster, as determined by organization module  126 . 
         [0127]    The event detected cluster may further be presented with the hashtags  612  used in the ingested data as detected by the organization module  126 , newsworthiness indication  618  as determined by newsworthiness module  133 . In one implementation, newsworthiness indication  618  might be depicted as a filled in star. 
         [0128]    The event detected cluster may also be presented with veracity score  614  as determined by verification module  150 . In one implementation, the veracity score may be in the form of filled-in circles indicative of the strength of the veracity determination, with 5 solid circles as near accurate. 
         [0129]    In yet another implementation, the user may select create new channel  620  based on concepts in an event detected cluster. The newly created channel is based on identified concepts  610 . 
         [0130]    Using the critical event detected cluster as an example, the selection of the cluster is illustrated in  FIG. 6 c   . The set of unit data  632   a - 632   n  corresponding to the selected event detected cluster  631  is presented. In a further implementation, the user may utilize link  634  to view a specific unit data. 
         [0131]    Returning back to  FIG. 6 b   , in another implementation, channel options  622  allows for filtering of the event detected cluster results presented by UI module  232  onto browser  172  of the access device  170 . The UI module  232  receives the filter designation as selected by the user in the application interface  600  and processes the request in accordance with the filters illustrated in relation to  FIG. 7 a   - 7   e.    
         [0132]    In one implementation, as shown in  FIG. 7 a   , filtering is available based on topic  710 , sort method  720 , category  730  and advance  740  filtering. 
         [0133]      FIG. 7 b    illustrates an exemplary topic filter  710 . The topic filter  710  contains list of topic filters  712   a - 712   n . They may be, but not limited to, topics pertaining to: business/finance, crisis, entertainment, hard news, health/medical, law/crime, life/society, politics, sports, technology, weather, or other as identified by the topic categorization module  131 . 
         [0134]      FIG. 7 c    illustrates an exemplary sort filter  720 . The sort filter  720  contains options  722   a - 722   n  and they may be but are not limited to sorting by: newest, updated, most popular, tending, newsworthy, and veracity. 
         [0135]      FIG. 7 d    illustrates an exemplary category filter  730 . The category filter  730  contains a list of category filters  732   a - 732   n . The category options may be but are not limited to: breaking news, conflict, disaster, dow, financial risks, geopolitical risks, legal, legal risks, markets, oil, politics, shootings, U.S. elections. 
         [0136]      FIG. 7 e    are the advanced options upon selection of advance  740  on application interface  600 . In one implementation, the advance options for the selected channel may be, reset defaults  744 , timeline  746  with a time frame selection, minimum posts  748  count, and three levels of strict  760 , medium  762  or loose  764  for fact  750 , newsworthiness  752  and veracity  754 . 
         [0137]      FIGS. 1 through 7   e  are conceptual illustrations allowing for an explanation of the present disclosure. Various features of the system may be implemented in hardware, software, or a combination of hardware and software. For example, some features of the system may be implemented in one or more computer programs executing on programmable computer. Each program may be implemented in a high level procedural or object-oriented programming language to communicate with a computer system or other machine. Furthermore, each such computer program may be stored on a storage medium such as read-only-memory (ROM) readable by a general or special purpose programmable computer or processor, for configuring and operating the computer to perform the functions described above. 
         [0138]    Notably, the figures and examples above are not meant to limit the scope of the present disclosure to a single implementation, as other implementations are possible by way of interchange of some or all of the described or illustrated elements. Moreover, where certain elements of the present disclosure can be partially or fully implemented using known components, only those portions of such known components that are necessary for an understanding of the present disclosure are described, and detailed descriptions of other portions of such known components are omitted so as not to obscure the disclosure. In the present specification, an implementation showing a singular component should not necessarily be limited to other implementations including a plurality of the same component, and vice-versa, unless explicitly stated otherwise herein. Moreover, applicants do not intend for any term in the specification or claims to be ascribed an uncommon or special meaning unless explicitly set forth as such.

Technology Classification (CPC): 6