Cross-domain topic space

Some examples include receiving a microblog entry from a social stream domain. Further, some implementations include determining, based on a topic space associated with the social stream domain and a media domain, a topic that is associated with the microblog entry. Some implementations include determining, based on the topic space, one or more media items that are associated with the topic.

BACKGROUND

Social media has become a disruptive platform for addressing many multimedia problems that could not be elegantly solved previously. For example, real-time social data may be utilized in semantic video indexing, video (or image) context annotation and visualizing political activity and flu outbreaks, among other examples. Social media streams (e.g., microblogging streams like Twitter®) may be a good indicator of crowd sourcing activity of a social community. The information in social streams is real time. As a result, such information may provide information about real life events quickly. World events such as the Egyptian Revolution, the London riots and the Japan Earthquake have been extensively captured using social streams such as Twitter® and Facebook® updates.

SUMMARY

Some implementations provide techniques and arrangements for determining, based on a topic space associated with a social stream domain and a media domain, a topic that is associated with a microblog entry. One or more media items in the media domain that are associated with the topic may be determined based on the topic space. Additionally, some implementations may generate an updated topic space based on a plurality of additional microblog entries that are received via the social stream domain. Based on the updated topic space, at least one different media item (e.g., a “fresh” media item) that is associated with the topic may be determined.

DETAILED DESCRIPTION

Overview

There are various kinds of social media. Some publish valuable information, some share information in real time, and some provide crowd sourcing options. There may be cross-correlations between media data in different domains that may be generated in response to the same events in the physical world. However, social media is distributed disparately, e.g., a social event could be reported via social media (e.g., via Twitter®) and via an online video search engine site (e.g., YouTube®) at the same time. Media on the Internet may be unevenly distributed depending on platforms, popularity and bias, among other factors. As such, the power of a media item may be limited by the domain where the media item originates. For example, video popularity may be judged by view count, but not by how trending the video topic is.

Viral videos, which spread primarily by sharing, do not usually contain any common topics with the trending topics in social media. That is, viral videos do not become suddenly popular because the video topic is popular. Another example is that microblog users may only see related media that is shared via the microblog site, not related media from external sources. As such, a user may perform unguided searches for related media in external resources manually. However, video search engine sites may include a large amount of video or image information that may limit the ability of the user to locate related media that is relevant to the topic.

One aspect of social microblogs like Twitter® is its short text format, which is fast, real-time, and allows events to be instantly reported and distributed online. Consider this in the light of a traditional media application like video recommendation, e.g., a user is watching an old video on “Egypt” (uploaded in 2008) on the eve of the “Egyptian revolutions” (Jan. 29, 2011). Also, consider a journalist who just uploaded a live video of the revolution to a video search engine site. A video recommendation system may not be able to easily relate these two videos. That is, a video recommendation system may not be able to relate the “seed” video that the user is watching and the related newly uploaded “fresh” video, in spite of their topical similarity. Socialized recommendation using social streams has the potential to model factual world events in substantially real time by topic extraction and to subsequently perform tasks such as video associations among the old and “fresh” videos belonging to similar topics. Identifying “fresh” video associations may improve the performance of multimedia applications, such as video recommendation, in terms of topical relevance and popularity.

The present disclosure describes connecting social media from disparate sources on the Internet using a common topic space. The topic space may represent a base context platform upon which multiple media applications can be constructed, representing a bidirectional “bridge” between a social stream domain (e.g., a microblog domain) and a multimedia domain (e.g., a video search engine domain). The topic space may be created based on historical data and updated in substantially real time by extending a Latent Dirichlet Allocation (LDA) model to utilize streaming online data (e.g., microblog entries such as “Tweets®”). The topical model of the present disclosure, referred to herein as an Online Streaming LDA (OSLDA) topic modeler, may extract, learn, populate, and update the topic space in substantially real time, scaling with streaming online data. By utilizing a common topic space that is updated in substantially real time, the present disclosure may allow for media recommendation applications such as microblog enrichment with relevant video recommendations or may allow for query suggestion in response to receiving a microblog entry. Further, the present disclosure describes utilizing social trends as a factor in video popularity, with a popularity ranking on a video publishing site boosted by the magnitude of social trend of the video topics.

Example Implementations

FIG. 1illustrates an example framework of a system100according to some implementations. The system100includes at least one computing device101that includes one or more processors (not shown) and that stores a topic space102between a social stream domain104(e.g., a microblog domain) and a media domain106(e.g., a video domain). It will be appreciated that the media domain106may represent not only a video domain, but also an image domain or an audio domain, among other alternatives. That is, the term “media item” as used herein may refer to video, image, audio, text, or a combination thereof.

The topic space102ofFIG. 1represents a common topic space between the social stream domain104(social stream side) and the media domain106(multimedia side). That is, the topic space102may act as a bidirectional bridge between microblog entries and media items, such as videos. The topic space102allows social stream data received from the social stream domain104(e.g., microblog entries such as “Tweets®”) to be connected to topics, and topics to be connected to media items (e.g., videos). As an illustrative example, the topic space102may include topics such as “Japan” containing words such as “volcano,” “earthquake,” etc. In the context of video recommendation, the topic space102may allow fresh socially relevant videos (e.g., Japan's Shinmoedake Volcano Eruption in 2012) to be associated with older videos (e.g., Japan Earthquake 2008) to facilitate social trend aware recommendations.

To build the topic space102, historical data from the social stream domain104(e.g., training data that includes a plurality of microblog entries) may be preprocessed, and topics may be extracted to build a topic space from the social stream side.FIG. 1illustrates the results of this preprocessing as a topic-microblog inverted index108. On the multimedia side, a set of video Click-Through (CT) data from a commercial video search engine may be used. Video information (e.g., attributes or features such as video tags) may be extracted from the CT data and stored in a library. Based on this information, a record level inverted index110may be generated. This index110may be further divided by categories for easier indexing.

The following provides an illustrative example of the process of populating from the social stream side. A Latent Dirichlet Allocation (LDA) topic modeler112may be used to extract topics (zεZ) from a stream of microblog entries (dεD) (represented inFIG. 1as microblog entries114), where Z represents the whole topic space102and D represents the entire corpus of microblog entries. The LDA topic modeler112generates two distributions: a topical word-topic distribution P(w|z) and a topics-microblog entries distribution P (z|d). The vocabulary consists of words wεW, where W represents the entire vocabulary. Parameters α and β are Dirichlet priors to the topic-tweet and the word-topic distributions, respectively. A Tweet® or other microblog entry114is a sequence of words, where wnis the nth word in the sequence.

Consider a k-dimensional Dirichlet random variable θ that can take values in (k−1) simplex. The LDA topic modeler112assumes the following generative process for each tweet d in the corpus D: (i) choose N from a Poisson distribution; (ii) choose θ˜Dirichlet(α); and (iii) for each of the N words wn: (a) Choose a topic zn˜Multinomial(θ), and (b) Choose a word wnfrom p(wn|zn, β), a multinomial probability conditioned on the topic zn.

The dimensionality k of the Dirichlet distribution is assumed known and fixed. Therefore, the joint distribution of the topic mixture θ, the set of topics N topics Z and N words in the vocabulary W, is given by Equation [1] below:
p(θ,Z,W|α,β)=p(θ|α)Πn=1Np(zn|θ)p(wn|zn,β)  [1]

Thus, the LDA topic modeler112may represent each of the microblog entries114as a random mixture over latent topics, whereas every topic has a distribution over the words. A topic is comprised of a set of topical words. As an illustrative example, one topic generated by the LDA topic modeler112may be: {egypt, mubarak, tahrir, army, revolution, . . . }, related to the concept of the Egyptian revolution.

On the video side, a set of videos (V), represented inFIG. 1as video library116, have related video identifiers. The membership strength that each video possesses with the set of topics in the topic space102may be determined. Note that a video tag may be a video identifier. For the j-th video, the set of tags is represented by Gj. As described above, there are a set of topical words that were previously extracted from the microblog entries114. Let the topical words in the k-th topic be represented by the set Tk. Then, treating the set of topics and videos as a bipartite graph, define a link weighting function U, given by Equation [2] below:

Thus, the more tags that a video has in common with the words of a topic, the higher the weight Uk,j; and thus the higher the membership of the video towards this topic.

The system of the present disclosure may learn in substantially real time by updating the topic space102with every incoming stream of microblog entries in a time slot (illustrated inFIG. 2). This updating of the topic space102in substantially real time is referred to herein as Online Streaming LDA (OSLDA), as the model leverages online LDA and also scales across streams of incoming microblog entries, updating microblog entry-topic and topic-video connections at the same time. Rather than update a word-topic prior distribution β with time, OSLDA updates the topic space102with time, using an active time decay function. Thus, the OSLDA model may be considered robust in that the model assumes the word-topic distribution can change significantly due to the dynamic nature of microblog entries.

With each time slot, OSLDA models incoming bursts of microblog entries and updates the topic space102. As an illustrative example, fixing the number of topics to 30 may be enough for 60K microblog entries per time slot (referring toFIG. 2). Processing more microblog entries may take more time, but the number of topics needed to be extracted from a sudden “burst” (e.g., 120K microblog entries) may be less, as such a sudden burst is typically caused by a single event (single topic). As such, the number of topics to be extracted may not double if the microblog burst doubles.

FIG. 2illustrates an illustrative example of periodically updating the topic space102. InFIG. 2, the topic space102is initially determined based on the training data (e.g., the microblog entries114ofFIG. 1). A video recommender202is also illustrated. OSLDA is used to generate a topic model204based on a plurality of incoming microblog entries received over a period of time. Updates to the topic model204based on the incoming microblog entries are reflected in an updated topic space102that may be utilized by the video recommender202.

Thus, a bidirectional connection is built between the social stream domain104and the multimedia domain106using the topic space102, which may be used to enrich microblog entries with video recommendations and to enrich videos with social popularity ranking.

Microblog entries may be “noisy” and difficult for users to understand. The user experience may be improved by recommending related and relevant media. From the user perspective, this may enrich the information surrounding the microblog entry (in terms of the topic of the entry), as media (e.g., image/video) may be easier for the user to comprehend. Once the LDA topic modeler112has been trained on a stream of microblog entries (e.g., the microblog entries114), the LDA topic modeler112may be used to connect a microblog entry (illustrated inFIG. 1as microblog entry118) to a topic and to connect the topic to a set of videos (illustrated inFIG. 1as a first video120, a second video122, and a third video124).

FIG. 3provides an illustrative example of recommending videos from a single microblog entry. Given a microblog entry d′ (e.g., the microblog entry118ofFIG. 1), the LDA topic modeler112may be used to determine the probability distribution of topics302(represented as zkinFIG. 3) for that microblog entry. In the example ofFIG. 3, k is 5, with the five different topics represented as z1, z2, z3, z4, and z5. A set of topical words304(represented as r1inFIG. 3for exemplary topic z1) is associated with each of the topics302, with each of the topical words304represented as circles inFIG. 3. Similarly, a set of video tags306(represented as c1inFIG. 3for exemplary video v1) is associated with each of the videos308(with six videos represented in the example ofFIG. 3) in the video library116. Subsequently, videos to be recommended are selected based on the optimization, represented by Equation [3] below:
v*=arg max0≦j<|V|Σ0≦k<|Z|P(zk|d′)·Uk,j[3]

In Equation [3], P(zk|d′) signifies the microblog entry-topic link weight, while Uk,jrepresents the topic-video link weight. Thus, the microblog entry118connects to those videos for which it has the strongest links through the topic space102. Thus, the present disclosure may empower microblog entries with related videos from another domain (e.g., the multimedia domain106).

Further, microblog social trends are also a distribution over topics in the topic space102. Such trends are a measure of real-time social popularity. By leveraging this observation, video popularity may be augmented based on socially trending topics.

It may be difficult to capture video popularity based on multimedia techniques alone. The present disclosure may allow video popularity in a video publishing site to depend not only on traditional factors (such as view count), but also on how socially important the topic for the particular video is. This effect may be referred to as the “social prominence” of a video. The overall new popularity metric, which is a combination of traditional and social, may be referred to as “Trend Aware Popularity (TAP).” Thus, the present disclosure may empower videos with social stream trend awareness, influencing social video popularity.

A microblog site may provide its users with a set of trending topics by geography. This set of trending topics may be calculated using a naive keyword (# hash tag) approach. Observing that trends simply represent topics with high weight over a stream of entries, the LDA topic modeler112may again be used for this purpose. The LDA topic modeler112may be modified to accept streams of incoming microblog entries (as illustrated inFIG. 4). In particular, by summing each column of the topic-microblog entries probability distribution matrix and comparing the result, it may be determined which topics in the whole topic space (Z) have the maximum membership over the set of microblog entries (D).

Thus, as streams of microblog entries are received over time, the maximum membership topics may be detected, as shown inFIG. 4. InFIG. 4, the boxes represent topics, with each topic including identified topical words. This membership may be considered as a sign of trending nature. The higher boxes denote stronger membership. For illustrative purposes,FIG. 4includes four topics402,404,406, and408in a first time slot410. For example, inFIG. 4, the topic on {Egypt, tahrir, etc.} (identified as topic402in the first time slot410ofFIG. 4) is more trending than the topic on {# nowplaying, music, etc.}, identified as topic404in the first time slot410ofFIG. 4. InFIG. 4, three topics410,412, and414are illustrated in a second time slot416, and three topics418,420, and422are illustrated in a third time slot424. In the example ofFIG. 4, the highest trending topic410in the second time slot416and the highest trending topic418in the third time slot424both correspond to the same topic as the highest trending topic402in the first time slot410(e.g., the Egyptian revolution). However, as illustrated FIG.4, the OSLDA model may periodically adjust the set of topical words that are associated with the same topic based on incoming microblog entries.

Trends are temporally dynamic entities. Usually, trends are most prominent in the first few minutes after they appear, and their attractiveness fades away in time. Therefore, the present disclosure's formulation of social prominence may use a time decay factor, such that the social prominence of a topic reduces in time. The social prominence of a topic z may be defined using a trending score, determined using Equation [4] below:

In Equation [4], δz=f(tcur,tonset) is the time decay factor. The system receives a set of D microblog entries in one time slot, where tcuris the current time slot and tonsetis the time slot when the trend was first observed.

The decay function may be modeled as either a blind decay or an active decay. Blind decay does not determine whether the trend increases or decreases in the next time slot in real life. For example, blind decay can be computed as shown in Equation [5] below:

On the other hand, an active decay identifies when a trend appears, falls and then rises again in real time. Active decay does not reduce social prominence when the trend rises for the second time. Active decay may be calculated using Equation [6] below:

Active decay may be appropriate for more dynamic topics, unlike those topics that have been trends for more than hours (some trending topics on microblog sites have this nature).

Thus, Trend Aware Popularity (TAP) may represent a combination of traditional popularity factors (like view counts or comments) and the social prominence of the video topic. For a video v, z* may be selected as the topic to which the video has maximum membership. Thus, z* is the principal topic for video v. The topic z* is connected to a set of videos V*, all of which have some sort of membership to z*, defined by Equation [2] above. Therefore, the final temporal popularity score to assign to the video v is given by Equation [7] below:

Thus, Equation [7] may represent the popularity of the video v based on both external social (trend) and internal direct activity (view count) factors. The parameter γ is a weighing parameter that may be used to balance social versus traditional control. That is, a weighting factor may be used to determine a contribution of a view count to a popularity ranking of a media item.

Thus, the present disclosure describes how cross domain media can be recommended based on social streams and how media popularity can be affected based on their social prominence. The same can be applied to images and other cross domain media across the web.

Referring toFIG. 5, an example of a method of determining media items (e.g., videos) in a media domain that are associated with a topic is illustrated and generally designated500.

At block502, the method500may include receiving a microblog entry from a social stream domain. For example, the received microblog entry may include the microblog entry118received from the social stream domain104ofFIG. 1.

At block504, the method500may include determining, based on a topic space associated with the social stream domain and a media domain, a topic that is associated with the microblog entry. For example, the topic space may include the topic space102ofFIG. 1that is associated with the social stream domain104(e.g., a microblog domain) and the media domain106(e.g., a video domain).

At block506, the method500may include determining, based on the topic space, one or more media items in the media domain that are associated with the topic. For example, the one or more media items may include one or more videos (e.g., the first video120, the second video122, and the third video124) in the media domain106ofFIG. 1.

Referring toFIG. 6, an example of a method of determining media items in a media domain that are associated with a topic is illustrated and generally designated600.

At block602, the method600may include receiving a microblog entry from a social stream domain. At block604, the method600may include determining, based on a topic space associated with the social stream domain and a media domain, a topic that is associated with the microblog entry. At block606, the method600may include determining, based on the topic space, one or more media items in the media domain that are associated with the topic.

At block608, the method600may include generating an updated topic space based on a plurality of microblog entries received from the social stream domain. For example, referring toFIG. 2, the method600may include utilizing the LDA modeler112to periodically update the topic space102based on microblog entries received via the microblog feed. Further,FIG. 4illustrates the dynamic nature of topics in the microblog feed.

At block610, the method600includes receiving a second microblog entry from the social stream that is associated with the topic (i.e., the same topic identified at block604).

At block612, the method600includes determining, based on the updated topic space, that at least one different media item in the media domain is associated with the topic.

Example Computing Device and Environment

FIG. 7illustrates an example configuration of a computing device700and an environment that can be used to implement the modules and functions described herein.

The computing device700may include at least one processor702, a memory704, communication interfaces706, a display device708(e.g. a touchscreen display), other input/output (I/O) devices710(e.g. a touchscreen display or a mouse and keyboard), and one or more mass storage devices712, able to communicate with each other, such as via a system bus714or other suitable connection.

The processor702may be a single processing unit or a number of processing units, all of which may include single or multiple computing units or multiple cores. The processor702can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the processor702can be configured to fetch and execute computer-readable instructions stored in the memory704, mass storage devices712, or other computer-readable media.

Memory704and mass storage devices712are examples of computer storage media for storing instructions which are executed by the processor702to perform the various functions described above. For example, memory704may generally include both volatile memory and non-volatile tangible memory devices (e.g., RAM, ROM). Further, mass storage devices712may generally include hard disk drives, solid-state drives, removable media, including external and removable drives, memory cards, flash memory, floppy disks, optical disks (e.g., CD, DVD), a storage array, a network attached storage, or a storage area network. Both memory704and mass storage devices712may be collectively referred to as memory or computer storage media herein, and may be computer-readable media capable of storing computer-readable, processor-executable program instructions as computer program code that can be executed by the processor702as a particular machine configured for carrying out the operations and functions described in the implementations herein.

The computing device700may also include one or more communication interfaces706for exchanging data with other devices, such as via a network, direct connection, or the like, as discussed above. The communication interfaces706can facilitate communications within a wide variety of networks and protocol types, including wired networks (e.g., LAN, cable, etc.) and wireless networks (e.g., WLAN, cellular, satellite, etc.), the Internet and the like. Communication interfaces706can also provide communication with external storage (not shown), such as in a storage array, network attached storage, storage area network, or the like.

The discussion herein refers to data being sent and received by particular components or modules. This should not be taken as a limitation as such communication need not be direct and the particular components or module need not necessarily be a single functional unit. This is not to be taken as limiting implementations to only those in which the components directly send and receive data from one another. The signals could instead be relayed by a separate component upon receipt of the data. Further, the components may be combined or the functionality may be separated amongst components in various manners not limited to those discussed above. Other variations in the logical and practical structure and framework of various implementations would be apparent to one of ordinary skill in the art in view of the disclosure provided herein.

A display device708, such as touchscreen display or other display device, may be included in some implementations. Other I/O devices710may be devices that receive various inputs from a user and provide various outputs to the user, and may include a touchscreen, such as a touchscreen display, a keyboard, a remote controller, a mouse, a printer, audio input/output devices, and so forth.

Memory704may include modules and components for execution by the computing device700according to the implementations discussed herein. In the illustrated example, memory704includes the topic-microblog inverted index108, the record level inverted index110, the video library116, the LDA topic modeler112and the topic space102as described above with regard toFIG. 1. Memory704may further include one or more other modules716, such as an operating system, drivers, application software, communication software, or the like. Memory704may also include other data718, such as data stored while performing the functions described above and data used by the other modules716. Memory704may also include other data and data structures described or alluded to herein. For example, memory704may include information that is used in the course of deriving and generating the topic space102as described above.

Although illustrated inFIG. 7as being stored in memory704of computing device700, the indexes108and110, the LDA topic modeler112, the video library116and the topic space102, or portions thereof, may be implemented using any form of computer-readable media that is accessible by computing device700. As used herein, “computer-readable media” includes computer storage media and communication media.

Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read only memory (ROM), electrically erasable programmable ROM (EEPROM), flash memory or other memory technology, compact disk ROM (CD-ROM), digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store information for access by a computing device.

In contrast, communication media may embody computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave. As defined herein, computer storage media does not include communication media.

FIG. 8illustrates an example framework of a system800according to some implementations. The system800includes a computing device802that includes one or more processors804and at least one memory806. The computing device802includes the LDA topic modeler112to build the topic space102based on training data and to periodically update the topic space102. In a particular embodiment, the computing device802ofFIG. 8may correspond to the computing device101ofFIG. 1.

To build the topic space102from the social stream side, the computing device802may receive training data808from a microblog site810. The training data808may represent a plurality of microblog entries collected over a period of time. The LDA topic modeler112may process the training data808and determine a plurality of topics812. The LDA topic modeler112may generate the topic-microblog inverted index108based on the plurality of topics812identified based on the training data808.

A video analysis component814of the computing device802may receive video Click-Through (CT) data816from a video search engine site818. The video analysis component814may extract video information (illustrated as video tags820inFIG. 8) from the CT data816and store the video tags820in the video library116. The video analysis component814may generate the record level inverted index110based on the video tags820stored in the video library116. The video analysis component814may further divide the record level inverted index110into categories822for easier indexing.

After initially building the topic space102based on the video CT data816and the training data808, the LDA topic modeler112may periodically update the topic space102in substantially real time based on the plurality of microblog entries114received from the microblog site810.

In contrast, communication media may embody computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transmission mechanism. As defined herein, computer storage media does not include communication media.

CONCLUSION

Although the subject matter has been described in language specific to structural features and/or methodological acts, the subject matter defined in the appended claims is not limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. This disclosure is intended to cover any and all adaptations or variations of the disclosed implementations, and the following claims should not be construed to be limited to the specific implementations disclosed in the specification. Instead, the scope of this document is to be determined entirely by the following claims, along with the full range of equivalents to which such claims are entitled.