Aggregating and Searching Social Network Images

Methods and systems directed to aggregating social content from social networking sites, and making the social content available for searching is disclosed. Social content, corresponding to multiple computer users, is obtained from a plurality of social networking sites. The social content is stored in a content store, making it available for searching. In response to receiving a search query (directed to social content), a set of search results is identified, the search results including at least one item of social content obtained from a social networking site. The social content in the search results is filtered according to privacy constraints. A presentation of the filtered search results is generated and provided to the requesting computer user.

DETAILED DESCRIPTION

For purposed of clarity, the use of the term “exemplary” in this document should be interpreted as serving as an illustration or example of something, and it should not be interpreted as an ideal and/or a leading illustration of that thing. Additionally, the term “social content” refers to content that is uploaded to a network site such that the content may be viewed by one or more other persons/computer users. While much of the description of the disclosed subject matter is made with regard to images, images are just an example of social content. Social content may include, by way of illustration and not limitation: images, videos, hyperlinks, textual posts, comments, audio files, check-in data, and likes. In obtaining social content from a social networking site, both the social content (i.e., the content that was posted to the social networking site) and content data (i.e., information that relates to the posted content) are obtained, though the two parts may not necessarily be obtained at the same time. A “social networking site” refers to a network location where persons/computer users are able to upload social content such that it may be viewed by one or more other persons/computer users.

Turning now to the figures,FIG. 1is a block diagram illustrating an exemplary networked environment100suitable for implementing aspects of the disclosed subject matter, including aggregating and searching for social content. The illustrative networked environment100includes one or more user computers, such as user computers102-106, connected to a network108, such as the Internet, a wide area network or WAN, and the like. Also connected to the network108is a search engine110configured to provide search results in response to search queries received from one or more computer users, such as computer user101by way of user computer102over the network108. According to aspects of the disclosed subject matter, the search engine110is configured such that it is able to aggregate social content from social networking sites and make the social content available for searching.

Those skilled in the art will appreciate that, generally speaking, a search engine110corresponds to an online service hosted on one or more computers, or computing systems, located and/or distributed throughout the network108. The search engine110receives and responds to search queries submitted over the network108from various computer users, such as computer user101using user computer102over the network108. In particular, responsive to receiving a search query from a computer user, the search engine110obtains search results information related and/or relevant to the received search query (as defined by one or more query terms of search query.) The search results information includes search results, i.e., references (typically in the form of hyperlinks) to relevant content available at various network locations located throughout the network108, such network sites112-116. The content sites may include (by way of illustration and not limitation): social networking sites, such as social networking sites112and116; online shopping sites, such as online shopping site114; news outlets (not shown); educational and research sites (not shown); and the like.

It should be appreciated that while much of the following discussion will be made with regard to a search engine110that is separate from a social networking site, such as social networking sites112and116, this should not be construed as limiting upon the disclosed subject matter. Indeed, aspects of the disclosed subject matter may be suitably implemented in the search services of the various social networking sites, such as social networking sites112and116. Moreover, social content may be obtained from any number of sources, not just social networking sites. For example, a computer user may post a comment/review with regard to a product available on a shopping site, such as shopping site114. This product review may be obtained by a search engine110and made available for searching in accordance to aspects of the disclosed subject matter.

To better appreciate how the search engine110aggregates social content from social networking sites, reference is made toFIG. 2.FIG. 2is a block diagram illustrating the flow of social content from social networking sites to a search service, such as a search engine110, where the content is made available for searching. For ease of description, the discussion ofFIG. 2, as well as addition description thereafter, will be made with regard to a specific embodiment of social content: images. However, while the subsequent description will be made in regard to processing images from social networking sites, this should be view as illustrative and not limiting of the novel aspects of the disclosed subject matter.

As shown inFIG. 2, a search engine110obtains image content202from one or more social networking sites, such as social networking sites112and116. The image content202will typically include both image204(either the image or a reference to the image) and related image data206. Of course, it should be noted that whileFIG. 2illustrates that the image204and related image data206are obtained together, this is for illustration purposes. In various embodiments, the image204and related image data206may be obtained separately.

While obtaining the image204or a link to the image at the social networking site is important for the search engine110(so that it can serve up the image in response to a search query), obtaining related image data206is also important. According to aspects of the disclosed subject matter, the search engine uses the related image data206to identify keys and other aspects or factors upon which the image can be indexed in a content store. Generally speaking, the search engine110uses numerous aspects of the related image data206to identify keys/criteria/factors that help identify the particular image. These aspects include (by way of illustration and not limitation): the album title in which the obtained image is found; album comments—from both the image poster as well as other persons allowed to comment on the album at the social networking site; tagged entities in the image, including people and/or places; image titles; images comments; location information regarding the geographic location of subject of the image; the date and/or time that the image was taken; the number of “likes” that the image has received; popularity of the item, e.g., the number of times that the image has been viewed; the number of times that the image has been shared (reposted) by others; and the date the image was uploaded to the social networking site. Also included in the related image data206is an Uploader ID (identifier), the Uploader ID being an identifier corresponding to the person/computer user that uploaded the image to the social networking site. As will be discussed below, the Uploader ID plays an important function in determining whether a requesting computer user has sufficient privileges to access a given image.

Upon receiving the image content202, the search engine110examines the related image data206and identifies and/or determines criteria, referred to as “key terms,” upon which the image may be indexed in a content store for efficient retrieval. According to various embodiments of the disclosed subject matter, the search engine110extracts the key terms from the image content202via a key extraction component208. Once the key terms are identified, the image content202, comprising the image204and related image data206, is stored in the content store210, indexed according to the identified key terms.

Turning now toFIG. 3,FIG. 3is a flow diagram illustrating an exemplary routine300for obtaining social content202from social networking sites and making the social content available for searching. Beginning at block302, an iteration loop is begun to iterate through each of a plurality of computer users associated with social networking sites. For clarity in the following discussion, the term “current computer user” refers to the computer user currently being processed in the iteration loop.

At block304, another iteration loop is begun in which the process iterates through each social networking site associated with the current computer user. In this manner, the process is able to obtain image data associated with the current computer user from each of the social networking sites with which the computer user is associated, and where image data regarding the current computer user is found. As with the prior iteration loop, the term “current social networking site” refers to the social networking site currently being processed in this iteration loop.

At block306, the process obtains image content202corresponding to the current user from the current social networking site. As mentioned above in regard toFIG. 2, the image content202includes both the image (or a reference to the image) and related image data. According to various embodiments, the image and the related image data may be obtained jointly or, in the alternative, the image data may be obtained separately from the current social networking site.

At block308, yet another iteration loop is begun in which the process iterates through each of the images obtained for the current computer user from the current social networking site. As with the other iteration loops, the image that is currently being processed in the iteration loop is referred to as the “current image.” At block310, key terms are extracted from the image data206associated with the current image. At block312, the image content202(including the image204or a link to the image, and the image data206) is added to the content store210and indexed according to the extracted key terms. At block314, if there are any additional images to be processed, the routine300returns to block308where the next image is selected as the current image for processing, and the loop repeats for the current image. However, if there are no more images for the current user from the current social networking site to be processed, the routine proceeds to block316.

At block316, if there are any additional social networking sites associated with the current computer user, the routine300returns to block304where the next social networking site is selected as the current social networking site for processing and the iteration loop repeats. However, if there are no more social networking sites corresponding to the current user from which image content202may be obtained, the routine proceeds to block318.

At block318, if there are any additional computer users to be processed, the routine300returns to block302where the next computer user is selected as the current computer user and the loop repeats as described above. However, if there are no more computer users for processing, the routine300terminates.

Those skilled in the art should appreciate that the process described in routine300is only one example of a routine suitable for processing image content (or, of a more general nature, social content). Indeed, those skilled in the art will appreciate that obtaining image content and then processing it may be performed in any number of manners. Significantly, image content202(or, more generally, social content) corresponding to a computer user is obtained from a plurality of social networking sites, the image content includes a source (Uploader ID), key terms are extracted from the image data, and the image content is stored in the content store210and indexed according to one or more of the extracted key terms.

While routine300describes a process for aggregating image content (or, more generally, social content) into a content store210,FIG. 4illustrates an exemplary routine400for responding to a search query with regard to an image/social content that has been uploaded to a social networking site. Beginning at block402, a search engine110receives a search query from a requesting computer user regarding an image relating to an identified entity. An exemplary search query may be “Image of my friend Steve at Lake of the Angels.”

At block404, the routine400parses the search query to identify the query topic (or topics) and conditions of the search query. With reference to the example of above, in parsing the search query the routine400identifies “my friend Steve” as the query topic, as well as identifying sub-topics and/or conditions such as “image” and “Lake of the Angels.” Of course, in this example the search engine110may consult with one or more networking sites associated with the requesting computer user to accurately identify “my friend Steve.” Thus, at block406, a set of images are identified from the content store210that are relevant to the search query, both the query topic as well as the conditions. These results are available for searching since the images have been stored and indexed in the content store210.

As those skilled in the art will appreciate, the images (or, more generally, social content) that are posted on social networking sites are often posted by people that have various access restrictions in place as to who is able to view the posts and who is not. Thus, at block408, the initial set of results/images may or may not be accessible to the requesting computer user based on the access restrictions of the person that posted the content in the first place. Thus, the initial set of search results/images is filtered according to privacy constraints.FIG. 5is a flow diagram illustrating an exemplary sub-routine500for filtering out search results to which the requesting user does not have current access permissions.

Beginning at block502, an iteration loop is begun to iterate through each of the images in the initial set of resulting images, with each image being processed in the iteration loop being referred to as the “current image.” Since each social networking site may have its own privacy policies and each user may establish different privacy settings for access to content that he/she posts on the social networking site, at block504the social networking site associated with the current image is identified from the image data206that was stored as part of the image content202in the content store210. Additionally, the Uploader ID (identifier for the user that uploaded the image to the social networking site) is also identified from the image content202in the content store210.

At decision block508, the identified social networking site is queried as to whether the requesting computer user (typically identified by a user identifier) has permission to access/view the current image (based on the Uploader ID). If the response is that the requesting computer user has permission to access the image, the routine500proceeds to block512where the process determines whether there are any additional images to examine in the process. However, if at decision block508it is determined that the requesting computer user does not have sufficient privileges to access the current image, the routine500proceeds to block510. At block510, the current image is removed from the set of images selected as potential search results for the search query.

At block512, if there are additional images in the initial set to validate, the routine500returns to block502where the next image in the set of images is selected as the current image and the described above repeats. Alternatively, if there are no additional images to validate (i.e., check whether the requesting computer user has sufficient privilege to access/view), the routine500has completed filtering the initial set of images and, correspondingly, terminates.

Returning again toFIG. 4, after having filtered the initial set of images according to privacy constraints, routine400proceeds to block410. At block410, a presentation of the filtered set of images is generated in response to the search query. While any number of presentations may be generated as the presentation of the filtered set of search results, including one or more typical search results pages, a discussion of exemplary presentations tailored to images is set forth below in regard toFIGS. 6 and 7. Thereafter, at block412the presentation of the filtered set of images responsive to the search query is provided to the requesting computer user, and the routine400terminates.

Regarding routines300,400and500, while these routines are expressed in regard to discrete steps, these steps should be viewed as being logical in nature and may or may not correspond to any actual and/or discrete steps of a particular implementation. Nor should the order in which these steps are presented in the various routines be construed as the only order in which the steps may be carried out. Moreover, while these routines include various novel features of the disclosed subject matter, other steps (not listed) may also be carried out in the execution of the routines. Further, those skilled in the art will appreciate that logical steps of these routines may be combined together or be comprised of multiple steps. Steps of routines300,400and500may be carried out in parallel or in series. Often, but not exclusively, the functionality of the various routines is embodied in software (e.g., applications, system services, libraries, and the like) that is executed on computer hardware and/or systems described below in regard toFIG. 800.

It should be appreciated that while many novel aspects of the disclosed subject matter are expressed in routines, applications (also referred to as computer programs), apps (small, generally single or narrow purposed, applications), and/or methods, these aspects may also be embodied as computer-executable instructions stored in computer-readable media, also referred to as computer-readable storage media. As those skilled in the art will recognize, computer-readable media can host computer-executable instructions for later retrieval and execution. When executed on a computing device, the computer-executable instructions stored on one or more computer-readable storage devices carry out various steps, methods and/or functionality, including those steps, methods, and routines described above in regard to routines300,400and500. Examples of computer-readable media include, but are not limited to: optical storage media such as Blu-ray discs, digital video discs (DVDs), compact discs (CDs), optical disc cartridges, and the like; magnetic storage media including hard disk drives, floppy disks, magnetic tape, and the like; memory storage devices such as random access memory (RAM), read-only memory (ROM), memory cards, thumb drives, and the like; cloud storage (i.e., an online storage service); and the like. For purposes of this disclosure, however, computer-readable media expressly excludes carrier waves and propagated signals.

Turning now toFIG. 6,FIG. 6is a pictorial diagram600illustrating an exemplary user interface showing the presentation of search results with regard to a search query directed to images (or, more generally, social content) available at a social networking site. As shown inFIG. 6, an exemplary search query, “Image of my friend Steve at Lake of Angels,” returns various images arranged in a “pinboard view” arrangement, including images604and606. As discussed above, the images were identified according to the key terms identified in the image content data. In this exemplary display, the image view608may be scrolled up and down via a touch interface, though this is a non-limiting example. In other embodiments (not shown), scrolling devices may be provided that enable the computer user to scroll the resultant images in the image view608, using input devices such as a mouse, keyboard, joystick, touchpad, and the like.

In contrast to the pinboard view arrangement ofFIG. 6,FIG. 7illustrates a “slideshow view” arrangement for the same exemplary search query702. As with the pinboard view arrangement ofFIG. 6,FIG. 7illustrates the images702-706resulting from the search query “Image of my friend Steve at Lake of the Angels.” In contrast to the pinboard view, the slideshow view enables the requesting computer user to scroll the images horizontally through the display view708using a swipe touch gesture. Of course, while not shown, other input paradigms may be used; e.g., scrolling devices may be provided that enable the computer user to scroll the resultant images in the image view608, using input devices such as a mouse, keyboard, joystick, touchpad, voice controls/commans, and the like.

FIG. 8is a block diagram illustrating exemplary components of a search engine configured according to aspects of the disclosed subject matter. As described above, the search engine110is configured to aggregate social content from one or more social networking sites and make the aggregated content available for searching. As shown inFIG. 8, the search engine800includes a processor802(or processing unit) and a memory804interconnected by way of a system bus810. As those skilled in the art will appreciated, memory804typically (but not always) comprises both volatile memory806and non-volatile memory808. Volatile memory806retains or stores information so long as the memory is supplied with power. In contrast, non-volatile memory808is capable of storing (or persisting) information even when a power supply is not available. Generally speaking, RAM and CPU cache memory are examples of volatile memory whereas ROM and memory cards are examples of non-volatile memory.

The processor802executes instructions retrieved from the memory804in carrying out various functions, particularly in aggregating social content from various social networking sites and making the aggregated content available for searching. The processor802may be comprised of any of various commercially available processors such as single-processor, multi-processor, single-core units, and multi-core units. Moreover, those skilled in the art will appreciate that the novel aspects of the disclosed subject matter may be practiced with other computer system configurations, including but not limited to: mini-computers; mainframe computers, personal computers (e.g., desktop computers, laptop computers, tablet computers, etc.); handheld computing devices such as smartphones, personal digital assistants, and the like; microprocessor-based or programmable consumer electronics; and the like.

The system bus810provides an interface for search engine's components to inter-communicate. The system bus810can be of any of several types of bus structures that can interconnect the various components (including both internal and external components). The illustrative search engine110further includes a network communication component812for interconnecting the search engine with other computers (such as user computers102-106and networking sites112-116) as well as other devices on a computer network108. The network communication component812may be configured to communicate with an external network, such as network108, via a wired connection, a wireless connection, or both.

The exemplary search engine110further includes a query parsing component814. As previously suggested, the query parsing component814parses search queries received from computer users in order to identify a query topic (or topics), sub-topics, conditions such that the search engine110is able it identify a set of search results of social content responsive to the query. These topics and conditions are used as keys by a content retrieval component820to identify and/or retrieve social content in the content store210that satisfies the search query. An access filter component816filters potential search results according to current access permissions before the search engine provides the identified search results to the computer user. A presentation generation component818generates a presentation of the identified search results for the requesting computer user, such as the pinboard or slideshow views described inFIGS. 6 and 7. The search engine110also includes a key extraction component208that extracts key terms from the social content202obtained from the various social networking sites112and116and stores the social content in the content indexed according to the extracted key terms.

Those skilled in the art will appreciate that the various components described above, including the query parsing component814, access filter component816, key extraction component208, presentation generation component818and others may be implemented as executable software modules within the search engines110, as hardware modules, or a combination of the two. Moreover, each of the various components may be implemented as an independent, cooperative process or device, operating in conjunction with the search engine110. It should be further appreciated, of course, that the various components described above in regard to the search engine110should be viewed as logical components for carrying out the various described functions. As those skilled in the art appreciate, logical components (or subsystems) may or may not correspond directly in a one-to-one manner to actual components. In an actual embodiment, the various components identified as being part of the search engine110inFIG. 8may be combined together or broke up across multiple actual components and/or implemented as cooperative processes on a computer network108.

While various novel aspects of the disclosed subject matter have been described, it should be appreciated that these aspects are exemplary and should not be construed as limiting. Variations and alterations to the various aspects may be made without departing from the scope of the disclosed subject matter.