Patent Description:
This invention relates generally to photographs taken by a mobile device operative in a networked environment. More particularly, this invention relates to supplying such a mobile device with content aware photo filters.

The number of digital photographs taken with mobile wireless devices is increasingly outnumbering photographs taken with dedicated digital and film based cameras. Thus, there are growing needs to improve the experience associated with mobile wireless digital photography.

<CIT>discloses a method of adjusting visual content. A client terminal extracts visual content data pertaining to the visual content, forwarding a request which includes the visual content data to a network node via a network, receiving, in response to the request, a list of a plurality of visual content editing functions from the network node, presenting, on the client terminal, the plurality of visual content editing functions to a user, receiving a selection of at least one member of the list from the user, adjusting the visual content using the at least one member, and outputting the adjusted visual content.

<CIT> discloses an Augmented Reality (AR) apparatus and method for recognizing objects and synthesizing a virtual object or virtual information with a real environment such that the virtual object or virtual information appears like a real object or real information that may exist in the real world environment.

<CIT> describes processing video data of an augmented reality image produced by adding information about objects in the image taken by the camera of the terminal in real time are provided. The method includes displaying (<NUM>) an image in augmented reality mode, requesting (<NUM>) augmented reality information of objects constituting the image, presenting (<NUM>), if the augmented reality information is received, the augmented reality information in relation to the respective objects, merging (<NUM>), if a message transmission is requested (<NUM>), the image and the augmented reality information; and transmitting (<NUM>) the merged image as an augmented reality image. The image processing method of the present invention is advantageous for the terminal to provide the user with information related to the objects presented in an image produced using the augmented reality function.

<CIT> concerns customization of a list of properties associated with media files based at least in part on user's preferences. In some examples, a method may include receiving, by a server, a plurality of user inputs that respectively identify the user's designated favorites from among a plurality of media files; determining, by the server, the user's preferences from among a plurality of properties associated with the user's designated favorites from among the plurality of media files, based at least in part on the received user inputs; and providing, by the server, the user with a list of the plurality of properties based at least in part on the user's preferences from among the plurality of properties.

<CIT> describes the provision of a communication between users of a computer network are provided, the method including the steps of storing, in a database, received geo-location information, object information, brand information or other information, from a user; providing to the user a subset of previously stored digital media templates from the database for user selection; combining a user-selected digital media template with a user-selected media data object file received from the user; and tagging the geo-location information, object information, brand information or other information, to the combined user-selected digital media template and user-selected media data object file for communicating between users of the computer network.

<CIT> concerns an electronic camera that comprises a display means further comprising an image display region for displaying an image, a console means for movably designating a prescribed region of the image display region, and a filter means for carrying out a filter process on an image corresponding to the prescribed region which is movably designated with the console means. Thus, it is possible, without carrying out a complex filter operation, to easily carry out an operation which overlays a plurality of filter effects or compare an image with and without a filter effect or with different filter effects.

The invention is a system, method and computer storage product as defined in the appended claims. In the system, a server includes a photo filter module with instructions executed by a processor to identify when a client device captures a photograph. Photograph filters are selected based upon attributes of the client device and attributes of the photograph. The photograph filters are supplied to the client device.

The invention is more fully appreciated in connection with the following detailed description taken in conjunction with the accompanying drawings, in which:.

<FIG> illustrates an electronic device <NUM> utilized in accordance with an embodiment of the invention. In one embodiment, the electronic device <NUM> is a Smartphone with a processor <NUM> in communication with a memory <NUM>. The processor <NUM> may be a central processing unit and/or a graphics processing unit. The memory <NUM> is a combination of flash memory and random access memory. The memory <NUM> stores a photo controller <NUM>. The photo controller <NUM> includes executable instructions to coordinate the capture, display and archiving of digital photographs. The photo controller <NUM> may include photo filter processing disclosed herein, which augments or replaces such photo filter processing that is described below in connection with a server based photo filter module.

The processor <NUM> is also coupled to image sensors <NUM>. The image sensors <NUM> may be known digital image sensors, such as charge coupled devices. The image sensors capture visual media, which is presented on display <NUM>, as coordinated by the photo controller <NUM>.

A touch controller <NUM> is connected to the display <NUM> and the processor <NUM>. The touch controller <NUM> is responsive to haptic signals applied to the display <NUM>. In one embodiment, the photo controller <NUM> monitors signals from the touch controller <NUM> to coordinate the capture, display and archiving of digital photographs. The electronic device <NUM> may also include other components commonly associated with a Smartphone, such as a wireless signal processor <NUM> to support wireless communications, a power control circuit <NUM> and a global positioning system processor <NUM>.

<FIG> illustrates a system <NUM> configured in accordance with an embodiment of the invention. The system <NUM> includes a set of client devices 100_1 through 100_N. The client devices <NUM> are connected to a network <NUM>, which is any combination of wireless and wired network communication devices. A server <NUM> is also connected to the network <NUM>. The server <NUM> includes standard components, such as a central processing unit <NUM> and input/output devices <NUM> connected via a bus <NUM>. The input/output devices <NUM> may include a keyboard, mouse, display and the like. A network interface circuit <NUM> is also connected to the bus <NUM> to provide connectivity to network <NUM>. A memory <NUM> is also connected to the bus <NUM>. The memory <NUM> includes modules with executable instructions, such as a photo filter module <NUM>. The photo filter module <NUM> implements photo evaluation and filter selection operations, as discussed below.

<FIG> illustrates processing operations associated with an embodiment of the invention. The operations are performed by the photo filter module <NUM> of server <NUM> in combination with one or more client devices <NUM>. Initially, the photo filter module <NUM> serves a photo prompt <NUM>. For example, the photo filter module <NUM> may form a segment of a network executed application that coordinates taking photographs and appending messages to such photographs for delivery from one user to another. In this context, client 100_1 accesses the photo filter module <NUM> over network <NUM> to activate the application, which serves the photo prompt to the client 100_1. A user at the client 100_1 takes a photo <NUM>.

<FIG> illustrates client device 100_1 with a display <NUM> that presents a photo prompt <NUM>. Activation of the photo prompt <NUM> results in a picture <NUM>. The photo filter module <NUM> monitors the client device activity to determine if a photo is taken <NUM>. If so, the attributes of the photograph and client device are evaluated <NUM>. Photo filters are selected and supplied <NUM> based upon the evaluation.

The attributes of the client device include geolocation of the client device, which is collected from the GPS processor <NUM>. The geolocation is used to designate photo filters relevant to the geolocation. For example, if the geolocation is proximate to a beach, then photo filters to augment a beach setting (e.g., a color filter for water, sand and/or sky) may be supplied. The geolocation may be used to select a filter with a brand associated with an establishment proximate to the geolocation. For example, a restaurant or store may sponsor a photo filter that includes a brand associated with the restaurant or store. In this case, in addition to the brand, the photo filter may include other indicia associated with the restaurant (e.g., an image of a hamburger or taco) or store (e.g., an image of a surf board or sun glasses).

The attributes associated with the client device may include established preferences associated with the client device. The established preferences may be defined by explicitly stated preferences supplied by a user. Alternately, the established preferences may be derived from prior use patterns. For example, explicitly stated or derived preferences may indicate that photo filters with a temperature overlay, date and/or time overlay be supplied.

The attributes of the photograph may include the physical environment captured in the photograph. For example, the photograph may be evaluated to identify an urban setting, a rural setting, a sunset a seascape and the like. Filters applicable to the physical environment may then be supplied.

The attributes of the photograph include an object depicted in the photograph. For example, the evaluation may identify a building, a building feature (e.g., door or roof), a flower, an individual, an animal and the like. Filters applicable to such objects are then be supplied.

The next operation of <FIG> is to apply the photo filters <NUM>. A swipe across the display of a client device 100_1 causes a photo filter to slide across the original photo. <FIG> illustrates the result of a first swipe motion, which results in a darkening filter <NUM> being applied to the original photo. Another swipe motion may result in another filter being presented. For example, <FIG> illustrates the result of a second swipe motion, which results in an object specific filter <NUM> being presented. In this case, the object specific filter <NUM> relates to the identification of a door in the photo. The identification of the door may result in the supply of a variety of filters for different door colors. Another swipe of the display may result in still another filter, such as shown in <FIG>. The filter of <FIG> includes an object specific filter <NUM>, in this case for a roof of a building. The filter also includes a brand component <NUM>. This filter also includes an overlay of the temperature <NUM> when the photo was taken. A time overlay <NUM> and date overlay <NUM> are also supplied.

Returning to <FIG>, the next operation is to select a photo filter <NUM>. Selection of a photo filter includes selection of one or more available filters. The photo may then be saved with the applicable filter or filters. The photo and filter are also sent to another user <NUM>. In this case, the server <NUM> routes <NUM> the photo to another client 100_2, which displays the photo with the filter <NUM>.

Photograph filters may also be selected based upon popular filters. Branded filters may be supplied based upon an auction mechanism. For example, vendors may bid on photo filters to be supplied based upon characteristics of a user, location of a user, content of a photograph and the like.

An embodiment of the present invention relates to a computer storage product with a non-transitory computer readable storage medium having computer code thereon for performing various computer-implemented operations. The media and computer code may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well known and available to those having skill in the computer software arts. Examples of computer-readable media include, but are not limited to: magnetic media, optical media, magneto-optical media and hardware devices that are specially configured to store and execute program code, such as application-specific integrated circuits ("ASICs"), programmable logic devices ("PLDs") and ROM and RAM devices. Examples of computer code include machine code, such as produced by a compiler, and files containing higher-level code that are executed by a computer using an interpreter. For example, an embodiment of the invention may be implemented using JAVA®, C++, or other object-oriented programming language and development tools. Another embodiment of the invention may be implemented in hardwired circuitry in place of, or in combination with, machine-executable software instructions.

Claim 1:
A system comprising a server (<NUM>) and a plurality of client devices (100_1 - 100_N),
wherein each client device comprises a processor (<NUM>), a touch controller (<NUM>) and a display (<NUM>), the touch controller being responsive to haptic signals applied to the display, and
wherein the server (<NUM>) comprises:
a processor (<NUM>); and
a memory (<NUM>) including a photo filter module (<NUM>) with instructions executable by the processor (<NUM>) to:
identify (<NUM>) when a client device (100_1) captures a photograph;
select a plurality of photograph filters based upon attributes of the client device and attributes of the photograph, wherein the attributes of the client device include geolocation of the client device and the attributes of the photograph include an object depicted in the photograph, and wherein the filters are selected so as to be relevant to the geolocation and so as to be applicable to the object;
supply (<NUM>) the photograph filters to the client device, wherein the photograph filters are configured to be independently selectable in response to a swipe gesture applied by the user via the touch controller (<NUM>) to the photograph as presented on the display (<NUM>) of the client device, wherein each photograph filter is an overlay on top of the photograph to augment the photograph, wherein:
a first filter of the plurality of photograph filters is a general filter configured to be selected in response to a first swipe motion detected by the touch controller (<NUM>), the general filter being applied to the whole of the photograph; and
a second filter of the plurality of photograph filters is an object specific filter configured to be selected in response to a second swipe motion detected by the touch controller (<NUM>), the object specific filter being applied to a portion of the photograph associated with the object;
wherein the instructions further include instructions executable by the processor to: in response to a photograph with an independently selected one or more photograph filters being received (<NUM>) from the client device, route (<NUM>) the one or more photograph filters and the photograph to another client device (<NUM>-<NUM>).