AUTOMATIC ITEM RECOMMENDATIONS BASED ON VISUAL ATTRIBUTES AND COMPLEMENTARITY

A user device is caused to display a visual attribute representation for a plurality of visual attributes. Each visual attribute is based at least in part on an image and each visual attribute representation is selectable. A processor is caused to identify a plurality of items, each item is associated with a visual attribute matching at least one of the plurality of visual attributes. The items are classified a first set and a second set. The items in the first and second sets are mutually exclusive and simultaneously displayed. If the processor receives a single selection of a first visual attribute representation of a first visual attribute of the plurality of visual attributes, the first set consist of items associated with a visual attribute matching the first visual attribute and the second set comprise items associated with a visual attribute matching at least one of the plurality of visual attributes.

FIELD

This disclosure relates generally to recommending items based on visual attributes associated with the items and complementarity of the items, and more particularly to recommending items based on visual attributes which match one or more visual attributes associated with an image or a palette and based on complementarity of items with other items.

BACKGROUND

Shopping for items online may enable users to purchase a large variety of items from a large variety of vendors. However, traditional online shopping vendors may present users with too many item options, may present users with items in a random order or a haphazard manner, or may present users with unrelated or irrelevant items. Traditional online shopping may also present users with items that do not meet current needs or desires, such as items which are behind current trends or which are not popular in a demographic. In such traditional online shopping environments, users may be unable to purchase desirable items and unable to visualize a desirable combination of items or a desirable ensemble.

SUMMARY

In one embodiment, there is provided a computer-implemented method involving causing at least one processor configured with specific computer-executable instructions to cause a user device to display a visual attribute representation for each visual attribute of a plurality of visual attributes generated from at least one image. Each visual attribute is based at least in part on the at least one image and each visual attribute representation is selectable by a user of the user device. The computer-implemented method further involves causing the at least one processor to identify a plurality of items based on information stored in an electronic database. Each item of the plurality of items is associated with a visual attribute matching at least one visual attribute of the plurality of visual attributes. The computer-implemented method further involves causing the at least one processor to classify the plurality of items into a first set and a second set. The items in the first set and the items in the second set are mutually exclusive. If the at least one processor receives a single selection of a first visual attribute representation of a first visual attribute of the plurality of visual attributes, the first set consist of items associated with a visual attribute matching the first visual attribute and the second set comprise items associated with a visual attribute matching at least one visual attribute of the plurality of visual attributes. The computer-implemented method further involves causing the at least one processor to cause the user device to simultaneously display the first set and the second set proximate to each other.

Other aspects and features of the present disclosure will become apparent to those ordinarily skilled in the art upon review of the following description of specific embodiments of the disclosure in conjunction with the accompanying figures.

DETAILED DESCRIPTION

Methods and systems for providing recommendations for items based on a plurality visual attributes generated from an image and/or a custom palette and on complementarity of the items with other items are disclosed.

A user, upon viewing an image or a palette with desirable visual attributes, may wish to locate and purchase items which have those desirable visual attributes. Such “visual attributes” include without limitation, colors, patterns, textures, reflectivity etc. Further, when a user is shopping or searching for a particular item, the user may be interested in “complementary” items which match, or are often purchased together with, the initially shopped for item. “Complementary” items include without limitation items which fall into macro-item categories that are often found together (tops and bottoms, shoes and socks, sofas and cushions etc. for example), items which fall into micro-item categories that are often found together (dress shirts and suits, dresses and heels, etc. for example), items which have a history of being selected and viewed by users when the users are searching for another item (“interaction history”), items which have a history of being combined by users (“combination history”), items which have a history of being purchased by users together (“purchase history”), and items which are simply associated with visual attributes which match one or more visual attributes of the desirable visual attributes of an image or a palette. Further, items may be “complementary” to other items at an item-level (a specific dress is complementary with a specific pair of shoes for example), and items may also be “complementary” to other items at an category-level (dresses are generally complementary with heels for example).

Presenting the user with multiple complementary items which have the desirable visual attributes may encourage the user to purchase more than one item or purchase more and different items than initially intended.

An illustrative embodiment of an item recommendation server is shown generally at100inFIG.1. The item recommendation server100includes a processor circuit, which in the embodiment shown includes at least one microprocessor102, and a clock104, an input/output (“I/O”) interface106, a program memory108, and a storage memory110, all in communication with the microprocessor102. In other embodiments, the item recommendation server100may include different components, a greater or a fewer number of components, and can be structured differently.

The clock104maintains values representing a current date and time and provides the values to the microprocessor102for storage in various data stores in the storage memory110as described below. The I/O interface106includes an interface for communicating, over components of a network shown generally at112, with at least one user device114and at least one vender server116, and in some embodiments, at least one payment processor117.

Although only a few user devices114and a few vendor servers116are shown inFIG.1, other embodiments may include a larger or a fewer number of user devices114or vendor servers116. In some embodiments, the microprocessor102may communicate with the user devices114and the vendor servers116without the network112.

The program and storage memories108and110may each be implemented as one or a combination of a random-access memory (“RAM”), a hard disk drive (“HDD”), and other computer-readable and/or -writable memory. In other embodiments, the item recommendation server100may be partly or fully implemented using different hardware logic, which may include discrete logic circuits and/or an application specific integrated circuit (“ASIC”), for example. In some embodiments, the microprocessor102may communicate with the storage or program memories110and108via the network112or another network.

The storage memory110may store information obtained by the microprocessor102and may be an information or data store. The program memory108includes various blocks of code, including codes for directing the microprocessor102to execute various functions of the item recommendation server100, such as image processing services, item processing services and item recommendation services. The program memory108also includes database management system (“DBMS”) codes120for managing an application database122and a representation database124in the storage memory110. In other embodiments, the program memory108may include additional or alternative blocks of code.

An embodiment of the application database122is shown generally inFIG.2; in the embodiment shown, application database122is a relational database including a plurality of tables. The various tables of the application database122can each store various. The various entries each include various fields, and an instance of such an entry can store specific values in such fields. In other embodiments, the application database122may include different components, a greater or a fewer number of components, can be structured differently, can be a graph database, and can be an unstructured database. The representation database124may store representations of different images, items, palettes and visual attributes.

Referring now toFIG.2, in the embodiment shown, the application database122includes a user table130that can store any number of instances of a user entry, an embodiment of which is shown generally at131inFIG.3. An instance of the user entry131stores data associated with a user registered with the item recommendation server100that may access the item recommendation server100through one or more user devices114(shown inFIG.1). In the embodiment ofFIG.3, the user entry131includes an identifier field132for storing an integer (a user identifier) assigned by the DBMS codes120(shown inFIG.1) to identify an instance of the user entry131uniquely in the user table130. The user entry131may also include an email field134for storing an electronic mail address, a password field136for storing a password and a username field137for storing a username. A user associated with the user entry131may access the item recommendation server100by using the email or username stored in fields134and137in combination with the password stored in field136. The user entry131may also include a user data field138for storing various demographic information associated with the user, such as “age”, “gender”, “address” and “income range” for example. In the embodiment shown, user data field138is a single field, but in other embodiments, the user data field138may be a plurality of fields. The user entry131may also include a user representation path field139for storing a uniform resource identifier (“URI”) identifying a storage location of a representation of the user in the representation database124(shown inFIG.1) to allow the microprocessor102to retrieve a representation of the user for display.

Referring back toFIG.2, the application database122also includes an image table140that can store any number of instances of an image entry, an embodiment of which is shown generally at141inFIG.4. An instance of image entry141represents a stored image which may be processed by the item recommendation server100to generate image posts and/or to extract one or more visual attributes. In the embodiment ofFIG.4, the image entry141includes an identifier field142storing an integer (an image identifier) assigned by the DBMS codes120(shown inFIG.1) to identify an instance of the image entry141uniquely in the image table140. The image entry141may also include a user identifier field144storing an user identifier stored in the identifier field132of an instance of the user entry131(shown inFIG.3), such that an instance of the image entry141identifies an instance of the user entry131to associate an image with a particular user (images which are uploaded by the user would be associated with that user for example). In the embodiment shown, multiple instances of the image entry141can identify a particular instance of the user entry131, indicating that a single user can upload multiple images. The image entry141may also include a visual attribute identifier field146storing a visual attribute identifier stored in an identifier field162of an instance of a visual attribute entry161(shown inFIG.6) described below, such that an instance of the image entry141identifies an instance of the visual attribute entry161to associate an image with a particular visual attribute (image is associated with a color, a pattern, or a texture). In the embodiment shown, an instance of the image entry141can identify multiple instances of the visual attribute entry161indicating that a particular image can be associated with more than one visual attribute. The image entry141may also include a description field148storing a description of the image, which may be a text string and may provide keywords associated with the image, such as “blue” or “cityscape” or “outfit” or “female”. The image entry141may also include an image representation path field150storing a URI identifying a storage location of the image in the representation database124(shown inFIG.1) to allow the microprocessor102to retrieve the image for display.

Referring back toFIG.2, the application database122also includes a palette table190that can store any number of instances of a palette entry, an embodiment of which is shown generally at191inFIG.5. An instance of the palette entry191represents a stored palette including a collection of visual attributes. In certain embodiments, the visual attributes associated with a palette may originate from an image stored in the representation database124. In the embodiment ofFIG.5, the palette entry191includes an identifier field192for storing an integer (a palette identifier) assigned by the DBMS codes120(shown inFIG.1) to identify an instance of the palette entry191uniquely in the palette table190. The palette entry191may also include a visual attribute identifier field146for storing a visual attribute identifier stored in the identifier field162of an instance of the visual attribute entry161(shown inFIG.6) described below, such that an instance of the palette entry191identifies an instance of the visual attribute entry161to associate a palette with a particular visual attribute. In the embodiment shown, an instance of the palette entry191can identify multiple instances of the visual attribute entry161indicating that a particular palette can be associated with more than one visual attribute. The palette entry191may also include a description field194for storing a description of the custom palette, which may be a text string and may provide some keywords associated with the palette, such as “rust” or “spring” for example. The palette entry191may also include a palette representation path field196for storing a URI identifying a storage location of representations of the custom palette in the representation database124(shown inFIG.1) to allow the microprocessor102to retrieve a representation of the custom palette for display.

Referring back toFIG.2, the application database122also includes a visual attribute table160that can store any number of instances of the visual attribute entry161, an embodiment of which is shown generally inFIG.6. An instance of the visual attribute entry161represents a visual attribute which may be associated with an image, a palette, or an item. In the embodiment ofFIG.6, the visual attribute entry161includes an identifier field162storing an integer (a visual attribute identifier) assigned by the DBMS codes120(shown inFIG.1) to identify an instance of the visual attribute entry161uniquely in the visual attribute table160. The visual attribute entry161may also include a definition field164for storing a definition of the visual attribute that can be compared to definitions of other visual attributes to determine if the two visual attributes match and the degree of match between the two visual attributes. For example, if the visual attribute is a color, the definition field164may store the red, green and blue pixel values associated with the color; if the visual attribute is a pattern or a texture, the definition field164may store a visual attribute representation representing the pattern or the texture. The visual attribute entry161may also include a description field166for storing a description of the visual attribute, which may be a text string and may provide some keywords associated with and for identifying the visual attribute, such as “green”, “periwinkle”, “aqua”, “yellow”, “strawberry”, “fuchsia”, “plaid”, or “floral” for example. The visual attribute entry161may also include a visual attribute representation path field168for storing a URI identifying a storage location of visual attribute representations in the representation database124to allow the microprocessor102to retrieve a representation of the visual attribute for display.

Referring back toFIG.2, the application database122also includes an item table170that can store any number of instances of the item entry, an embodiment of which is shown generally at171inFIG.7. An instance of item entry171represents a stored item which may be obtained by the microprocessor102from one or more vendor servers116(or websites operated by one or more vendors) and which may be recommended by the microprocessor102to a user. In the embodiment ofFIG.7, the item entry171includes an identifier field172storing an integer (an item identifier) assigned by the DBMS codes120(shown inFIG.1) to identify an instance of the item entry171uniquely in the item table170. The item entry171may also include a vendor identifier field173for storing an vendor identifier stored in an identifier field232of an instance of a vendor entry231(shown inFIG.8) described below, such that an instance of the item entry171identifies an instance of the vendor entry231to associate an item with a particular vendor (items which are retrieved by the microprocessor102from a vendor server116operated by a particular vendor would be associated with that vendor for example). In the embodiment shown, a particular instance of the item entry171can identify a particular instance of the vendor entry231, indicating that a particular item is associated with one vendor. In other embodiments, a particular item may be associated with more than one vendor. The item entry171may also include: a vendor description field178for storing a description of the item from the vendor, and which may be presented to a user when the user is examining the item to determine whether the user would like to purchase the item; a price field179for storing a price of the item retrieved from the vendor, and which may be presented to the user; and an options field180for storing various options associated with item provided by the vendor, such as clothing size, shoe widths, and furniture configurations for example, and which may be presented to the user. The item entry171may also include a purchase path field181for storing information to facilitate purchase of the item by a user from the vendor, such as a link to a vendor's webpage for purchasing the item or information which facilitates direct communication between the item recommendation server100and the payment processor117associated with the vendor sever116to facilitate direct purchase of the item through the item recommendation server100.

The item entry171may also include a taxonomy identifier field174for storing a taxonomy identifier stored in an identifier field222of an instance of a taxonomy entry221(shown inFIG.9), such that an instance of the item entry171identifies an instance of the taxonomy entry221to associate an item with a particular item category represented by the taxonomy entry221. In the embodiment shown, the item entry171can identify multiple instances of the taxonomy entry221, indicating that an item can be associated with more than one item category. The item entry171may also include a visual attribute identifier field175, storing a visual attribute identifier stored in the identifier field162of an instance of the visual attribute entry161(shown inFIG.3), such that an instance of the item entry171identifies an instance of the visual attribute entry161to associate an item with the visual attribute. In the embodiment shown, an instance of the item entry171can identify multiple instances of the visual attribute entry161, indicating that a particular item can be associated with one or more visual attributes. The item entry171may also include a description field176for storing a description of the item, which may be a text string and may provide some keywords associated with the item, such as “shoes” or “heels” or “SS2019” for example. The item entry171may also include an item representation path field177for storing a URI identifying a storage location of one or more representations of the item in the representation database124(shown inFIG.1) to allow the microprocessor102to retrieve one or more representation of the item for display. The item entry171may also include a complementary item identifier field182for storing an item identifier stored in the identifier field172of another instance of the item entry171that is complementary to the current item entry171and a complementary item order field183, which may store a level of complementarity of the item identified in the complementary item identifier field182with the current item.

Referring back toFIG.2, the application database122also includes a vendor table230that can store any number of instances of the vendor entry231, an embodiment of which is shown inFIG.8. An instance of vendor entry231represents information about a vendor and may represent a source of items obtained by the microprocessor102. In the embodiment ofFIG.8, the vendor entry231includes the identifier field232storing an integer (a vendor identifier) assigned by the DBMS codes120(shown inFIG.1) to identify an instance of the vendor entry231uniquely in the vendor table230. The vendor entry231also includes description field234, storing a description of the vendor, and which may include a link back to a vendor website.

Referring back toFIG.2, the application database122also includes a taxonomy table220that can store any number of instances of the taxonomy entry221, an embodiment of which is shown inFIG.9. An instance of the taxonomy entry221stores information about an item category and may identify complementary item categories. In the embodiment shown inFIG.9, the taxonomy entry221includes the identifier field222storing an integer (a taxonomy identifier) assigned by the DBMS codes120(shown inFIG.1) to identify an instance of the taxonomy entry221uniquely in the taxonomy table220. The taxonomy entry221may also include a macro-item category field224and a micro-item category field226. In certain embodiments, the macro-item category field224may store an indication of a broad item category, such as “clothing”, “cosmetics”, “home & garden” or “shoes” for example, and the micro-item category field226may store an indication of narrower item types, such as “dress”, “skirt”, “sofa”, “bed” or “heels” for example. In some embodiments, the taxonomy entry221may include one of the macro-item category field224and the micro-item category field226. The taxonomy entry221may also include: a complementary taxonomy identifier field228for storing a taxonomy identifier stored in the identifier field222of another instance of the taxonomy entry221complementary to the current taxonomy entry221, such that an instance of the taxonomy entry221identifies one or more other instances of complementary taxonomy entries221; and a complementary taxonomy order field229for storing a level of complementarity of the taxonomy identified in the complementary taxonomy identifier field228with the current taxonomy.

Referring back toFIG.2, the application database122also includes an interaction history table250that can store any number of instances of an interaction history entry, an embodiment of which is shown generally at251inFIG.10. An instance of the interaction history entry251represents a record of a user interacting with a particular item via an interface implemented by the item recommendation server100, and may more generally function as a record of which items are being interacted with by users in general, which items may be interacted with by a particular user, and/or which items may be interacted with when recommended by the item recommendation server100in association with a particular image or palette. The interaction history entries251may be utilized by a complementarity model to determine which items are complementary with other items and/or a ranking model to determine which items should be ordered first in a first set of items and a second set of items as described below in connection with recommend items codes650ofFIGS.27A-27C.

In the embodiment shown inFIG.10, the interaction history entry251includes an identifier field252storing an integer (an interaction history identifier) assigned by the DBMS codes120(shown inFIG.1) to identify an instance of the interaction history entry251uniquely in the interaction history table250. The interaction history entry251may also include: a user identifier field253for storing a user identifier stored in the identifier field132of an instance of the user entry131(shown inFIG.3), an item identifier field254for storing an item identifier stored in the identifier field172of an instance of the item entry171(shown inFIG.7), and an image identifier field255for storing an image identifier stored in the identifier field142of an instance of the image entry141(shown inFIG.4) or a palette identifier stored in the identifier field192of a palette entry191(shown inFIG.5). The interaction history entry251may further include an item set field258for storing an indication of whether the item identified in the item identifier field254was classified within the first set or the second set as described below in connection with recommend items codes650ofFIGS.27A-27C. As briefly described above, an instance of the interaction history entry251functions as a record associating a particular user with a particular item and a particular image (or a particular palette) and represents an indication of a user's interest in a particular item. In the embodiment shown, an instance of the interaction history entry251can identify one user, one item and one image (or one palette), indicating a single interaction of a user with an item. In other embodiments, the interaction history entry251can identify more than one user, more than one item or more than one image. The interaction history entry251may further include: a visual attribute identifier field256for storing a visual attribute identifier stored in the identifier field162of an instance of the visual attribute entry161(shown inFIG.6) in embodiments where a user selects a particular visual attribute associated with the image or the palette identified in the image identifier field255; and a taxonomy identifier field257for storing a taxonomy identifier stored in the identifier field222of an instance of the taxonomy entry221(shown inFIG.9) in embodiments where a user enters a particular text query that match or correspond to an instance of the taxonomy entry221. The interaction history entry251may further include a created field259and a modified field260for storing a date and time at which the instance of the interaction history entry251was created and modified respectively. For example, a user may interact with a particular item at a particular time, and that particular time may be stored in the created field259; the user may then interact with that same item at a different point in time, and that different time may be stored in the modified field260. In some embodiments, the interaction history entry251only includes the created field259, such that a new instance of the interaction history entry251is created each time a user interacts with a same item.

Referring back toFIG.2, the application database122also includes a combination history table290that can store any number of instances of a combination history entry, an embodiment of which is shown generally at291inFIG.11. An instance of the combination history entry291represents a record of a user interacting with more than one item at the same time via an interface implemented by the item recommendation server100, and may function as a record of which items are often combined by users in general, which items are often combined by a particular user, and which items may be combined by users when recommended by the item recommendation server100in association with a particular image for example. The combination history entries291may be utilized by a complementarity model to determine which items are complementary with other items and/or a ranking model to determine which items should be ordered first in a first set of items and a second set of items as described below in connection with the recommend items codes650ofFIGS.27A-27C.

In the embodiment shown inFIG.11, the combination history entry291includes an identifier field292storing an integer (a combination history identifier) assigned by the DBMS codes120(shown inFIG.1) to identify an instance of the combination history entry291uniquely in the combination history table290. The combination history entry291may also include: a user identifier field293storing a user identifier stored in the identifier field132of an instance of the user entry131(shown inFIG.3), a first set item identifier field294storing at least one item identifier stored in identifier fields172of instances of the item entry171(shown inFIG.7) classified within the first set as described below in connection with the recommend items codes650ofFIGS.27A-27C, a second set item identifier field295storing at least one item identifier stored in identifier fields172of instances of the item entry171(shown inFIG.7) classified within the second set as described below in connection with the recommend items codes650ofFIGS.27A-27C, and an image identifier field296storing an image identifier stored in the identifier field142of an instance of the image entry141(shown inFIG.4) or a palette identifier stored in the identifier field192of an instance of the palette entry191(shown inFIG.5). As described briefly above, an instance of the combination history entry291thus functions as a record associating a particular user with at least one item from the first set and/or at least one item from the second set and a particular image (or a particular palette), and represents that the user's simultaneous interest in at least two items or the user's combination of at least two items. In the embodiment shown, an instance of the combination history entry291can identify one user, one or more items and one image (or one palette), indicating a combination of one or more items by a user. In other embodiments, the combination history entry291can identify more than one user, only one item or more than one image. The combination history entry291may further include: a visual attribute identifier field297for storing a visual attribute identifier stored in the identifier field162of an instance of the visual attribute entry161(shown inFIG.6) in embodiments where a user provides a selection of a visual attribute associated with the image or the palette identified in the image identifier field296; and a taxonomy identifier field298for storing a taxonomy identifier stored in the identifier field222of an instance of the taxonomy entry221(shown inFIG.9) in embodiments where a user enters a particular text query that may match or correspond to a taxonomy entry221. The combination history entry291may further include a created field299and a modified field300for storing a date and time at which the instance of the combination history entry291was created and modified respectively. For example, a user may make a combination of two or more items at a particular time, and that particular time may be stored in the created field299; the user may then add another item to the combination, or remove an item from the combination at a later point in time, and that later time may be stored in the modified field300. In some embodiments, the combination history entry291only includes the created field299, such that a new instance of the combination history entry291is created each time a user combines one or more items.

Referring back toFIG.2, the application database122also includes a purchase history table270that can store any number of instances of a purchase history entry, an embodiment of which is shown generally at271inFIG.12. An instance of the purchase history entry271represents a record of a user purchasing a particular item or a plurality of items in combination with each other, via an interface implemented by the item recommendation server100, and may more generally function as a record of which items are often purchased by users in general, which items are often purchased by a particular user, which items may be purchased together, and which items may be purchased by users when recommended by the item recommendation server100in association with a particular image or a particular palette for example. The purchase history table entries271may be utilized by a complementarity model to determine which items are complementary with other items and/or a ranking model to determine which items should be ordered first in a first set of items and a second set of items as described below in connection with the recommend items codes650ofFIGS.27A-27C.

In the embodiment shown inFIG.12, the purchase history entry271includes an identifier field272storing an integer (a purchase history identifier) assigned by the DBMS codes120(shown inFIG.1) to identify an instance of the purchase history entry271uniquely in the purchase history table270. The purchase history entry271may also include: a user identifier field273for storing a user identifier stored in the identifier field132of an instance of the user entry131(shown inFIG.3), an item identifier field274for storing an item identifier stored in the identifier field172of at least one instance of the item entry171(shown inFIG.7), and an image identifier field275for storing an image identifier stored in the identifier field142of an instance of the image entry141(shown inFIG.4) or a palette identifier stored in the identifier field192of an instance of the palette entry191(shown inFIG.5). As briefly described above, an instance of the purchase history entry271thus functions as a record associating a particular user with one or more items and a particular image (or a particular palette), and represents that the user purchased the one or more items. In the embodiment shown, an instance of the purchase history entry271can identify one user, one or more items and one image (or one palette), indicating a purchase of one or more items by a user. In other embodiments, the purchase history entry271can identify more than one user or more than one image. The purchase history entry271may further include: a visual attribute identifier field276for storing a visual attribute identifier stored in the identifier field162of an instance of the visual attribute entry161(shown inFIG.6) in embodiments where a user provides a selection of a visual attribute associated with the image or the palette identified in the image identifier field275; and a taxonomy identifier field277for storing a taxonomy identifier stored in the identifier field222of an instance of the taxonomy entry221(shown inFIG.9) in embodiments where a user enters a particular text query that may match or correspond to a taxonomy entry221. The purchase history entry271may further include a created field278for storing a date and time at which the instance of the purchase history entry271was created. For example, a user may purchase a particular item at a particular time, and that particular time may be stored in the created field278.

Referring back toFIG.1, the program memory108includes user interface codes330for communicating with user interfaces of the user devices114and for displaying information on displays of the user devices114. For example, the user interface codes330may include various codes to enable a user of the user device114to interact with the item recommendation server100via a mobile application, and descriptions of embodiments below illustrate various mobile application interfaces for display on the user device114. Other configurations may allow the item recommendation server100and the user device114to interact in a similar manner. For example, the user may access a web page hosted by the item recommendation server100using an internet browser installed on the user device114.

Referring toFIG.13, a login page produced by the user interface codes330for display by the user device114is shown generally at350. The login page350may be accessed by opening a mobile application installed on the user device114. In the embodiment shown, the login page350includes an email input field352, a password input field354and a login button356.

When the user is registered with the item recommendation server100, the user may enter an electronic mail address into the email input field352, enter a password into the password input field354, and select the login button356. When the user selects the login button356, the user interface codes330may direct the user device114to transmit the entered email and password to the microprocessor102in a user login request. The microprocessor102may respond to the user login request by determining whether the user table130(shown inFIG.2) includes a user entry131storing the transmitted email address in the email field134and storing the transmitted password in the password field136. If the transmitted email and address do not both match an instance of the user entry131, the user interface codes330may transmit an error message to the user device114indicating that, for example, the email entered could not be found or that the password entered is incorrect or that the user needs to register with the item recommendation server100. If the user is not already registered with the item recommendation server100, the user may be prompted to navigate to a separate registration page (not shown) which includes fields prompting the user to enter an email, a password, a username, and certain user data (such as demographic data or a user photo for example).

When the user has entered and submitted the required information, the user interface codes330may direct the user device114to transmit the information to the microprocessor102in an add user request. The microprocessor102may respond to the add user request by adding a new instance of the user entry131(shown inFIG.3) to the user table130(shown inFIG.2). This new instance of the user entry131may store the email, the password, the username, and the user data entered by the user in the email field134, the password field136, the username field137and the user data field138respectively.

If the transmitted email and address do both match an instance of the user entry131, the user interface codes330may direct the user device114to display a home page, an embodiment of which is shown generally at360inFIG.14. Referring toFIG.14, in the embodiment shown, the home page360includes a header region362, a feed region364and a navigation region366. The header region362and the navigation region366may be common to a number of different pages that the user interface codes330direct the user device114to display.

In the embodiment shown, the header region362includes a palette selection button370and a search button372. When the user selects the palette selection button370, the user interface codes330may direct the user device114to display a palette selection page750described below in connection withFIG.33. When the user selects the search button372, the user interface codes330may direct the user device114to display a search page (not shown) with a query region (not shown) operable to receive a text query from the user. If the user enters a query in the query region, the microprocessor102may search the application database122for, and retrieve, for example, user entries131in the user table130, image entries141in the image table140, item entries171in the item table170, palette entries191in the palette table190, vendor entries231in the vendor table230, and visual attribute entries161in the visual attribute table160, which match or correspond to the query entered by the user in the query region. An entry may match or correspond to the query entered by the user by storing, in one of the entry's fields, text matter which is identical, similar to, or synonymous with, the text of the query entered by the user. For example, if the user searches for the query “kitten” in the query region, the microprocessor102may locate and retrieve: (1) user entries131(shown inFIG.3) which store “kitten”, “cat” or “kitty” in the email field134, the username field137, and/or the user data field138; (2) image entries141(shown inFIG.4) which store similar terms in the description field148; (3) item entries171(shown inFIG.7) which store similar terms in the description field176or the vendor description field176; (4) palette entries191(shown inFIG.5) which store similar terms in the description field194; and/or visual attribute entries161(shown inFIG.6) which store similar terms in the description field166. The user interface codes330may then direct the user device114to display the entries retrieved by the microprocessor102on the search page (not shown). In certain embodiments, the search page may display the entries from different tables of the application database122as a group on the search page. For example, the retrieved user entries131under a header of “People”, the retrieved image entries141under a header of “Posts”, and the retrieved item entries171under a header of “Items”.

The feed region364displays images and associated visual attributes posted by users. In the embodiment shown inFIG.14, the feed region364may display images and associated visual attributes by displaying a plurality of image posts380,390, wherein each image post380,390includes an image384,394, a user indicator382,392, a visual attribute array386,396, and a shop image button388. In the embodiment shown, the feed region364is vertically scrollable to view the image posts380,390. In other embodiments, the feed region may be horizontally scrollable, or may have a page flip format where each page corresponds to an image post.

Each image post380,390may correspond to an image entry141(shown inFIG.4) stored in the image table140. The image384,394may display the image representation stored in the representation database124(shown inFIG.1) directed to by a URI in the image representation path field150of that image entry141. The user indicator382,392may display an indication of the user that uploaded the image384,394, and may display the username stored in the username field137of the user entry131identified in the user identifier field144of the image entry141. The visual attribute arrays386,396generally represent the visual attributes associated with the image384,394. In the embodiment shown, each visual attribute array386,396includes a plurality of visual attribute representations. Each visual attribute representation may display a visual attribute representation stored in the representation database124(shown inFIG.2) directed to by a URI in the visual attribute representation path field168of one or more visual attribute entries161(shown inFIG.6) identified in the visual attribute identifier field175of the image entry141(shown inFIG.4). The shop image button388is selectable by a user and allows the item recommendation server100to recommend items associated with visual attributes which match the visual attributes associated with the image384,394(such visual attributes of the visual attribute array386,396). If a user selects the shop image button388, the user interface codes330may direct the user device114to display a shop image page630described below in connection withFIG.25.

Still referring toFIG.14, in the embodiment shown, the navigation region366includes a home button400, a user profile button402, an upload image button404and a shopping cart button406. When the user selects the home button400, the user interface codes330may direct the user device114to display the home page360.

When the user selects the user profile button402, the user interface codes330may direct the user device114to display a user profile page as shown generally at410inFIG.15. Referring toFIG.15, the user profile page410displays information associated with a particular user corresponding to a particular user entry131(shown inFIG.3) in the user table130, and images posted by that user. When the user selects the user profile button402of the navigation region310, the user interface codes330direct the user device114to display the user profile page410corresponding to the current user (such as the user that initially logged on using the login page350(shown inFIG.13) for example). However, when the user selects a user indictor of another user, such as the user indicator382,392(shown inFIG.14) for example, the user interface codes330may display the user profile page410associated with that other user. In the embodiment shown inFIG.15, the user profile page410includes a profile region412which can be vertically scrolled to view data associated with the user corresponding to the user profile page410and image posts posted by the user. In other embodiments, the profile region412may be horizontally scrolled or may have a page-flip format.

The profile region412may include a user data region414and a user post region416. The user data region414may display various user data associated with the user, such as information stored in the user entry131representing that user. For example, the user data region414includes a user representation420, which may display a user representation stored in the representation database124(shown inFIG.2) directed to by a URI in the user representation path field139of the user entry131(shown inFIG.3). The user data region414also includes a user indicator422, which may display the username stored in username field137of the user entry131. In other embodiments, the user data region414may include more or less user data. For example, in other embodiments, the user data region414may include a biography or other information stored in the user data field138of the user entry131.

The user post region416displays a plurality of image posts430associated with the user in a manner similar to the feed region364(shown inFIG.14). For example, each image post430of the user post region416may correspond to an image entry141(shown inFIG.4) which identify the user entry131representing the user in the user identifier field144. The user post region416may display, as image posts430, every image entry141associated with the user entry131. In some embodiments, the user post region416may only display a subset of the image entries141associated with the user entry131. The image posts430may be displayed chronologically, such that the image post430corresponding to the most recent image entry141is displayed first; the image posts430may also be displayed in some other order, such as popularity with other users or user preference for example.

Each image post430displays information associated with an image corresponding to an image entry141(shown inFIG.4) of the image table140. In the embodiment shown inFIG.15, each image post includes an image432, a visual attribute array434, and a shop image button436. The image432may be similar to image384,394(shown inFIG.14), and may display the image representation stored in the representation database124directed to by a URI in the image representation path field150of that image entry141. The visual attribute array434may be similar to visual attribute arrays386,396(shown inFIG.14), and may display a plurality of visual attributes associated with the image entry141as a plurality of visual attribute representations. For example, the visual attribute array434may display visual attribute representations stored in the representation database124directed to by a URI in the visual attribute representation path field168of the visual attribute entries161(shown inFIG.6) identified in the visual attribute identifier field146of the image entry141. The shop image button436may be similar to the shop image button388(shown inFIG.14) and may be is selectable by a user. If a user selects the shop image button436, the user interface codes330may direct the user device114to display the shop image page630(shown inFIG.25).

When the user selects an image of an image post, such as the images384,394(shown inFIG.14) or432(shown inFIG.15), the user interface codes330may direct the user device114to display an image page440, an embodiment of which is shown inFIG.16. Referring toFIG.16, the image page440displays information associated with an image post corresponding to a particular image entry141(shown inFIG.4) in the image table140. In the embodiment shown, the image page440includes an image region442which can be vertically scrolled to view the image, the visual attributes, and other information, all associated with that particular image entry141. In other embodiments, the image region442may be horizontally scrolled or may have a page flip format. The image region442may include an image post444and an image post region shown generally at446.

The image post444includes an image450, a visual attribute array452, and a shop image button454. The image450may be similar to images384,394(shown inFIG.14) and432(shown inFIG.15), and may display an image stored in the representation database124(shown inFIG.2) directed to by a URI in the image representation path field150of the image entry141. The visual attribute array452may be similar to visual attribute arrays386,396(shown inFIG.14) and434(shown inFIG.15) and may display a plurality of visual attributes associated with the image entry141as a plurality of visual attribute representations. Each visual attribute representation may display a visual attribute representation stored in the representation database124(shown inFIG.2) directed to by a URI in the visual attribute representation path field168of visual attribute entries161(shown inFIG.6) identified in the visual attribute identifier field146of the image entry141. The shop image button454may be similar to the shop image buttons388(shown inFIG.14) and436(shown inFIG.15) and may be is selectable by a user. If a user selects the shop image button454, the user interface codes330may direct the user device114to display the shop image page630(shown inFIG.25).

The image data region446includes a user indicator460and a description462. The user indicator460may display an indicator of the user that uploaded the image450and may specifically display a username stored in username field137of a user entry131identified in the user identifier field144of the image entry141. The user entry131may correspond to the user that uploaded the image post444for example. The user indicator460may be selectable, and when selected by the user, the user interface codes330may direct the user device114to display the user profile page410(shown inFIG.15) which corresponds to the user entry131. The description462may display a description stored in the description field148(shown inFIG.4) of the image entry141. In other embodiments, the image data region446may include more or less data. For example, the image data region446may include comments posted by other users associated with the image450(not shown) and alternative indications that other users like or otherwise appreciate the image450(not shown).

The item recommendation server100may allow a user to search for images based on visual attributes associated with the image. In some embodiments, when the user selects one or more visual attribute representations from a visual attribute array associated with an image (such as the visual attribute representations from the visual attribute arrays386,396(shown inFIG.14),434(shown inFIG.15) and452(shown inFIG.16) for example), the user interface codes330may direct the microprocessor102to search for, and retrieve, a visual attribute entry161(shown inFIG.6) in the visual attribute table160which corresponds to the selected visual attribute representation. In embodiments where more than one visual attribute representation is selected, the microprocessor102may retrieve more than one visual attribute entry161. A visual attribute entry161may correspond to the selected visual attribute representation when the visual attribute entry161stores a URI in the visual attribute representation path field168which identifies the selected visual attribute representation in the representation database124(shown inFIG.2). The microprocessor102may then search for, and retrieve, image entries141(shown inFIG.4) in the image table140which identify the retrieved visual attribute entry161in the visual attribute identifier field146. In embodiments where more than one visual attribute entry161is retrieved, the microprocessor102may retrieve image entries141which identify, in the visual attribute identifier field146, (1) any one of the more than one retrieved visual attribute entries161and/or (2) each one of the every retrieved visual attribute entries161. After the image entries141are retrieved, the user interface codes330may direct the user device114to display a visual attribute search page470shown inFIG.17to display the retrieved image entries141as search results, and may display the retrieved image entries141as image posts.

Referring toFIG.17, the visual attribute search page470displays results of a visual attribute search as noted above, and may further allow a user to refine an initial search by adding or removing visual attributes, or to search for additional or alternative images based on visual attributes. In the embodiment shown, the visual attribute search page470includes a visual attribute results region472. The visual attribute results region472includes a visual attribute query region474and a plurality of image posts490.

The visual attribute query region474may be automatically populated with one or more visual attribute queries corresponding to the visual attribute initially selected by the user. For example, when the user selects a visual attribute representations from a visual attribute array associated with an image (such as the representations from the visual attribute arrays386,396(shown inFIG.14),434(shown inFIG.15), or452(shown inFIG.16)), the user interface codes330may display visual attribute query region474automatically populated with a selected visual attribute query480. In embodiments where the user selects more than one visual attribute, the visual attribute query region474may be automatically populated with more than one selected visual attribute query.

The visual attribute results region472can be vertically scrolled to view the plurality of image posts490. In other embodiments, the visual attribute results region472may be horizontally scrolled or may have a page flip format. Each image post490may correspond to an image entry141(shown inFIG.4) of the image entries141retrieved by microprocessor102based on the initially selected visual attribute representation. The image post490includes a user indicator496, an image492, a visual attribute array494, and a shop image button498. The user indicator496may be similar to the user indicators382,392(shown inFIG.14) and460(shown inFIG.16), and may display an indication of the user that uploaded the image492, and may specifically display the username stored in the username field137of the user entry131identified in the user identifier field144of the image entry141. The image492may be similar to image384,394(shown inFIG.14),432(shown inFIG.15), and the image450(shown inFIG.16), and may display an image stored in the representation database124(shown inFIG.2) directed to by a URI in the image representation path field150of the image entry141. The visual attribute array494may be similar to the visual attribute arrays386,396(shown inFIG.14),434(shown inFIG.15) and452(shown inFIG.16), and may display visual attributes associated with the image entry141as a plurality of visual attribute representations. Each visual attribute representation may display a visual attribute representation stored in the representation database124directed to by a URI in the visual attribute representation path field168of visual attribute entries161(shown inFIG.6) identified in the visual attribute identifier field146of the image entry141. As described above, because the visual attribute results region472displays image entries141which are associated with the visual attribute initially queried for, at least one visual attribute representation of the visual attribute array494of each image post490will match the visual attribute initially queried for. For example, in the embodiment shown inFIG.17, only a single visual attribute was selected as the visual attribute query480, and thus a visual attribute representation495aof the visual attribute array494matches the visual attribute query480. In embodiments where a plurality of visual attributes are selected as the visual attribute query, (1) one visual attribute of the visual attribute array494of each image post490may match any visual attribute of the selected visual attributes or (2) the visual attribute array494of each image post490may include visual attributes which match each visual attribute of the selected visual attributes. The shop image button498may be similar to the shop image buttons388(shown inFIG.14),436(shown inFIG.15) and454(shown inFIG.16) and may be selectable by a user. If a user selects the shop image button498, the user interface codes330may direct the user device114to display the shop image page630(shown inFIG.25).

In some embodiments, the visual attribute query region474may be operable to receive additional, or modifications of, the visual attribute query. For example, a user may delete the visual attribute query or enter one or more descriptions or definitions of visual attributes in the visual attribute query region474. If the user adds or modifies the visual attribute query, the user interface codes330may direct the microprocessor102to search the visual attribute table160for one or more visual attribute entries161(shown inFIG.6) which match the visual attribute query as added or modified by the user. For example, the user may delete the visual attribute query480(shown inFIG.17) pre-populated in the visual attribute query region474and may type in “red”, “plaid” or “R:255 B:94 G:120” instead. The microprocessor102may search for, and retrieve, visual attribute entries161which match the added or modified visual attribute query from the visual attribute table160. Such visual attribute entries161may store, in the definition field164or in the description field166, text matter which is identical to, similar to, or synonymous with, the added or modified visual attribute query. For example, if the visual attribute query entered by the user in the visual attribute query region474is “red”, the microprocessor102may retrieve all visual attribute entries161which store “red” in the description field166or which store red, blue and green values in the definition field164that result in a color which generally corresponds to common definitions of “red”.

Referring generally to the navigation region366(labelled inFIG.14), when the user selects the upload image button404, the user interface codes330may direct the user device114to display sect image page is shown generally at500inFIGS.18and19. The select image page500allow a user to search for, capture and select an image that the user may wish to upload to the item recommendation server100and generate visual attributes from. When a user selects the upload image button404, the microprocessor102may communicate with the user device114to access and retrieve photos and other images from a storage memory (not shown) of the user device114(“album mode”, embodiment shown inFIG.18) or to access and command cameras (not shown) of the user device114to capture new photos and new images (“camera mode”, embodiment shown inFIG.19). In certain embodiments, the user interface codes330may direct the user device114to display the album mode when the user initially selects the upload image button404by default; alternatively, the item recommendation server100may have a record of the preferred mode of the user, and the user interface codes330may direct the user device114to display the preferred mode when the user initially selects the upload image button404.

Referring toFIG.18, in the embodiment of the album mode shown, the select image page500includes a mode selection region504and an album display region506. The mode selection region504allows the user to switch between the album mode and the camera mode and includes an album mode button510and a camera mode button512. When a user selects the album mode button510, the user interface codes330direct the user device114to display the album mode of the select image page500(shown inFIG.18). When the user selects the camera mode button512, the user interface codes330direct the user device114to display the camera mode of the select image page500(shown inFIG.19).

The album display region506may display a plurality of images and photos retrieved by the microprocessor102from the storage memory of the user device114. In the embodiment shown, the album display region506can be vertically scrolled to view the retrieved images and photos, and more specifically includes a first column514displaying photo representations516a,516band516c, and a second column515displaying photo representations516d,516eand516f. The first and second columns514and515may be scrolled simultaneously or may be independently scrollable. In other embodiments, the album display region506may be horizontally scrolled or may have a page flip format.

The user can select one of the retrieved photos and images by selecting the photo representation516a-516f. When the user selects a photo representation516b, the user interface codes330may modify the selected representation516band direct the user device114to display a modified representation516b′. For example, non-selected representations516a,516c,516d,516e, and516fmay be displayed with rectangular outlines; in other embodiments, the non-selected photo representations516a-516fmay be displayed with circular outline, a square outline, etc. Selection of one of the representations516bmay cause the user interface codes330to direct the user device114to display the modified photo representation516b′, wherein the modified photo representation516b′ is a rectangular outline with a folded bottom-right corner. In other embodiments, different portions of the outlines of the representations may be folded when the representation is selected, such as the entire bottom half of the outline or different corners of the outline, such as the top-left, top-right or bottom-left corners for example. In other embodiments, the user interface codes330may direct the user device114to modify the selected representation in an additional or an alternative manner, such as coloring the outline of the selected representation with a specific or a random color for example. A user may re-select a modified photo representation516b′ to de-select a photo, and the user interface codes330may direct the user device114to re-display the unmodified photo representation516bin response to user re-selection.

When the user has selected at least one retrieved photo or image, the user interface codes330may also direct the user device114to display a “Next” button (not shown). If the user further selects the “Next” button, the user interface codes330may then direct the user device114to display a confirm image page530described below in connection withFIG.20.

Referring now toFIG.19, in the embodiment of the camera mode shown, the select image page500includes the mode selection region504and a camera region522. When the user interface codes330directs the user device114to display the camera mode of the select image page500shown inFIG.19, the microprocessor102may communicate with at least one camera of the user device114to display images acquired by the at least one camera and to control functionality of the at least one camera to capture new images and photos using the at least one camera. For example, in the embodiment shown, the camera region522may display, in real-time, the images acquired by the at least one camera of the user device114. The camera region522may also include a switch camera button524and a capture image button526. The switch camera button524may allow a user to switch between different cameras of the user device114. For example, the user device114may include a front facing camera and a rear facing camera (not shown); selecting the switch camera button524toggle between displaying images acquired by the front facing camera and the rear facing camera in the camera region522. The capture image button526allows a user to capture an image acquired by the at least one camera and displayed in the camera region522as a photo. When the user selects the capture image button526, the user interface codes330may then direct the user device114to display the confirm image page530(shown inFIG.20).

Referring now toFIG.20, the confirm image page530may allow a user to confirm their selection or capture of an image and to perform minor edits to the image, such as to change zoom or dimension of the image. With respect to the latter functionality, the user interface codes330may display the images of image posts (such as images384,394(shown inFIG.14),432(shown inFIG.15),450(shown inFIG.16), and492(shown inFIG.17)) at a set dimension to ensure that both the images and visual attribute arrays (such as visual attribute arrays386,396(shown inFIG.14),434(shown inFIG.15),452(shown inFIG.16), and492(shown inFIG.17)) of the image posts are be simultaneously displayed on the display of the user device114. In the embodiment shown inFIG.20, the confirm image page530includes a confirm selection region532. The confirm selection region532includes an image534, a use button536and a retry button538.

The image534may display the image selected by the user using the album mode of the select image page500shown inFIG.18or the captured by the user using the camera mode of the select image page500shown inFIG.19. The user interface codes330may require the user to crop the selected or captured image to the set dimension for images noted above to enable subsequent display in image posts, and may also allow the user to move the image534to select a specific region of the image534or to modify zoom of the image534. When the user selects the use button536, the user interface codes330may transmit the image534to process image codes550stored in the program memory108(shown inFIG.1) in a process image request. When the user selects the retry button538, the user interface codes330may direct the user device114to re-display the select image page500to enable the user to select a different image or photo using the album mode (shown inFIG.18) or capture a different new image or photo using the camera mode (shown inFIG.19).

The process image codes550generally includes blocks of code for directing the microprocessor102to generate and add an instance of the image entry141(shown inFIG.4) to the image table140, the new instance of the image entry141representing the image uploaded by the user using the select image page500and the confirm image page530. The process image codes550also include blocks of code for processing the image uploaded by the user to associate the new instance of image entry141with new or existing visual attribute entries161(shown onFIG.6). An illustrative embodiment of the process image codes550is shown inFIG.21. In the embodiment shown, the process image codes550begin at block552, which include code for directing the microprocessor102to store an image representation of the image, which may be contained in the process image request, in the representation database124(shown inFIG.1).

The process image codes550then continue to block554, which include code for directing the microprocessor102to add a new instance of the image entry141(shown inFIG.4) to the image table140(shown inFIG.2). This new instance of the image entry141stores an identifier identifying the instance of the user entry131representing the user that uploaded the image in the user identifier field144. This new instance of the image entry141also stores a URI directing to the location in the representation database124where the image representation was initially stored in block552in the image representation path field150.

The process image codes550then continue to block556, which includes codes for directing the microprocessor102to process the image representation request to extract certain visual attributes associated with that image. As described above, “visual attributes” include without limitation colors, patterns, textures, and reflectivity. Block556may thus include code for directing the microprocessor102to generate visual attributes by extracting color of the image, patterns included in the image, textures included in the image, and reflectivity of the image. In different embodiments, block556may include different blocks of code for generating different visual attributes from the image representation in different manners.

For example, visual attributes which are colors of the image may be generated from the image representation using K-means clustering, median-cut clustering, or binning via color histogram. In one embodiment, where the visual attributes to be extracted from the image representation are colors defined by RBG values, the codes of block556may extract the dominant colors from the image representation by plotting each pixel of the image representation in three-dimensional pixel space, wherein the x-axis are the red pixel values, the y-axis are the green pixel values, and the z-axis are the blue pixel values. Block556may then specify six clusters (k=6); in other embodiments, block556may specify different numbers of clusters. Block556may then assign each pixel of the image representation to a specific cluster, depending on which centroid pixel of a cluster is the minimum distance away from the pixel to be assigned. As more pixels are assigned to a particular cluster, the centroid pixel of that cluster changes to correspond to the mean of the pixels in the cluster. After each pixel extracted from the image representation has been assigned to a cluster, the centroid pixel of the six clusters may each be designated as the visual attributes generated based at least in part on the image representation, and block556may direct the microprocessor102to extract the RBG value of the corresponding six centroid pixels of the six clusters as the value or definition of the generated visual attributes.

Alternatively, visual attributes which are patterns or textures may be extracted from the image using manual cropping or using a pattern/texture classification model. For example, the pattern/texture classification model may be trained on pixel matrices of a plurality of training visual attribute representations which are previously labelled with known patterns and textures (such as “floral”, “plaid” or “stripes” for example) and may be adapted to clustering together different training visual attribute representations associated with similar labels. For example, visual attribute representations which correspond to a “floral” pattern may be clustered together in a first cluster, while visual attribute representations which correspond to a “stripe” pattern may then be clustered together in a second cluster. The pattern/texture classification model may be trained to increase the distance between clusters associated with different patterns or textures by, for example, weighing different pixels of the pixel matrices differently. After training, the pattern/texture classification model may be capable of outputting a pattern label or a texture label based on an input of the pixel matrix of a visual attribute representation, and may be able to predict the pattern label or the texture label based on the pixel matrix of a particular visual attribute representation. The pattern label or the texture label associated with the visual attribute representation by the pattern/texture may be defined as the visual attribute generated based at least in part on the image representation. The codes of block556may thus include code for extracting a portion of the image representation as a visual attribute representation, generating a pixel matrix thereof, and inputting the generated pixel matrix into the pattern/texture classification model. The codes of block556may then receive the text of the pattern label or the texture label from the pattern/texture classification model as the value or definition of the generated visual attribute. In other embodiments, the pixel matrix extracted from the visual attribute representation may be defined as the value or definition of generated visual attribute.

The process image codes550then continue to block558, which include code for directing the microprocessor102to search the visual attribute table160(shown inFIG.2) to determine whether it already stores instances of the visual attribute entry161(shown inFIG.6) representing each visual attribute of the generated visual attributes. In some embodiments, the microprocessor102may search for visual attribute entries161which store a value or definition in the definition field164which matches the generated value or definition of each visual attribute. For example, the codes of block558may direct the microprocessor102to search for, and retrieve, visual attribute entries161which store the extracted RBG values, the extracted pattern labels or texture labels, or the extracted pixel matrices in the definition field164; and/or visual attribute entries161which store a description in the description field166which matches the value or other definition of each generated visual attribute (such as a stored description corresponding to the extracted RBG values, the extracted pattern labels, or the extracted texture labels).

If the microprocessor102determines, at block558, that one of the generated visual attributes does not have a corresponding existing visual attribute entry161in the visual attribute table160, the process image codes550then continue to block560which includes codes for directing the microprocessor102to add a new instance of the visual attribute entry161for such an generated visual attribute. The new instance of the visual attribute entry161stores a value or a definition which corresponds to the value or other definition of the generated visual attribute in the definition field164, and may further store a portion of the value or the definition also in the description field166. Block560may also include code for causing the microprocessor102to store a representation, such as image data or other data, of the generated visual attribute in the representation database124(shown inFIG.1) and a URI identifying the storage location of the visual attribute representation in the representation database124in the visual attribute representation path field168. For example, where the generated visual attribute is a pattern or a texture, the stored visual attribute representation may be a cropped image of that pattern or texture; and where the generated visual attribute is a color, the stored visual attribute representation may be an image filled with that color. Block560may also include code for causing the microprocessor102to automatically generate an appropriate description to populate the description field166. The generated description may be based at least in part on the values or other definition of the generated visual attribute in the definition field164. For example, if the definition field164stored particular ranges of RBG values generally corresponding to the color blue, the description field166may be populated with different human readable descriptions of the color blue, such as “baby blue”, “navy blue” or “periwinkle”; if the definition field164stored pixel matrices of particular patterns and textures, the description field166may be populated with human-readable labels of patterns and textures which are generated by inputting the pixel matrix into pattern/texture classification model described above. The process image codes550then continue at block562as described below. If the microprocessor102determines at block558that the visual attribute table160does include an existing visual attribute entry161representing the generated visual attribute, process image codes550then continue at block559, which include code for directing the microprocessor102to retrieve the existing visual attribute entry161from the visual attribute table160. The process image codes550may cycle through block558and then either block560(add new instance of visual attribute entry) or block559(retrieve existing instance of visual attribute entry) for each visual attribute generated by the microprocessor102based at least in part on the image at block556. The process image codes550then continue at block562, which includes codes for directing the microprocessor102to display the image with the visual attributes extracted from the image. In certain embodiments, the user interface codes330may then direct the user device114to display an upload image page shown generally at580inFIG.22.

Referring nowFIG.22, the upload image page580may generally allow a user to: (1) confirm that certain visual attributes should be associated with an image, (2) to modify which visual attribute are associated with the image, and (3) to upload the image as an image post. In the embodiment shown, the upload image page580includes an upload region582. The upload region582includes an image584, a visual attribute array585including plurality of visual attribute representations586a-586f, a description field588, and an upload button590.

The image584corresponds to the image uploaded by the user using the select image page500(shown inFIGS.18and19) and the confirm image page530(shown inFIG.20), and represented by the new instance of the image entry141added at block554. The image584may display the image stored in the representation database124directed to by the URI in the image representation path field150of the new instance of the image entry141. The plurality of representations586a-586fof visual attributes may represent the plurality of visual attributes automatically generated by the microprocessor102from the image584at block556and may display visual attribute representations stored in the representation database124directed to by the URIs in the visual attribute representation path field168of the visual attribute entries161identified at block558and/or added at block560(shown inFIG.21). The user may enter a text string in the description field588of the upload region582describing or otherwise captioning the image584.

Referring back toFIG.21, the process image codes550continue at block564, which includes codes for directing the microprocessor102to determine whether the user modifies or overrides any of the visual attributes automatically generated from the image representation at block556. For example, referring now toFIGS.22and23, the user may modify or override the automatically generated visual attributes by selecting the visual attribute representation586a-586fof the visual attribute that the user wishes to modify or override, such as representation586cin the embodiment shown, which may cause the user interface codes330to direct the user device114to display the upload region582(shown inFIG.22) as a modified upload region582′ (shown inFIG.23).

The modified upload region582′ includes the image584, the visual attribute array585including the plurality of visual attribute representations586a-586f, wherein the selected visual attribute representation is a modified visual attribute representation586c′, a representation of a sampled visual attribute592, a confirm button594and a cancel button596. The modified visual attribute representation586c′ may include an indication that the visual attribute represented by the visual attribute representation586cis being modified or overridden by the user. In the embodiment shown, the modified representation586c′ includes a dropper icon591overlaying the representation586c; in other embodiments, the modified representation586c′ may include additional or alternative indications. The user may modify or override the visual attribute represented by the modified representation586c′ by selecting a portion of the image584, and the user interface codes330may direct the microprocessor102to extract the visual attribute represented by the selected portion of the image584in a manner similar to that described above in connection with block556(shown inFIG.21). The user interface codes330may further direct the user device114to display a representation of that visual attribute as the sampled visual attribute representation592. For example, if the selected portion of the image584corresponds to a pixel having a color with a particular RBG value, the user interface codes330may display the color with that RBG value as the sampled visual attribute representation592; if the selected portion of the image584corresponds to a pattern or texture, the user interface codes330may crop the selected portion and display the selected portion as the sampled visual attribute representation592.

When the user is satisfied with the visual attribute displayed by the sampled visual attribute representation592, the user may select the confirm button594. Referring back toFIG.21, selecting the confirm button594may direct the microprocessor102to determine at block564that the user has modified or overridden at least one of the visual attributes initially extracted at block556, and direct the process image codes550to return to block558and proceed from block558as described above. For example, block558directs the microprocessor102to determine whether the visual attribute table160includes a visual attribute entry161(shown inFIG.6) representing the visual attribute selected by the user and displayed in the sampled visual attribute representation592. The process image codes550continue from block556to either retrieve an existing instance of the visual attribute entry161at block559or add a new instance of the visual attribute entry161at block560as described above, before proceeding again to block562to display the image with the visual attributes as modified or overridden. For example, block562may also include code for re-displaying the upload image page580(shown inFIG.22) with the visual attribute selected by the user from the image584using the modified upload image page580′ (shown inFIG.23) as the visual attribute representation586c.

If the user decides against modifying or overriding a particular visual attribute after selecting a particular visual attribute representation586a-586f, the user may select the cancel button596. Selecting the cancel button596may cause the user interface codes330to direct the user device114to re-display the upload image page580(shown inFIG.22) with no modification of the visual attributes associated with the image584.

If the user is satisfied with the image584, the visual attributes associated with the image584represented by the plurality of visual attribute representations586a-586f, and description entered in the description field588, the user may select the upload button590. Referring back toFIG.21, selecting the upload button590may cause the microprocessor102to determine at block564that the user has not modified or overridden any of the visual attribute entries161(shown inFIG.6) retrieved at block559or added at block556. The process image codes550then continue at optional block565, which include code for directing the microprocessor102to update the image entry141(shown inFIG.4) added at block554by storing the text string entered in the description field588of the upload image page580(shown inFIG.22) in the description field148, before continuing to block566. If the user did not enter any text in the description field588, then the process image codes550may continue directly to block566.

Block566includes codes for associating the visual attribute entries161(shown inFIG.6) retrieved at block559or added at block560(and displayed on the upload image page580shown inFIG.21) with the image entry141(shown inFIG.4) added at block554to persistently associate a particular image uploaded by a user with one or more visual attributes.

In the embodiment shown, block566may direct the microprocessor102to store visual attribute identifiers from the identifier field162of the newly added or existing instances of the visual attribute entries161in the visual attribute identifier field146of the new instance of the image entry141. The process image codes550then end.

As a result of the process image codes550, the image initially uploaded by the user via the select image page500(shown inFIGS.18and19) and the confirm image page530(shown inFIG.20) is stored in the image table140as an image entry141and is persistently associated with one or more visual attributes. The user interface codes330may display the image and the associated visual attributes as an image post on various pages of the mobile application, such as on the home page360(shown inFIG.14) and on the user profile page410associated with the user (shown inFIG.15). User selection of the image on such pages may also direct the user interface codes330to display the image page440(shown inFIG.16).

Referring back toFIG.1, the program memory108further stores process item codes600, which may be executed by the item recommendation server100intermittently, such as when the item recommendation server100receives a process item message from a vendor server116, or at set intervals, such as when the item recommendation server100retrieves the process item message from a vendor website or the vendor server. Each process item message may correspond to one or more items offered for sale by a vendor operating the vendor server116, such as on a vendor website hosted by the vendor server116for example. The process item message may include at least one item representation of the item, a description of the item from the vendor, a price for the item, different options associated with the item (such as size of the item or different lengths of the item for example) and a link to the vendor webpage for purchasing the item or other information which facilitates direct communication between the item recommendation server100and the payment processor117associated with the vendor for direct purchase of the item through the item recommendation server100.

The process item codes600generally include blocks of code for generating and adding an instance of the item entry171(shown inFIG.7) to the item table170(shown inFIG.2), the new instance of the item entry171representing an item offered for sale by a vendor and included in the process item message. The process item codes600also include blocks of code for processing an item representation (such as the at least one item representation in the process item message) to associate the new instance of the item entry171with new or existing visual attribute entries161(shown onFIG.6) and with new or existing taxonomy entries221(shown inFIG.9).

An illustrative embodiment of the process item codes600is shown inFIG.24. In the embodiment shown, the process item codes600begin at block602, which include code for directing the microprocessor102to store the at least one item representation contained in the process item message in the representation database124(shown inFIG.1).

The process item codes600then continue at block604, which include code for directing the microprocessor102to add a new instance of the item entry171(shown inFIG.7) to the item table170(shown inFIG.2). This new instance of the item entry171may store, in the vendor identifier field173, a vendor identifier identifying an instance of the vendor entry231representing the vendor operating the vendor server116from which the item recommendation server100received the process item message and, in the item representation path field177, URI(s) directing to location(s) in the representation database124where the at least one item representation was initially stored in block602. This new instance of the item entry171may also store information included in the process item message, such as the description of the item from the vendor in the vendor description field178, price of the item in the price field179, different sizes of the item in the options field180and the link to the vendor webpage or the other information for facilitating purchase of the item in the purchase path field181.

The process item codes600then continue to block606, which include code for directing the microprocessor102to process the information in the item message (such as the at least one image representation) to generate visual attributes associated with the item. For example, the block606may include code for directing the microprocessor102to extract colors, patterns, textures and reflectivity of the item from the at least one item representation as visual attributes which are similar to the codes of block556of the process image codes550(shown inFIG.21). For example, block606may also extract different values or definitions of generated visual attributes from the at least one item representation using the K-means cluster and the pattern/texture classification model.

The process item codes600then continue at block608, which include code for directing the microprocessor102to search the visual attribute table160to determine whether it already stores instances of the visual attribute entry161representing a visual attribute generated at block606, and may be similar to the codes of block558of the process image codes550(shown inFIG.21). For example, block608may also direct the microprocessor102to search for visual attribute entries161which: store a definition in the definition field164which matches the value or other definition of the generated visual attribute; or store a description in the description field166which matches the value or other definition of the generated visual attribute.

If the microprocessor102determines, at block608, that one of the generated visual attributes does not have a corresponding existing visual attribute entry161in the visual attribute table160, the process item codes600then continue to block610, which include code for directing the microprocessor102to add a new instance of the visual attribute entry161for such an generated visual attribute and may be similar to the codes of block560of the process image codes550(shown inFIG.21). For example, the new instance of the visual attribute entry161may store a value or a definition which corresponds to the value or other definition of the generated visual attribute in the definition field164and/or the description field166, a representation of the generated visual attribute in the representation database124(shown inFIG.1), and a URI identifying the storage location of a visual attribute representation of the generated visual attribute in the visual attribute representation path field168. The process item codes600then continue at block612. If the microprocessor102determines, at block608, that the visual attribute table160does include a visual attribute entry161representing the visual attribute, the process item codes600then continue at block609, which include code for directing the microprocessor102to retrieve the existing visual attribute entry161from the visual attribute table160. The process item codes600may cycle through block608and then either block610(add new instance of visual attribute entry) or block609(retrieve existing instance of visual attribute entry) for each visual attribute generated by the microprocessor102from the process item message (such as the at least one item representation) at block606.

The process item codes600then continue to block612, which may include code for directing the microprocessor102to associate the visual attribute entries161(shown inFIG.6) retrieved at block609or added at block610with the new instance of the item entry171(shown inFIG.7) added at block604to persistently associate a particular item received or retrieved from a vendor with one or more visual attributes. The codes of block612may be similar to the codes of block566of the process image codes550(shown inFIG.21). For example, block612may include code for directing the microprocessor102to store the visual attribute identifiers from the identifier fields162of the retrieved or added visual attribute entries161in the visual attribute identifier field175of the new instance of the item entry171.

The process item codes600then continue to block614, which may include code for directing the microprocessor102to process the information in the process item message to classify or categorize the item into a particular macro-item category and/or into a particular micro-item category. In some embodiments, the microprocessor102may analyze the description of the item from the vendor or the at least one item representation received in the process item message. For example, if a description of an item from the vendor included “Cardigan; Embroidered Cashmere; Camel; P62535 K48069 13E367”, then block614may include code for directing the microprocessor102extract text from the description and label the item with “top”, “sweater”, “cardigan” and “cashmere” category labels; alternatively, if the description of an item from the vendor included “Necklace; Metal, Glass Pearls, Imitation Pearls & Resin; Gold, Blue, Pearly White; AB2394 Y47901 Z8798” then block614may include code for directing the microprocessor102to extract text from the description and label the item with “jewelry”, “necklace”, “resin”, “pearl” and “glass” category labels. Alternatively or additionally, block614may include code for implementing an item category classification model that automatically classifies an item into a macro-item category and/or a micro-item category based on a representation of the item. For example, the classification model may be trained on pixel matrices of a plurality of training item representations which are previously labelled with known item category labels, such as “tops”, “pants”, “sweaters”, or “shoes” and may be adapted to cluster together different training item representations associated with similar labels using the pixel matrix. For example, item representations which correspond to the “tops” category may be clustered together in a first cluster, while item representations which correspond to a “pants” category may then be clustered together in a second cluster. The item category classification model may be trained to increase the distance between clusters associated with different item categories by, for example, weighing different pixels of the pixel matrices differently or by considering additional information, such as descriptions received from the vendor. After training, the item category classification model may be capable of outputting at least one item category label based on an input of the pixel matrix of an item representation, and may be able to predict the item category label based on the pixel matrix of a particular item representation of. The item category label may be defined as the item category generated based at least in part on the item representation. The microprocessor102may extract one or more category labels from the information in the process item message.

The process item codes600then continue to block616, which include code for directing the microprocessor102to search the taxonomy table220to determine whether it stores instances of the taxonomy entries221(shown inFIG.9) representing each item category label of the one or more extracted item category labels. For example, the microprocessor102may search for, and retrieve, taxonomy entries221which store a text string in the macro-item category field224and/or the micro-item category field226which matches the text string of the item category label extracted from the information in the process item message at block614.

If the microprocessor102determines, at block616, that one of the one or more extracted item category labels does not have a corresponding existing taxonomy entry221in the taxonomy table220, the process item codes600then continue to block620, which include code for directing the microprocessor102to add a new instance of the taxonomy entry221for such an extracted item category label. This new instance of the taxonomy entry221may store the extracted item category label in the macro-item category field224and/or the micro-item category field226. In certain embodiments, block620may also include code for directing the microprocessor102to automatically generate an item category label for the macro-item category field224if the extracted item category label is stored in the micro-item category field226and vice versa. For example, if “lucite” is the extracted item category label in the micro-item category field226, the microprocessor102may store “jewelry” or “acrylic” in the macro-item category field224. Block620may include code for directing the microprocessor102to automatically generate corresponding item category labels utilizing an item ontology. The process item codes600then continue at block622. If the microprocessor102determines, at block616, that the taxonomy table220does include an existing taxonomy entry221representing the extracted item category label, the process item codes600proceed to block618, which include code for directing the microprocessor102to retrieve the existing taxonomy entry221. The process item codes600may cycle through block616and then either block620(add new instance of taxonomy entry) or block618(retrieve existing instance of visual attribute entry) for each item category label extracted by the microprocessor102at block614. The process item codes600then continue to block622.

Block622includes codes for directing the microprocessor102to associate the taxonomy entries221(shown inFIG.9) retrieved at block618or added at block620with the new instance of the item entry171(shown inFIG.7) added at block604to persistently associate a particular item received or retrieved from a vendor with one or more item categories. In the embodiment shown, block622may include code for directing the microprocessor102to store the taxonomy identifiers from the identifier field222of the newly added or existing instances of the taxonomy entries221in the taxonomy identifier field174of the new instance of the item entry171. The process item codes600then end.

As a result of the process item codes600, the item received or retrieved from a vendor in the process item message is stored in the item table170(shown inFIG.2) as an item entry171and is persistently associated with one or more visual attributes and one or more taxonomies. As described above, the process image codes550associated an image uploaded by a user persistently with one or more visual attributes. The combination of the process item codes600and the process image codes550thus associate both items and images with visual attributes, which enables item recommendation server100to retrieve both items and images based on the visual attributes which are associated with the item and the image. A user whom identifies an image having an attractive or desirable color scheme or patterns (visual attributes), may search for items that have similar colors or similar patterns and vice versa. More particularly, the item recommendation server100allow the user to search for items which are associated with at least one visual attribute which match at least one visual attribute associated with an image.

In the embodiment shown, image posts (such as the image posts380,390(shown inFIG.14),430(shown inFIG.15),444(shown inFIG.16), and490(shown inFIG.17) for example) include a shop image button (such as shop image buttons388,436,454and498for example). When a user selects such shop image buttons, the user interface codes330may direct the user device114to display the shop image page630shown inFIG.25.

In other embodiments, as will be described below in connection withFIGS.33and34, the user may be directed to different embodiments of the shop image page630by selecting the palette selection button370(labelled inFIG.14) of the header region362, which may direct the user to select visual attributes associated with a palette. The palettes may be selected by a vendor or a host of the item recommendation server100or other users of the item recommendation server100or the current user at a previous point in time. The palette may be based on at least one image representation stored in the representation database124. In other embodiments (not shown), the item recommendation server100may enable a user to search for items using visual attributes which are associated with images which do not correspond to images uploaded by users via the process image codes550. For example, a user may select visual attributes from a representation of all possible colors, such as from an image of a color wheel with all possible RBG or hexadecimal values, or from a representation of a plurality of possible colors in a color hue, such as from an image of different shades of red or a different shades of orange. The user may also select visual attributes from a representation of a plurality of textures and/or patterns, such as from a list of different textures and/or patterns. Such representations of visual attributes may be at least one image representation stored in the representation database124or may be an image generated by the microprocessor102automatically based on all or a portion of the visual attribute entries161stored in the visual attribute table160.

Referring now toFIG.25, in the embodiment shown, the shop image page630includes a shop image region632. The shop image region632displays an image634, a visual attribute array635including a plurality of visual attribute representations636a-636fassociated with the image634, a query field638, and a shop button640.

The image634corresponds to the image (such as images384,394(shown inFIG.14),432(shown inFIG.15),450(shown inFIG.16), and492(shown inFIG.17)) of an image post displayed on the home page360, the user profile page410, the image page440or the visual attribute search page470and selected by the user. The user interface codes330may display, as the image634, the image representation stored in the representation database124(shown inFIG.2) directed to by a URI in the image representation path field150of the image entry141(the “selected image entry”, shown inFIG.4) corresponding to the image post selected by the user.

The visual attribute array635includes the visual attributes which are associated with the image634. In certain embodiments, the user interface codes330may display, as the plurality of visual attribute representations636a-636f, the representations stored in the representation database124(shown inFIG.2) directed to by a URI in the visual attribute representation path field168of the visual attribute entries161(shown inFIG.6) identified in the visual attribute identifier field146of the selected image entry141corresponding to the image post selected by the user. In the embodiment shown, the visual attribute array635includes six visual attribute representations636a-636f, indicating that six instances of visual attribute entries161are identified by the selected image entry141. In other embodiments, the visual attribute array635may include a greater or a fewer number of visual attribute representations, the number of representations indicating the number of visual attributes that are associated with the selected image. In embodiments where the number of visual attribute representations is greater, the visual attribute array635may include two rows of visual attribute representations or may be horizontally or vertically scrollable by the user.

Each of the plurality of visual attribute representations636a-636fmay be selectable, and user selection of one or more of the visual attribute representations636a-636fmay indicate that the user is interested in finding items which are associated with visual attributes that match or correspond to the selected visual attribute. The user may not select any of the visual attribute representations636a-636f, single select one of the visual attribute representations636a-636f, double select one of the visual attribute representations636a-636for select more than one of the visual attribute representations636a-636f. In some embodiments, the user may only select a single one of the visual attribute representations636a-636f.

When the user single selects a particular visual attribute representation636a-636f, the user interface codes330may direct the user device114to display a modification of the selected visual attribute representation. For example, referring toFIG.26A, in the embodiment shown, non-selected visual attribute representations636a,636b,636c,636e, and636fhave rectangular outlines, but the single-selected visual attribute representation636dis displayed as a modified selected visual attribute representation636d′ having a rectangular outline with a folded bottom-right corner. In other embodiments, the non-selected visual attribute representations636a-636fmay be displayed with circular outline, a square outline, etc. Further, different portions of the outlines of the visual attribute representations may be folded when the visual attribute representation is single-selected, such as the entire bottom half of the outline or a different corner of the outline for example. In some other embodiments, the user interface codes330may modify the single-selected visual attribute representation in an additional or an alternative manner than folding the outline of the visual attribute representation. For example, the outline of the visual attribute representation may be colored with a specific or random color. When the user double selects a particular visual attribute representation, the user interface codes330may direct the user device114to display a modification of the double-selected visual attribute representation, as well as a modification of every other non-selected visual attribute representation in the visual attribute array635. For example, referring toFIG.26B, in the embodiment shown, the double-selected visual attribute representation636dis displayed as the modified selected visual attribute representation636d′ having a rectangular outline with a folded bottom-right corner and the non-selected representations636a,636b,636c,636eand636fare all displayed as modified non-selected representations636a′,636b′,636c′,636e′ and636f′ having rectangular outline with a grayed-out color. In other embodiments, the outlines of the representations may be different, different portions of the outlines of the representations may be folded when the representation is double-selected, and the non-selected representations may be modified in alternative or additional manners.

Referring back toFIG.25, the query field638is operable to receive a text query from the user entered via the user device114. For example, the user may enter the item category that the user is interested in, such as “dress”, “sweater” or “jewelry”, as the text query. The text query may correspond to one or more macro-item categories or micro-item categories stored in, respectively, the macro-item category field224and the micro-item category field226of one or more instances of the taxonomy entry221(shown inFIG.9) stored in the taxonomy table220. In other embodiments, the user may enter a name of a clothing brand, a name of a vendor, a clothing style, or any other text string, as the text query. For example, the text query may correspond to one or more descriptions stored in the description field234of one or more instances of the vendor entry231(shown inFIG.8). The user may not enter any text query in the query field638, may enter one text query in the query field638, or may enter more than one text query in the query field638.

When the user selects the shop button640, the user interface codes330may transmit information from the shop image page630, including the image634, the visual attribute array635, any selection of the visual attribute representations636a-636fand any text query entered in the query field638, to the recommend items codes650(shown inFIG.1) stored in the program memory108of the item recommendation server100in a recommend items request.

The recommend items codes650generally include code for retrieving a plurality of items associated with visual attributes that match one or more of the visual attributes associated with the image (or palette) that the user selected to shop from. The recommend items codes650also include code for (1) classifying the retrieved plurality of items into a first set and at least one separate second set, wherein items in the first set and the second set are mutually exclusive, and then (2) simultaneously displaying the first set and the second set proximate to each other. In certain embodiments, the items in the second set may be complementary to the items in the first set or may be complementary to an entered text query. Displaying different and/or complementary items proximate to each other may encourage a user to purchase matching items and may further encourage a user to provide combinations of items to the item recommendation server100. The recommend items codes650may also include code for ordering the items within the first set and within the second set and displaying the first and second set items in the specific orders.

An illustrative embodiment of the recommend items codes650is shown inFIGS.27A-27C. In the embodiment shown, the recommend items codes650begin at block652, which include code for directing the microprocessor102to identify and retrieve a plurality of items associated with a visual attribute which match at least one visual attribute associated with the image (or palette) that the user initially selected to shop from. For example, block652may include code for directing the microprocessor102to identify and retrieve a plurality of item entries171(shown inFIG.7) identifying a visual attribute entry161(shown inFIG.6) in the visual attribute identifier fields175which matches with least one of the visual attribute entries161identified by the selected image entry141(in the visual attribute identifier field146for example, shown inFIG.4) representing the image the user selected to shop from (such as image634for example) or identified by a selected palette entry191(in the visual attribute identifier field193for example, shown inFIG.5) representing a palette a user selected to shop from. In different embodiments, block652may include different codes for directing the microprocessor102to determine whether a visual attribute identified by an item entry171“matches” a visual attribute identified by an image entry141. For example, block652may use different codes depending on whether the visual attribute represented by the visual attribute entries161are colors or patterns/textures.

If the two visual attributes are colors defined by pixel values stored in the respective definition fields164of corresponding visual attribute entries161(shown inFIG.6), block652may include code for directing the microprocessor102to determine whether the two visual attributes match by determining whether a distance between the two pixel values exceed a threshold. For example, where the visual attributes are defined by RGB values, block652may include code for directing the microprocessor102to determine the distance in 3-dimensional RGB space between a first color corresponding to a first visual attribute entry161and a second color corresponding to a second visual attribute entry161using formula (1) below.

wherein:Rc1, Gc1, and Bc1represent R, G and B pixel values stored in the definition field164of the first visual attribute entry161representing the first color, andRc2, Gc2, and Bc2represent the R, G and B values stored in the definition field164of the second visual attribute entry161representing the second color.

The smaller the value of D, the closer the color match. A perfect color match between the first color and the second color occurs if D=0. The threshold for determining that the first color and the second color match may be set at D<30 for example.

If the two visual attribute are patterns or textures, block652may include code for directing the microprocessor102to determine matches between a first pattern corresponding to a first visual attribute entry161and a second pattern corresponding to a second visual attribute entry161by finding matches of descriptions stored in the description fields166of the first and second visual attribute entries161. For example, the microprocessor102may determine that the first and second patterns match if both the corresponding first and the second visual attribute entries161store the term “floral” or “plaid” in the description field166. As described in connection with block556of the process image codes550and block606of the process item codes600, the description field166of the visual attribute entries161may be automatically populated using a pattern label or texture label outputted by the pattern/texture classification model, and block652may thus determine that two patterns match if the pattern/texture classification model outputted the same pattern label based the visual attribute representations of the two patterns. In other embodiments, the microprocessor102may determine whether the first and second patterns match by determining whether a distance between the first pattern and the second pattern exceed a threshold. For example, block652may include code for directing the microprocessor102to extract a pixel matrix of the visual attribute representation of the first pattern and a pixel matrix of the visual attribute representation of the second pattern (directed to by a URI in the visual attribute representation path168of the first and second visual attribute entries161) and then determine the distance between the two pixel matrices utilizing a model which calculates an edit distance or a graph edit distance (such as the Wagner-Fischer algorithm, the Jaro-Winkler distance algorithm or the Hamming distance calculator for example). The first and second patterns may match if the distance is below a threshold and may not match if the distance is above the threshold.

The recommend items codes650then continue at block654, which includes codes for directing the microprocessor102to determine whether the user selected any of the visual attribute representations636a-636fusing the shop image page630(shown onFIG.25), and if the user did select a visual attribute representation636a-636f, whether the selection was a single selection or a double selection. Any such visual attribute selection may be included in the recommend items request.

Referring toFIG.27A, if the microprocessor102determines at block654that the user provided a single selection of one of the visual attribute representations636a-636f, the recommend items codes650continue at block656, which include code for directing the microprocessor102to determine whether the user entered any text query (such as in the query field638of the shop image page630shown onFIG.25). Any such text query entered may also be included in the recommend items request.

If the microprocessor102determines at block656that the user entered a text query—such that the user both (a) provided a single selection of a visual attribute representation representing a selected visual attribute and (b) entered a text query—the recommend items codes650then continue at block658, which include code for directing the microprocessor102to identify those item entries171of the item entries171initially retrieved at block652which (1) are associated with visual attributes which match the selected visual attribute and (2) match or correspond to the text query. The microprocessor102classifies the item entries171which meet criteria (1) and (2) in a first set of items or as first set items.

With respect to criteria (1) above, block658may include code for directing the microprocessor102to identify items associated with visual attributes which match the single-selected visual attribute by directing the microprocessor102to identify, from the item entries171initially retrieved at block652(identifying visual attribute entries161which match any visual attribute associated with the image (or palette)), those item entries171which identify visual attribute entries161in the visual attribute identifier field175that match the visual attribute entry161representing the single-selected visual attribute. Block658may determine whether two visual attributes match in a manner similar to block652described above.

With respect to criteria (2) above, in different embodiments, block658may include different codes for enabling the microprocessor102to identify items matching or corresponding to the text query. For example, block658may include code for directing the microprocessor102to identify item entries171storing a text string or other descriptive matter which matches or corresponds to the text query. For example, where the text query is “green” or “wool” for example, item entries171which store “green” or “wool” in either the description field176or the vendor description field178may be identified. Alternatively or additionally, in embodiments where text query corresponds to an item category that the user is interested in purchasing, such as “dress” or “sweater” for example, the text query may match or correspond to certain taxonomy entries221(shown inFIG.9). Block658may include code for directing the microprocessor102to (a) determine at least one taxonomy which matches or corresponds to the entered text query, such as by identifying at least one taxonomy entry221storing a text string matching or corresponding to the entered text query in either the macro-item category field224or the micro-item category field226and then to (b) determine, of those item entries171identifying visual attributes which match the single-selected visual attribute (satisfies criteria (1) above), which also identifies one or more of the taxonomy entries221identified at (a) above in the taxonomy identifier field174.

The recommend items codes650then continue at block660which include code for directing the microprocessor102to identify those item entries171of the item entries171initially retrieved at block652which (1) are associated with a visual attribute matching any visual attribute of the image (or palette) the user initially selected to shop from, (2) are complementary to at least one first set item and/or complementary to the text query, and (3) is not in the first set. The microprocessor102may then classify the item entries171which meet the criteria (1), (2) and (3) above within a second set of items or as second set items. In such embodiments, the first set items includes items associated with the single-selected visual attribute and matching the text query and the second set items includes items associated with at least one visual attribute of the image (or palette) and which is complementary to the first set items and/or the text query.

With respect to criteria (1) and (3) above, in different embodiments, block660may include different codes for enabling the microprocessor102to identify items associated with a visual attribute matching at least one visual attribute of the image and which are not in the first set. For example, block660may direct the microprocessor102to identify items associated with visual attributes which match any of the visual attributes of the image, but which are not in the first set items, by excluding, from the item entries171initially retrieved at block652, those item entries171which were classified within the first set at block658. In other embodiments, block660may direct the microprocessor102to identify items associated with visual attributes which match any visual attribute of the image that is not the single-selected visual attribute and which are not in the first set, by: (a) identifying, from the item entries171initially retrieved at block652, those item entries171identifying visual attributes entries161that match visual attribute entries161representing any visual attribute of the image (or palette) other than the single-selected visual attribute, and (b) exclude from the item entries171identified at (a), those item entries171which were classified within the first set at block658. Block660may determine whether two visual attributes match in a manner similar to block652described above.

With respect to criteria (2) above, in different embodiments, block660may include different codes for enabling the microprocessor102to determine which items are complementary to at least one first set item and/or complementary to the text query. For example, block660may include code for directing the microprocessor102to determine complementary items utilizing a curated determination or a model-based determination, or utilizing a combination of the curated determination and the model-based determination.

In embodiments where block660directs the microprocessor102to utilize the curated determination, the complementarity of different item entries171may be explicitly programmed in the application database122(shown inFIG.2). For example, the microprocessor102may identify complementary second set items by (a) identifying item entries171identified in the complementary item identifier field182(shown inFIG.7) of the item entries171representing the first set items, and/or (b) identifying item entries171which identify, in their complementary item identifier field182, item entries171representing the first set items. Such codes may identify complementary between specific items, and thus complementarity at the item-level (a specific dress is complementary with a specific pair of shoes for example). The specific combination of complementarity between items may be curated by the vendor selling the items or by a host of the item recommendation server100. Alternatively or additionally, block660may identify complementary categories, and thus complementarity at the category-level (dresses are generally complementary with heels for example). In such embodiments, the microprocessor102(a) identify taxonomy entries221identified in the taxonomy identifier fields174of the item entries171representing the first set items, the identified taxonomy entries221corresponding to item categories of the first set items, (b) identify any complementary taxonomy entries221which are identified in the complementary taxonomy identifier fields228(shown inFIG.9) of the taxonomy entries221identified at (a), and (c) identify, as the complementary second set items, item entries171which identify the complementary taxonomy entries221identified at (b) in their taxonomy identifier fields174. In embodiments where the entered text query itself matches or substantially corresponds to certain taxonomy entries221, block660may further include code for directing the microprocessor102to (a) identify taxonomy entries221which matches or substantially corresponds to the text query in a manner similar to block658described above, (b) identify any complementary taxonomy entries221which are identified in the complementary taxonomy identifier field228(shown inFIG.9) of the taxonomy entries221identified at (a), and (c) identify, as the complementary second set items, item entries171which identify the complementary taxonomy entries221identified at (b) in their taxonomy identifier fields174.

Block660may also direct the microprocessor102to identify, as complementary items, item entries171which: (b) identify vendor entries231(shown inFIG.8) in the vendor identifier field173that are also identified by the item entries171representing the first set items (items sold by a same vendor as the first set items); (c) store descriptions in the description field176or in the vendor description field178which match or correspond to the description stored in the description field176or the vendor description field178of the item entries171representing the first set items; or (d) store descriptions in the description field176or in the vendor description field178which match the description store in the macro-item category field224or the micro-item category field226of taxonomy entries221identified in the taxonomy identifier field174of the item entries171corresponding to the first set items.

In embodiments where block660directs the microprocessor102to identify complementary items utilizing the model-based determination, the complementarity of different items may be determined based on a complementarity model which processes historical entries stored in the application database122(shown inFIG.2) to determine complementary items or complementary item categories. The historical entries may be entries stored in the interaction history table250, the combination history table290, and the purchase history table270for example. In such embodiments, the items which are complementary to the first set items may shift over time depending on user interaction with items, user combination of items and user purchase of items. In certain embodiments, the complementarity model may categorize items into clusters based on prior user interaction with items, prior user combination of items and prior user purchase of items which are used as a proxy to indicate that certain items are complementary.

For example, the complementarity model may be trained on the interaction history entries251(shown inFIG.10), wherein different items that a particular user interacts with within a short time frame may be classified as “complementary” items. In this respect, as described in greater detail below in association with a recommend items page shown generally at700inFIG.28, a new instance of the interaction history entry251may be created each time a user interacts with an item representation716a-716cand719a-719cof an item entry171displayed on the recommend items page700. The complementarity model may categorize the item entries171identified in the item identifier field254of a single such interaction history entry251as “complementary” items. Alternatively, the complementarity model may categorize different item entries171identified in respective item identifier fields254of a plurality of such interaction history entries251as “complementary” items if the interaction history entries251identify a same user in the user identifier fields253, a same image (or palette) in the image identifier fields255, and/or store times in the created fields259that are separated by a time gap below an interaction time gap threshold. The interaction time gap threshold may be 6 minutes, 10 minutes, 19 minutes or 60 minutes for example.

Alternatively or additionally, the complementarity model may be trained on the combination history entries291(shown inFIG.11), wherein different items that a particular user combines and adds to the shopping cart may be classified as “complementary” items. For example, as described in greater detail below in association with the recommend items page700(shown inFIG.28), a new instance of the combination history entry291may be created each time a user interacts with at least one item representation716a-716cof at least one item classified within the first set and displayed in a first set region712of the recommend items page700and/or at least one item representation719a-719cof at least one item classified within the second set and displayed in a second set region714of the recommend items page700to add the items from the first item set and/or the second item set to the shopping cart. The complementarity model may categorize item entries171identified in the first set item identifier field294and/or in the second set item identifier field295of such combination history entries291as “complementary” items.

Alternatively or additionally, the complementarity model may be further be trained on the purchase history entries271(shown inFIG.12), wherein items that a particular user purchases at the same time, or within a short time frame of each other, may be classified as “complementary” items. For example, as described in greater detail below in association with a shopping cart page shown generally at780inFIG.35, a new instance of the purchase history entry271may be created each time a user purchases items or an item collection. The complementarity model may categorize item entries171identified in the item identifier field274of a single such purchase history entry271as “complementary” items. Alternatively, the complementarity model may also categorize different item entries171identified in respective item identifier fields274of a plurality of such purchase history entries271as “complementary” items, if the purchase history entries271also identify a same user in the user identifier fields273, a same image (or palette) in the image identifier fields275, and/or store times in the created fields278that are separated by a time gap below a purchase time gap threshold, to the complementarity model as records of the user purchasing “complementary” items. The purchase time gap threshold may be one hour, 11 hours, 23 hours, 48 hours, a week or a month for example.

After the complementarity model is trained on the historical entries, different items may be clustered into groups of item entries171considered to be “complementary” to each other, and such clusters may identify complementarity between specific item entries171, and thus complementarity at the item-level. The complementarity model may then extract the taxonomy entries221identified in the taxonomy identifier field174of item entries171that are grouped into a cluster and to categorize such taxonomies as “complementary” to each other, which allows the complementarity model to identify complementarity at the category-level. Block660may thus include code for identifying items complementary to the first set items and/or complementary to the text query by determining, for example: (a) which “complementary” item entries171are clustered together with the first set items, (b) which taxonomy entries221are clustered together with the taxonomy entry221matching or corresponding to the text query and then “complementary” item entries171identifying such taxonomy entries221in their taxonomy identifier fields173(items which are in the item categories clustered with the text query) and/or (c) which taxonomy entries221are identified in the taxonomy identifier field174of the item entries171representing the first set items and then “complementary” item entries171identifying such taxonomy entries221in their taxonomy identifier fields173(items which are in the item categories clustered as the first set items). Block660may also include code for determining which items are similar to the items clustered together with the first set items and determining that such items are also complementary to the first set items, even if such items are not directly clustered together with the first set items. For example, block660may identify item entries171which identify taxonomy entries221in the taxonomy identifier field174also identified by clustered item entries171(items in a same item category as the items clustered together with the first set items).

The recommend items codes650the continue to block661, which include code for directing the microprocessor102to order the items classified within the first set at block658and to order the items classified within the second set at block660. In different embodiments, block661may include different codes for directing the microprocessor102to determine the order of the items within the first set and the within second set. Block661may include code for directing the microprocessor102to order the items within the first and second sets independently, or to order the items such that the order of the items in the first set affects the order of the items in the second set and vice versa. Block661may also include code for directing the microprocessor102to order the items classified within each set of items utilizing a curated ranking, utilizing a model-based ranking or utilizing a combination of the curated ranking and the model-based ranking.

In embodiments where the block661directs the microprocessor102to order items utilizing the curated ranking, the ranking of different item entries171may be explicitly programmed in the application database122(shown inFIG.2). For example, block661may order items classified within the first set based, at least in part, on how closely the visual attribute associated with the first set item matches the selected visual attribute. Block661may include code similar to block652described above for determining how closely two visual attributes match. For example, in embodiments where the selected visual attribute is a color defined by pixel values, block661may order item entries171associated with visual attribute entries161having a distance of D=0 from the visual attribute entry161representing the single-selected visual attribute first, and item entries171associated with visual attribute entries161that are more distant subsequently. In embodiments where the selected visual attribute is a pattern or a texture, block661may order item entries171associated with visual attribute entries161that have pixel matrices which have a small edit distance from the pixel matrix of the single-selected visual attribute first, and item entries171associated with visual attribute entries161that have a greater edit distance subsequently. Additionally or alternatively, block661may order items classified within the first set based, at least in part, on how closely first set item matches or corresponds to the entered text query. For example, block661may order item entries171associated with a taxonomy entry221that is a perfect match or exactly corresponds to the entered text query (such as the macro-item category field224and the entered text query both being “dress” for example) first, and item entries171associated with a taxonomy entry221does not exactly match or correspond subsequently.

Block661may also order items classified within the second set based, at least in part, on how closely the visual attributes associated with the second set item matches (a) any of the visual attributes of the image (or palette) or (b) any visual attribute of the image (or palette) that is not the single-selected visual attribute. As noted above, block661may include code similar to block652described above for determining how closely two visual attributes match. Block661may also order items classified within the second set based, at least in part, on the number of the visual attributes of the image (or palette) which match the visual attributes associated with the second set item. For example, block661may order item entries171representing a second set item identifying visual attribute entries161in the visual attribute identifier field175which match more than one visual attribute entry161identified in the visual attribute identifier field146of the image entry141(or the visual attribute identifier field193of the palette entry191) representing the image (or palette).

Block661may also order items classified within the second set based, at least in part, on explicitly programmed levels of complementarity with the first set items. As noted above, item entries171may explicitly specify complementary items in the complementary item identifier field182and may further a level of complementarity for each complementary item in the complementary item order field183to identify complementarity at the item-level. The levels of complementarity stored in the complementary item order field183may be based on specific curated combinations of items selected by the vendor selling the items or by the host of the item recommendation server100. In this respect, second set items which are (1) particularly complementary to a particular first set item or (2) complementary to a large number of the first set items may be ordered first. As noted above in association with block660, item entries171representing second set items “complementary” to first set items may be identified as the item entries171identified in complementary item identifier field182of the item entries171representing the first set items. Block661may then order such second set items according to the levels of complementarity stored in the corresponding complementary item order fields183of item entries171representing the first set items, such that second set items having a high level of stored complementary may be ordered first, and second set items having a lower level of stored complementary may be ordered subsequently. Block661may also order items within the second set based on the number of first set items that the second set item is complementary with, and may thus order second set items that are identified in the complementary item identifier fields182of a large number of item entries171representing first set items first, and second set items which are identified in a fewer number subsequently.

Additionally, as noted above, taxonomy entries221may also explicitly specify both complementary taxonomies in the complementary taxonomy identifier field228and a level of complementarity for each complementary taxonomy in the complementary taxonomy order field229, to identify complementarity at the category-level. The levels of complementarity stored in the complementary taxonomy order field229of the taxonomy entries221may be based on specific curated combinations of item categories selected by the vendor selling items or by the host of the item recommendation server100. In this respect, second set items associated with taxonomies that are complementary to: (1) item categories of a particular first set item, (2) an item category associated with a large number of the first set items, and/or (3) item categories which match or correspond to the entered text query, may be ordered first; while second set items associated with a taxonomy that is not complementary to any item categories associated with any of the first set items or any item categories which match or correspond to the entered text query may be ordered subsequently. In this respect, as noted above in association with block660, second set items may be “complementary” to first set items when their respective item entries171identify complementary taxonomy entries221in their respective taxonomy identifier fields174; and second set items may also be “complementary” to the text query when the item entry171representing the second set item identifies a taxonomy entry221in the taxonomy identifier field174which is complementary to the taxonomy entry221matching or corresponding to the entered text query. Block661may then order items within the second set based the level of complementarity between taxonomy entries221identified by the second set items and the taxonomy entry221identified by the first set items or matching or corresponding to the entered text query, as stored in the complementary taxonomy order fields229for example; such that second set items associated with taxonomies which are highly complementary to the taxonomy of first set items or matching or corresponding to the entered text query are ordered first. Alternatively or additionally, block661may also order items within the second set based on the number of taxonomy entries221identified by the first set items or matching or corresponding to the entered text query which are complementary with the taxonomy entry221identified by the second set item, and may order items within the second set identifying a taxonomy entry221that is identified in the complementary taxonomy identifier fields228of a large number of the taxonomy entries221representing first set items or matching or corresponding to the entered text query first.

In embodiments where block661directs the microprocessor102to order items in the first set and to order items in the second set via the model-based ranking, the ranking of different item entries171may be determined based on a ranking model which processes historical entries stored in the application database122(shown inFIG.2) to determine order of items. The historical entries may be entries stored in the interaction history table250, the combination history table290, and the purchase history table270of the application database122for example. Specifically, in certain embodiments, the ranking model may order items based, at least in part, on prior user interaction with items, prior user combination of items and prior user purchase of items.

For example, the ranking model may order items classified within the first set based, at least in part, on processing interaction history entries251(shown inFIG.10). The ranking model may order first set items corresponding to item entries171that are identified by a large number of interaction history entries251in the item identifier fields254first (such items being frequently interacted with by users) and order first set items identified in a fewer number of interaction history entries251subsequently. The ranking model may assign different weights for different interaction history entries251, and the highly weighted interaction history entries251may be more relevant for determining order of items within the first set. For example, interaction history entries251which identify a same image entry141in the image identifier field255as the image the user initially selected to shop from (indicating the frequency that users interact with an item after selecting a same image), or which identify a user entry131in the user identifier field253that is the same as the current user (indicating the frequency that the current user interacts with this item), or which identify a visual attribute entry161in the visual attribute identifier field256that is the same as or matches the single-selected visual attribute (selected at block654, and indicating the frequency that users interact with an item after selecting a same visual attribute), or which identify a taxonomy in the taxonomy identifier field257that matches or substantially corresponds to the entered text query (determined at block656, and indicating the frequency that users interact with an item when searching for the same item category or the same text query), may be more highly weighted than interaction history entries251which identify different images, different users, different visual attributes or different taxonomies.

Additionally or alternatively, the ranking model may order items classified within the first set based, at least in part, on processing purchase history entries271(shown inFIG.12). The ranking model may order first set items corresponding to item entries171that are identified by a large number of purchase history entries271in the item identifier field274first (such items being frequently purchased by users) and order first set items that are identified by a fewer number of purchase history entries271subsequently. The ranking model may also assign different weights for different purchase history entries271and the highly weighted purchase history entries271may be more relevant for determining order of items within the first set. For example, purchase history entries271which identify a same image entry141in the image identifier field275as the image the user initially selected to shop from (indicating the frequency that users purchase an item after selecting a same image), or which identify a visual attribute entry161in the visual attribute identifier field276that is the same as or matches the single-selected visual attribute (selected at block654, and indicating the frequency that users purchase an item after selecting a visual attribute), or which identify a taxonomy entry221in the taxonomy identifier field277that matches or substantially corresponds to the entered text query (determined block656, indicating the frequency that user purchase an item when searching for the same item category or the same text query), may be more highly weighted that purchase history entries271which identify different images, different visual attributes or different taxonomies. The ranking model may also decrease the weight of items that the user has already purchased, based on the assumption that a user would not wish to purchase the same item more than once. For example, purchase history entries271which identify a user entry131in the user identifier field273that is the same as the current user may be given a low or a negative weight.

The ranking model may also order items classified to be within the second set based, at least in part, on processing interaction history entries251(shown inFIG.10). The ranking model may order second set items corresponding to item entries171identified by a large number of interaction history entries251in the item identifier fields254first. The ranking model may also assign different weights for different types of interaction history entries251similar to that described above in connection with ordering first set items based on processing interaction history entries251, and interaction history entries251which identify a same image entry141as the image the user initially selected to shop from, a user entry131that is the same as the current user, a visual attribute entry161that is the same as or matches the single-selected visual attribute, or a taxonomy entry221that matches or substantially corresponds to the entered text query, may be more highly weighted than interaction history entries251which identify different images, different users, different visual attributes or different taxonomies. Specific to ordering items in the second set, interaction history entries251which indicate that a user interacted with a second set item within a short time of interacting with a first set item may also be highly weighted by the ranking model, such as interaction history entries251which identify an item entry171representing a second set item and an item entry171representing a first set item in the item identifier fields254, a same image entry141in the image identifier fields255and a same user entry131in the user identifier fields253, wherein the time stored in the created field259of the interaction history entry251identifying the second set item is within a time gap below the interaction time gap threshold of the time stored in the created field259of the interaction history entry251identifying the first set item. The second set items identified by a large number of such interaction history entries251(indicating that a large number of users interact with that second set item within a short time gap of interacting with a first set item) may be ordered first.

Additionally or alternatively, the ranking model may order items classified within the second set based, at least in part, processing combination history entries291(shown inFIG.11). The ranking model may order second set items corresponding to item entries171identified by a large number of combination history entries291(in the second set item identifier fields295) that also identify an item entry171representing at least one first set item (in the first set item identifier field294) first, and the second set items identified by a smaller number of such combination history entries291subsequently. A large number of such combination history entries291indicate that a particular second set item is commonly combined with any of the first set items by users, whereas a small number of such combination history entries291indicate that a particular second set item is less commonly combined with any of the first set items by users. In other embodiments, rather than considering how commonly a particular second set item is combined with any first set item, the ranking model may instead consider how commonly a particular second set item is combined with a particular first set item (such as a highly ordered first set item for example), and may thus order second set items corresponding to item entries171identified by a large number of combination history entries291(in the second set item identifier fields295) that also identify an item entry171representing a specific first set item (in the first set item identifier fields294) first, and the second set items identified by a smaller number of such combination history entries291subsequently.

Additionally or alternatively, the ranking model may also order items classified within the second set based, at least in part, on processing purchase history entries271(shown inFIG.12). The ranking model may order second set items corresponding to item entries171identified by a large number of purchase history entries271in the item identifier field274first, and order second set items identified by a fewer number of purchase history entries271subsequently. The ranking model also may assign different weights for different purchase history entries271in a manner similar to that described above in connection with ordering first set items based on processing purchase history entries271, wherein purchase history entries271which identify a same image entry141as the image the user initially selected to shop from, a visual attribute entry161that is the same as or matches the single-selected visual attribute, or a taxonomy entry221that matches or substantially corresponds to the entered text query, may be more highly weighted than purchase history entries271which identify different images, different visual attributes or different taxonomies. The ranking model may also decrease the weight of second set items that the user has already purchased, based on the assumption that a user would not wish to purchase the same item more than once, and purchase history entries271which identify a user entry131in the user identifier field273that is the same as the current user may be given a low or a negative weight. Specific to ordering items in the second set, purchase history entries271which indicate that a large number of users other than the current user purchased a second set item together with a first set item, or within a short time of purchasing the first set item, may also be highly weighted. For example, a purchase history entry271which identifies item entries171representing both a first set item and a second set item in the item identifier field274may be weighted highly (such a second set item being purchased at the same time as the first set item). Similarly, purchase history entries271which identify an item entry171representing a second set item and an item entry171representing a first set item in the item identifier fields274and identifies a same user entry131in the user identifier fields273(that is not the current user for example), wherein the time stored in the created field278of the purchase history entry271identifying the second set item is within a time gap below the purchase time gap threshold of the time stored in the created field278of the purchase history entry271identifying the first set item may also be weighted highly (such a second set item purchased within a short time of the purchase of first set items). The second set items that a large number of users purchase together with (or within a short time gap of purchasing) first set items may be considered to be popular complementary items.

As described above, block661may also order items within the second set based at least in part on the order of items within the first set. For example, second set items which are complementary to a highly ordered item in the first set may be ordered before second set items which are complementary to a less highly ordered item in the first set. In certain specific embodiments, where the order of items within the first set includes first set item A ordered first and first set item B ordered second, block661may order second set items which are complementary to first set item A first, and then second set items which are complementary to first set item B subsequently. Alternatively or additionally, second set items which have a high level of complementarity to any of the items in the first set may be ordered first, and second set items which have a lower level of complementarity to any of the items in the first set may be ordered subsequently. In certain embodiments, block661may order second set items which have a high level of complementarity to either first set item A or first set item B first, and then order second set items which have a lower level complementarity to either first set item A or first set item B subsequently.

The recommend items codes650then continue to block662, which include code for directing the microprocessor102to display the first set of items and the second set of items to be simultaneously and proximate each other. For example, block662may include code for causing the user interface codes330to direct the user device114to display the recommend items page700(shown inFIG.28), described in greater detail below. The recommend items codes650then continue at block663, which include code for directing the microprocessor102to determine whether the user modifies either the selection of the visual attribute or the entered text query. The user may modify either the selection of the visual attribute or the entered text query using the recommend items page700for example.

Referring now toFIG.28, the recommend items page700allow a user to view both the first set of items and the second set of items simultaneously and proximate to each other. This allows a user to simultaneously browse items of the first set and items of the second set. The recommend items page700may further allow the user to: (a) interact with and view different first set items and different second set items; (b) combine different first set items with other first set items or with second set items and combine different second set items with other second set items or with first set items; (c) purchase items from at least one of the first set and the second set. In the embodiment shown, the recommend items page700includes a modify query region702and an item display region704.

The modify query region702includes a visual attribute array706which substantially corresponds to the visual attribute array635displayed on the shop image page630(shown onFIG.25). The visual attribute array706includes a plurality of visual attribute representations708a-708fwhich substantially corresponds to the plurality of visual attribute representations636a-636fdisplayed on the shop image page630(shown inFIG.25), and thus the visual attributes associated with the image (or palette) that the user initially selected to shop from. Each of the visual attribute representations708a-708fdisplayed on the recommend items page700may also be selectable by the user to modify which visual attribute is selected or the selection of the visual attribute (single selection, double selection or no selection) similar to the visual attribute representations636a-636fdisplayed on the shop image page630. When the user selects a particular visual attribute representation708a-708f, the user interface codes330may direct the user device114to display a modification of the selected representation and/or a modification of the non-selected representations in a manner similar to that described above in connection withFIGS.26A and26B. The modify query region702also includes a query field710operable to receive a text query from the user or receive a modification of the entered text query.

In the embodiment shown inFIG.28, the recommend items page700is displayed after block661of the recommend items codes650(shown inFIG.27A). Thus, the modify query region702displays the single selection of the selected visual attribute (single selection determined at block654), such as by automatically causing the modify query region702to display the visual attribute representation708dcorresponding to the single-selected visual attribute as a modified visual attribute representation708d′. The modify query region702also displays the entered text query (entered text query determined at block656), such as by automatically pre-populating the entered text query in the query field710. If the user interacts with the modify query region702to: (a) modify the selection of the visual attribute, such as by selecting another one of the plurality of visual attribute representations708a-708c,708e, or708for modifying the selection of the selected visual attribute by double selecting the visual attribute representation708dor de-selecting the visual attribute representation708dfor example; and/or (b) modify the entered text query, such as by modifying the text string entered in the query field710or by deleting the text string entered in the query field710for example, the user interface codes330may transmit the modification to the recommend items codes650. Referring briefly back toFIG.27A, upon receipt of the modification, block663may direct the microprocessor102to determine that the user has modified the selection of the visual attribute and/or the entered text query. The recommend items codes650then return to block654, which include code for directing the microprocessor102to determine whether the transmitted modification includes a single selection, a double selection or no selection of the visual attribute representations708a-708fas described above. The recommend items codes650then continue from block654as described above and below.

Referring back toFIG.28, the item display region704of the recommend items page700includes a first set region712displaying items classified within the first set and a second set region714, located proximate the first set region712, displaying items classified the second set simultaneously. In the embodiment shown, the first set region712is displayed as a first vertical scrollable column including a plurality of item representations716a-716ccorresponding to respective first set items and the second set region714is displayed as a second vertical scrollable column including a plurality of item representations718a-718ccorresponding to respective second set items and displayed immediately adjacent the first set region712. The first set region712and the second set region714may be independently scrollable. In other embodiments, at least one of the first set region712and the second set region714may displayed in a different format, such as horizontally scrollable rows, or in a page flip format where each page corresponds to an item from the first or second set. By displaying the first set items and the second set items simultaneously and proximate each other, the recommend items page700may promote user interaction with first set items and complementary second set items, may promote user combination of first set items and complementary second set items, and may encourage users to purchase more than the items that the users initial searched for (the first set items) as users are automatically and simultaneously presented with different and complementary items (the second set items) that could lead to impulse purchases. Further, by having the first set region712and the second set region714be independently scrollable, a user can browse a variety of different items (from the second set items) which may be complementary to an item that the user initially set out to purchase (from the first set items). As described above and below, user interaction, combination and purchase of first and second set items creates historical entries stored in the application database122(shown inFIG.2), which may then be used by the microprocessor102to determine complementary items or complementary item categories via the complementarity model, or the order of items in the first set and the second set via the ranking model. By displaying the first set items and the second set items simultaneously and proximate each other, the recommend items page700may promote generation of such historical entries by users of the item recommendation server100, and such historical entries may allow the microprocessor102to determine more relevant complementary items or item categories and more relevant item orders, which may in turn encourage greater user interaction, combination and purchase.

The item representations716a-716cmay correspond to at least one of the item representations stored in the representation database124(shown inFIG.1) directed to by the URIs in the item representation path fields177of the item entries171classified within the first set by various blocks of the recommend items codes650(including block658described above, and blocks664,672,678,686and692described below). The item representations718a-718cmay correspond to at least one of the item representations stored in the representation database124directed to by the URIs in the item representation path fields177of the item entries171classified within the second set by various blocks of the recommend items codes650(including block660described above, and blocks666,674,680,688and694described below). The displayed order of the item representations716a-716cin the first set region712and the displayed order of the item representations718a-718cin the second set region714may correspond to the order of the first and second set items as determined by various blocks of the recommend items codes650(including block661described above, and blocks667,676,682,690and696described below). As noted above, in the embodiment shown inFIG.28, the recommend items page700is displayed after block660(shown inFIG.27A) and thus, item representations716a-716cdisplayed in the first set region712represent the item entries171classified within the first set by block658and are displayed in the order determined by block661and item representations718a-718cdisplayed in the second set region714represent the item entries171classified within the second set by block660and are displayed in the order determined by block661.

Each of the item representations716a-716cand718a-718cmay be selectable to view additional information associated with the item represented by the item representation. For example, when a user selects any item representation716a-716c,718a-718c, the user interface codes330may direct the user device114to display a modified recommend items page shown generally at700′ inFIG.29. Referring toFIG.29, the modified recommend items page700′ includes an item detail region730which displays information associated with the selected item, which may allow the user to determine whether the user wishes to purchase the selected item. In the embodiment shown, the item detail region730display information from the item entry171(shown inFIG.7) representing the selected item, including item representations732, which may correspond to the representations stored in the representation database124(shown inFIG.1) directed to by a URI in the item representation path fields177of the item entry171representing the selected item. In embodiments where there is more than one item representation stored in the representation database124, the item detail region730may allow a user to scroll or click through the different item representations732. The item detail region730also displays: an item description734which may display the text stored in at least one of the description field176or the vendor description field178of the item entry171representing the selected item; a vendor description736which may correspond to the vendor entry231(shown inFIG.8) identified in the vendor identifier field173of the item entry171representing the selected item (identifying the vendor offering the selected item); and a price738which may display the price stored in the price field179of the item entry171representing the selected item. The item detail region730may also include an option selector740and an add-to-cart button742. The option selector740may allow the user to select different options associated with the item as stored in the options field180of the item entry171representing the selected item. When the user selects the add-to-cart button742, the user interface codes330may direct the microprocessor102to add the item displayed on the modified recommend items page700′ to a shopping cart for later purchase, and to cause the shopping cart page780(shown inFIG.35) to display a item representation representing the selected item when the user navigates to the shopping cart page780.

Referring back toFIG.28, as described above and below in connection with determining items complementary with other items or with the text query (see block660described above, and blocks674and688described below, for example) and determining the order of items in the first and second sets (see block661described above, and blocks667,676,682,690and696described below, for example), each time the user interacts with an item representation716a-716c,718a-718cby selecting that item representation, the user interface codes330may direct the microprocessor102to add a new instance of the interaction history entry251(shown inFIG.10) to the interaction history table250(shown inFIG.2). The new instance of the interaction history entry251may function as a record indicating that a particular user interacted with a particular item after being directed to that item from a particular image (or palette). The interaction history entry251may also function as a record indicating that (a) a particular user interacted with a particular item after being directed to that item from a selected visual attribute and/or from an entered text query, and (b) a particular user interacted with a particular item classified within either the first set or the second set based on a selected visual attribute and/or an entered text query.

The new instance of the interaction history entry251stores a identifier identifying the user entry131(shown inFIG.3) representing the user who selected the item (such as the user who logged on using the login page350(shown inFIG.13) for example) in the user identifier field253. The new instance of the interaction history entry251also stores: an identifier identifying the image entry141(shown inFIG.6) representing the image that the user initially selected to shop from (image634(shown inFIG.25) for example) or an identifier identifying the palette entry191(shown inFIG.5) representing a customized palette that the user initially selected to shop from (a “rust” palette selected from the palette selection page750(shown inFIG.34) for example) in the image identifier field255; an identifier identifying the item entry171(shown inFIG.7) representing the item that the user interacted with (item represented by the item representation716a-716c,718a-718cselected by the user (shown onFIG.28) for example) in the item identifier field254; and an indication of whether the item identified in the item identifier field254was classified within the first set or second set by the recommend items codes650(shown inFIG.27) in the item set field258. The new instance of the interaction history entry251may also store, in the created field259and the modified field260, a time obtained from the clock104(shown inFIG.1) corresponding to the time the new instance of the interaction history entry251was created or modified.

In embodiments where the user single selects or double selects a visual attribute (block654of the recommend items codes650), using either the visual attribute representations636a-636f(shown inFIG.25) or708a-708f(shown inFIG.28) for example, the new instance of the interaction history entry251may further store identifier(s) identifying the visual attribute entry (entries)161(shown inFIG.6) representing the selected visual attribute(s) in the visual attribute identifier field256. Further, in embodiments where the user enters a text query (blocks656,670and684of the recommend items codes650), using either the query fields638(shown inFIG.25) or710(shown inFIG.28) for example, the new instance of the interaction history entry251may further store an identifier identifying the taxonomy entry221(shown inFIG.9) matching or corresponding to the entered text query. The microprocessor102may determine taxonomy entries221matching or corresponding to the entered text query in a manner similar to block658described above.

Still referring toFIG.28, the item representations716a-716ceach include a respective add-to-cart button717aand717band the item representations718a-718calso each include a respective add-to-cart button719aand719b. When the user selects one or more of the add-to-cart buttons717a,717b,719a,719b, the user interface codes330may direct the microprocessor102to add the item represented by the item representation716a-716bb,718a-718bassociated with the selected add-to-cart buttons717a,717b,719a,719bto the shopping cart for later purchase, and to cause the shopping cart page780(shown inFIG.35) to display a item representation representing the selected items when the user navigates to the shopping cart page780. Further, when the user selects an add-to-cart button717a,717b,719a,719bassociated with an item representation716a-716c,718a-718c, the user interface codes330may direct the user device114to display a modified item representation as shown inFIG.30. For example, when the user selects the add-to-cart button717bassociated with item representation716bdisplayed in the first set region712and the add-to-cart button719aassociated with item representation718bdisplayed in the second set region714, user interface codes330may display the item representations716band718aas modified item representations716b′ and718a′. The modified item representations716b′ and718a′ may be displayed with a folded bottom-right corner. In other embodiments, the item representations may be modified in a different or alternative manner, such that the entire bottom half may be folded, different corners (such as the top-left, top-right or bottom-left corners) may be folded, or the item representation may be colored in a specific or a random color. If the user re-selects a selected and a modified item representation716a′,718a′, the user interface codes330may direct the user device114to re-display the modified item representation716a′,718a′ as the unmodified item representation716a,718a(shown inFIG.28) and may further direct the microprocessor102to remove the item represented by the item representation716a,718afrom the shopping cart such that the user device114does not display an item representation representing the selected items on the shopping cart page780(shown inFIG.35).

As described above and below in connection with determining items complementary with other items or with the text query (see block660described above, and blocks674and688described below, for example) and determining the order of items within the first and second sets (see block661described above, and blocks667,676,682,690and696described below, for example), each time the user selects an add-to-cart button717associated with an item representation716representing a first set item and displayed in the first set region712and an add-to-cart button719associated with an item representation718representing a second set item and displayed in the second set region714, the user interface codes330may direct the microprocessor102to add a new instance of the combination history entry291(shown inFIG.11) to the combination history table290(shown inFIG.2). The new instance of the combination history entry291may function as a record indicating that a particular user decided to combine at least one first set item with at least one second set item after being directed to those items from a particular image. The combination history entry291may also function as a record indicating that a particular user combined at least one first set item and at least one second set item after being directed to these items from a selected visual attribute and/or from an entered text query.

The new instance of the combination history entry291stores an identifier identifying the user entry131(shown inFIG.3) representing the user whom combined the items (such as the user who logged on using the login page350(shown inFIG.13) for example) in the user identifier field293. The new instance of the combination history entry291also stores: identifier(s) identifying item entry (entries)171(shown inFIG.7) representing the first set item(s) (represented by the item representations716a-716cin the first set region712(shown onFIG.28) for example) if the current user selected at least one first set item, in the first set item identifier field294; and identifier(s) identifying the item entry (entries)171representing the second set item (represented by the item representations718a-718cin the second set region714(shown onFIG.28) for example) if the current user selected at least one second set item, in the second set item identifier field295. In embodiments where the current user does not select any first set item or any second set item, such as if the current user only combines items within the first set or within the second set, one of the first set item identifier field294and the second set item identifier field295may not store any identifiers and the other one may store more than one identifier. In embodiments where the current user selects more than one first set item or more than one second set item, the first set item identifier field294and the second set item identifier field295may store identifiers identifying more than one item. In other embodiments, the first set item identifier field294and the second set item identifier field295may each store only a single identifier identifying a single item, and a new instance of the combination history entry291may be added each time a user combines a first set item with a second set item. For example, if the user selects the add-to-cart button717aof the first set item representation716aand the add-to-cart buttons719band719aof the second set item representations718band718a, two instances of the combination history entry291may be added to the combination history table290: (1) a first instance identifying the item entry171represented by the first set item representation716ain the first set item identifier field294and identifying the item entry171represented by the second item representation718bin the second set item identifier field295, and (2) a second instance identifying the item entry171represented by the first set item representation716ain the first set item identifier field294and identifying the item entry171represented by the second set item representation718ain the second set item identifier field295. The new instance of the combination history entry291may also store: an identifier identifying the image entry141(shown inFIG.6) representing the image that the user initially selected to shop from (image634(shown inFIG.25) for example) or an identifier identifying the palette entry191(shown inFIG.5) representing a customized palette that the user initially selected to shop from (the “rust” palette selected from the palette selection page750(shown inFIG.34) for example) in the image identifier field296; and a time obtained from the clock104(shown inFIG.1) corresponding to the time the instance of the combination history entry291was created and modified in the created field299and the modified field300.

In embodiments where the user single selects or double selects a visual attribute (block654of the recommend items codes650), using either the visual attribute representations636a-636f(shown inFIG.25) or708a-708f(shown inFIG.28) for example, the new instance of the combination history entry291may further store an identifier identifying the visual attribute entry161(shown inFIG.6) representing the selected visual attribute in the visual attribute identifier field297. Further, in embodiments where the user enters a text query (blocks656,670and684of the recommend items codes650), using either the query fields638(shown inFIG.25) or710(shown inFIG.28) for example, the new instance of the combination history entry291may further store an identifier identifying the taxonomy entry221(shown inFIG.9) matching or corresponding to the entered text query. The microprocessor102may determine taxonomy entries221matching or corresponding to the entered text query in a manner similar to block658described above.

Referring back toFIGS.27A and27B, if the microprocessor102determines at block656that the user did not enter a text query—such that the user (a) provided a single selection of a visual attribute but (b) did not enter a text query—the recommend items codes650then continue at block664shown inFIG.27B, which include code for directing the microprocessor102to identify those item entries171of the item entries171initially retrieved at block652which (1) are associated with visual attributes which match the single-selected visual attribute. The microprocessor102may classify the item entries171which meet the criteria (1) above in the first set of items or as first set items. With respect to criteria (1), block664may include code similar to block658and may thus also direct the microprocessor102to identify, from the item entries171initially retrieved at block652, those item entries171associated with visual attribute entries161that match only the double-selected visual attribute. Block664may determine whether two visual attributes match in a manner similar to block652described above.

The recommend items codes650then continue at block666, which include code for directing the microprocessor102to identify those item entries171of the item entries171initially retrieved at block652which (1) are associated with a visual attribute matching any visual attribute of the image the user initially selected to shop from that is not the single-selected visual attribute, and (2) are not in the first set. The microprocessor102may then classify the item entries171which meet the criteria (1) and (2) above within the second set of items or as second set items. In such embodiments, the first set items include items associated with the single-selected visual attribute, whereas the second set items include items associated with visual attributes of the image (or palette) other than the single-selected visual attribute. With respect to criteria (1) and (2) above, block666may include code for directing the microprocessor102to (a) identify, from the item entries171initially retrieved at block652, those item entries171associated with visual attributes entries161that match any visual attribute of the image (or palette) other than single-selected visual attribute, and (b) exclude from the item entries171identified at (a), those item entries171which were classified within the first set at block664. Block666may determine whether two visual attributes match in a manner similar to block652described above.

The recommend items codes650the continue to block667, which include code for directing the microprocessor102to order the items classified within the first set at block664and to order the items classified within the second set at block666, such as in a manner similar to block661described above for example. Block667may thus also direct the microprocessor102to order the items classified within each set utilizing the curated ranking, the model-based ranking or a combination thereof.

In embodiments where block667directs the microprocessor102to order items utilizing the curated ranking, the ranking of different item entries171may be explicitly programmed in the application database122(shown inFIG.2) in a manner similar to block661described above.

Block667may thus order items classified within the first set based, at least in part, on how closely the visual attribute associated with the first set item matches the single-selected visual attribute. Block667may thus also order items classified within the second set based, at least in part, on: (a) explicitly programmed levels of complementarity associated with the first set items and the second set items; (b) explicitly programmed levels of complementarity of the item category (taxonomy) associated with the second set items with the item category (taxonomy) associated with the first set items; (c) how closely the visual attributes associated with the second set item matches any visual attribute of the image (or palette) that is not the single-selected visual attribute; and/or (d) how many visual attributes of the image (or palette) that is not the single-selected visual attribute match the visual attributes associated with the second set item. Block667may retrieve explicitly programmed levels of complementarity in a manner similar to block661described above and may determine whether (and how closely) two visual attributes match in a manner similar to block652described above.

In embodiments where block667directs the microprocessor102to order items utilizing the model-based ranking, the ranking of different item entries171may be determined using the ranking model in a manner similar to block661described above. Block667may thus also utilize the ranking model to processes historical entries stored in the application database122(shown inFIG.2), such as interaction history entries251(shown inFIG.10) in the interaction history table250, combination history entries291(shown inFIG.11) in the combination history table290, and purchase history entries271(shown inFIG.12) in the purchase history table270for example, to determine order of items within both the first and second sets.

The recommend items codes650then return to block662, which include code for directing the microprocessor102to display the first set of items and the second set of items simultaneously and proximate each other, via the recommend items page700(shown inFIGS.28and30) for example. In embodiments where the recommend items page700is displayed after block667(shown inFIG.27B), the modify query region702may: (a) display the single selection of the selected visual attribute (single selection determined at block654) by automatically displaying the visual attribute representation708dcorresponding to the single-selected visual attribute as a modified visual attribute representation708d′ in a manner similar toFIG.26Adescribed above; and (b) not display any entered text query (no text query entered as determined at block656) in the query field710. Further, the item display region704may: (a) display item representations716a-716crepresenting the item entries171classified within the first set by block664in the first set region712in the order determined by block667; and (b) display item representations718a-718crepresenting the item entries171classified within the second set by block666in the second set region714in the order determined by block667. The recommend items codes650then continue from block662as described above and below.

Referring back toFIGS.27A and27B, if the microprocessor102determines at block654that the user provided a double selection of one of the visual attributes, using either the visual attribute representations636a-636f(shown inFIG.25) or708a-708f(shown inFIG.28) for example, the recommend items codes650then continue at block670(shown inFIG.27B), which includes code for directing the microprocessor102to determine whether the user entered any text query, using either the query fields638(shown inFIG.25) or710(shown inFIGS.28and30).

If the microprocessor102determines at block670that the user entered a text query—such that the user both (a) provided a double selection of a visual attribute and (b) entered a text query—the recommend items codes650then continue at block672, which include code for directing the microprocessor102to identify those item entries171of the item entries171initially retrieved at block652which (1) are associated with a visual attribute which match the double-selected visual attribute and (2) match or correspond to the text query. The microprocessor102may classify the item entries171which meet the criteria (1) and (2) above in the first set or as first set items.

With respect to criteria (1), block672may include code for directing the microprocessor102to identify, from the item entries171initially retrieved at block652, those item entries171associated with visual attribute entries161that match only the visual attribute entry161representing the double-selected visual attribute. Block672may determine whether two visual attributes match in a manner similar to block652described above. With respect to criteria (2), block672may determine whether an item matches or corresponds to a text query in a manner similar to block658described above. Block672may thus also direct the microprocessor102to identify, from the item entries171initially retrieved at block652, item entries171which: (a) store a text string or other descriptive matter in the description field176or the vendor description field178which matches or corresponds to the text query; and/or (b) identify at least one taxonomy entry221in the item taxonomy identifier field174which matches or corresponds to the text query.

The recommend items codes650then continue at block674, which include code for directing the microprocessor102to identify those item entries171of the item entries171initially retrieved at block652which (1) are associated with a visual attribute which match the double-selected visual attribute, (2) are complementary to a first set item classified at block672and/or the text query determined to be entered at block670, and (3) are not in the first set. The microprocessor102may classify the item entries171which meet the criteria (1), (2) and (3) above within the second set or as second set items. In such embodiments, the first set items include items associated with the double-selected visual attribute and matching the text query, whereas the second set items include items also associated with the double-selected visual attribute but which is complementary to the first set items and/or the text query. With respect to criteria (1) and (3) above, block674may include code for directing the microprocessor102to (a) identify, of the item entries171initially retrieved at block652, those item entries171associated with visual attribute entries161that match the double-selected visual attribute, and (b) exclude, from the item entries171identified at (a), those item entries171already classified within the first set at block672. Block674may determine whether two visual attributes match in a manner similar to block652described above. With respect to criteria (2) above, block674may determine whether an item is complementary to at least one first set item and/or complementary to the text query in a manner similar to block660described above. For example, block674may also direct the microprocessor102to determine complementary second set items utilizing the curated determination or the model-based determination, or a combination thereof.

In embodiments where block674directs the microprocessor102to identify complementary items utilizing the curated determination, the complementarity of different item entries171may be explicitly programmed in the application database122(shown inFIG.2) in a manner similar to block660described above. Block674may thus direct the microprocessor102to identify complementary second set items based at least in part on: (a) complementary items explicitly associated with the first set items and/or (b) items which explicitly identify first set items as complementary items. Block674may also direct the microprocessor102to identify complementary second set items by identifying items associated with item categories (taxonomies) which are complementary to the item categories associated with the first set items or complementary to the item categories matching or corresponding to the entered text entry.

In embodiments where block674directs the microprocessor102to identify complementary items utilizing the model-based determination, the complementarity of different item entries171may be determined using the complementarity model in a manner similar to block660described above. Block674may thus also utilize the complementarity model to process historical entries stored in the application database122(shown inFIG.2), such as the interaction history entries251(shown inFIG.10) in the interaction history table250, combination history entries291(shown inFIG.11) in the combination history table290, and purchase history entries271(shown inFIG.12) in the purchase history table270for example, to determine complementary items or complementary item categories.

The recommend items codes650the continue to block676, which include code for directing the microprocessor102to order the items classified within the first set at block672and to order the items classified within the second set at block674, such as in a manner similar to block661described above for example. Block676may thus also direct the microprocessor102to order the items classified within each set utilizing the curated ranking, the model-based ranking or a combination thereof.

In embodiments where block676directs the microprocessor102to order items utilizing the curated ranking, the ranking of different item entries171may be explicitly programmed in the application database122(shown inFIG.2) in a manner similar to block661described above. Block676may thus order items classified within the first set based, at least in part, on: (a) how closely the visual attribute associated with the first set item matches the double-selected visual attribute, and/or (b) on how closely first set item matches or corresponds to the entered text query. Block676may thus also order items classified within the second set based, at least in part, on: (a) explicitly programmed levels of complementarity associated with the first set items and the second set items; (b) explicitly programmed levels of complementarity of the item category (taxonomy) associated with the second set items with the item category (taxonomy) associated with the first set items or the item category (taxonomy) matching or corresponding to the entered text query; and/or (c) how closely the visual attributes associated with the second set item matches the double-selected visual attribute. Block676may retrieve explicitly programmed levels of complementarity and how closely an item matches or corresponds to the entered text query in a manner similar to block661described above and may determine whether (and how closely) two visual attributes match in a manner similar to block652described above.

In embodiments where block676directs the microprocessor102to order items utilizing the model-based ranking, the ranking of different item entries171may be determined based on the ranking model in a manner similar to block661described above. Block676may thus also utilize the ranking model to processes historical entries stored in the application database122(shown inFIG.2), such as interaction history entries251(shown inFIG.10), combination history entries291(shown inFIG.11), and purchase history entries271(shown inFIG.12) for example, to determine order of items within the first and second sets.

The recommend items codes650then return to block662, which include code for directing the microprocessor102to display the first set of items and the second set of items simultaneously and proximate each other, via the recommend items page700(embodiments shown inFIGS.28and31) for example. Referring now toFIG.31, in the embodiment shown, the recommend items page700is displayed after block676(shown inFIG.27B). The modify query region702displays the double selection of the double-selected visual attribute (double selection determined at block654) by displaying the visual attribute representation708drepresenting the double-selected visual attribute as a modified selected visual attribute representation708d′ and the visual attribute representations708a-708c,708eand708frepresenting the non-selected visual attributes as modified non-selected visual attribute representations708a′-708c′,708e′ and708f′, in a manner similar toFIG.26Bdescribed above. The modify query region702also displays the entered text query (text query entered as determined at block670) by automatically pre-populating the entered text query (text query entered as determined at block670) in the query field710. Further, the item display region704may: (a) display item representations716a-716crepresenting the item entries171classified within the first set at block672in the first set region712in the order determined by block676; and (b) display item representations718a-718crepresenting the item entries171classified within the second set by block674in the second set region714in the order determined by block667. The recommend items codes650then continue from block662as described above.

Referring back toFIG.27B, if the microprocessor102determines at block670that the user did not enter a text query—such that the user (a) provided a double selection of a visual attribute but (b) did not enter a text query—the recommend items codes650then continue at block678, which include code for directing the microprocessor102to identify those item entries171of the item entries171initially retrieved at block652which (1) are associated with visual attributes which match the double-selected visual attribute. The microprocessor102may classify the item entries171which meet the criteria (1) above in a first set of items or as first set items. With respect to criteria (1), block678may include code similar to block658and may thus also direct the microprocessor102to identify, from the item entries171initially retrieved at block652, those item entries171associated with visual attribute entries161which match only the double-selected visual attribute. Block678may determine whether two visual attributes match in a manner similar to block652described above. Block678may also include code for directing the microprocessor102to retrieve only a subset of the items associated with visual attributes which match the double-selected visual attribute. For example, block678may direct the microprocessor102to retrieve only item entries171which are also identified in a number of historical entries stored in the application database122(such as interaction history entries251, combination history entries291and purchase history entries271for example) above a certain threshold, indicating that such items are popularly interacted with, combined by, or purchased by users for example, or to retrieve only item entries171associated with visual attributes which very closely match the double-selected visual attribute.

The recommend items codes650then continue at block680, which include code for directing the microprocessor102to identify those item entries171of the item entries171initially retrieved at block652which (1) are associated with a visual attribute matching the double-selected visual attribute but (2) are not in the first set. The microprocessor102may then classify the item entries171which meet the criteria (1) and (2) above within the second set of items or as second set items. In such embodiments, the first set and second set both include items associated with the double-selected visual attribute, but the second set does not include any of the items included in the first set. With respect to criteria (1) and (2) above, block680may include code for directing the microprocessor102to: (a) identify, from the item entries171initially retrieved at block652, those item entries171associated with visual attributes entries161that match only the double-selected visual attribute, and (b) exclude from the item entries171identified at (a), those item entries171which were classified within the first set at block678. Block680may determine whether two visual attributes match in a manner similar to block652described above.

The recommend items codes650the continue to block682, which include code for directing the microprocessor102to order the items classified within the first set at block678and to order the items classified within the second set at block680, such as in a manner similar to block661described above for example. Block682may thus also direct the microprocessor102to order the items classified within each set utilizing the curated ranking, the model-based ranking or a combination thereof.

In embodiments where block682directs the microprocessor102to order items utilizing the curated ranking, the ranking of different item entries171may be explicitly programmed in the application database122(shown inFIG.2) in a manner similar to block661described above. Block682may thus order items classified within the first set based, at least in part, on how closely the visual attribute associated with the first set item matches the double-selected visual attribute. Block682may also order items classified within the second set based, at least in part, on: (a) explicitly programmed levels of complementarity associated with the first set items and the second set items; (b) explicitly programmed levels of complementary of the item category (taxonomy) associated with the second set items with the item category (taxonomy) associated with the first set items; and/or (c) how closely the visual attributes associated with the second set item match the selected visual attribute. Block682may retrieve explicitly programmed levels of complementarity in a manner similar to block661described above and may determine whether (and how closely) two visual attributes match in a manner similar to block652described above.

In embodiments where block682directs the microprocessor102to order items utilizing the model-based ranking, the ranking of different item entries171may be determined using the ranking model in a manner similar to block661described above. Block682may thus also utilize the ranking model to processes historical entries stored in the application database122(shown inFIG.2), such as interaction history entries251(shown inFIG.10), combination history entries291(shown inFIG.11), and purchase history entries271(shown inFIG.12) for example, to determine order of items within both the first and second sets.

The recommend items codes650then return to block662, which include code for directing the microprocessor102to display the first set of items and the second set of items simultaneously and proximate each other, such as via the recommend items page700for example. In embodiments where the recommend items page700is displayed after block682(shown inFIG.27B), the modify query region702may: (a) display the double selection of the selected visual attribute (double selection determined at block654) by displaying the visual attribute representation708dcorresponding to the double-selected visual attribute as the modified selected visual attribute representation708d′ and the visual attribute representations708a-708c,708eand708fcorresponding to the non-selected visual attributes as modified non-selected visual attribute representations708a′-708c′,708e′ and708f′ in a manner similar toFIG.26Bdescribed above; and (b) not display any entered text query (no text query entered as determined at block670) in the query field710. Further, the item display region704may: (a) display item representations716a-716crepresenting the item entries171classified within the first set at block678in the first set region712in the order determined by block682; and (b) display item representations718a-718crepresenting the item entries171classified within the second set by block680in the second set region714in the order determined by block682. The recommend items codes650then continue from block662as described above and below.

Referring toFIGS.27A and27C, if the microprocessor102determines at block654that the user provided no selection of one of the visual attributes, using either the visual attribute representations636a-636f(shown inFIG.25) or708a-708f(shown inFIGS.28and31) for example, the recommend items codes650then continue at block684(shown inFIG.27C), which include code for directing the microprocessor102to determine whether the user entered any text query, using either the query fields638(shown inFIG.25) or710(shown inFIGS.28,30and31) for example.

If the microprocessor102determines at block684that the user entered a text query—such that the user (a) provided no selection of a visual attribute, but (b) did enter a text query—the recommend items codes650then continue at block686, which include code for directing the microprocessor102to identify those item entries171of the item entries171initially retrieved at block652which (1) are associated with visual attributes which match any visual attribute of the image (or palette) user initially selected to shop from and (2) match or correspond to the text query. The microprocessor102may classify the item entries171which meet the criteria (1) and (2) above in the first set or as first set items. With respect to criteria (1) above, block686may include code for directing the microprocessor102to identify each of the item entries171initially retrieved at block652. With respect to criteria (2) above, block686may determine whether an item matches or corresponds to a text query in a manner similar to block658described above. Block686may thus also direct the microprocessor102to identify, from the item entries171initially retrieved at block652, item entries171which: (a) store a text string or other descriptive matter in the description field176or the vendor description field178which matches or corresponds to the text query; and/or (b) identify at least one taxonomy entry221in the item taxonomy identifier field174which matches or corresponds to the text query.

The recommend items codes650then continue at block688, which include code for directing the microprocessor102to identify those item entries171of the item entries171initially retrieved at block652which (1) are associated with visual attributes which match any visual attribute of the image (or palette) the user initially selected to shop from, (2) are complementary to a first set item classified at block688and/or the text query entered at block684, and (3) are not in the first set. The microprocessor102may classify the item entries171which meet the criteria (1), (2) and (3) above within the second set or as second set items. In such embodiments, the first set items include items associated with any visual attribute of the image (or palette) and matching the entered text query, whereas the second set items include items also associated with any visual attribute of the image (or palette) but which is complementary to the first set items and/or the text query. With respect to criteria (1) and (3) above, block688may include code for directing the microprocessor102to (a) identify each of the item entries171initially retrieved at block652, and (b) exclude, from the item entries171identified at (a), those item entries171classified within the first set at block688. With respect to criteria (2) above, block688may determine whether an item is complementary to at least one first set item and/or complementary to the text query in a manner similar to block660described above. For example, block688may also direct the microprocessor102to determine complementary second set items utilizing the curated determination, the model-based determination, or a combination thereof.

In embodiments where block688directs the microprocessor102to identify complementary items utilizing the curated determination, the complementarity of different item entries171may be explicitly programmed in the application database122(shown inFIG.2) in a manner similar to block660described above. Block688may thus direct the microprocessor102to identify complementary second set items by at least in part on: (a) complementary items explicitly associated with the first set items and/or (b) items which explicitly identify first set items as complementary items. Block688may also direct the microprocessor102to identify complementary second set items by identifying items associated with item categories (taxonomies) which are identified as complementary to the item categories (taxonomies) associated with the first set items or complementary to the item categories matching or corresponding to the entered text entry.

In embodiments where block688direct the microprocessor102to identify complementary items utilizing the model-based determination, the complementarity of different item entries171may be determined based on the complementarity model in a manner similar to that described above in connection with block660, wherein the complementarity model processes historical entries stored in the application database122(shown inFIG.2) to determine complementary items or complementary item categories. The historical entries may be entries stored in the interaction history table250, the combination history table290, and the purchase history table270for example.

The recommend items codes650the continue to block690, which include code for directing the microprocessor102to order the items classified within the first set at block686and to order the items classified within the second set at block688in a manner similar to block661described above for example. Block690may thus also direct the microprocessor102to order the items classified within each item set utilizing the curated ranking, the model-based ranking or a combination of the curated ranking and the model-based ranking.

In embodiments where block690directs the microprocessor102to order items utilizing the curated ranking, the ranking of different item entries171may be explicitly programmed in the application database122(shown inFIG.2) in a manner similar to block661described above. Block690may thus order items classified within the first set based, at least in part, on: (a) how closely the visual attributes associated with the first set item matches any of the visual attributes of the image (or palette); (b) how many of the visual attributes of the image (or palette) the visual attributes associated with the first set item matches; and/or (c) how closely the first set item matches or corresponds to the item category (taxonomy) matching or corresponding to the entered text query. Block690also may order items classified within the second set based, at least in part, on: (a) explicitly programmed levels of complementarity associated with the first set items and the second set items; (b) explicitly programmed levels of complementarity of the item category (taxonomy) associated with the second set items with the item category (taxonomy) associated with the first set items or the item category (taxonomy) matching or corresponding to the entered text query; (c) how closely the visual attributes associated with the second set item matches any of the visual attributes of the image (or palette); and/or (d) how many of the visual attributes of the image (or palette) the visual attributes associated with the second set item matches. Block690may determine how closely two visual attributes match in a manner similar to block652described above. Block690may determine how closely an item matches or corresponds to the entered text query in a manner similar to that described above in connection with blocks658.

In embodiments where block690directs the microprocessor102to order items utilizing the model-based ranking, the ranking of different item entries171may be determined based on the ranking model in a manner similar to block661described above. Block690may thus also utilize the ranking model to processes historical entries stored in the application database122(shown inFIG.2), such as interaction history entries251(shown inFIG.10), combination history entries291(shown inFIG.11), and purchase history entries271(shown inFIG.12) for example, to determine order of items within the first and second sets.

The recommend items codes650then return to block662, which include code for directing the microprocessor102to display the first set of items and the second set of items simultaneously and proximate each other on the user device114, such as via the recommend items page700, (embodiments shown inFIGS.28,31and32). Referring now toFIG.32, in the embodiment shown, the recommend items page700is displayed after block690(shown inFIG.27C). The modify query region702displays the no selection of the selected visual attribute (no selection determined at block654) by displaying the visual attribute representations708a-708fin an unmodified state. The modify query region702also displays the entered text query (text query entered determined at block684) by automatically pre-populating the entered text query (text query entered as determined at block684) in the query field710. Further, the item display region704may: (a) display item representations716a-716crepresenting the item entries171classified within the first set at block686in the first set region712in the order determined by block690, and (b) display item representations718a-718crepresenting the item entries171classified within the second set by block688in the second set region714in the order determined by block690. The recommend items codes650then continue from block662as described above and below.

Referring back toFIG.27C, if the microprocessor102determines at block684that the user did not enter a text query—such that the user (a) provided no selection of any visual attribute and (b) did not enter a text query—the recommend items codes650then continue at block692, which include code for directing the microprocessor102to identify those item entries171of the item entries171initially retrieved at block652which (1) are associated with a visual attribute matching any visual attribute of the image (or palette) the user initially selected to shop from. The microprocessor102may classify the item entries171which meet the criteria (1) above in the first set or as first set items. With respect to criteria (1) above, block692may include code for directing the microprocessor102to identify each of the item entries171initially retrieved at block652. In certain embodiments, block692may further include code for directing the microprocessor102to retrieve only a subset of the item entries171initially retrieved at block652. For example, block678may direct the microprocessor102to retrieve only item entries171which are also identified in a number of historical entries stored in the application database122(such as interaction history entries251, combination history entries291and purchase history entries271for example) above a certain threshold.

The recommend items codes650then continue at block694, which include code for directing the microprocessor102to identify those item entries171of the item entries171initially retrieved at block652which (1) are associated with a visual attribute matching any visual attribute of the image (or palette) the user initially selected to shop from but (2) are not in the first set. The microprocessor102may then classify the item entries171which meet the criteria (1) and (2) above within the second set of items or as second set items. In such embodiments, the first set and second set both include items associated with any visual attribute of the image (or palette), but the second set does not include any of the items included in the first set. With respect to criteria (1) and (2) above, block694may include code for directing the microprocessor102to (a) identify each of the item entries171initially retrieved at block652, and (b) exclude, from the item entries171identified at (a), those item entries171classified within the first set at block692.

The recommend items codes650the continue to block696, which include code for directing the microprocessor102to order the items classified within the first set at block692and to order the items classified within the second set at block694, such as in a manner similar to block661described above for example. Block690may thus also direct the microprocessor102to order the items classified within each set utilizing the curated ranking, the model-based ranking or a combination thereof.

In embodiments where block696directs the microprocessor102to order items utilizing the curated ranking, the ranking of different item entries171may be explicitly programmed in the application database122(shown inFIG.2) in a manner similar to block661described above. Block696thus may order items classified within the first set based, at least in part, on (a) how closely the visual attributes associated with the first set item matches any of the visual attributes of the image (or palette); and/or (b) how many of the visual attributes of the image (or palette) the visual attributes associated with the first set item matches. Block696may also order items classified within the second set based, at least in part, on: (a) explicitly programmed levels of complementarity associated with the first set items and the second set items; (b) explicitly programmed levels of complementary of the item category (taxonomy) associated with the second set items with the item category (taxonomy) associated with the first set items; (c) how closely the visual attributes associated with the second set item matches any of the visual attributes of the image (or palette); and/or (d) how many visual attributes of the image (or palette) match the visual attributes associated with the second set item. Block690may determine how closely two visual attributes match in a manner similar to block652described above. Block682may retrieve explicitly programmed levels of complementarity in a manner similar to block661described above and may determine whether (and how closely) two visual attributes match in a manner similar to block652described above.

In embodiments where block696directs the microprocessor102to order items utilizing the model-based ranking, the ranking of different item entries171may be determined using the ranking model in a manner similar to block661described above. Block696may thus also utilize the ranking model to processes historical entries stored in the application database122(shown inFIG.2), such as interaction history entries251(shown inFIG.10), combination history entries291(shown inFIG.11), and purchase history entries271(shown inFIG.12) for example, to determine order of items within both the first and second sets.

The recommend items codes650then return to block662, which include code for directing the microprocessor102to display the first set of items and the second set of items simultaneously and proximate each other on the user device114, such as via the recommend items page700for example. In embodiments where the recommend items page700is displayed after block696(shown inFIG.27C), the modify query region702may display: (a) the no selection of any visual attributes (no selection determined at block654) by displaying the visual attribute representations708a-708fin an unmodified state, and (b) the no entered text query (no text query entered as determined at block684) in the query field710. Further, the item display region704may: (a) display item representations716a-716crepresenting the item entries171classified within the first set at block692in the first set region712in the order determined by block696, and (b) display item representations718a-718crepresenting the item entries171classified within the second set by block694in the second set region714in the order determined by block696. The recommend items codes650then continue from block662as described above.

As briefly described above, in some embodiments (not show), the item recommendation server100may enable a user to search for items using visual attributes which are associated with images which do not correspond to images uploaded by users via the process image codes550(shown inFIG.21). For example, a user may select visual attributes from a representation of all possible colors, or from a representation of a plurality of possible colors in a color hue, or from a representation of a plurality of textures and/or patterns. Such representations may be at least one image representation stored in the representation database124or may be an image generated by the microprocessor102automatically based on the visual attribute entries161stored in the visual attribute table160.

As also briefly described above, in some embodiments, rather than a user searching for items based on an image and the visual attributes associated with the image via the shop image page630(shown inFIG.25), item recommendation server100may allow a user to search for items based on a palette, and visual attributes selected from the palette. In some embodiments, the palette may originate from at least one image representation stored in the representation database124and the visual attributes of palette may represent visual attribute pre-generated from at least one image representation by a host of the item recommendation server100.

Referring generally to the header region362(labelled inFIG.14), when the user selects the palette selection button370, the user interface codes330direct the user device to display the palette selection page750shown inFIG.33. The palette selection page750allows a user to select a custom palette and generally displays a plurality of custom palettes.

Referring now toFIG.33, in the embodiment shown, the palette selection page750includes a palette selection region752. The palette selection region752may be vertically scrollable and displays a plurality of palette representations753-757which may each represent a palette entry191(shown inFIG.5) stored in the palette table190(shown inFIG.2). In certain specific embodiments, the palette representations753-757may correspond to palette representations stored in the representation database124(shown inFIG.1) directed to by a URI in the palette representation path fields196of the corresponding palette entries191. Each of the palette representations753-757may be selectable, and when the user selects one of the palette representations754a-754d, the user interface codes330may direct the user device114to display a custom palette page shown generally at760inFIG.34. In the embodiment shown, the user may select palette representation756, corresponding to a “rust” custom palette for example.

Referring now toFIG.34, the custom palette page760displays information stored in a selected palette entry191and may further allow a user to search for items associated with visual attributes that match visual attributes included in the palette or for images associated with visual attributes that match visual attributes included in the palette. The custom palette region762displays a visual attribute array763associated with the custom palette, a select all button765, a deselect all button766, a query field767, a shop button768and a search button770.

The visual attribute array763includes a plurality of visual attribute representations764a-764lrepresenting the visual attributes associated with the custom palette selected by the user. More specifically, the user interface codes330may display, as the visual attribute representations764a-764l, the representations stored in the representation database124(shown inFIG.2) directed to by a URI in the visual attribute representation path field168of the visual attribute entries161(shown inFIG.6) identified in the visual attribute identifier field193of the palette entry191representing the user selected palette. In the embodiment shown, the visual attribute array763includes ten visual attribute representations764a-764l, indicating that ten instances of visual attribute entries161are identified in the visual attribute identifier field193of the palette entry191representing the user selected “rust” palette756(shown inFIG.33). In other embodiments, the visual attribute array763may include a greater or a fewer number of visual attribute representations.

Each of the visual attribute representations764a-764lmay be selectable, and user selection of one or more of the visual attribute representations764a-764lmay allow the user to create a smaller visual attribute array from the large visual attribute array763. For example, the user may (1) not select any of the visual attribute representations764a-764l, wherein every visual attribute of the visual attribute array763forms the smaller visual attribute array; (2) select one of the visual attribute representations764a-764l, wherein only the selected visual attribute forms the smaller visual attribute array; (3) select more than one of the visual attribute representations764a-764l, wherein the selected visual attribute forms the smaller visual attribute array. The user may select all unselected visual attribute representations764a-764lof the visual attribute array763by selecting the select all button765and may de-select all selected visual attribute representations764a-764lby selecting the deselect all button766.

Similar to visual attribute representations636a-636f(shown inFIG.25) and708a-708f(shown inFIGS.28,31and32), when the user selects a particular visual attribute representation764a-764l, the user interface codes330may direct the user device114to display a modification of the selected visual attribute representation. For example, referring toFIG.34, in the embodiment shown, non-selected visual attribute representations764a-764fand764j-764khave circular outlines, but the selected visual attribute representations764g-764iare displayed as modified selected representations764g′-764i′ having a circular outline with a folded bottom-left corner. In other embodiments, the outlines of the representations may be different, different portions of the outlines of the representations may be folded when the representation is selected, and the selected and non-selected representations may be modified in an alternative or an additional manner.

Similar to the query fields638(shown inFIG.25) and710(shown inFIGS.28and30-32), the query field767is operable to receive a text query from the user entered via the user device114. The text query may correspond to one or more macro-item categories or micro-item categories stored in, respectively, the macro-item category field224and the micro-item category field226of one or more instances of the taxonomy entry221(shown inFIG.9). In certain embodiments, the user may (1) not enter any text query in the query field767; (2) may enter one text query in the query field767, or (3) may enter more than one text query in the query field767. In other embodiments, the user may not enter more than one text query in the query field767.

When the user selects the shop button768, the user interface codes330may direct the microprocessor102to search for items which are associated with visual attributes that match any selected visual attributes of the selected palette and which match or correspond to the entered text query, and the microprocessor102may be directed to executed the recommend items codes650(shown inFIGS.27A-27C). For example, the user interface codes330may transmit information from the custom palette page760, including the visual attribute array763, any selection of the visual attribute representations764a-764land any text query entered in the query field767by the user, to the recommend items codes650in a recommend items request, and the recommend items codes650may process the received recommend items request in a manner similar to that described above.

For example, in embodiments where the user does not select any of the visual attribute representations764a-764land every visual attribute of the large visual attribute array763forms the smaller visual attribute array, the recommend items codes650may (1) identify and retrieve, at block652, a plurality of items associated with a visual attribute which match at least one visual attribute of the large visual attribute array763, (2) determine at block654that the user did not single-select or double-select any of the visual attributes, and (3) continue from block684as described above depending on whether the user entered a text query in the query field767. Additionally or alternatively, in embodiments where the user selected one of the visual attribute representations764a-764land only the single visual attribute forms the smaller visual attribute array, the recommend items codes650may, (1) identify and retrieve, at block652, a plurality of items associated with a visual attribute which match the single visual attribute, (2) determine at block654that the user did not single-select or double-select the single visual attribute, and (3) continue from block684as described above depending on whether the user entered a text query in the query field767. Additionally or alternatively, in embodiments where the user selected more than one visual attribute representations764a-764land the selected visual attributes form the smaller visual attribute array, the recommend items codes650may (1) identify and retrieve, at block652, a plurality of items associated with a visual attribute which match at least one visual attribute of the smaller visual attribute array, (2) determine at block654that the user did not single-select or double-select any visual attributes, and (3) continue from block684as described above depending on whether the user entered a text query in the query field767.

When the user selects the search button770, the user interface codes330may direct the microprocessor102to search for images which are also associated with any selected visual attributes of the selected palette in a manner similar to that described above in connection with the visual attribute search page470(shown inFIG.17). For example, the user interface codes330may direct the microprocessor102to: (1) identify and retrieve, a visual attribute entry161(shown inFIG.6) corresponding to each of the selected visual attribute representations764a-764l; (2) identify and retrieve image entries141(shown inFIG.4) which identify the retrieved visual attribute entry161in the visual attribute identifier field146; and (3) direct the user device114to display the visual attribute search page470(shown inFIG.17) to display the retrieved image entries141as search results.

Referring generally to the navigation region366(labelled inFIG.14), when the user selects the shopping cart button406, the user interface codes330direct the user device114to display the shopping cart page generally at780inFIG.35. The shopping cart page780allows a user to view and edit items that the user selected for purchase. In some embodiments, the shopping cart page780may more specifically display such items in association with the image (or palette) that the user initially selected to shop from. The user may add certain items to the shopping cart by selecting the add-to-cart buttons717a,717b,719a,719b(shown inFIG.28) or742(shown inFIG.29) for example. Such items may be displayed, in the shopping cart page780, as items of an item collection which is associated with the image or palette that the user initially selected to shop from (such as the image634(shown inFIG.25) or the “rust” custom palette756(shown inFIG.33) for example) and the visual attributes associated with the image or palette (such as the visual attribute array635(shown inFIG.25) associated with the image634, or the large visual attribute array763(shown inFIG.34) associated with the “rust” custom palette756). Displaying items as item collections associated with an image or palette and/or the visual attributes associated with the image or palette may encourage the user to buy multiple different items which may be complementary to each other and which each match a visual attribute associated with the image or palette to form a visually appealing combination that is similar to the image or the palette.

Referring now toFIG.35, in the embodiment shown, the shopping cart page780includes a shopping cart region782. The shopping cart region782includes a plurality of item collections784and786, and a buy all button812. Each item collection784,786is displayed in the shopping cart region782as corresponding to a specific image or a specific palette. In the embodiment shown, each item collection784,786includes an image source indicator788, item indicators794and796, a subtotal indicator804, an edit collection button806, and a buy collection button810.

The image source indicator788displays information associated with the image entry141(shown inFIG.4) representing the image that the user initially selected to shop from (image634(shown inFIG.25) for example) or the palette entry191(shown inFIG.5) representing the palette that the user initial selected to shop from (the “rust” custom palette756(shown inFIG.33) for example). In the embodiment shown, the image source indicator788includes an image post link789, a user indicator790and a visual attribute array791. In other embodiments, the image source indicator788may display more or less information associated with the image entry141representing the initially selected image or the palette entry191representing the initially selected palette.

The image post link789may allow the user to navigate back to the image or palette that the user initially selected. When the user selects the image post link789, the user interface codes330may direct the user device114to display the image page440(shown inFIG.16) displaying information associated with the image that the user initially selected or to display the custom palette page760(shown inFIG.34) displaying information associated with the palette that the user initially selected. The user indicator790may display the username of another user that uploaded the image or customized the palette the current user initially selected. For example, in specific embodiments, the user indicator790may display the username stored in the username field137of a user entry131identified in the identifier stored in the user identifier field144of the image entry141representing the image the user initially selected. The user indicator790may also be selectable to allow the user to navigate to the user profile of the other user to view other images uploaded by the other user or other palettes customized by the user.

For example, in specific embodiments, when the user selects the user indicator790, the user interface codes330may direct the user device114to display the user profile page410(shown inFIG.15) of the other user. The visual attribute array791may include a plurality of visual attribute representations792a-792frepresenting visual attributes associated with the image or palette the user initially selected. The visual attribute representations792a-792fmay more specifically correspond to the visual attribute representations stored in the representation database124(shown inFIG.1) directed to by a URI in the visual attribute representation path fields168of visual attribute entries161(shown inFIG.6) identified in the visual attribute identifier field146of the image entry141representing the image or by a URI in the visual attribute representation path fields168of visual attribute entries161identified in the visual attribute identifier field193of the palette entry191representing the palette.

The item indicators794and796display information associated with the item entries171(shown inFIG.7) representing the items placed into the shopping cart by the user, such as by selecting the associated add-to-cart buttons717a,717b,719a,719bon the recommend items page700(shown inFIG.28) or the add-to-cart button742on the modified recommend items page700′ (shown inFIG.29). Each item indicator794,796corresponds to an item, and in embodiments where more than one item was placed into the shopping cart by the user, the item collection784includes more than one item indicator794,796. In the embodiment shown, each item indicator794,796includes an item representation797,800, an item name798,801and an item price799,802. In other embodiments, the item indicator794,796may display more or less information associated with the item entry171representing the item placed into the shopping cart.

The item representation797,800provides a visual representation of the item and may specifically correspond to the item representation stored in the representation database124(shown inFIG.1) directed to by a URI in the item representation path field177of the item entry171(shown inFIG.7). The item name798,801may display a name of the item and may specifically display at least a portion of the description stored in the description field176or the vendor description field178of the item entry171. The item price799,802displays the price of the item, and may specifically display the price stored in the price field179of the item entry171.

In the specific embodiment shown inFIG.35, the user navigates to the shopping cart page780after the user selected to shop from the image634on the shop image page630(shown inFIG.25), and then selected item representation716b′, representing a first item, from the first set region712and then item representation718a′, representing a second item, from the second set region714(shown inFIG.28). The shopping cart region782displays an item collection784as including a first item indicator794representing the first item and a second item indicator796representing the second item, and further displays the item collection784as associated with the image634and the visual attributes of the image634. More specifically, in the embodiment shown, the first and second item indicators794and796may display, from the item entries171representing the first item and the second item respectively, (a) the item representation stored in the representation database124(shown inFIG.1) directed to by a URI in the item representation path field177as the item representation797,800, (b) a portion of the description stored in the vendor description field178as the item name798,801, and (c) a price stored in the price field179as the item price799,802. The image source indicator788may display, from the image entry141(shown inFIG.4) representing the image634, (a) a hyperlink back to the image page440(shown inFIG.16) of the image entry141as the image post link789, (b) a username stored in the username field137of a user entry131identified in the identifier stored in the user identifier field144as the user indicator790and (c) visual attribute representations stored in the representation database124(shown inFIG.1) directed to by the URIs stored in the visual attribute representation path fields168of visual attribute entries161(shown inFIG.6) identified in the visual attribute identifier field146as the representation of visual attributes792a-792f.

Still referring to the item collection784, the subtotal indicator804provides a subtotal of the prices displayed in the item prices799,802of the item indicators794,796and thus generally corresponds to the total price to purchase all of the items in a particular item collection784,786. The edit collection button806allows a user to remove items from a particular item collection784, and may further allow a user to change options associated with the item (such as clothing size, shoe widths, and furniture configurations stored in the options field180of the item entry171representing the item) and/or to change a purchased quantity of the item.

The buy collection button810enables the user to purchase the all items in a particular item collection784,786. For example, when the user selects the buy collection button810, the user interface codes330may direct the microprocessor102to communicate with the vendor server116(shown inFIG.1) or directly with the payment processor117(shown inFIG.1) via information stored in the purchase path field181of the item entries171representing each item of the item collection784,786, to facilitate purchase of each item in that item collection784,786.

Still referring toFIG.35, the buy all button812enables the user to purchase all items in the shopping cart, and thus all items of all item collections784,786displayed on the shopping cart page780. For example, when the user selects the buy all button812, the user interface codes330may direct the microprocessor102to communicate with the vendor server116, or directly with the payment processor117, via information stored in the purchase path field181of the item entries171representing each item of each item collection784,786displayed on the shopping cart page780, to facilitate purchase of each item.

As described above in connection with determining which items are complementary with other items or with the text query (see blocks660,674and688described above for example) and determining the order of items in the first and second sets (see blocks661,667,676,682,690and696described above for example), each time the user selects either the buy collection button810or the buy all button812of the shopping cart page780, the user interface codes330may direct the microprocessor102to add a new instance of the purchase history entry271(shown inFIG.12) to the purchase history table270(shown inFIG.2). The new instance of the purchase history entry271functions as a record indicating that a particular user decided to purchase at least one item after being directed to that at least one item from a particular image (or palette). In certain embodiments, the purchase history entry271may also function as a record indicating that the particular items were purchased based on a selected visual attribute and/or an entered text query.

The new instance of the purchase history entry271stores, in the user identifier field273, an identifier identifying the user entry131(shown inFIG.3) representing the user whom purchased the items (such as the user who logged on using the login page350(shown inFIG.13) for example). The new instance of the purchase history entry271also stores, in the item identifier field274, at least one identifier identifying at least one item entry171(shown inFIG.7) representing the at least one item purchased by the user. In embodiments where the user purchases more than one item, the item identifier field274may store more than one identifier identifying more than one item entry171. The new instance of the combination history entry291also stores, in the image identifier field296, an identifier identifying the image entry141(shown inFIG.6) representing the image that the user initially selected to shop from (image634of the shop image page630(shown inFIG.25) for example) or an identifier identifying the palette entry191(shown inFIG.5) representing a customized palette that the user initially selected to shop from (a “rust” palette selected from a palette selection page750(shown inFIG.34) for example). The new instance of the purchase history entry271may also store, in the created field278, a time obtained from the clock104(shown inFIG.1) generally corresponding to the time the instance of the purchase history entry271was created.

In certain embodiments, and specifically in embodiments where the user single selects or double selects a visual attribute (block654of the recommend items codes650) using either the visual attribute representations636a-636fdisplayed in the shop image page630(shown inFIG.25) or using the visual attribute representations708a-708fof the recommend items page700(shown inFIG.28), the new instance of the purchase history entry271may further store, in the visual attribute identifier field276, the identifier identifying the visual attribute entry161(shown inFIG.6) representing the selected visual attribute. Further, in certain embodiments, and specifically in embodiments where the user enters a text query (blocks656,670and684of the recommend items codes650) using either the query fields638(shown inFIG.25) or the710(shown inFIGS.28and30-32), the new instance of the purchase history entry271may further store, in the taxonomy identifier field298, an identifier identifying the taxonomy entry221(shown inFIG.9) representing or generally corresponding to the text query entered by the user.

In general, embodiments such as those described above may facilitate customized recommendation of items based on visual attributes associated with the items and complementarity of items. Embodiments such as those described above may use (1) images received from users and (2) items representations of items received or retrieved from vendors to generate visual attributes associated with the images and the items. Embodiments described above then present items in an automatically generated first set of items and in an automatically generated second set of items different from the first set of items. The first set and the second set are displayed simultaneously and proximate each other to facilitate user perusal of options for an item that they searched for (the first set items for example) as well as different and possibly complementary items (the second set items for example). Some embodiments described above automatically retrieve first set items associated with visual attributes which match one visual attribute of an image and second set items associated with visual attributes which match other visual attributes of the image, such that all items of the first and second sets may have generally cohesive and complementary visual attributes that are based on a specific image. Further, some embodiments described above may automatically retrieve second set items which are different from the first set items but are complementary to the first set items. The user may then be automatically presented with, and may select or combine or purchase, items associated with complementary visual attributes (color, pattern, textures, etc.) and which are also complementary (of a same item category, of a matching item category, from a same vendor, etc.) to each other.

By presenting items in a first set and a complementary second set, the embodiments described above may facilitate an improved online shopping experience by allowing the user to view items in a manner that allows the user to visualize desirable ensembles, may further increase the likelihood that a user will purchase more items than the items the user initially searched for, and may provide a more efficient and targeted online shopping experience when compared to other methods of online shopping.

While specific embodiments have been described and illustrated, such embodiments should be considered illustrative of the subject matter described herein and not as limiting the claims as construed in accordance with the relevant jurisprudence.