Systems and methods for personalized banner generation and display

One or more computing devices, systems, and/or methods for personalized banner generation and display is provided. For example, an image comprising a product object depicting a product is identified. A background of the image is identified and removed. Features of the product object are evaluated to determine a product type of the product. A new background is selected based upon the product type. A banner comprising the new background and the product object positioned over the new background at a position within the banner is generated. The banner is rendered on a display of a computing device through a user interface.

BACKGROUND

Many users access content through computing devices, such as smart watches, mobile phones, computers, and/or a variety of other electronic devices. A user may access content that is displayed through a user interface, such as a web browser displaying a website or an application displaying application content. While a user accesses content through the user interface, it may be efficient to show additional relevant and/or interesting content to the user through the user interface. For example, information about a product that may be interesting to the user may be displayed through a banner populated within the user interface. Because there may be millions of different products and interests of each user, manual generation of individual banners for every situation, product, and user is not feasible, and would consume a large amount of computing resources and human manual effort to create and store the banners.

SUMMARY

In accordance with the present disclosure, one or more computing devices and/or methods for personalized banner generation and display are provided. In an embodiment, a determination is made that a user is accessing content through a user interface displayed on a computing device. Information about the user, such as browsing history, purchase history, location, profile information, age, gender, social network data, AD interactive data, mail content, calendar data, etc. is used to determine that the user has an interest. An image relating to the interest may be obtained. The image is processed, such as by an unsupervised image segmentation process to remove a background of the image and identify a product object, within the image, depicting a product. Other content, such as logos, text, or other objects are also removed from the product as part of removing the background. In this way, the product object is isolated for further processing.

Features of the product object are evaluated to determine a product type of the product. For example, an image recognition net (e.g., an image recognition net VGC-19) processes features of the product object (e.g., shape, color, and/or other product identifying features extractable from the image) to select the product type of the product from a list of product types. A new background is selected based upon the product type or a theme which is manually assigned. Supplemental content (e.g., a price of the product, a product description, etc.) and/or decorator objects (e.g., overlay imagery, such as a sun object, a cloud object, a tree object, etc. that may be overlaid the new background) may be acquired based upon the product type. In this way, a banner is generated to comprise the new background. The product object, the supplemental content, and/or the decorator objects are inserted into the banner at select locations.

A visual adjustment is applied to the banner, such as to modify a brightness, a saturation, a size, a position, a color, and/or other visual properties of the new background, the product object, the decorator objects, and/or the supplemental content. The banner is machine generated by a computer in a manner that is aesthetic and personalized to the user. In this way, the banner is rendered through the user interface on the display of the computing device.

DETAILED DESCRIPTION

1. Computing Scenario

FIG. 1is an interaction diagram of a scenario100illustrating a service102provided by a set of servers104to a set of client devices110via various types of networks. The servers104and/or client devices110may be capable of transmitting, receiving, processing, and/or storing many types of signals, such as in memory as physical memory states.

The servers104of the service102may be internally connected via a local area network106(LAN), such as a wired network where network adapters on the respective servers104are interconnected via cables (e.g., coaxial and/or fiber optic cabling), and may be connected in various topologies (e.g., buses, token rings, meshes, and/or trees). The servers104may be interconnected directly, or through one or more other networking devices, such as routers, switches, and/or repeaters. The servers104may utilize a variety of physical networking protocols (e.g., Ethernet and/or Fiber Channel) and/or logical networking protocols (e.g., variants of an Internet Protocol (IP), a Transmission Control Protocol (TCP), and/or a User Datagram Protocol (UDP). The local area network106may include, e.g., analog telephone lines, such as a twisted wire pair, a coaxial cable, full or fractional digital lines including T1, T2, T3, or T4 type lines, Integrated Services Digital Networks (ISDNs), Digital Subscriber Lines (DSLs), wireless links including satellite links, or other communication links or channels, such as may be known to those skilled in the art. The local area network106may be organized according to one or more network architectures, such as server/client, peer-to-peer, and/or mesh architectures, and/or a variety of roles, such as administrative servers, authentication servers, security monitor servers, data stores for objects such as files and databases, business logic servers, time synchronization servers, and/or front-end servers providing a user-facing interface for the service102.

Likewise, the local area network106may comprise one or more sub-networks, such as may employ differing architectures, may be compliant or compatible with differing protocols and/or may interoperate within the local area network106. Additionally, a variety of local area networks106may be interconnected; e.g., a router may provide a link between otherwise separate and independent local area networks106.

In the scenario100ofFIG. 1, the local area network106of the service102is connected to a wide area network108(WAN) that allows the service102to exchange data with other services102and/or client devices110. The wide area network108may encompass various combinations of devices with varying levels of distribution and exposure, such as a public wide-area network (e.g., the Internet) and/or a private network (e.g., a virtual private network (VPN) of a distributed enterprise).

In the scenario100ofFIG. 1, the service102may be accessed via the wide area network108by a user112of one or more client devices110, such as a portable media player (e.g., an electronic text reader, an audio device, or a portable gaming, exercise, or navigation device); a portable communication device (e.g., a camera, a phone, a wearable or a text chatting device); a workstation; and/or a laptop form factor computer. The respective client devices110may communicate with the service102via various connections to the wide area network108. As a first such example, one or more client devices110may comprise a cellular communicator and may communicate with the service102by connecting to the wide area network108via a wireless local area network106provided by a cellular provider. As a second such example, one or more client devices110may communicate with the service102by connecting to the wide area network108via a wireless local area network106provided by a location such as the user's home or workplace (e.g., a WiFi (Institute of Electrical and Electronics Engineers (IEEE) Standard 802.11) network or a Bluetooth (IEEE Standard 802.15.1) personal area network). In this manner, the servers104and the client devices110may communicate over various types of networks. Other types of networks that may be accessed by the servers104and/or client devices110include mass storage, such as network attached storage (NAS), a storage area network (SAN), or other forms of computer or machine readable media.

1.2. Server Configuration

FIG. 2presents a schematic architecture diagram200of a server104that may utilize at least a portion of the techniques provided herein. Such a server104may vary widely in configuration or capabilities, alone or in conjunction with other servers, in order to provide a service such as the service102.

The server104may comprise one or more processors210that process instructions. The one or more processors210may optionally include a plurality of cores; one or more coprocessors, such as a mathematics coprocessor or an integrated graphical processing unit (GPU); and/or one or more layers of local cache memory. The server104may comprise memory202storing various forms of applications, such as an operating system204; one or more server applications206, such as a hypertext transport protocol (HTTP) server, a file transfer protocol (FTP) server, or a simple mail transport protocol (SMTP) server; and/or various forms of data, such as a database208or a file system. The server104may comprise a variety of peripheral components, such as a wired and/or wireless network adapter214connectible to a local area network and/or wide area network; one or more storage components216, such as a hard disk drive, a solid-state storage device (SSD), a flash memory device, and/or a magnetic and/or optical disk reader.

The server104may comprise a mainboard featuring one or more communication buses212that interconnect the processor210, the memory202, and various peripherals, using a variety of bus technologies, such as a variant of a serial or parallel AT Attachment (ATA) bus protocol; a Uniform Serial Bus (USB) protocol; and/or Small Computer System Interface (SCI) bus protocol. In a multibus scenario, a communication bus212may interconnect the server104with at least one other server. Other components that may optionally be included with the server104(though not shown in the schematic architecture diagram200ofFIG. 2) include a display; a display adapter, such as a graphical processing unit (GPU); input peripherals, such as a keyboard and/or mouse; and a flash memory device that may store a basic input/output system (BIOS) routine that facilitates booting the server104to a state of readiness.

The server104may operate in various physical enclosures, such as a desktop or tower, and/or may be integrated with a display as an “all-in-one” device. The server104may be mounted horizontally and/or in a cabinet or rack, and/or may simply comprise an interconnected set of components. The server104may comprise a dedicated and/or shared power supply218that supplies and/or regulates power for the other components. The server104may provide power to and/or receive power from another server and/or other devices. The server104may comprise a shared and/or dedicated climate control unit220that regulates climate properties, such as temperature, humidity, and/or airflow. Many such servers104may be configured and/or adapted to utilize at least a portion of the techniques presented herein.

1.3. Client Device Configuration

FIG. 3presents a schematic architecture diagram300of a client device110whereupon at least a portion of the techniques presented herein may be implemented. Such a client device110may vary widely in configuration or capabilities, in order to provide a variety of functionality to a user such as the user112. The client device110may be provided in a variety of form factors, such as a desktop or tower workstation; an “all-in-one” device integrated with a display308; a laptop, tablet, convertible tablet, or palmtop device; a wearable device mountable in a headset, eyeglass, earpiece, and/or wristwatch, and/or integrated with an article of clothing; and/or a component of a piece of furniture, such as a tabletop, and/or of another device, such as a vehicle or residence. The client device110may serve the user in a variety of roles, such as a workstation, kiosk, media player, gaming device, and/or appliance.

The client device110may comprise one or more processors310that process instructions. The one or more processors310may optionally include a plurality of cores; one or more coprocessors, such as a mathematics coprocessor or an integrated graphical processing unit (GPU); and/or one or more layers of local cache memory. The client device110may comprise memory301storing various forms of applications, such as an operating system303; one or more user applications302, such as document applications, media applications, file and/or data access applications, communication applications such as web browsers and/or email clients, utilities, and/or games; and/or drivers for various peripherals. The client device110may comprise a variety of peripheral components, such as a wired and/or wireless network adapter306connectible to a local area network and/or wide area network; one or more output components, such as a display308coupled with a display adapter (optionally including a graphical processing unit (GPU)), a sound adapter coupled with a speaker, and/or a printer; input devices for receiving input from the user, such as a keyboard311, a mouse, a microphone, a camera, and/or a touch-sensitive component of the display308; and/or environmental sensors, such as a global positioning system (GPS) receiver319that detects the location, velocity, and/or acceleration of the client device110, a compass, accelerometer, and/or gyroscope that detects a physical orientation of the client device110. Other components that may optionally be included with the client device110(though not shown in the schematic architecture diagram300ofFIG. 3) include one or more storage components, such as a hard disk drive, a solid-state storage device (SSD), a flash memory device, and/or a magnetic and/or optical disk reader; and/or a flash memory device that may store a basic input/output system (BIOS) routine that facilitates booting the client device110to a state of readiness; and a climate control unit that regulates climate properties, such as temperature, humidity, and airflow.

The client device110may comprise a mainboard featuring one or more communication buses312that interconnect the processor310, the memory301, and various peripherals, using a variety of bus technologies, such as a variant of a serial or parallel AT Attachment (ATA) bus protocol; the Uniform Serial Bus (USB) protocol; and/or the Small Computer System Interface (SCI) bus protocol. The client device110may comprise a dedicated and/or shared power supply318that supplies and/or regulates power for other components, and/or a battery304that stores power for use while the client device110is not connected to a power source via the power supply318. The client device110may provide power to and/or receive power from other client devices.

In some scenarios, as a user112interacts with a software application on a client device110(e.g., an instant messenger and/or electronic mail application), descriptive content in the form of signals or stored physical states within memory (e.g., an email address, instant messenger identifier, phone number, postal address, message content, date, and/or time) may be identified. Descriptive content may be stored, typically along with contextual content. For example, the source of a phone number (e.g., a communication received from another user via an instant messenger application) may be stored as contextual content associated with the phone number. Contextual content, therefore, may identify circumstances surrounding receipt of a phone number (e.g., the date or time that the phone number was received), and may be associated with descriptive content. Contextual content, may, for example, be used to subsequently search for associated descriptive content. For example, a search for phone numbers received from specific individuals, received via an instant messenger application or at a given date or time, may be initiated. The client device110may include one or more servers that may locally serve the client device110and/or other client devices of the user112and/or other individuals. For example, a locally installed webserver may provide web content in response to locally submitted web requests. Many such client devices110may be configured and/or adapted to utilize at least a portion of the techniques presented herein.

2. Presented Techniques

One or more systems and/or techniques for personalized banner generation and display are provided. In particular, the present system provides for machine banner design by a computer that can automatically design and generate personalized banners (e.g., on the fly in response to a user accessing a user interface through which the banner will be displayed) with human aesthetic using machine intelligence. Personalized banners provide an improved user experience because users will be more likely to interact with banners personalized to the interests and design tastes of the users. Otherwise, it is too manually and computer resource intensive for humans to use computers to generate millions of banners personalized for every possible situation that may occur, such as due to millions of different products and users with different tastes and interests, product information such as price or quantity that quickly change, etc.

As will be further discussed, the present system is configured to automatically generate personalized banners by performing background removal of an image depicting a product, identifying suitable new background candidates and decorators, arranging and constructing a banner based upon a product type of the product, and applying color schemes to the banner. The present system is configured to generate personalized banners in real-time when users access user interfaces, such as browsing to a website. The present system is configured to automatically gather mass layer data for the purpose of machine learning training for how to design and generate banners. The present system is configured to construct an ecosystem (repository) of personalized banners, such that user feedback (e.g., interaction with a banner or a user ignoring a banner) is used to train and modify a recommendation module used to design and generate banners.

The present system is configured to auto-resize and/or auto-crop personalized banners based upon user browsing behaviors and/or attributes of a device (e.g., a screen size, a view orientation, a resolution, etc.). The present system is configured to automatically generate and personalize banners for content campaign execution when needed (e.g., each action performed for generating and personalizing banners can be turned on or off as required). The present system is configured to generate banner layouts based on user scenarios, such as where different banner layouts are used based upon location within a user interface at which a banner is to be displayed. The present system is configured to generate banner layouts based on user feedback, such as where different banner layouts are used for different users based upon user reactions to banners in the past.

The present system is configured to perform background removal by detecting a main item (e.g., a product object) within an image so as to define background areas to be removed from the image. Areas in the image are segmented based upon saturation and brightness. Background areas are removed based upon item detection and the segmented areas, while an area comprising the main item is retained for further processing.

The present system is configured to apply a color scheme to a banner. In particular, items, backgrounds, decorators, text, and/or other elements (objects) are separated and colored based upon different hues on a color wheel. The present system is configured to determine a location at which the main item (e.g., the product object) is to be positioned in the banner based upon a geometric center of the item in an aesthetic manner. The present system is configured to distribute elements on the banner in an aesthetic manner. The present system is configured to create and review banners through user feedback to enhance aesthetics of banner generation using machine learning to process the user feedback.

An embodiment of personalized banner generation and display is illustrated by an example method400ofFIG. 4, which is described in conjunction withFIGS. 5A-5F. In an example, a user may use a computing device, such as a mobile device, a computer, a wearable device, a smart watch, etc., to access a user interface populated with content of a swimming team website, as illustrated byFIG. 5A. In an embodiment, a recommendation module504(e.g., hardware and/or software executing one or more computers for designing, generating, and displaying/rendering personalized banners) may detect508the user accessing the user interface. In an embodiment, an identity of the user is identified, if possible, so that user information can be obtained and evaluated to determine interests of the user (e.g., a browsing history of visiting music websites, a social network post about wanting recommendations for a guitar to purchase, an email from a music store, a calendar entry about band practice, user profile information such as age and gender, etc.). For example, the user information may indicate that the user has an interest in guitars.

The recommendation module504is configured to identify510an image512depicting a product, as illustrated inFIG. 5B. For example, the recommendation module504identifies510the image512based upon the image512depicting a guitar and the user having the interest in guitars. The image512may be identified from an image repository of images depicting products or identified from any other content source. In this way, images are identified based upon the images relating to interests of users.

The image512is processed in order to generate a personalized banner that the particular user will likely find to be aesthetic and comprising relevant content, thus increasing a likelihood that the user will want to interact with the banner.

At402, a background of the image512is removed516, as illustrated byFIG. 5C. In an embodiment, an unsupervised image segmentation process514is executed by the recommendation module504to distinguish a first portion of the image512as depicting the product, such as a guitar product object518depicted within the image512, from a second portion of the image512depicting the background (e.g., a sun object, a tree object, a sky object, a grass object, a cloud object, and/or other objects other than the guitar product object518). In particular, the unsupervised image segmentation process514performs edge detection to detect edges of objects depicted within the image512, such as the cloud object, the guitar product object518, the sun object, etc. The unsupervised image segmentation process514calculates areas of each object, such as a pixel count of pixels representing each object. In some examples, images for a product may depict the product at a size that is larger than other objects depicted within the images, the guitar product object518is identified as depicting a product based upon the guitar product object518having an area larger than areas of other objects depicted within the image512, for example. It may be appreciated that a variety of other techniques may be used to detect areas of an image as depicting a product (e.g., a machine learning model may be used for image recognition such as feature extraction and entity recognition). In an embodiment, multiple product objects may be identified if there is a set of objects having a relatively larger area than other objects (e.g., an image depicts multiple products).

The objects other than the guitar product object518are removed from the image512, such as removal of the sun object, the cloud object, the grass object, the tree object, etc. as part of removing516the background. An object may be removed by assigning an average color of surrounding pixels to pixels of the removed object.

At404, features of the guitar product object518are evaluated to determine522a product type of the product, as illustrated byFIG. 5D. In particular, the recommendation module504maintains a product type list506of various product types and/or features indicative of such product types (e.g., punk, high tech, sweet pink, or any other type of product). The features may correspond to shape, size, color, and/or a wide variety of other features that can be extracted from images for detecting a product type of a product. In an example, an image recognition net520(e.g., image recognition net VGC-19) or other machine learning technique is used to process the features extracted from the image512to identify a corresponding product type from the product type list506for the guitar product object518, such as a music product type.

The product type may be used to select a new background and acquire524decorators and supplemental content to use for constructing a personalized banner to display to the user, at illustrated byFIG. 5E. In an embodiment, the new background is selected based upon a theme that can be manually assigned. In an embodiment of acquiring524supplemental content, a term frequency inverse document frequency algorithm is executed to calculate scores for a plurality of supplemental content (e.g., images, text, audio, a video, product pricing, available product quantity, product description, keywords about a product, and/or other content available from various content sources such as a repository of supplemental content, a website, a web service, etc.) based upon how much each supplemental content corresponds to a product title of the guitar product or other identifying information of the guitar product. In this way, one or more supplemental content items are selected based upon the one or more supplemental content items having scores above a threshold (e.g., highest scores). For example, the phrase “Axe Guitar!!” is acquired524from a website to use as supplemental content for generating the banner.

In an embodiment of acquiring decorators, one or more decorators are selected from a data source (e.g., a repository of decorators, a website, a web service, etc.) based upon the one or more decorators corresponding to the product type or other identifying information of the guitar product. For example, music note objects are acquired524from the data source to use as decorators for generating the banner. In an embodiment, the new background is selected based upon the product type of the guitar product, at406. The new background may comprise shapes, patterns, drawings, colors, video, imagery, or other content that relates to the music product type.

At408, a banner528is constructed526using the new background, the guitar product object518, and/or any decorators or supplemental content acquired for the banner528, as illustrated byFIG. 5F. In an embodiment, a layout for the banner528is determined, such as sizes and locations of where the guitar product object518, the decorators such as the music note objects, and/or the supplemental content such as the phrase “Axe Guitar!!” are to be inserted into the banner528. The layout may be based upon a location of where the banner528will be displayed within the swimming website502. The layout may be based upon prior user interaction with content, such as where the user previously interacted with banners having product objects positioned in the middle of banners and having an extra-large size and where the user previously ignoring banners having smaller sized product objects not positioned in the middle of banners. Thus, the guitar product object518may be placed in the middle of the banner528, and a size of the guitar product object518may be increased. In this way, the user may be more likely to interact with the banner528because the layout of the banner528is personalized to the user. In an embodiment, the banner528is automatically sized and/or positioned within the swimming website502based upon user browsing behavior of the user (e.g., the user may generally not scroll down through websites, and thus the banner528may be positioned towards a top of the swimming website502so that the user will notice the banner528; the user may generally ignore larger sized banners, and thus the banner528size may be decreased; etc.).

In an embodiment of constructing526the banner528, a visual adjustment is applied to the banner528, such as to adjust a saturation, a brightness, or other visual property (e.g., color, tone, size, etc.). The visual adjustment may be applied to the new background, the guitar product object518, decorators, supplemental content, etc. In an example, a main color of the guitar product object518is detected. The main color is transferred from a red, green, blue (RGB) color space to a huge, saturation, value (HSV) color space. Complementary colors and/or analogous colors of the main color are identified using the HSV color space (e.g., complementary colors are within 180 degree difference of the main color and analogous colors are within 120 degree difference of the main color). The visual adjustment is applied to the banner528based upon the complementary colors and/or the analogous colors, such as by recoloring objects, decorators, supplemental content, and/or the new background based upon the colors.

The banner528is transmitted530, such as over a network, to the computing device of the user for rendering through the swimming website502on a display of the computing device. In an embodiment, user interaction with the banner528through the swimming website502is monitored, such as whether the user clicks on the banner528, quickly scrolls past the banner528, scrolls to a position where the banner528is prominently displayed/viewable through the swimming website502, etc. The recommendation module504, such as the various machine learning techniques implemented by the recommendation module504to select the new background, generate the banner528, apply a visual adjustment to the banner528, select decorator objects or supplemental content, identify the product type, etc., is updated based upon the user interaction. For example, parameters and/or other logic of the recommendation module504may be adjusted based upon the user interaction (e.g., values of parameters used to create the banner528may be maintained or weighted higher based upon the user interacting with the banner528or the values may be changed or weighted lower based upon the user not interacting with the banner528). In this way, the creation of personalized banners is improved.

FIG. 6is an illustration of a scenario600involving an example non-transitory machine readable medium602. The non-transitory machine readable medium602may comprise processor-executable instructions612that when executed by a processor616cause performance (e.g., by the processor616) of at least some of the provisions herein. The non-transitory machine readable medium602may comprise a memory semiconductor (e.g., a semiconductor utilizing static random access memory (SRAM), dynamic random access memory (DRAM), and/or synchronous dynamic random access memory (SDRAM) technologies), a platter of a hard disk drive, a flash memory device, or a magnetic or optical disc (such as a compact disk (CD), a digital versatile disk (DVD), or floppy disk). The example non-transitory machine readable medium602stores computer-readable data604that, when subjected to reading606by a reader610of a device608(e.g., a read head of a hard disk drive, or a read operation invoked on a solid-state storage device), express the processor-executable instructions612. In some embodiments, the processor-executable instructions612, when executed cause performance of operations, such as at least some of the example method400ofFIG. 4, for example. In some embodiments, the processor-executable instructions612are configured to cause implementation of a system, such as at least some of the example system500ofFIGS. 5A-5F, for example.

3. Usage of Terms