Is AI Disrupting Online Advertising?
By: Pankaj Kohli
Tags: Online Advertising, Artificial Intelligence, Convolutional Network, Click Through Rate, Return On Investment

Introduction

Deep learning based artificial intelligence (AI) is disrupting the insurance sector. Can it also disrupt online advertising ?

We start by reviewing the AI led innovations in the insurance sector. We outline the image based deep learning algorithms or convolutional neural networks (CNNs) that are leading these innovations and enabling us to recognize images in our daily lives. The online advertising business model and its dependence on being able to understand image ads to improve its prediction engine is highlighted. We further explain how CNNs are being trained to decode ads and the challenges of cultural context in these efforts.

AI and Insurance

A recent New York Times article (link to the article) highlighted how deep learning algorithms were disrupting the insurance sector.

It described how insurers are in the process of leveraging submitted vehicle damage photographs to automatically generate initial repair estimates. This is in contrast to the established practice of using an adjuster to inspect the damage either through photographs or in person, and then generate those initial estimates.

Image recognition deep algorithms were used to parse the provided automobile damage photographs to generate those initial estimates. The algorithmic approach reduces the claim settlement cycle considerably by eliminating the role of an adjuster, with it taking only a few seconds now to generate the initial claim estimates.

Image Recognition Deep Learning Algorithms

These developments highlight how image recognition deep learning algorithms or convolutional neural networks (CNNs) are making significant inroads in our transactional lives.

CNNs are already enabling us to process images and multi-media videos by leveraging complex interactions through additional layers. They are classifying various objects in images (including recognizing hand written digits and facial expressions).

As a result, CNNs are at the forefront of making self-driving cars a reality in the near future.

Similarly, video hosting websites are leveraging CNN to better recognize, index, search, and recommend an increasing number of uploaded videos.

Many researchers have also started leveraging CNN on breast cancer image data in efforts to improve detection rates through visual inspection. CNN driven apps are also being used to scan X-rays for signs of tuberculosis early on in India in remote locations where qualified radiologists are in short supply.

Most recently, Microsoft released a free deep learning app Lobe supporting image classification.

What is a CNN

CNN is basically a classifier that processes an image and classifies it to an image class. For example, CNN may learn from thousands of different labeled tree images to be able to correctly classify an unlabeled tree image.

Object Detection with CNN

In the shown figure (source), the CNN is able to correctly identify the imbedded objects of a dog, a person, a horse and a car after being trained on thousands of such tagged images.

How does it do that? The algorithm breaks the image into different square pixels of various gradations (i.e., it pixelates the image) and these unique pixels with unique locations serve as predictors for the algorithm.

In our earlier insurance example, CNN pixelates the provided photographs and is able to recognize the damaged part and its extent of damage and then recommends the claim estimate.

Online Advertising Trends

Thanks to near universal internet access, digital advertising spends have exceeded the combined offline media spends including TV advertising, print ads, and billboards. Digital advertising now accounts for roughly 55% of the market share in the United States.

Most of these online advertising spends are devoted to search and display advertising. Roughly $55 billion was spent on search ads and $36 billion on display ads in the US in 2019. By some estimates, over 1700 banner ads are shown to an average consumer in a month with a 50% click through rate (CTR).

Display ad spending is growing at a faster rate than search ad spends. This is expected since display advertising allows excellent customer targeting and richer multimedia executions in comparison to search advertising.

These ads play a very important role in generating brand awareness and moving the prospective customer along the discovery stage of the sales funnel.

Both search and display ads are sold through real time bidding (RTB) auctions where advertisers bid for the placement of their ad at a favored rank / location at the search engine results page, the publisher websites or social network feeds etc.

Search engines / publishers get paid by advertisers whenever the targeted customer clicks the shown ad and reaches the advertiser website. Therefore, both the search engines and the publishers are interested in showing ads that have a higher probability of the ad being clicked when shown to the target.

Ad platforms predict and rank the CTR of the available ads and show ads with the highest CTR.

Better CTR prediction allows the search engines / publishers to show the relevant ad to the target that results in click through behavior and revenue for the advertisers. Displaying a relevant ad also increases web experience.

Thus, CTR prediction plays an important role in defining the keyword / webpage location and customer profile combination bid price and is considered a core task in ad allocation.

How do we predict CTR?

Various machine learning algorithms (including logistic regression and decisions trees) are used to predict ad level CTR. These algorithms leverage online behavior related features including:

keyword actionability (searching for hotels in San Francisco vs San Francisco)

ads with higher historical CTRs

display rank

other contextual features such as text and traffic volume for the landing page.

target average CTRs across different search ads

earlier target search / browsing / purchase / reviews behavior where available

target demographic characteristics such as locations, age, and gender and

more general constructs such as target interests topics and target intent: browsing, knowledge building, search, or buying.

Essentially, behavioral targeting (BT) is used to show the most relevant ads or ads with the highest CTR to the targeted consumer. BT leverages the prospective consumer’s past history including searches, pages visited and past purchases.

Target user interests can also be derived by combining different elements of behavior including websites visited, past searches, and reviews. User interests in combination with demographic details can provide insights to the user profile and underlying segment affiliation. This can enable further customization of the shown advertisement.

CNN and CTR

Till recently, ad level features had not played a significant role in the crucial task of CTR predictions. Context and user level features including demographics and historical online behavior were generally used to make such predictions.

Advances in CNN are now helping advertisers leverage ad level features in these CTR predictions.

It is easy to train a CNN to identify various imbedded objects (and at what location) in a display ad.

More exploratory analysis can be done by identifying the imbedded objects algorithmically and asking:

which ads are similar to each other?

which ads are more text oriented and which ads are more image oriented:

among the image oriented ads, which ads are focusing on the product and which ads are focusing more on lifestyle?

what imbedded objects are repeated most across ads in a given product category?

Most importantly, CNN enables advertisers to understand how ad level features including various imbedded objects in different positions are driving the CTR of an ad. Following common types of ads have been observed from this perspective:

text ads, in which most of the ads contain mainly text, rather than images;

models ads, in which images of people are the primary focus of the ad, and in many cases these people are performing various actions or standing in various poses; and

product ads, in which an image of the product is the focus of the ad, something especially common in ads under the Auto category.

Understanding Ads

Leveraging CNN to understand persuasiveness of ads or any image is confronted by significant challenges.

User impression of an image (including ads) is driven by the persuasiveness of that image. Understanding the persuasiveness of an image is a much more challenging task than simply identifying the imbedded objects in that image.

Persuasiveness of ad images is not defined by the mere presence / absence of different objects and at what location. It is defined by how the imbedded objects, text, and sound (in case of multimedia visuals) interact to make the ad topical and relevant and deliver various emotions / sentiments for the target audience. These latent sentiments / emotions then combine nonlinearly to drive the overarching appeal of those visuals.

Decoding the causality from the imbedded objects in various rendered forms to the evoked emotions and then from those emotions to the overall appeal is a challenging task for any CNN algorithm. It requires the CNN to be trained on thousands of images that are tagged on various aspects including:

the imbedded objects

the sentiments / emotions conveyed by the visual

symbolism / rhetoric of the visual if any

the topicality of the visual or what the image is conveying

what action the visual wants the viewer to take and

why should the viewer take that action.

Visual Examples

Two Visual Examples

Let us take examples of two visuals to motivate the process and challenges of CNN learning.

The first picture consists of a tree with branches and leaves and a white background.

The second images shows a boy lying on the beach with a paper boat and moonlit waters in the background.

Recognizing imbedded objects

The trained CNN algorithm would be able to generally identify the imbedded objects of:

leaves, branches, and a tree against a grayish background in the first image and

a seafront, a sleeping boy, a paperboat, and the moon in the second image.

What is the ad saying

Beyond recognizing the individual objects, CNNs are being trained to convey the description of the image or what the image is saying.

In our examples, a trained CNN would describe

the first image as that of a tree and the second example as that of a child sleeping on the beach after possibly playing with a paper boat.

Sentiments

The algorithm needs to understand how the identified objects are interacting to generate different sentiments / emotions.

Sentiments drive the persuasiveness of the image in combination with its topicality / context.

Thus, image sentiments can help in understanding the ad beyond the location of the objects or the description of the image. Algorithmic image description can be modified by the sentiments and the rhetoric of the image that are defined by the imbedded objects.

In our examples, the tree image is evoking the sentiments of quiet, sparseness and gravity. Similarly, the boy on the beach image is conveying the sentiments of stillness, darkness and mystery.

Visual Rhetoric

Visual rhetoric flows from the choice of how the objects are shown and why they are portrayed so. It can completely change the intent of the imbedded objects and the photorealistic story line. Therefore, the ability to understand ads is critically dependent on the algorithm’s ability to automatically decode this rhetoric.

In our first example, symbolism is imbedded with the tree being shaped as two lungs of the planet. A deep learning algorithm can understand this symbolism only if it recognizes the branches shaped as lungs and is aware of saving the environment context. With this symbolism, description of the image would be updated as that of trees being portrayed as lungs and life of our planet / environment.

The algorithm finds it very difficult to understand the symbolism of the second image as the tragedy of boat refugees, since the image was created in response to the disturbing news of a 3-year-old Kurdish boy Aylan Kurdi washed up dead on a Turkish beach. With this symbolism, now the image description becomes that of the tragedy a dead child refugee washed on the shores.

Topicality

More generally, the algorithm is also being trained to recognize the context or topicality of the ad image that plays a critical role in defining its persuasiveness.

The topic of the image gains clarity once the visual rhetoric has been decoded.

The topic context for the first image is that of saving the environment and the plight of refugees in the second image.

Thus, in the first image, the topic of environment is being combined with the sentiments of urgency and seriousness with trees being portrayed as lungs and life of the environment / planet.

Similarly, in the second image, the topic of refugees is being paired with the sentiments of tragedy and sadness.

What does the ad image want us to do?

CNNs will further have to be trained to derive actionability from the combination of the topic and its sentiments.

The first image is suggesting to us to act on saving the environment. The second image is asking us to develop a better understanding of the tragedy and suffering of the refugees.

Conclusions

As the New York Times article highlighted, CNNs are disrupting the insurance sector by making it easier to understand the extent of claimed damage without active human intervention.

Are CNNs ready to disrupt online advertising as it continues to grow on the wings of more multimedia content? That depends on how effectively CNNs can contribute to predicting CTR by understanding the persuasiveness of the image or multimedia content exposed to the target.

AI and CNN architectures are rapidly evolving in their ability to decode images and multimedia creatives and understand their persuasiveness in eliciting desired targeted response.

CNNs have obtained excellence in identifying various imbedded objects in their varying degree of presence, e.g., recognizing different objects, recognizing damaged vehicle parts, recognizing cancer disturbance in x-rays etc.

Thus, CNNs can help CTR prediction incrementally where the presence / absence of objects are relevant to the ad.

At the same time, CNNs are still not fully evolved in decoding and understanding the persuasiveness of an ad image.

The biggest challenge CNNs face is to understand how the imbedded objects and rhetoric update different sentiments and topicality and then persuasiveness.

Decoding rhetoric requires a monumental task for a CNN to understand the cultural context changing over time. This can be seen as a possible problem of underspecification in the training samples with the samples not fully representing our cultural experience.

This suggests that we are a few years away before CNNs may be able to replicate the human eye appreciation of an ad image and its persuasiveness. It is only then that CNNs will be able to contribute significantly in CTR prediction and in impacting the online advertising business model.

Acknowledgement

The editorial contributions of Parvathi Kamath, Ph.D. are gratefully acknowledged. A fuller version of this article can be read here.