metadata: authors: 0: author: Berg,Gunhild
1: author: Zia,Bilal Husnain
count: World,South Africa
docna: 0: docna: Harnessing emotional connections to improve
            financial decisions : evaluating the impact of financial
            education in mainstream media
docty: Policy Research Working Paper
theme: Other social protection and risk management
majtheme: Social protection and risk management
subtopic: Private Sector Economics,Strategic Debt Management,External Debt,Educational Sciences,Gender and Development,Private Sector Development Law,Marketing,Debt Markets,Financial Literacy
teratopic: Finance and Financial Sector Development,Health, Nutrition and Population
abstracts: cdata!: This paper exploits the emotional
            connections and viewer attentiveness of mainstream media to
            evaluate the economic impact of financial education messages
            on debt management delivered through a popular television
            soap opera in South Africa. The study uses a symmetric
            encouragement design to compare outcomes of individuals who
            were randomly assigned to watch a soap opera with financial
            messages, "Scandal!" to those of individuals who
            were invited to watch a similar soap opera without financial
            messages, "Muvhango." Both shows overlapped in
            evening primetime and had similar past viewership profiles.
            The financial storyline spanned two months and featured one
            of the leading characters of the show borrowing excessively
            and irresponsibly through hire-purchase, gambling, and
            ending up in financial distress; and eventually seeking help
            to find her way out. Two intermediate and one final
            follow-up surveys were conducted as part of the study. The
            analysis finds individuals assigned to watch Scandal had
            significantly higher financial knowledge of the issues
            highlighted in the soap opera storyline, in particular
            messages delivered by the leading character. On behavior,
            Scandal viewers were almost twice more likely to borrow from
            formal sources, less likely to engage in gambling, and less
            prone to enter hire purchase agreements. Messages promoting
            a national debt mediation helpline delivered by an external
            character did not sustain traction beyond immediate
            interest. Three qualitative focus groups highlight the
            importance of emotional connections with the leading
            character in motivating behavior change.
geo_reg_key: 517195,517191
repnme: repnme: Harnessing emotional connections to improve
            financial decisions : evaluating the impact of financial
            education in mainstream media
display_title: Harnessing emotional connections to
            improve financial decisions : evaluating the impact of
            financial education in mainstream media
listing_relative_url: /research/2013/04/17561771/harnessing-emotional-connections-improve-financial-decisions-evaluating-impact-financial-education-mainstream-media
topicv3: Private Sector Development m1326060 1921,Education m1327048 470,Debt Management m1327211 451,Gender m1327621 615,Financial Literacy m1504979 3391
publishtoextweb_dt: 2016-09-12T00:00:00Z
historic_topic: Finance and Financial Sector Development,Health, Nutrition and Population
no_of_pages: 51
author_keys: 0: author_key: 1023788
1: author_key: 642766
model_extractions: WPS6407 Policy Research Working Paper 6407 Harnessing Emotional Connections to Improve Financial Decisions Evaluating the Impact of Financial Education in Mainstream Media _Gunhild Berg Bilal Zia_ The World Bank Africa Region Finance and Private Sector Development Unit & Development Research Group Finance and Private Sector Development Team April 2013
Policy Research Working Paper 6407

# **Abstract**

This paper exploits the emotional connections and viewer attentiveness of mainstream media to evaluate the economic impact of financial education messages on debt management delivered through a popular television soap opera in South Africa. The study uses a symmetric encouragement design to compare outcomes of individuals who were randomly assigned to watch a soap opera with financial messages, “Scandal!” to those of individuals who were invited to watch a similar soap opera without financial messages, “Muvhango.” Both shows overlapped in evening primetime and had similar past viewership profiles. The financial storyline spanned two months and featured one of the leading characters of the show borrowing excessively and irresponsibly through hire-purchase, gambling, and ending up in financial distress; and eventually seeking help to find her way out. Two intermediate and one final follow-up surveys were conducted as part of the study. The analysis finds individuals assigned to watch Scandal had significantly higher financial knowledge of the issues highlighted in the soap opera storyline, in particular messages delivered by the leading character. On behavior, Scandal viewers were almost twice more likely to borrow from formal sources, less likely to engage in gambling, and less prone to enter hire purchase agreements. Messages promoting a national debt mediation helpline delivered by an external character did not sustain traction beyond immediate interest. Three qualitative focus groups highlight the importance of emotional connections with the leading character in motivating behavior change.

This paper is a joint product of the Finance and Private Sector Development Teams of the Development Research Group and Africa Region. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http:// econ.worldbank.org. The authors may be contacted at gberg@worldbank.org and bzia@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Harnessing Emotional Connections to Improve Financial Decisions: Evaluating the Impact of Financial Education in Mainstream Media**

Gunhild Berg and Bilal Zia<sup>1</sup>

**JEL Codes:** C93, D03, D14, O12, O17

**Keywords:** Encouragement Design, Randomized Evaluation, Debt Management, Financial Literacy, Financial Education **Sector Board:** FSE

1 Both authors are from the World Bank. Corresponding author: bzia@worldbank.org. We would like to thank our direct project counterparts, Ochre Media, the production company of Scandal!, in particular Stan Joseph and Lebogang Ramafoko who coordinated the development of the storyline; the National Debt Mediation Association for supporting the project, in particular Magauta Mphalele; and TNS South Africa, in particular Bobby Berkowitz who offered keen intellectual insights into the survey design. We are also grateful for comments and input received from Claus Astrup, Michael Fuchs, Richard Hinz, Ruth Kagia, Elaine Kempson, Lawrence Kincaid, Thomas LosseMueller, Matthias Lundberg, David McKenzie, Margaret Miller, Florentina Mulaj, Leopold Sarr, Robert Walker, and the South Africa Advisory Panel consisting of representatives from National Treasury, the Financial Services Board, the National Credit Regulator, and the Credit Ombudsman. Special thanks also to Christian Salas Pauliac and Julia Thelen for exceptional research assistance. We gratefully acknowledge funding received from the World Bank Russia Trust Fund on Financial Literacy and Education.
# **1. Introduction**

Financial education is important, yet there is a considerable knowledge gap in determining how best to deliver it. The literature on careful evaluations of financial literacy is small but growing, and has moved away from classroom based interventions, such as Cole et al. (2011), to more innovative delivery mechanisms, such as videos and DVDs (Carpena et al., 2011). Yet, the scope and reach of even the best produced DVDs is limited on the supply side, and attracting viewership can be significantly challenging on the demand side.

Entertainment media – television and radio – offers not only broader outreach since nearly every household nowadays has a TV, but also a captive audience. Furthermore, as emotional connections are established between a show and its audience, the program provides a potentially powerful platform for communicating messages and influencing behavior.

There is considerable anecdotal and some survey evidence on the impact of mass media on behavior, especially in the areas of health and education. For instance, as Brazil’s Rede Globo television network grew through the 1970s and 1980s, women began having fewer children, and experienced the same decrease in fertility as with two extra years of education (La Ferrara, Chong and Duryea, 2008). Similarly, one of the most successful public health campaigns in Egypt, the central component of which was a soap opera, was highly successful in improving the use of Oral Rehydration Therapy and reducing infant mortality rates by 70% (Abdulla, 2004). In other studies, the introduction of television has been linked to decreased domestic violence and fertility rates in India (Jensen and Oster, 2009), and lower adolescent drug use and increased contraception adoption in Brazil (Verner and Cardoso, 2007).

The emotional connections channel is particularly central to media campaigns. Advertisers, for example, consistently and successfully use influences of emotions to affect consumer choices (Sitgler, 1987; Russo, Carlson, and Meloy, 2006; and Bagwell, 2007). In the financial context, Bertrand et al. (2010) and Agarwal and Ambrose (2007) show that persuasive advertising and direct mail solicitation can influence consumer loan contract choices. Perhaps the most direct anecdotal evidence on the power of emotional connections with television personalities comes from the US where Kennedy et al. (2004) find that the greatest number of calls to the Center for 2
Disease Control’s AIDS hotline in one year came not as a result of a million dollar public awareness campaign or through news and documentary broadcasts, but rather in response to a beloved soap opera character dealing with his HIV positive diagnosis.<sup>2</sup> This paper contributes to the literature by carefully analyzing the impact of financial education messages delivered through a popular South African television soap opera, “Scandal!” In partnership with the production company of Scandal, the soap storyline features one of the main characters borrowing excessively through hire-purchase, gambling, and falling into a debt trap; and eventually seeking help to find her way out and manage her debt responsibly. The storyline plays out over a two month period, and the study evaluates the effectiveness of these messages through two intermediate surveys and one final follow-up survey conducted four months later. Further, three qualitative focus groups provide insight into the mechanisms behind measured impacts.

The methodological challenge associated with such an evaluation is non-trivial. Critically, it is impossible to externally restrict viewership since the soap opera is nationally televised with no variation in TV signal strength, at least in urban areas where this study focuses. Even an encouragement design methodology is limited in scope as we can only encourage a set of individuals to view Scandal, but in no way can we prevent others (particularly those in the control group) from doing the same.

To overcome this constraint, we employ a symmetric encouragement design in this study. Specifically, while a randomly selected treatment group is encouraged through financial incentives to view Scandal, another set of individuals – the control group – is similarly encouraged through identical financial incentives to view a comparable soap opera (“Muvhango”) that overlaps with Scandal in television primetime. Hence, we are able to create an artificial separation between treatment and control groups, which enables us to identify the treatment effects of financial literacy delivered through Scandal.

> 2 See also Chong and Ferrara (2009) and Kenny (2009) for similar findings.

3
Our results are quite encouraging. First, we confirm that the overwhelming majority of individuals (96%) assigned to watch the two soap operas in fact watched them, with very little overlap – less than 12% of Muvhango viewers claim to have also watched Scandal during the study period, and a third of these viewers watched several other shows too. Moreover, such crossovers will only underestimate treatment effects.

We measure and study outcomes along the causal chain of impact, from financial knowledge to financial behavior, as per Carpena et al. (2011). The analysis shows significant improvements in content specific financial knowledge, a 4.5 percentage point increase, and no differences in knowledge of financial concepts that were external to the soap storyline.

Central to the theme of the soap was highlighting the high interest charges and hidden costs of borrowing through hire-purchase. Our analysis on financial behaviors finds a significant and substantial increase in the likelihood of borrowing through formal channels – 22 percent compared to 13 percent in the control group, and a greater likelihood to borrow for productive purposes such as investments in household durable goods and vehicles. We also directly measure hire-purchase use and find a significant reduction in the likelihood of entering such contracts – 15 percent compared to 19 percent in the control group. Further, we find a 5 percentage point lower propensity to gamble among the treatment group, yet another theme highlighted in the show. These are promising results and point to successful dissemination of themes depicted in the soap storyline.

One aspect of the soap storyline that did not sustain traction beyond immediate effects was a public call to action for the South African National Debt Mediation Association (NDMA). The soap featured an NDMA counselor who helps the main character build a financial plan to enable her to save enough and repay her debts. The actual NDMA toll-free helpline also appeared at the end of several shows. The findings on this public call to action are quite stark. While we see a huge uptake in the number of calls received within days of the NDMA being featured in the soap, the longer term analysis shows no recall of the NDMA or even memory that a debt counselor appeared in the soap storyline.

4
These quantitative results hint at the mechanism of our measured impacts. Specifically, the behavior change in our study tracks well with the storyline that featured the main soap character, who herself depicted poor financial behavior before transforming her habits. This character was part of the show prior to the two-month financial storyline and remained part of the show afterwards. The non-result on knowledge of the NDMA could at least partly be explained by the fact that the NDMA messages were delivered not by the main character, but by a different character who was external to the main soap cast and who appeared in 2-3 episodes as an NDMA counselor, and then disappeared. Hence, the audience did not get the opportunity to establish or maintain a connection or following with this character.

In order to further assess the merits of this emotional connections channel, we conducted three qualitative focus group discussions with study participants three months after the final quantitative survey. In these meetings, the focus group leaders were unable to elicit recall of NDMA from the participants and only a few women seemed to remember, and they associated their memory more with the fact that the NDMA character was female rather than her representation of the NDMA. In contrast, all participants readily recalled the main soap character and aspects of the financial storyline delivered by her. Upon further inquiry, they attributed better recall to the fact that the main character was a mainstay in the soap opera, and was popular and likeable.

Overall, our findings suggest an important role for entertainment media as an accessible and important tool for policymakers to deliver carefully designed educational messages that resonate with the audience and can potentially influence financial knowledge and behavior. Further, our findings suggest that emotional connections and familiarity with media personalities certainly play a role in motivating knowledge and behavior change among viewers, and that harnessing such potential can be an important channel for achieving development impact.

This paper is organized as follows. Section 2 provides a brief literature review on financial literacy and mass media. Section 3 focuses on financial education in South Africa and explains the study context and the soap opera storyline. Section 4 outlines the research methodology and 5
study timeline, and Section 5 presents summary statistics. Section 6 provides the main analysis and discussion of results, and Section 7 concludes.

# **2. Background and Literature**

# _2.1. General Financial Education Evaluations_

Making informed financial decisions and evaluating complex financial choices remains a challenge for large parts of the population across the world (Lusardi and Mitchell, 2007; Lusardi et al., 2010). While financial literacy training has been heavily promoted as a solution, evidence on its effectiveness is limited. A recent study finds muted evidence that financial education influences the demand for saving accounts in Indonesia (Cole et al., 2011). A more comprehensive video-based financial education program in India is associated with large and statistically significant improvements on knowledge and awareness of financial products and services, and some moderate improvements in savings rates (Carpena et al., 2011).

To date, debt literacy research has been limited to survey-based studies. Lusardi and Tufano (2009) find that debt literacy levels are generally low in the US, especially among women, and some groups perceive their knowledge to be higher than it actually is. More work has been published on the effects of mortgage counseling,<sup>3</sup> however, such facilities are rarely available in developing countries.

# _2.2. Mass Media for Edutainment_

Mass media outlets, including television and radio, have been used to successfully deliver educational messages in developing and developed countries (Kenny, 2009). For example, Verner and Cardoso (2007) find that access to TV lowers the probability of teen pregnancy and risk-taking in Brazilian favelas. Similarly, Jensen and Oster (2007) analyze the power of TV to shift cultural roles of women by studying the effects of satellite or cable service installation in Indian villages. Access to TV increases women’s autonomy and female school enrollment while fertility, tolerance of domestic violence, and the preference for sons appear to decrease.

> 3 See for example, Hartarska and Gonzalez-Vega, 2005; Agarwal et al., 2009, 2010; Hirad and Zorn, 2001; and Elliehausen et al., 2007.

6
Entertainment education through soap operas is quite popular in Spanish language telenovelas. The Peruvian series “Simplemente Maria” became a success in educating and inspiring people to take actions to improve their own lives (Singhal et al., 1995). Similarly, the U.S. soap opera “The Bold and the Beautiful” successfully reached at-risk minority women when a storyline was accompanied by display of a free hotline number (Spader et al., 1993). Further, after an HIV positive character was diagnosed – a subplot that lasted only two shows – the number of calls to the U.S. Center for Disease Control’s AIDS hotline spiked substantially.

Examples of education through entertainment media come from other parts of the world as well. The radio soap opera “Apwe Plezi” was broadcast between February 1996 and September 1998 in St. Lucia, with themes including family planning and sexually transmitted disease prevention. Vaughan et al. (2000) carried out personal interviews (753 pre-test and 1,238 post-test) and find that although only small changes were observed, the program had a positive effect on knowledge and change of attitudes with a minimal change in behavior.

Similarly, Paluck and Green (2009) evaluate the effectiveness of a radio program on conflict resolution and intergroup tolerance in Rwanda using a group-randomized research design. For one year the treatment group was exposed to the reconciliation program while the control group listened to a program about health and HIV. The authors observe increased dissent from popular opinion and also increased collective action among the treatment group.

In Tanzania, the soap opera “Twende Na Wakati” was broadcast between 1993 and 1997, with messages promoting adoption of family planning methods and HIV/AIDS prevention. Rogers et al. (1999) and Rogers and Vaughan (2000) evaluate the effectiveness of this program and find that the soap opera had strong behavioral effects on family planning.

Yet another example is from Brazil, where La Ferrara et al. (2008) and La Ferrara and Chong (2009) study the effects of “novelas” on fertility rates in the 1970s and 1980s. The authors find that as a result of the expansion of the Rede Globo network (which enjoyed a monopoly position as the supplier of “novelas” until the 1990s), family size was reduced, women began naming their children after soap opera characters, and separation and divorce figures went up in the areas 7
that were exposed to the network. The results can partly be explained by the strong influence of television in Brazil.

In Kazakhstan, “Seeking Happiness”, a six-episode soap opera funded by USAID, was used to explain capitalism, especially privatization and rule of law, to the former soviet country. According to Mandel (1998), the project failed since the local production company was unfit to produce a social marketing soap opera, the budget was too small to produce the series well and the USAID decision-makers lacked experience in rural Kazakhstan. Another soap opera, “Crossroads”, funded by the British government, took a different approach to social engineering in Kazakhstan and it ended up becoming one of the most popular Kazakhstani television programs in 1998. The most important element for success was ensuring high quality at all production levels, especially a good storyline and convincing actors.

Finally, Brown and Cody (2006) explore the effects of a pro-social soap opera “Hum Log” on promoting the status of women in India. The authors find that although the soap was popular, it did not significantly influence society’s perception of women.

Overall, the use of mass media for spreading social messages is quite common both in developed and developing countries, with mixed evidence on effectiveness.

# _2.3. Mass Media and Financial Education_

The use of mass media and specifically soap operas to communicate financial capability-related messages is a relatively new approach to financial education, and the few existing financial literacy mass media programs have not yet been rigorously tested for their effectiveness.

The telenovela “Nuestro Barrio” in the US was one of the earliest soap operas to incorporate financial literacy messages. It was aimed at educating Hispanic immigrants about financial matters and promoting the use of bank accounts. It was primarily directed towards households outside the formal financial system with the goal of making these products and services attractive.

8
Similarly, “Makatuno Junction”, a Kenyan soap opera that is still in production includes financial literacy themes. Since 2005, the series has reached large audiences in Kenya (over 7 million viewers in 2009), Uganda (3 million viewers in 2009) and Tanzania (2 million viewers). Among the many storylines featured in the soap, the series has encouraged viewers to open bank accounts, to save, and to budget. The soap has also featured other social messages, such as malaria prevention, contraception, and schooling. Yet, to date, no careful evaluation of these messages has been conducted.

Apart from television, the online world has stepped into the fray of financial education. The most prominent example is the US non-profit, Doorway to Dreams (D2D), which offers online games that teach players important lessons on managing finances. Tufano et al. (2010) analyze the impact of these video games on low-income and minority adults and find that the games improved financial knowledge and confidence, although the sample size and scope of this study were fairly limited.

Yet another entertainment outlet, theatre, has been used to spread financial messages. The VISA Financial Literacy Road Show is a traveling theatrical performance that provides educational messages on financial matters in un-banked communities in Sub-Saharan Africa. However, the program has never been evaluated.

# _2.4. Evidence on Emotional Connections through Media_

The evidence on using emotional connections to connect with target audiences is varied. Several factors and external impulses have been shown to impact consumer decisions. Numerous factors, from the messenger, to the setting, to the method of delivery and phrasing of the message matter when communicating information.

The clearest example of persuasive messaging comes from advertising. Advertisers regularly use influences of emotions to affect and complement consumer preferences (Stigler, 1987; and Bagwell, 2007). Russo, Carlson, and Meloy (2006) further argue that persuasive advertising can lead decision-makers to make inferior choices. Bertrand et al (2010) are among the first to systematically study how persuasive advertising influences consumer choice. The authors carry 9
out a large-scale direct-mail field experiment to study the effects of advertising content on real decisions in South Africa, and find that factors such as framing and phrasing, such as including a picture of an attractive female accounts manager, influences loan demand. Similarly Agarwal and Ambrose (2007) study the impact of direct mail advertising on financial decisions and find that advertising campaigns have a persuasive effect on loan contract choices.

Outside of the financial realm, Evans (1963) finds that demographic similarities between the client and the salesperson influences retail demand. Similarly, an experiment involving door-todoor charitable fundraising finds that attractive female solicitors secure significantly more donations (Landry et al. 2006). Further evidence on the influence of emotions through media comes from Anschutz et al. (2008), who find that body size of television actors affects personal body dissatisfaction and actual food intake.

# **3. Financial Education in South Africa**

# _3.1. The South African Context_

Household debt has been a large and growing problem for South Africa over the last decade. The ratio of household debt to disposable income was 76.3% in the second quarter of 2012, up from 50% in 2002.<sup>4</sup> In June 2012, the National Credit Regulator (NCR) reported that of the 19.6 million consumer borrowers in its records, 9.22 million (47%) had impaired records.<sup>5</sup> Out of the consumers with impaired records, 19.5% were in arrears for three months or more, 13.3% had adverse listings, and 14.2% had judgments and administration orders issued against them. What is particularly troubling is that the percentage of consumers with impaired records has been above 45% since December 2009.

Financial woes in South Africa are not limited to indebtedness. The savings rate of households is with 1.7 percent in the second quarter of 2012 very low.<sup>6</sup> In fact, according to FinScope (2012), a large percentage of the South African population (67%) does not save at all, even though the majority of adults believe that saving is important. In addition, out of those who save, only 22% save through formal channels, either with banks or other formal financial institutions.

> 4 South African Reserve Bank, Quarterly Bulletin, September 2012, No. 265. 5 National Credit Regulator, Credit Bureau Monitor, June 2012. 6 South African Reserve Bank, Quarterly Bulletin, September 2012, No. 265.

10
The Financial Services Board (FSB), the statutory regulator of the non-banking financial sector in South Africa, undertook a nationally representative study on financial literacy levels in South Africa in 2011.<sup>7</sup> The survey focused on four key areas of financial literacy: day-to-day money management; financial planning; choosing appropriate products; and financial knowledge and understanding. One of the central findings from this survey was that financial literacy is to a large extent dependent on educational attainment and income levels (FSB, 2011). For example, the presence of a household budget, indicative of a higher awareness towards financial management, is strongly related to income levels, with 79% of households in the top income quintile having household budgets relative to 36% in the lowest income quintile.

Income differences are also prevalent in the sources of information that influence individuals’ decisions on financial products. While individuals in the richest quintile are far more likely to consult the internet, independent financial advisers or friends in the financial services industry, the poor are more likely to rely on public resources such as newspapers, radio and television. In general, the importance of public media for financial decisions is high across all income groups, with 33% of respondents indicating that TV and radio programs or adverts were the source of information that most influenced their decisions about financial products **,** showing the potential media has to influence financial decisions and behavior in South Africa.

Against this backdrop, financial education through popular media is potentially a powerful tool to stimulate improvements in financial knowledge and behavior. This paper exploits this potential to study the effectiveness of financial messages delivered through a popular soap opera.

# _3.2. Scandal!_

South Africa has a well-developed media industry with popular television shows reaching millions of viewers. Further, educational messages in health and education have previously been incorporated in soap operas such as “Soul City”.

> 7 A total of 3,112 South Africans (individuals aged 16 and above living in South Africa) participated in the survey. The study was undertaken in collaboration with the International Network on Financial Education (INFE) of the Organization for Economic Co-operation and Development (OECD).

11
The target soap opera for this study is “Scandal!” (henceforth, Scandal). Scandal has been running for eight years (more than 1,500 episodes) and produces four weekly episodes. The show is broadcast on eTV, the second most popular station in South Africa,<sup>8</sup> and is especially popular among low-income South Africans.

The storyline for this study was developed by the production company of Scandal, Ochre Media, together with the National Debt Mediation Association (NDMA), and a team of financial capability and social marketing experts. Messages in the storyline were tested through focus groups to ensure that they were correctly understood by the target group, that the plot appeared realistic, and that the target group could identify with the characters and their problems. All suggested changes and recommendations by the focus group members were included in the final storyline.

The storylines focused on debt aspects of financial capability including sound financial management (e.g. using credit wisely and borrowing from formal sources), avoiding debt traps (e.g. impulse buying and living beyond one’s means) and getting out of debt (e.g. practical steps for seeking help). The financial education storyline stretched over a period of two consecutive months, a time that was deemed necessary for the viewers to emotionally connect with the soap operas’ characters and their problems. Specifically, between 13<sup>th</sup> of February 2012 and 27<sup>th</sup> of March 2012, 26 episodes with financial literacy messages were aired.

The objective of the storyline was to positively affect the behavior of medium- to low-income South Africans around financial literacy and management through increasing their knowledge and understanding regarding personal financial management. The story highlighted five major themes:

1. Getting into debt. Caused by financial mismanagement, for example, impulse buying and / or living beyond means.

2. The effects of financial mismanagement and debt on family and home life.

3. Acknowledgment that a problem exists with managing finances which has led to debt, which in turn has led to other problems.

> 8 For more information see http://www.etv.co.za/soapies/index/scandal_/

12
4. Getting out of debt. Practical steps for seeking help, for example, debt management, assessment tools and debt recovery.

5. Sound financial management, for example, using credit wisely, borrowing responsibly, feeling confident.

In addition, the NMDA was featured prominently in the storyline, and the main character and her husband seek advice from an NDMA employee who helps them find their way back to financial responsibility. At the end of each episode in which the NDMA was highlighted, the actual NDMA toll-free number was displayed on screen with a public announcement offering NDMA’s free financial advisory service.

A detailed synopsis of the show is available in Appendix 1 of this paper. Further, a short summary video clip of the storyline is available in the Web Appendix.

# **4. Research Design and Timeline**

# _4.1 Research Design_

Evaluating the impact of a nationally televised soap opera is quite challenging. First, it is difficult to separate the effect of the soap opera’s message from other messages on similar issues that individuals and families may be receiving from alternate sources. Second, certain types of individuals may self-select into watching a particular soap opera, and hence any subsequent behavior change is confounded by these selection issues. Third, it is difficult to identify the cause of any subsequent behavior change as the message itself because the effect of the message is confounded by the mode of delivery. Specifically, since on-screen actors typically have a mass following, many of those watching may simply follow what their favorite actors advocate irrespective of the content of the message.

To overcome these concerns, the project used a random encouragement design methodology with the following setup. First, a listing exercise of the study population was conducted during which basic socio-economic and TV viewership information was gathered from all households. The evaluation and hence the listing exercise was concentrated in the area of urban Gauteng (with 2,500 respondents from Johannesburg and 500 respondents living in Pretoria). The listing 13
was done for 3,000 people through physical visits (face-to-face). It was ensured that all individuals listed were responsible for their household’s spending decisions, watched television soap operas, were 20 years of age or older, and belonged to medium to low income groups (i.e. classified as LSM 5 to 7 on the Living Standard Measurement (LSM) scale).<sup>9</sup> To avoid individuals in the sample communicating with each other about the soap operas that they watch (and thus potentially influencing each other), only one interview per home/family was conducted, only one interview per street was carried out, and two streets were mechanically skipped before visiting another household.

Next, the study sample of 1,031 individuals was chosen and randomized into treatment and control groups. The original study design called for stratification by gender, income level, education, and the likelihood of watching soap operas to allow for measuring heterogeneous impacts, however, due to an undetected coding error the randomization occurred unconditionally rather than within strata. Nevertheless, as the summary statistics in Table 1 show, the sample is balanced across treatment and control on these variables.

Since the soap operas were nationally televised, we could not control or restrict viewership. In order to encourage individuals to abide by our assignments and to create a separation between treatment and control groups, a financial incentive was provided for individuals in both groups to watch their respective assigned show. Initial telephone calls were made to explain these details to the study sample. They were told that we would call them every 3-4 weeks during the next 3 months and ask them questions about the soap storyline, and if they answered three out of four questions correctly they would win 60 Rand (US$7) each time. This amount was agreed upon after consulting with the survey and production firm and was deemed high enough to encourage participation. An auxiliary advantage of providing financial incentives to both treatment and

> 9 The South African Audience Research Foundation (SAARF) LSM (Living Standards Measure) is the most widely used marketing research tool in Southern Africa. It divides the population into 10 LSM groups, 10 (highest) to 1 (lowest). The LSM cuts across race, gender, age or any other variable used to categorize people. Instead, it groups people according to their living standards using criteria such as degree of urbanisation and ownership of cars and major appliances.

14
control groups was that any confounding effects of the incentive itself on subsequent financial attitude or behavior change (i.e. Hawthorne-type effects) would be netted out.<sup>10</sup> During these calls, in addition to questions about the storyline, the surveyors asked questions related to financial knowledge, attitudes, and behavior. Since the scope of questions asked over the telephone was limited, a face-to-face follow-up survey was conducted four months after conclusion of the storyline with more comprehensive knowledge and behavior questions. In addition, administrative daily call data was obtained from the NDMA so we could track incoming calls to the financial advisory call center following the public call to action. Finally, in order to be able to understand mechanisms of impact, three qualitative focus groups were conducted after the final follow-up survey.

An important aspect of the above research design is the comparability of viewers across Scandal and Muvhango. Clearly, if the demographic profiles varied substantially, we could have significant differences in take-up among our random sample of treatment and control groups. Hence, prior to the study, we researched all available alternative soap operas to Scandal and found Muvhango to be very similar in content and in viewership profile.<sup>11</sup> Both series had approximately 2.5 million adult viewers who came predominantly from mid to low income groups (LSM 2-4, LSM 5-6 and LSM 7-8), and viewership was distributed similarly across race with most viewers being non-white and primarily Sotho speaking.

# _4.2 Study Timeline_

The study spanned a period of one year, from December 2011 to December 2012. The initial listing exercise described in the previous sub-section was conducted in early December 2011, after which our study sample was selected. An initial Computer Assisted Telephone Interview (CATI) was conducted in February 2012, prior to the airing of the storyline. The soap opera storyline then began airing from mid February to end March.

> 10 This design is similar to an experimental evaluation of radio programming to change social norms and behavior regarding dispute resolution in Rwanda (Paluck and Green, 2009).

> 11 Muvhango has run for the last three years on SABC2. It is set in a Venda community both in rural Venda and in urban Soweto. For more information see http://www.tvsa.co.za/showinfo.asp?showid=540.

15
The first follow-up CATI was conducted in early March, the midpoint of the storyline, and the second one in early April, at the conclusion of the storyline. The final face-to-face follow-up survey was conducted four months later, in late July, 2012.

The results from these surveys were then analyzed and issues highlighted for the qualitative study. Three focus group sessions were finally conducted in mid November, 2012 to conclude the study. Figure 1 illustrates the study timeline.

# **5. Summary Statistics, Randomization, and Take-up**

Table 1 presents baseline summary statistics that were gathered as part of the listing exercise. The table first presents means and standard deviations for the full sample, then separately for treatment and control groups, and finally presents p-values for a test of differences in means between treatment and control groups. None of the differences are significant, which confirms that the randomization was successful.

Overall, the sample consists of 70% females, which is unsurprising since soap opera viewership is predominantly female. At the same time, however, we strived to include males in the sample and the surveyors made the extra effort to find males who watched soap operas during the listing phase. Since our sample is entirely urban, the level of formal education was relatively high, with 70% of the sample having at least graduated from secondary school. A little more than 50% scored above median in the financial literacy test, which highlights that the mean and median are almost the same, thus a nice bell-shaped distribution. A little less than half the sample (43-46%) belongs to the low LSM group, thus providing variation in income levels. Half the sample also has a full time job or other paid work. Nearly 60% of the sample had not borrowed money in the past 6 months at the time of the baseline survey, and finally, about 50% of the sample had watched Scandal in the past.

Table 2 presents attrition results and notes two important points. First, the general level of attrition is very low – in the final follow-up survey from which we obtain knowledge and behavioral outcomes for our analysis, we were able to interview more than 93% of the baseline 16
sample.<sup>12</sup> Second, there is no significant difference between response rates of treatment and control groups. Some of the final interviews could not be conducted face-to-face and we investigate whether the likelihood of face-to-face interview is correlated with treatment status, and reassuringly it is not (column 2). Nevertheless, we control for mode of interview in all subsequent analysis.

Finally, the take-up within both treatment and control groups was very high. Out of the sample of individuals assigned to either group, 96% reported watching their assigned soap during the study period, and there was no variation across treatment and control groups. Furthermore, Table 3 investigates whether there was substantial crossover or treatment contamination, with the control group also watching the treated show, Scandal. In either follow-up 1 or 2, we find that less than 12% of Muvhango viewers claim to have also watched Scandal in the last four weeks, and a third of these viewers watched several other shows too. The tabulations show that 2% of respondents watched Scandal and another show, about 1% watched two other shows, and 0.5% watched three or more other shows. A large proportion – 38% in follow-up 1 and 29% in follow-up 2 watched entirely other shows and did not watch Scandal. While these are reassuring findings, we note that such crossovers will only underestimate treatment effects in our subsequent analysis.

# **6. Results and Discussion**

This section presents the main analysis on financial knowledge and behavior. Since our research methodology uses random assignment, we compare the cross-sectional averages between the treatment and control groups in a regression framework, and conclude that any difference between the two groups is due to viewing the financial messages in Scandal. Additionally, we control for mode of interview and use robust standard errors.

Note that while the two intermediate CATI surveys provide insight into the immediate effects of our intervention, the outcomes measured in these surveys are not directly comparable with the outcomes in the final face-to-face survey since a richer set of questions were asked in the latter.

> 12 Note that the response rates on the first two telephonic surveys were relatively lower, 85% and 76%, respectively.

17
Hence, our analysis focuses primarily on outcome measures obtained from the final follow-up survey, though we briefly discuss intermediate results also.

Finally, the discussion below also explains our qualitative results from the three focus groups. Note that the focus groups were held after we analyzed the quantitative data and were designed and directed at understanding mechanisms and heterogeneity of some of our quantitative findings.

# _6.1. Financial Knowledge_

The first outcome of interest is financial knowledge. While our intermediate CATI surveys did ask questions on financial knowledge, we could only ask very basic questions and do not detect any significant differences between treatment and control groups in these surveys.

However, in the final face-to-face survey, we were able to ask more comprehensive questions, and further were able to distinguish between general financial literacy and content specific financial literacy. General financial literacy refers to questions that test financial concepts that were not at the forefront of the soap opera storyline, such as separating business and personal expenses, probabilities in lotteries (although gambling was highlighted, less attention was paid on probabilities), and writing down budgets. In contrast, content specific financial literacy refers to questions that focused on knowledge of hire purchase agreements, which was the central contract through which the main soap character overspent and ended up with unaffordable high monthly payments.

Table 4 presents our findings. Column 1 shows no significant differences between treatment and control groups on general financial literacy, but significantly higher scores on content specific financial literacy in column 2 – compared to 42.5% in the control group, 47% of respondents in the treatment group answered these questions correctly, a 4.5 percentage point increase. Comparing across subgroups of gender, ex-ante financial literacy, schooling, and ex-ante income status, we do not see significant differences on these literacy measures.

_6.2 Debt Management_ 18
Next, we turn to behavior. Clearly the most important financial behavior highlighted in the soap storyline was borrowing and debt management. The first outcome we study is borrowing propensity – whether there was an increase or decrease in borrowing due to the storyline. Exante, we did not expect any particular effect on this margin since the storyline did not discourage borrowing _per se_ , rather only discouraged _irresponsible_ borrowing. And the results confirm our prior as we do not observe any significant difference in borrowing likelihood between the treatment and control groups, as shown in Table 5.

While borrowing is similar across the two groups, we identify a very strong treatment effect in the composition of borrowing. Column 2 of Table 5 shows that compared to 13% of respondents in the control group, 22% of treated respondents reported borrowing primarily from a formal bank. This is a very large effect size, equivalent to a 69 percent increase from the control group. Further, this result is directly relatable to the storyline as the main soap character falls into financial distress precisely because she underestimates the financial charges associated with borrowing outside of formal financial institutions – in her case hire purchase borrowing.

Further, we detect significant differences in the length of loan with the treatment group obtaining loans 4.5 months longer in term than the control group (column 3 of Table 5). This difference in loan terms complements the finding of a shift towards formal borrowing as non-formal borrowing is typically of shorter duration. Nevertheless, we do not detect any differences in missed payments between the treatment and control groups.

Finally, we examine reasons for borrowing in Table 6 and find that the treatment group is more likely to borrow for investment purposes, which includes loans for vehicles, home improvement, or business development – all productive loans.

Some interesting findings do emerge on heterogeneous margins. First, the effect of borrowing formally is driven entirely by men, while the coefficient for women is positive but not significant. Similarly, those with high ex-ante financial literacy and high education also have larger (and significant) effect sizes. The latter results make sense since more educated folks would be more likely to be eligible and approach formal financial institutions for loans. The 19
gender difference is interesting and the qualitative analysis below sheds more light into this finding. Finally, though not statistically significant because of small sample size and large standard errors, we do see point estimates higher for women in borrowing for unexpected events, and higher for men in borrowing for consumption purposes.

On the qualitative side, women generally came across as being incredibly reluctant to borrow at all, and only preferred to borrow as a last resort. This helps explain the muted impact on formal borrowing and borrowing only for unexpected emergencies. One female response was, “If you do loans you will end up paying a lot of money back, they are very costly. It is better to budget and save for what you want.” Another female reacted to information from a colleague about her multiple loans, “Three accounts? That is damaging!” Men on the other hand were more willing to use borrowing, often to pay for consumption items and consumer electronics. What is particularly interesting is that men themselves recognized that women tend to be more responsible in their borrowing decisions. One male offered his insight, “Women are good with money most of the time. When they need something and they are short 500 [Rand], you should know that she really needs 1,500 [Rand] but she already has the 1,000 [Rand].” Another male concurred, “Women don’t like borrowing money, they have a conscience.” Perhaps as a consequence of these differences in borrowing preferences, women in our focus group tended to have more credibility among friends and family, and reported less resistance on the rare occasions they had to borrow in the past. One female summed up, “Men are so scared [to borrow from friends and family]. If my brother does not have money he will phone me over and over again and then I must make a plan for him.” Finally, despite thinking differently about reasons to borrow, both male and female respondents agreed that formal borrowing was key to successful financial management. An older female provided her opinion, “Sometimes people are negligent in using money, as human being we have to manage every cent we get, we have to. If you are negligent you will end up with loan sharks and in trouble.” The men concurred with this opinion, with one male responding, “When I started 20
working I bought a TV and the salesman advised me to buy it over a 6 month credit. And because hire purchase is expensive you pay for years, and when you don’t pay, they come and take your stuff.”

# _6.3 Hire Purchase and Gambling_

The discussion of qualitative interviews above leads to our next set of results – use of hire purchase and gambling. The severe consequences of both these financial decisions were highlighted centrally in the soap storyline, with the main character purchasing items on hire purchase and gambling and ending up in significant financial distress.

Our quantitative results confirm the effectiveness of the soap opera’s message. As shown in Table 7, respondents in the treatment groups were significantly less likely to have used hire purchase in the past six months (19% in control vs. 15% in treatment) and to have gambled in the past six months (31% in control vs. 26% in treatment). These effect sizes are also large, representing an increase of 27 percentage points and 19 percentage points, respectively.

When we examine heterogeneous effects, we see that these effects are strongest for respondents with low initial financial literacy and low formal schooling, both of which are consistent with having strong treatment effects given the low base level of financial knowledge and schooling.

Turning to the qualitative study, we find a strong disdain for gambling among both men and women in our focus groups. One woman expressed her frustration, “People are crying, it is addictive. There is a lady I work with, when the money hits her account and she is working the midnight shift with me, when we finish in the morning she doesn’t go home, she goes to Gold Reef City [a casino].” The men agreed, with one relating a personal experience with gambling, “I regretted that day, my whole salary went down the drain.”

# _6.4. Savings and Wellbeing_

Next, we investigate whether there were any treatment impacts on other financial outcomes, such as savings, budgeting, and feeling of financial well-being in Table 8. We should note that we did not expect any significant treatment effects on these margins ex-ante since the soap opera 21
storyline did not focus on these issues. However, there may have been indirect impacts on these measures as a result of improved financial behavior elsewhere. Our results do not find significant differences except in feeling confident about solving financial problems.

The non result on savings is, in fact, encouraging as it provides some evidence against survey bias – that individuals were not simply providing answers that please the surveyors, else we would have seen a positive treatment effect here also.

# _6.5 Seeking Financial Advice_

Finally, we turn to an important feature of the soap opera – a public call to action through the National Debt Mediation Association (NDMA). As explained in previous sections, the NDMA was highlighted prominently in the storyline and their toll free number was broadcast at the end of each episode which featured an NDMA representative helping the main soap character.

We examine quantitative outcomes in two separate ways. First, we obtained daily call volume data from the NDMA call centers and observe a huge spike in incoming calls immediately following the episode where the NDMA was introduced into the soap storyline. Figure 2 plots the daily call volume, with the red vertical line representing the date of the soap feature. The call volume immediately jumped from an average of 120 calls per day to over 500, a more than 300 percent increase.

Unfortunately, we cannot classify and match callers to our sample and hence can only present a before-after picture. However, our second intermediate survey, conducted just a short while later included a question on consumer awareness of financial advice outlets. The regression analysis confirms a significant increase in awareness of formal avenues for financial advice (such as the NDMA). Specifically, as shown in column 1 of Table 9, compared to an average of 69% in the control group, nearly 80% respondents in the treatment group stated that they would seek financial advice from a formal source such as the NDMA if it was needed.

Surprisingly, however, this effect dissipated in the long run. Columns 2-4 of Table 9 shows that while the average proportion of respondents in treatment and control who would seek formal 22
financial advice remained relatively high (about 60%), there was no discernable statistical difference between the two groups. Also, the treatment group was no less likely to believe that there was nowhere to go for financial advice.

The qualitative analysis provides key insight into these differences in short and long term effects. In the focus groups, very few respondents recognized the NDMA logo and no link was made between the Scandal storyline and the NDMA until there was more probing into the related soap characters. Men in particular needed more prompting to recall the show, while women could remember more but primarily based on the fact that the debt advisor on the show was female rather than her affiliation with the NDMA. However, neither women nor men could accurately recall the role of the NDMA advisor in the storyline.

One leading explanation for this lack of recall is that the NDMA advisor only appeared in the show for 2-3 episodes, was an external character, and never reappeared after her short stint. Hence, the audience did not get the opportunity to establish or maintain a connection or following with this character. In contrast, all participants readily recalled the main soap character and aspects of the financial storyline delivered by her. Further, they attributed better recall to the fact that the main character was a mainstay in the soap opera, and was popular and likeable.

# **7. Conclusion**

This paper analyzes the economic impact of financial literacy messages on debt management delivered through a primetime television soap opera in South Africa. Using a symmetric randomized encouragement design methodology, the results show significant improvements in content specific financial knowledge, strong affinity towards borrowing formally, moving away from hire purchase deals, and gambling less – all messages that were conveyed in the soap storyline. A complementary qualitative analysis confirms these findings and further highlights some key gender differences in the way men and women think about borrowing, with women generally using debt as a last resort while men being more willing to borrow for purchases.

Interestingly, while a televised public call to action towards seeking financial advice through the National Debt Mediation Association leads to significant upsurge in calls immediately after the 23
messages are shown on TV, this effect dissipates over time. The qualitative discussions suggest that lack of emotional connections with an external character, acting as the NDMA agent, was a leading cause for the erosion of memory of the NDMA. Conversely, respondents easily and fondly recalled other aspects of the show delivered by the main soap character, who was part of the show prior to the financial literacy storyline and remained part of it afterwards.

These findings suggest that emotional connections and familiarity with media personalities certainly play a role in motivating knowledge and behavior change among viewers, and that erosion of memory is an important concern when incorporating public service messages in mass media productions. Hence, incorporating messages through mainstay actors, delivering complementary interventions, or scheduling regular reappearances of external characters could potentially lead to greater retention and continuation of impact.

Overall, unlike most other financial literacy evaluations, our analysis shows significant and favorable impacts on financial knowledge and behavior, which highlights the importance of delivery mechanisms in financial education. Indeed, entertainment media has the power to capture the attention of individuals unlike any other medium, and thereby provides policy makers with an effective and accessible vehicle to deliver carefully designed educational messages.

24