#OperationFFS:
Fake Face
Swarm
A Joint Report by Graphika& the Atlantic Council’s Digital Forensics Research Lab

12.2019
Ben Nimmo, C. Shawn Eib, L. Tamora, Kate Johnson, Ian Smith fromGraphika; Eto Buziashvili, Alyssa Kann, Kanishk Karan, Esteban Poncede León Rosas, Max Rizzuto from DFRLab
Edited by Camille François & Iain Robertson

Operation #FFS:Fake Face Swarm
Facebook Takes Down Network
Tied to Epoch Media Group That
Used Mass AI-Generated Profiles

1

Executive Summary
On December 20, 2019, Facebook ​took action against a network of over 900 pages, groups, andaccounts on its own platform and on Instagram that were associated with “The Beauty of Life”(TheBL),1 ​reportedly an offshoot ​of the Epoch Media Group (EMG).2 These assets were removed forengaging in large-scale ​coordinated inauthentic behavior (CIB)3. The takedown also encompassed a
set of Vietnamese assets that were linked to TheBL and to EMG.
TheBL Facebook page claimed that “Truth in Content is our purpose,”4 but Facebook said that theoperators behind this network “made widespread use of fake accounts — many of which had been
automatically removed by our systems — to manage Pages and Groups, automate posting at very
high frequencies and direct traffic to off-platform sites. Some of these accounts used profile photos
generated by artificial intelligence and masqueraded as Americans to join Groups and post the BL
content.”
The network was not the largest (by number of assets) Facebook has taken down this year: atakedown in ​Honduras in July included over 1,400 pages.5 But the assets in TheBL portfolio spent over
9 million US dollars on advertising and amassed about 55 million followers, making it nominally thelargest takedown by those metrics, although it is not clear how many of the followers were real users.
Most of the followers were outside the United States.
Before the action, Facebook shared a list of page and group names with Graphika and the Atlantic
Council’s Digital Forensic Research Lab (DFRLab) for independent analysis. Our research focused on
the assets’ content and behavior and highlighted the following key points:●

Facebook groups in this set were heavily populated by fake accounts.6 In some cases,
every​ administrator of a group was a fake account.

1

Nathaniel Gleicher, “Removing Coordinated Inauthentic Behavior From Georgia, Vietnam and the US”,
Facebook, December 20, 2019,https://about.fb.com/news/2019/12/removing-coordinated-inauthentic-behavior-from-georgia-vietnam-an
d-the-us/​.
2A​ lex Kasprak and Jordan Liles, “EXCLUSIVE: Expanding Pro-Trump Outlet ‘The BL’ Is Closely Linked to The
Epoch Times,” Snopes, October 11, 2019,https://www.snopes.com/news/2019/10/11/pro-trump-outlet-linked-epoch-times/​.
3​Nathaniel Gleicher, “Coordinated Inauthentic Behavior Explained,” Facebook, December 6, 2018,https://about.fb.com/news/2018/12/inside-feed-coordinated-inauthentic-behavior/
4“All citizens strive for TRUTH IN CONTENT,” states their description in all caps.
5
Nathaniel Gleicher, “Removing Coordinated Inauthentic Behavior in Thailand, Russia, Ukraine andHonduras,” Facebook, July 25, 2019,https://about.fb.com/news/2019/07/removing-cib-thailand-russia-ukraine-honduras/​.
6
In this context, “a fake account is an account where someone is pretending to be something or someonethat doesn't exist. Fake accounts can include accounts for fake or made up people, pets, celebrities ororganizations.” See: h​ ttps://www.facebook.com/help/306643639690823?helpref=uf_permalink​.

2

●
●
●
●

●

●
●

Dozens of these fake accounts had profile pictures generated by artificial intelligence (AI),in the first large-scale deployment of fake faces known to the authors of this report.
Other fake accounts used profile pictures taken from elsewhere online, sometimes withlittle regard to the apparent sex of the persona they were adopting.
Some of these fake accounts were created in batches just a few minutes apart.
Automation was also used for posting: some of the fake accounts used an automation
tool called Postcron, allowing them to maintain their rates of posting with no humanintervention.
Many of the fake accounts almost exclusively shared posts from the TheBL, with a small
mixture of content from the Epoch Times, pointing real users toward its website andFacebook Page.
Many of the groups focused on US politics but were partly or primarily managed from
Vietnam.
The campaign extended beyond Facebook. Notably, TheBL Twitter accounts,@TheBLNews and @TheBLcom, showed signs of inauthentic amplification. Bothaccounts were significantly retweeted by accounts that have since been suspended. A
YouTube channel branded to TheBL also showed a pattern of viewing that suggests
inauthentic boosting.

This activity was pervasive across a significant number of groups and appeared to constitute alarge-scale and systematic attempt to boost TheBL properties on Facebook and the wider
internet using a range of inauthentic techniques. The strong focus on supporting President Trumpfrom pages managed in Vietnam may also indicate a desire amplify pro-Trump messaging
throughout the 2020 election.

The Beauty of Life
The Beauty of Life was an online content provider that ​described its mission as “to present to theworld the most beautiful aspects of life. We focus on content that represents fundamental moral
standards and values, to enlighten all who prefer to be inspired. We uphold traditional cultures andupright beliefs.”7 Its mission statement made a strong case against misinformation: “Inaccurate anddegenerate information can be easily channeled toward vulnerable or uninformed people, who thentransmit these corrupt messages back into society, often creating a vicious cycle of misinformation.”7h​ ttps://www.facebook.com/pg/TheBLcom/about/?ref=page_internal​.

3

TheBL ran three main Facebook pages in English, ​TheBL.com​, ​TheBLTV​,8 and ​The BL Stories​,9
together with pages in ​Spanish​,10 ​Portuguese​,11 and ​Chinese​.12 It also ran a cluster of websites:
theBL[.]com in English, BLes[.]com in Spanish, and BLdaily[.]com in Chinese.
The content across the network varied widely. Some was apolitical, potentially clickbait content,
featuring heartwarming stories of animal and human devotion.

Post from ​BLNoticias​ sharing a “feel good” article from the Spanish BLes.com.
Some TheBL content was more political and focused on China and the ruling Chinese Communist
Party (CCP). Coverage on these issues was consistently hostile toward China. It included accusations
that China is interfering in Taiwan’s elections and that the CCP is launching World War Three.

8h​ ttps://www.facebook.com/TheBLTV/​.
h​ ttps://www.facebook.com/The-BL-Stories-991079371077992/​.
10​https://www.facebook.com/BLNoticias​.
11​https://www.facebook.com/Belezas-da-Vida-352542148583765/​.
12​https://www.facebook.com/BeautiesLifeInspired​.
94

Posts from BeautiesOfLifeInspired on the CCP, posted just 12 minutes apart on December 18, 2019. The
articles shared were all from BLdaily[.]com.
The anti-Chinese content featured across languages, as in this video (subtitled in Portuguese) on
Chinese prisons.

Post from TheBL Portuguese Page, sharing a video from TheBL.
The English-language assets focused heavily on positive coverage of US President Donald Trump andhis supporters, interspersing articles from TheBL.com with pro-Trump memes and inviting users tocomment on some of the more partisan content. This activity also included 86 groups with explicitly
pro-Trump names, such as “Patriots for President Trump 2020 #KAG,” “President Trump for President
2020 # KAG,” and “Hispanos por Trump.”

5

.
Posts by TheBL.com Facebook Page on December 17-18, 2019.
The content was strongly partisan, referring to Trump as “our president” and providing substantial
coverage of his public appearances.

Posts about Trump from TheBLcom on December 19, 2019.

6

Alongside pro-Trump material, these assets posted large quantities of material attacking his critics
and rivals, often presenting users with a partisan statement and urging them to respond if theyagreed.

”Say true if you agree”: posts on TheBLcom on December 19, 2019.
Facebook took action against TheBL assets for engaging in coordinated inauthentic behavior, notbecause of the content they posted on platform. The above posts are therefore primarily intended forillustration, not to analyze the reasons for the action. In one way, however, they are significant: thecombination of pro-Trump and anti-CCP content closely matches the editorial line of the Epoch MediaGroup (EMG), publisher of the Epoch Times newspaper.
In October 2019, researchers at the US-based fact-checking organization Snopes ​reported that TheBL
was connected with the Epoch Media Group, based on commonalities in their personnel and technicalinfrastructure.13
Both the Epoch Times and TheBL denied any connection, but Facebook’s takedown statement said,“Although the people behind this network attempted to conceal their identities and coordination, ourinvestigation linked this activity to Epoch Media Group, a US-based media organization, and individuals
in Vietnam working on its behalf.” Some of the assets in the set appeared to confirm the link.

Epoch Times Vietnam
Facebook’s takedown also encompassed a number of Vietnamese-language pages that were directly
linked to the Epoch Times. One ​verified page with almost 9 million followers was called “Đại KỷNguyên - EpochTimes Vietnam”14; ​another​, with 12.6 million followers, was called “Đại Kỷ Nguyên.”15
13​Alex Kasprak and Jordan Liles, “EXCLUSIVE: Expanding Pro-Trump Outlet ‘The BL’ Is Closely Linked to
The Epoch Times,” Snopes, October 11, 2019,https://www.snopes.com/news/2019/10/11/pro-trump-outlet-linked-epoch-times/​.
14​https://www.facebook.com/DaiKyNguyenVietnam2/​.

7

Another ​page​, “Đại Kỷ Nguyên - Văn hóa,” directly mentioned its affiliation with Epoch Times in its
about section, listing the Epoch Times Online as a “product.”16Home sections for the two main assets (above and next page).

15
16​https://www.facebook.com/daikynguyenvietnam/​.
​https://www.facebook.com/pg/DaiKyNguyenVanhoa/​.

8

Screenshot of the About section for “Đại Kỷ Nguyên - Văn hóa” listing The Epoch Times Online as a
product, along with newspapers globally.
Alongside this affiliation to the Epoch Times, the pages were also linked to TheBL. The About sectionof all three pages linked to the same website, dkn[.]tv, whose full title is “Đại Kỷ Nguyên,” the same as
the Facebook pages.
At the foot of dkn[.]tv’s homepage, a “links” section pointed to five websites. The first, “Tin nhanh” or“fast news,” linked to a subdomain of the main dkn[.]tv domain: ​tinnhanh.dkn.tv/​. The other four werethe various language versions of TheBL. They also appeared to use a similar theme for both dkn.tv
and thebl[.]com, the latter of which shows dkn.tv as its Vietnamese-language outlet.

9

Screenshot of the footer of dkn[.]tv, showing the links to TheBL assets.

Web banners for dkn[.]tv (above) and thebl[.]com.
These Vietnamese-language pages covered a wide range of topics, lifestyle and culture having a heavy
presence in addition to political and world news. Their political posts on US politics, when translated,
often seemed to avoid taking a position, and instead stayed fairly neutral in their language; the articles
they linked to were often much more slanted. The US political content tended to be supportive of
Donald Trump; the top Trump-related headline on the site on December 19, 2019 regarded his
fundraising haul of $5 million on the day of his impeachment. The pages’ posts on Facebook reflected
this, using the pro-Trump outlet Breitbart as a source and repeating Trump’s statements on the
impeachment.

Dkn[.]tv headline the day following Trump’s impeachment discussing his fundraising.

10

Example posts from “Đại Kỷ Nguyên - Tin nhanh,” their “Quick News” outlet.
These pages’ focus on China and surrounding issues, such as the Hong Kong protests, was likewise
somewhat tempered in their Facebook posts and not nearly as prominently featured as it was on therelated website. Across the different pages, the most recent article related to Hong Kong was about
the suspension of civil servants during an investigation, and the site’s top headline was about theHong Kong protesters wanting other world leaders to follow the US in supporting the protests.

Translated version of the top headline on dkn[.]tv on December 19, 2019.

Behavioral Patterns
Some of the engagement on TheBL’s properties appeared authentic (that is, appeared to haveoriginated from real users engaging with this content), but most comments, reactions, and likes were11

greatly boosted by inauthentic behavior and inauthentic accounts, on an industrial scale. Fakeaccounts served as the administrators of Facebook groups, increased the membership numbers of
those groups, liked posts on the Pages, and posted large quantities of content from TheBL-related
assets.
This structure constituted a large-scale artificial amplification factory whose only observable function
was to boost content from TheBL and, to a lesser extent, from the Epoch Times.

Clusters of fake admins
Fake accounts were a major feature of this network. Their frequency across the set varied from region
to region.
The most massive collection of fake accounts in this set was concentrated in a cluster of over 80
groups and pages dedicated to US President Donald Trump. Almost all the accounts administering
these assets were fakes. These assets had strongly patriotic American names, such as “America
Needs President Trump,” “Trump for America's President,” and “WE STAND WITH TRUMP & PENCE!”,but transparency tools showed that the administrators for these pages were primarily based in
Vietnam.

12

Page transparency for “​America Needs President Trump​,” “​TRUMP MAGA 2020​,” and “​Make America
Great Again​,” showing the predominance of admin locations in Vietnam.17
Most of the groups had between eight and 12 admins, although the number fluctuated as fakeaccounts were suspended; at least one group had no admins left at all by the time this analysis began.
Many of the accounts managed multiple groups, with some accounts involved in running more than a17​https://www.facebook.com/America-Needs-President-Trump-113411710067544/​;https://www.facebook.com/TRUMP-MAGA-2020-106083217394997/​;
https://www.facebook.com/Make-America-Great-Again-105692740827005/​.

13

dozen groups each. These fake accounts formed tightly connected clusters, with the same accounts
managing the same sets of groups and posting content from TheBL (and occasionally the Epoch
Times) into those groups.

Network graph of the relationship between core accounts that managed US-focused groups in TheBL
network, with the size of the node indicating the number of groups the account administered (filtered bydegree range 2 and k-core 6).

Network graph of the relationship between core accounts that managed groups focused on Taiwan,
Hong Kong, and China, with the size of the node indicating the number of groups the accountadministered (filtered by degree range 2 and k-core 6). The three main accounts at the top left, Yahui Jin,
Xia Dong, and Wade Peggy, and the main accounts at the bottom right, all used stolen profile pictures
and are discussed below.

14

The overwhelming majority of all the content these pages and groups posted came from one of TheBL
assets, either on Facebook or on external websites. In effect, other assets served as an amplification
system, boosting content from TheBL through systematic and large-scale false activity. The networkof fake admins were central to that effort.

Shares of content from TheBL[.]com, BLDaily[.]com, and The BL Video to a US-focused group, aTaiwan-focused group, and a US-focused page.

People who never were: AI-generated faces
Many of the fake admins and group members had profile pictures that appear to have been generated
by a type of artificial intelligence (AI) called generative adversarial networks (GAN)18: they were
pictures of people who have never existed.
GANs are neural networks that compete against another to generate new data, for instance images
that look like real human faces, although they do not belong to a real person. Although this innovationin machine learning has been around since 2014, the ability to generate such pictures became more
visible to the general public in February 2019, with the launch of a website called
thispersondoesnotexist[.]com that uses GAN technology to generate fake faces. It is unclear fromforensic traces whether TheBL operators used this website to source the picture, as they could havealso used a GAN on their own.
TheBL network is the first time the authors have seen AI-generated pictures deployed at scale togenerate a mass collection of fake profile pictures deployed in a social media campaign.
This technology is rapidly evolving toward generating more believable pictures, but a few indicators
still give these profile pictures away. Images generated by the current state of GANs are notorious for
struggling with features that should be symmetrical on the human face, such as glasses or earrings,
and with background details. Profile pictures from the network showed telltales of all three.

18

See ​Goodfellow, Ian, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron
Courville, and Yoshua Bengio. "Generative adversarial nets." In ​Advances in neural information processing
systems​, pp. 2672-2680. 2014. Or for the shorter version, an explainer on YouTube:
https://www.youtube.com/watch?v=8Tvn67gO8DQ&feature=emb_title​.

15

Profile picture for “​Alfonzo Macias​,”19 an admin on a group in this cluster. Note the asymmetry in theglasses and also the poorly defined background. The authors checked this image with experts at the
University Federico II of Naples,20 who assessed that this image was GAN-generated with 99.9 percent
certainty.21

19​https://www.facebook.com/alfonzo.macias.589
We’re deeply grateful to Luisa Verdoliva and Francesco Marra. Readers who are hungry for more
technical details are encouraged to read the paper describing the methodology that was applied to this​ ​set:
F. Marra, D. Gragnaniello, D. Cozzolino, L. Verdoliva, “Detection of
GAN-generated fake images over social networks”, 1st IEEE
International Workshop on "Fake MultiMedia", Miami (USA), April 2018.
21
The authors typically blur the faces of individuals involved in online influence operations, to avoid
violating their privacy. In this case, however, there is no current indication that any such individual actuallyexists, rendering the question of privacy moot.
20

16

Profile picture for “​Teresa Taylor​,”22 an admin on a group in this cluster. Note the asymmetry in theearrings. The authors checked this image with experts at the University Federico II of Naples, who
assessed that this image was GAN-generated with 99.9 percent certainty.

Profile pictures for “​Mary Keen​”23 and “​Jacobs Guillermo​,”24 both admins on groups in this cluster. Notethe strange dimensions of Mary’s neck and the distorted backgrounds. The authors checked theseimages with experts at the University Federico II of Naples, who assessed that both images were
GAN-generated with 100 percent certainty.
22​https://www.facebook.com/profile.php?id=100044137683587​.
​https://www.facebook.com/mary.keen.5201​.
24​https://www.facebook.com/jacobs.guillermo​.
23

17

Despite the ease of generating fake pictures, on at least one occasion the network featured two
admins with the ​same​ AI-generated picture.

Profile pictures for “Jacobs Guillermo” and his identical twin “Duncan Gilbert.”25
Note that in some cases, female profile pictures were attached to male names, and vice versa(perhaps a marker of progressive views in fake profiles, perhaps an error from the operators).

Profiles for “​Dillinger Johnny​” 26 and “​Davidson Susan​.” 27
Other forms of analysis can help open-source researchers determine whether an image has been
generated by an AI classifier. For instance, in the case of Mary Keen, illustrated above, it was possible
to identify uneven color compression and a partially formed background using Error Level Analysis(ELA), a forensics technique often used to detect manipulation in images. ELA works by identifying thedifferent compression levels in an image.28

25​https://www.facebook.com/duncan.gilbert.568​https://www.facebook.com/dillinger.johnny.927​https://www.facebook.com/davidson.susan.1
28
See​: h​ ttps://en.wikipedia.org/wiki/Error_level_analysis
26

18

.
Error Level Analysis for the “Mary Keen” profile picture.
One noteworthy feature of such profile pictures, exposed by open-source analyst ​@conspirator029 andfurther examined by authenticity analyst Sarah Thompson of ​leadstories.com​,30 who both
independently analyzed TheBL Facebook groups and came to the same conclusion, is that the eyes of
all the profile pictures occur in the same place, regardless of the orientation of thecomputer-generated face. When the faces are superimposed, the eyeballs match - unlike with real
profile pictures, which show greater variation. The authors of this report tried a small-scale
experiment, comparing the profiles of five of the research team with five of the fake profile pictures.

Comparison of the research team’s faces (top row) with the AI-generated faces (bottom row).
29​https://twitter.com/conspirator0/status/1096877542521344002/photo/1​.
Sarah Thompson, “ Fake Faces: People Who Do Not Exist Invade Facebook To Influence 2020 Elections,”
leadstories[.]com, December 12, 2019,https://analysis.leadstories.com/3471185-fake-faces-people-Who-Do-Not-Exist-Invade-Facebook-To-Influe
nce-2020-Elections.html​.
30

19

The authors then compared the pictures of the fake profiles with five AI-generated images taken from
thispersondoesnotexist[.]com. This time, the eyeballs aligned precisely, even when all 10 images weresuperimposed on one another.

Comparison of the two sets of AI-generated images.
These fake admins formed the administrative backbone of these groups, often making up more thanhalf of all the admins.

Admins on “​Patriots for President Trump​”31 and “​President Trump KAG 2020​.”32 ​On the former group, 9of 15 admins used AI-generated faces. On the latter, 8 of 16 did.

31
32

​https://www.facebook.com/groups/839685489742960/admins/​.
​https://www.facebook.com/groups/450880889092181/admins/​.

20

The ease with which the operation managed to generate so many synthetic pictures, in order to giveits fake accounts (mostly) convincing faces, is a concern. Further research is needed to find ways toidentify AI-generated profile pictures reliably and at scale, so that platforms and researchers can
automate their detection.

Stolen pictures and accounts
AI-generated faces were the most innovative forms of fake account in this network, but they were by
no means the only type. Some accounts used the more traditional method of taking their profile
pictures from stock photos and glamor shots across the web. For example, one ​group that Facebook
identified as belonging to this network, “#台灣政治新聞 Taiwan political news,”33 featured five admins.
One’s profile picture appeared to be machine-generated; ​three of ​the ​others34 had profile pictures
taken​ ​from​ e​ lsewhere​ online, including a “photoshoot businessman corporate.”35

The admins for the “Taiwan Political News” group, showing the profile pictures of three members of theteam, and earlier websites featuring the same pictures. These are the ​Yahui Jin​,36 ​Xia Dong​,37 and W​ ade
38
Peggy mentioned in the network map on page 5, above.
33​https://www.facebook.com/groups/1287048571474181​.
“Xia Dong”:​ h​ ttps://www.facebook.com/thihongtham.pham.10​;​ “Wade Peggy”:
https://www.facebook.com/profile.php?id=100040268111460​; ​“Yahui Jin”:
https://www.facebook.com/profile.php?id=100040395064377​.
35“Xia Dong”:​ h​ ttps://www.maxpixels.net/Photoshoot-People-Businessman-Corporate-Model-2911325​;“Wade Peggy”:
https://www.fimela.com/lifestyle-relationship/read/3668025/agar-tidak-gampang-stres-menghadapi-deadli
ne-kerjaan-yuk-ikuti-tipsnya​;​ “Yahui Jin”:
https://lifestyle.uzone.id/sebuah-renungan-pentingnya-berdamai-dengan-diri-sendiri-ketika-dunia-tidak-berp
ihak-padamu​.
36​https://www.facebook.com/profile.php?id=100040395064377​.
37​https://www.facebook.com/thihongtham.pham.10​.
38​https://www.facebook.com/profile.php?id=100040268111460​.
34

21

Adding to the evidence that these accounts were indeed fake, the same three accounts all originallyfeatured profile pictures of bearded white men: these were all uploaded on September 16, 2019, all in aperiod of less than an hour. They then changed to their current profile pictures on November 4, 2019,in less than a two-hour period.

The same admins for the Taiwan Political News group, showing the original profile picture uploads on
September 16, 2019, and the replacements on November 4.

22

Joining in bursts

Example of accounts joining groups in batches, marked by day, from May 1, 2019 to December 14,
2019.

The example accounts detailed above all joined as part of a much larger batch of accounts. The
Facebook process for joining a group entails an approval by an admin, and an “approve all” option
is available, so there is a reasonable explanation for the pattern. Some days saw very highnumbers of accounts joining, however, even though another batch had joined recently. This may
suggest a sudden surge in organic interest, but given the number of fake accounts across the
network, it may also indicate batches of fake accounts joining the groups.
Another pattern within the data showed that many of these batches were made very early in themorning, US Eastern time: between 2 am and 4 am. This would correspond to regular working
hours for admins located in Vietnam or Taiwan, 12 hours ahead of the East Coast of theUnited States.
The scale of these bursts of new members led to a huge increase in the groups’ membership. The
following graph illustrates the cumulative membership numbers from just one group, ​“​Stand With
President Trump 2020​,” from May until December 2019. On more than 30 different days, over 200 new
members joined in a single day; on the busiest day of all, the group gained 1,000 new members. The
uptick started in July and then continued at a slightly lower but still very inflated rate after that.

23

Total new members gained by “Stand With President Trump 2020” from May to December.

Industrial and automated
The way that the three apparently Asian accounts Yahui Jin, Xia Dong, and Wade Peggy uploaded theirfirst profile pictures within an hour of each other was not an isolated incident. On several occasions,the authors of this report found accounts that followed similar behavior patterns, joining the samegroups at the same time, or uploading their profile pictures within very short timespans.39
On some occasions, multiple accounts with AI-generated faces joined the same groups at the sametime and almost in the same order.

39

Facebook does not show the date on which accounts were created: the upload date of the first profile
picture is a useful proxy that can at least indicate the beginning of activity by the account.

24

Multiple AI-generated image accounts joined the same groups on the same day.
On other occasions, the admins of specific groups uploaded their AI-generated profile pictures on thesame day, only a few minutes apart, suggesting that the accounts were created and stocked withprofile pictures in a batch. Many of them used Facebook’s suggested stock cover images as a
background.

Profile uploads for “​Houston Heather​,”40 “​Howell Vanessa​,”41 “​Bass Frankie​,”42 and “​Patterson Clinton​,”43
all admins of the group “Patriots for President Trump.”44 Note the upload times between 4:10 and 4:45
pm on December 5, 2019. “Bass” and “Patterson” were suspended before December 17, 2019.
40​https://www.facebook.com/houston.heather.5​.
​https://www.facebook.com/howell.vanessa.3​.
42​https://www.facebook.com/bass.frankie​.
43​https://www.facebook.com/patterson.clinton.5​.
44​https://www.facebook.com/groups/839685489742960​.
41

25

Other admin accounts also appear to have been created in batches. These did not have AI-generated
faces or, indeed, any faces at all: they featured flowers and nature scenes as their profiles andFacebook’s stock images for their cover photos.

Profile uploads for “​Nancy Myers​,”45 “​Shei Shei​,” 46 and “​Maya Sharma​,”47 all on December 10, 2019,
between 2:02 pm and 4:21 pm, and all admins on groups that boosted TheBL in particular.
The same behavior featured on pro-Trump groups in America and Chinese-language groups that
focused on “daily news” in China, Hong Kong, and Taiwan. One Chinese-language group identified by
Facebook, “​每日新聞 @​” (“Daily News @,” according to Google Translate),48 featured 10 admins. All ofthem had profile pictures taken from elsewhere online.

​https://www.facebook.com/nancy.myers.182940​.
​https://www.facebook.com/shei.shei.129794​.
47​https://www.facebook.com/profile.php?id=100044750432520​.
48​https://www.facebook.com/groups/1090051461359736/​.
45
46

26

The profile pictures and their originals. Left-hand column of profiles, top to bottom: u​ nsplash.com​,49
unsplash.com​,50 ​unsplash.com​,51 ​jewelry4man.com​,52 u​ nsplash.com53; right-hand side, top to bottom:
54
55
unsplash.com​, ​pxhere.com​, Vietnamese fashion outlet t​ hoitrangtichtac.com​,56 a P​ interest post​ on
57
Vietnamese “ao dai” dresses, and Vietnamese website ​tienphong.vn​.58​https://unsplash.com/photos/5pPxsuAbYVY​; account.
​https://unsplash.com/photos/U4JDjYmjn1g​; account ​https://www.facebook.com/xue.yin.7946​.
51​https://unsplash.com/photos/aXDj_NiS05o​; account ​https://www.facebook.com/zhi.liang.9699​.
52​https://jewelry4man.com/a-man-guide-to-wearing-a-bracelet/​, accounthttps://www.facebook.com/huan.hao.509​.
53​https://unsplash.com/photos/GBgp6Iy16lc​;account.
54​https://unsplash.com/photos/H14pfhlfr24​; accounthttps://www.facebook.com/photo.php?fbid=104806857606212​.
55​https://pxhere.com/fr/photo/656877​; accounthttps://www.facebook.com/photo.php?fbid=118460482897566​.
49
50

56

https://www.thoitrangtichtac.com/thoi-trang-nam/ao-khoac-nam/14188-ao-khoac-kaki-nam-gia-vest-mau-x
anh-den-b4188.html​; account ​https://www.facebook.com/photo.php?fbid=100334014613058​.
57​https://www.pinterest.co.uk/pin/524669425333567196/​; accounthttps://www.facebook.com/hanh.nga.146612​.
58

https://www.tienphong.vn/suc-khoe/cong-dung-ngon-hanh-cua-vo-keo-chong-ngoai-tinh-tro-ve-700835.tp
o​; account ​https://www.facebook.com/profile.php?id=100038797278619​.

27

As the network map on page 13 demonstrates, the same accounts administered three other groups
whose names translate as “​Young People Today and Tomorrow​,”59 “​Taiwan News​,”60 and “​Taiwan Daily
News​.”61

Admin lists for “Taiwan Daily News @,” “Young people today and tomorrow,” and “Daily News @,”showing the same admins with the same stolen profile pics, in the same order.
These accounts, too, appear to have been created in bursts, albeit further apart: three on July 4, 2019,
between 2:34 and 3:04 am; three on October 14, spread out across the day; and four on November
15-16.
As we have seen, the Chinese-language accounts “Wade Peggy,” “Yahui Jin,” and “Xia Dong” also
appear to have been created in a batch: t​ hey​ ​administered​ ​at​ ​least​ ​seven​ ​different​ ​groups​.62​https://www.facebook.com/groups/2336026449858512/​.
​https://www.facebook.com/groups/520705002012874​.
61​https://www.facebook.com/groups/387103835311765/​.
62​https://www.facebook.com/groups/426190564912141/admins/​;
https://www.facebook.com/groups/2446995738679703/admins/​;
https://www.facebook.com/groups/376878212988491/admins/​;
https://www.facebook.com/groups/899453053769505/members/​;
https://www.facebook.com/groups/1287048571474181/members/​;
https://www.facebook.com/groups/2449674968643963/members/​;
https://www.facebook.com/groups/394974761398565/members/​.
59
60

28

The same admins across six groups.
The batch behavior of such admin accounts was not limited to joining and managing groups: theadmin accounts also ​posted in batches, spreading the same content across multiple groups in shortorder. Almost invariably, the posts they shared came from TheBL[.]com or one of its offshoots. On
many occasions, different fake accounts posted the same link to different groups in a production
chain of fake amplification.

29

The same post from TheBL[.]com, shared by different admins to different groups between 6:00 am and7:05 am on December 14, 2019. Note the admins include “Maya Sharma” and “Skyler Reyes,” describedabove.

The same post from BLdaily[.]com, shared to two different groups by one fake admin, both posted within
15 minutes of each other.

30

Shares of the same article from TheBL[.]com by two different fake accounts with AI-generated faces tothe same two groups noted above.
This network appears to engage in a large-scale case of coordinated inauthentic behavior, with fakeaccounts being created in batches, assigned to administer groups and pages in batches, and posting
content to those groups in batches - with the end effect of primarily boosting TheBL online footprint.

Automated postingSome of this batch behavior can be explained by automated posting tools: in several of the
US-focused groups, the admins used an automation service called ​postcron.com​.63 This website
offers users the ability to “Post up to 1,000 posts, pins, or tweet at once from an Excel or Google Docs
file.”The service works across platforms, automating Instagram, Facebook, Twitter, Pinterest, andYouTube. It offers plans ranging from a single person automating up to 10 accounts across platforms
to a plan allowing for control of 140 accounts by up to 15 team members. TheBL also uses it toautomate both of its Twitter accounts, @TheBLNews and @TheBLcom, though their engagement on
Twitter is much lower than that seen on Facebook. The highest tier service allows for up to
3,000 scheduled posts.
The service allows both group and page management, though there appear to be some limitations inusing it with multiple groups and pages, noting on their site, “*Note: Due to Facebook Policies, you can
only add the Facebook groups that are connected with the Postcron's app.”64 The precise meaning of
this is unclear, but the implication is that there are limitations to a client’s ability to cross-post acrossgroups. The accounts seen using Postcron for their posts were admins of the group in which it was
posting, though not all admins appear to have posted via the app. All of the posts observed that weremarked as having been posted using Postcron came from individual accounts, rather than otherpages.
63
64​https://postcron.com/en/​.
​https://postcron.com/en/pricing​.

31

Links to theBL[.]com made via Postcron to different groups.
Automation is not in itself a violation of terms of service, although the very high numbers that
Postcron advertises (up to 1,000 posts) appear likely to run the risk of qualifying as spam. The
Postcron feature across Facebook and Twitter is interesting for the light it shines on how thisoperation worked.

Advertising clusters
Some of TheBL Facebook assets advertised extensively; according to Facebook, total ad spend acrossthe banned accounts amounted to less than $9.5 million. According to Facebook’s ad library, by far themost active advertiser was TheBL.com, which posted around 1,000 ads in its lifetime, although many
were removed for violating Facebook’s ad policies.

Ad library entry for TheBL. Note the removed ads.

32

The ads were not spread out evenly across the year: they came in bursts, with dozens of very similarads running on the same day, perhaps as a form of A/B testing to see which would work best.

Three ads run by TheBLcom on June 7, 2019. The page ran 43 ads that day, 56 ads on April 4, 74 on
February 15, and 78 on January 25.
This pattern of silence followed by sudden bursts of intense activity was repeated across the network,
though none of the other assets was as active as TheBLcom.

Graph showing the distribution of ads from June 2018 through June 2019.
The overall ad spend was substantial, although Facebook pointed out that it “includes the entirety ofadvertising activity by both inauthentic and authentic accounts removed as part of this network

33

engaged in coordinated inauthentic behavior.” Hitherto, the highest ad spending that featured in a
Facebook takedown was ​$1.6 million​.65

Twitter and YouTube
TheBL was less active and successful on Twitter. As of December 19, 2019, one of its official
accounts, @TheBLcom, which joined in September 2018, only had 711 followers. @TheBLNews,
which joined in June 2019, had 664 followers.
As of December 19, 2019, neither account showed high numbers of retweets: typical posts received
no engagement at all, and a few scored in the range of 1-3 likes and/or retweets.. All the most recent
posts as of that date were automatic shares from TheBL[.]com, showing the same stories in the sameorder on both accounts.

Timelines of @TheBLcom and @TheBLNews as of December 19, 2019, showing the same posts in thesame order​.

65

Nathaniel Gleicher, “Removing Coordinated Inauthentic Behavior From Iraq and Ukraine,” Facebook
newsroom, September 16, 2019,https://about.fb.com/news/2019/09/removing-coordinated-inauthentic-behavior-from-iraq-and-ukraine/​.

34

However, some tweets scored retweets on the day they were posted from Twitter accounts that werelater suspended for violating Twitter’s terms of service in some way, most likely spam activity.
For example, on November 25, @TheBLNews ​tweeted a short link to a video article on TheBL.com.66
The shortened link read ​https://t.co/HTXoBQnlJo​. As of December 19, 2019, the tweet did not show
any retweets or likes.

The tweet by @TheBLNews. Note the lack of reactions.
The December 19 figure did not take into account the fact that the tweet had initially been retweeted
nine times, but every single account that retweeted it had later been suspended on an unknown date.
Similarly, a ​tweet on November 13 concerned the whistleblower who first reported President Trump’s
controversial call with the Ukrainian president, a report that launched the process leading to the US
president’s impeachment.67 As of December 19, this tweet showed six retweets, but on the day it wasposted, accounts that have since been suspended gave it eight more.

66​https://twitter.com/TheBLNews/status/1198824412822290432​. The link resolves tohttps://thebl.com/video/an-south-asian-child-grimaced-in-agony-after-being-tear-gassed-in-tsim-sha-tsui/​.
67​https://twitter.com/TheBLNews/status/1194410850813186048​.

35

The same pattern occurred on @TheBLcom. For example, on November 24, a ​tweet from this accountshared an article on TheBL[.]com that was, in turn, about a tweet by President Trump. As of
December 19, the tweet showed just one like and no retweets; however, on the day it was tweeted, itscored 10 retweets from other accounts, all since suspended.

Tweet from @TheBLcom, showing its last lonely like.

These incidents were smaller in scale than the activity on Facebook: Sysomos registered just under
2,000 mentions of @TheBLNews between December 2018 and December 2019 and 1,230 mentions
of @TheBLcom. That the accounts have already been suspended limits the evidence available toassess the scale of the activity. It is clear, however, that the amplification of both accounts benefited
from a substantial proportion of engagement from fake accounts, when compared with theengagement from real ones.

36

On YouTube, TheBL showed little presence, but the little that it showed exhibited an unusual featurethat may indicate inauthentic amplification on that platform, too. The namesake channel, created on
April 18, 2019, featured just three videos and had 51 subscribers as of December 19, 2019.68“About” details and videos from TheBL on YouTube.
Two of the videos promoted TheBL’s video content,69 promising to “highlight the positive” and provide“honest coverage of President Donald Trump and the achievements of the Trump administration
which are not reported by the mainstream media.” These were both uploaded on June 16, 2019, andscored 3,259 views and 3,305 views, respectively.
A third video was uploaded on July 13, 2019. Headlined “Get Real Free News,” it featured aWhite House video of Trump insulting journalists.70 It generated six replies, 15 reactions, and over
1.8 million views.
68​https://www.youtube.com/channel/UCxeRM7W8zt873p3flyVZAUw/​.
​https://www.youtube.com/watch?v=OsjyKwHCix0​ ​andhttps://www.youtube.com/watch?v=pcn1wjxvg_8​.
70​https://www.youtube.com/watch?v=Dvs4WoQqLJ0​.
69

37

The most successful BL video, watermarked to the White House.
This disparity between the numbers of subscribers, reactions, and views is so stark that it raises thequestion of whether the video was artificially boosted in some way.

Ben Nimmo (Director of Investigations at Graphika, non-resident Senior Fellow at DFRLab) led thisinvestigation, with teams at both Graphika and DFRLab: C. Shawn Eib​, L. Tamora​, ​Kate Johnson​, ​Ian
Smith from Graphika; ​Esteban Ponce de León Rosas​, ​Kanishk Karan​, ​Eto Budziashvili​, ​Alyssa Kann​,
Max Rizzuto​ from DFRLab. ​Camille François​ and​ Iain Robertson​ served as Editors.
The authors also thank ​Sam Gregory at Witness, and Luisa Verdoliva and Francesco Marra at
University Federico II of Naples for their kind help in analyzing the AI-generated pictures in this set.

🤖📸🙏
38