FOR PUBLICATION



### UNITED STATES COURT OF APPEALS FOR THE NINTH CIRCUIT

R E Y N A L D O G O N Z A L E Z ; T H E E S T A T E O F N O H E M I G O N Z A L E Z ; B E A T R I Z G O N Z A L E Z , Individually and as Adm inistrator of the Estate of Nohem i Gonzalez; J O S E H E R N A N D E Z ; R E Y G O N Z A L E Z ; P A U L G O N Z A L E Z ,

Plaintiffs-Appellants

,

v.

G O O G L E LLC, Defendant-Appellee. No. 18-16700

D.C. No. 4:16-cv-03282-
D MR

Appeal from

the United States District Court for the Northern District of California Donna M. Ryu, Magistrate Judge, PresidingG O N Z A L E Z V

. G O O G L E

2

M E H I E R T A

A M N E H ; L A W

R E N C E T A A M N E H ; S A R

A T A A M N E H ; D I M A N A T A A M N E H , Plaintiffs-Appellants

,

v.

T W I T T E R , I N C .; G O O G L E LLC;
F A C E B O O K , I N C
.,

Defendants-Appellees. No. 18-17192

D.C. No. 3:17-cv-04107- EMC

Appeal from

the United States District Court for the Northern District of California
E dward M. Chen, District Judge, PresidingG O N Z A L E Z V

. G O O G L E 3

G R E G O R Y C L A Y B O R N , Individually and as Successor-In-Interest of the Estate of S I E R

R A C L A Y B O R N ; K I M C L A Y B O R N ; T A M I S H I A C L A

Y B O R N ; V A N E S S A N G

U Y E N , Individually and as Successor-In-Interest of the Estate of T I

N N G U Y E N ; T R U N G D O ; J A C O B T H A L A S I

N O S ; J A M E S T H A L A S I

N O S
, Plaintiffs-Appellants

,

v.

T W I T T E R , I N C .; F A C E B O O K , I N C .; G O O G L E LLC,

Defendants-Appellees. No. 19-15043

D.C. Nos. 3:17-cv-06894-LB 3:18-cv-00543-LB

OPINION

Appeal from

the United States District Court for the Northern District of California Laurel D. Beeler, Magistrate Judge, Presiding

A rgued and Subm itted March 26, 2020 San Francisco, California

Filed June 22, 2021

B efore: Ronald M. Gould, Marsha S. Berzon, and Morgan Christen, Circuit Judges.

Opinion by Judge Christen; Concurrence by Judge Berzon; Partial Concurrence and Partial Dissent by Judge GouldG O N Z A L E Z V

. G O O G L E

4

SUMMARY *

Anti-Terrorism Act

The panel addressed appeals from the district c ourt’s d is missal o f three actions seeking dam ages under the AntiTerrorism Act against Google, Twitter, and Facebook on the basis that defendants’ social m edia p latform s allowed ISIS to post videos and other content to comm unicate the ter r o rist group’s m essage, to radicalize new recruits, and to g enerally further its m ission. The panel affirm ed the judgm ents in the Gonzalez and Clayborn appeals and reversed and rem anded in Taamneh

.

Mem bers of the fam ilies of victim s of terrorism in Paris, Istanbul, and San Bernardino alleged that Google, Twitter, and Facebook were directly and secondarily liable for ISIS’s

acts of international terrorism

. The Gonzalez plaintiffs brought claim s for both d irect and secondary liability against Google. The district court concluded that m ost of plaintiffs’ claim s were b arre d pursuant to 47 U.S.C. § 230 of the Com m unications Decency A ct, and the direct liability claim s failed to adequately allege proxim ate cause. In the Taamneh and Clayborn cases, the district co u rt co n c lu d ed th at p laintiffs failed to plau s i b ly allege a secondary liability claim against Google, Twitter, and Facebook.

The panel held that the district court in Gonzalez properly ruled that § 230 barred m o st of plaintiffs’ claim s. The panel further held that the Gonzalez plaintiffs failed t o s t ate an

* This sum m ary constitutes no part of the opinion of the court. It h as been prepared by court staff for the convenience of the reader.G O N Z A L E Z V

. G O O G L E 5

actionable claim as to their rem aining theories of liability. In Taamneh

, the panel held that the district court erred by ruling that plaintiffs failed to state a claim for aiding-and-abe tt i ng liability under the ATA. In Clayborn

, the panel c on c l ud ed that the d istrict court c o r r e ctly held that plaintiffs failed to plausibly plead their claim for aiding-and-abetting liability.

Addressing Gonzalez

, the panel held that th e c ivil

rem edies section of the ATA perm its United States nationals

to recover dam ages for injuries suffered “by reason of acts o f international terrorism .” The Just ice Against Sponsors o f International Terrorism Act of 2016 (JASTA) am ended the ATA to include secondar y civil liability for aiding and abetting, or conspiring to comm it, acts o f international terrorism

. Section 230 of the Com m unications Decency A ct protects websites from liability for m ater i a l p osted on the website by som eone else. The panel h e ld that the

presum ption against the extraterritorial application of federal

statutes did not prevent § 2 30 from applying to the Gonzalez plaintiffs’ claim s because the relevant conduct took place in the United States. The panel concluded that JASTA did not im pliedly repeal § 230. Agreeing with the First and Second Circuits, the panel held that the exception s et fo r t h in § 230(e)(1), concerning im pairm ent o f the enforcem e n t o f federal crim inal statutes, does not extend to actions for civil dam ages

. Th us, the Gonzalez plaintiffs’ claim s were not categorically excluded from the reach of § 230 im m unity.

T h e G o n za lez p lain

tiffs argued that th e im

m u n ity af f o rd ed by § 230 did not bar their claim s becaus e § 230 im m unizes only those who publish content cr e at e d by third parties, and their claim s were d irected to content creat e d b y Google. Google argued that the plaintiffs im perm issibl y sought to
treat Google as a publisher of content created by third parties,G O N Z A L E Z V

. G O O G L E

6

presum ably ISI S, o n YouTube. In Part III.E of its opinion, the p anel affirm ed the district court’s ruling that § 230 barred all of plaintiffs’ claim s except to the ex t e n t t heir com plaint presented claim s prem ised on the allegation that G oogle shared adverti s i n g revenue with ISIS. Section 230(c) precludes liability for “(1) a p rovider or user of an interactive

com puter service (2) whom a plaintiff seeks to treat

. . . as a publisher or speaker (3) of info rm ation provided by another inform ation conte n t p rovider.” The panel concluded that plaintiffs’ claim s did not inherently require the court to treat Go o g le as the publisher or speaker o f content provided by ISIS, and the duty that plaintiffs alleged Google violated did not derive from Google’s status or conduct as a publisher or speaker. The panel concluded that Google d id not crea te o r develop content by m aking a m aterial contribution to i ts alleged u n lawfulness when it created the “m o saics” by which ISIS videos were d elivered. The panel held that the c o urt’s

case law foreclo s e d the argum ent that Google’s pairing o f ISIS content with selected ad vertising and o ther videos vitiated § 230 im m

unity. A ccepting as true plaintiffs’ allegation th at Google’s algorithm s recom m ended ISIS

conten t t o users, and agreeing with the Second Circuit, the panel wrote t ha t the algorithm s d id not treat ISIS-created

content differently than any other third-party created content, and thus were entitled to § 230 im m unity.

In P art III.F of its o pinion, the p anel held that § 230 did

not b ar the Gonzalez plaintiffs’ claim s prem ised on the

a l l e gation that because it shared advertising revenue wit h ISIS, Google should b e held directly lia b le for providing

m aterial support to ISIS and secondarily liable for providing substantial assistance to ISIS.G O N Z A L E Z V

. G O O G L E 7

In Parts IV and V, the panel held th a t the Gonzalez plaintiffs did not adequately allege clai ms fo r direct o r secondary liab ility u n d er th e A T A b ased o n a revenue-sharing theory. As to direct liability, pla intiffs failed to plausibly allege that Google directly perpetrated an act of international terrorism because they did not allege that Go o g le’s actions

were m otivated by anything other than econom ic selfenrichm ent. As to secondary liability, plaintiffs did not state

a c l a im on either a theory of aiding and abetting or a theory of conspiracy liability.

In P art VI, reversing the d istrict court’s d ism issal o f the Taamneh action, the panel held t h a t t he Taamneh plaintiffs adequately stated a claim for aiding-and-abetting liability.

In Part VII, affirm ing the dism issal of the Clayborn action, the panel held that because the Clayborn plaintiffs did

n o t plausibly allege that ISIS comm itted, planned o r authorized the terrorist attack in San Bernardino, they d id not adequately state a claim for aiding and abetting an act o f international terrorism

.
Judge B erzon c o nc u r red in the m ajority o p inion in full. She wrote separately to explain that, although the panel was bound by Ninth Circuit precedent com pelling the outcom e in this case, she joined the growing chorus of voices calling for

a m ore lim ited reading of the scope of § 2 3 0 i m m unity. Judge B erzon urged the court to r e c onsider its precedent en banc to the extent that it holds that § 230 im m unity extends

to the use of m achine-learning a lgorithm s to recom m end content and connections to users.

Judge Gould concurred in the m ajority opinion in its Parts
I and I I

, Part III.A through III.D, Part III.F, and Part VI andG O N Z A L E Z V

. G O O G L E

8

dissented in part as to Part III. E and Parts IV, V, and VII. Judge Gould w rote that he concurred insofar as the m ajority would reverse in p art the dism issal of revenue-sharing claim s in Gonzalez

, and insofar as it would reverse t h e d istrict court’s judgm ent in Taamneh that the com plaint fa il e d to adequately state a claim under the ATA. Judge Gould wrote that he dissented as to the m ajority’s dism issal of the Gonzalez claim s on grounds of § 230 im mu n i ty, and o f failure to state a claim for direct or secondary liability under the ATA, because of the m ajority’s m istaken conclusion that there was n o act of international terrorism

, and he also would hold t h a t the com

plaint adequately alleged that there was proxim ate cause supporting dam ages on those claim s. Judge Gould agreed that claim s could proceed in the Taamneh case, and accordingly agreed with reversing and rem anding in that

case. On the Clayborn case, Judge G ould dissented because the m ajority’s conception of an attack authorized by ISIS was inconsistent with the allegations o f the o p e rative com plaint and well-established principles of tort and agency law.G O N Z A L E Z V

. G O O G L E 9

C OUNSEL

Keith Altm an (argued) and Daniel W. W eininger (argued), Excolo Law, Southfield, Michigan, Plaintiffs-Appellants Reynaldo G onzalez, Mehier Taam neh, Lawrence T aam neh, Sara Taam neh, Dim ana Taam neh, Gregory C layborn, Kim Clayborn, Tam ishia Clayborn, Vane s s a Nguyen, Trung Do, Jacob Thalasinos, and Jam es Thalasinos.

Ro b e r t J. Tolchin (argued) and Meir Katz, Berkm an La w Offi c e LLC, B rooklyn, New York; for Plaintiffs-Appellants
Estate of Nohe mi Gonzalez; Beatriz Gonzalez, Jose Hernandez, Rey Gonzalez, and Paul Gonzalez.

Brian M. W illen (argued), W ilson Sonsini Go odrich & Rosati, New York, Ne w York; David H. Kram er, Lauren Gallo W h i te
, an d Kelly M. Knoll, W ilson Sonsini Goodrich & Rosati, Palo Alt o, California; for Defendant-Appellee Google LLC.

Kristin A. Linsley (argued) and J a c ob T. Spencer, Gibson
Dunn & Crutcher LLP, San Francisco, California; for Defendant-Appellee Facebook Inc.

Seth P. W axm an, Patrick J. Carom e, and Ari Holtzblatt,
W ilm er Cutler Picke ring Hale & Dorr LLP, W ashington, D.C., for Defendant-Appellee Twitter Inc.

Aaro n M ackey and Sophia Cope, Electronic Frontier Foundation, San Francisco, Cali fo rnia, for Am icus Curiae Electronic Frontier Foundation.G O N Z A L E Z V

. G O O G L E

10

OPINION

CHRISTEN, Circuit Judge:

W e ad dress three appeals arising from separate acts o f terroris m— one in Paris, one in Istanbul, and one in San Bernardino—in whi c h Nohem i Gonzalez, Nawras Alassaf, Sierra Clayborn, Tin Nguyen, and N icholas Thalasinos lost their lives. The foreign terrorist organization known as ISIS took responsibility for the attacks in Paris and I s t a n bul and lauded the attack in San Bernardino after the fact. Plaintiffs

are m em bers of the victim s’ fam ilies.

Plaintiffs seek dam ages pursuant to the Anti-Terrorism Act (ATA), 1 8 U.S.C. § 2333. The ATA allows United States nationals to recover dam ages for injuries suffered “by

reason of an act of international terrorism ,” id. § 2333(a), but the defendant in th e s e c a ses is n ot ISIS. Instead, plaintiffs allege that Google, Twi t t er, and Facebook are directly and secondarily liable for the five m urders at issue in these cases. The com plaints alleg e t

h at d e f en d an ts’ so cial m ed ia p latf o rm s allowed ISIS to post videos and other content to com m unicate the terrorist group’s m essage, to radicalize new recruits, and

to generally further its m ission. Pl a i nt iffs also claim that Google placed p a i d advertisem ents in proxim ity to ISIS-created content and shared the resulting ad revenue with ISIS. In these and other ways, all t h ree com plaints allege defe n dants are directly liable for com m itting acts o f international terrorism pursuant to § 2333(a) of the ATA, and secondarily liable for conspiri n g with, and aiding andG O N Z A L E Z V

. G O O G L E 11

abetting, ISIS’s acts of internat io n a l terrorism pursuant to § 2333(d). 1

This opinion addresses three separate appeals. The Gonzalez appeal concerns claim s for bo t h direct and secondary liability against G oogle. In that case, the d i s trict

court granted Google’s m otion to dism iss, concluding that

m ost of the Gonzalez Plaintiffs’ claim s were barred p u rsuant
to 47 U.S. C. § 230 of the Com m unications Decency A ct (CDA), and that the Gonzalez Plaintiffs’ direct liability claim s failed to adequately a l l e g e proxim ate cause. The
Taam neh and Clayborn appeals concern claim s for secondary liability against Google, Twitter, and Facebook. In both o f these cases, the district court granted defendants’ m o tions to dism iss on t h e g r ounds that the plaintiffs failed to plausibly allege a secondary liability claim under the ATA.

W e have jurisdiction purs u ant to 28 U.S.C. § 1291. W e conclude the district co urt in Gonzalez properly ruled that § 230 bars m ost of the Go n za l e z Plaintiffs’ claim s, and that the Gonzalez Plaintiffs fai l ed to state an actionable claim as to their rem aining theories of liability asserted pursuant to the ATA. In Taamneh

, we concl u d e t he district court erred by ruling the Plaintiffs f

ailed to state a c l aim for aiding-andabetting liability unde r t h e A TA. The district court d id not

r e a c h § 230 im m

unity in Taamneh

. In Clayborn

, we conclude the d istrict court correctly held that Plaintiffs failed

1 The acronym “ISIS” r efers to “The I slam ic State o f Iraq and S yria.” ISIS is occasionally referred to as “ISIL” or “ The Islam i c S tate of Iraq and the L evant. ” Both nam es are derived from the Arabic “ad-Dawlah alIslam iyah fil-‘Iraq wash-Sham .” The o rganization late r s hortened its

nam e t o “ ad-Dawlah al-Islam iyah” (“The Islam ic State” or “IS”). For sim plicity, we use the nam e ISIS.G O N Z A L E Z V

. G O O G L E

12

to plausibly plead their claim f o r aid in g

- a

n d -ab ettin g liability.
W e theref o re af f irm the judgm ents in Gonzalez and Clayborn

, and reverse and rem and for further proceedings in T aamneh

.

I

A

Nohem i Gonzalez, a 2 3-year-old U.S. citizen, studied in Paris, France during the f all of 2015. On Novemb e r 1 3, 2015, when Nohem i was enjoying an evening m eal with her friends at a café, three ISIS terrorists—Ab d elh am id A b aao ud, Brahim Abdeslam

, and Chakib Akrouh—fired into the crowd of diners, killing her. This tragic event occurred wit h i n a broader se ries of attacks perpetrated by ISIS in Paris on Novem ber 13 (the “Paris Attacks”). ISIS carried o ut several suicide bomb i n gs and m ass shootings in Paris that day, including a m assacre at the Bataclan theatr e

. Th e day after the Paris Attacks, ISIS c laim ed responsibility by issuing a written statem ent and releasing a YouTu be video.

The operative Gonzalez com p laint alleges that at the tim e of the Paris A ttacks, ISIS had becom e o n e of the largest and

m ost widely recognized terrorist organizations in the world. Th e c om plaint also alleges that ISIS carried out violent terrorist attacks as a m eans of instilling terror in the p ublic and com m

unicating its broader obje ctives, and that ISIS’s

m essages—com m

unicated b efore, during and after its terror attack s—are essential com ponents of generating the ph y sical,

em otional, and psychological im p act ISIS desires to achieve.

Google o wns YouTube, a global online service u s e d t o post, share, v iew, and com m ent on videos related to a vast
range of topics. Users can post content d irectly on YouTube,G O N Z A L E Z V

. G O O G L E 13

though G oogle has the ability to rem ove any content. W h en Google receives a com

plaint about a video

, i t r e views the video and rem oves it if it violates Google’s content policies.

The Gonzalez com

plaint alleges that YouTu b e “has becom e an essential and integral part of ISIS’ s p r o g ram o f terrorism

, ” and that ISIS u ses YouTube to r e c r uit m em bers, plan terrorist at t acks, issue terrorist threats, instill fear, and intim idate civilian populations. A ccording to the Gonzalez Plaintiffs, YouTube provides “a unique and p o werful tool o f

com m unication that enables ISIS to achieve [its] goals.”

W ith regard to the P aris Attacks in partic u l ar, the Gonzalez Plaintiffs all ege that two of the twelve ISIS terrorists who carried out the attacks used online social m edia platform s to p ost links t o ISIS recruitm ent YouTube videos and “ jihadi YouTube videos.” Abaaoud, one of the attackers in the café shooting, appeared in an ISIS YouTube video from March 2014, and d elivered a m onologue aim ed at recruiting jihadi fighters to join ISIS.

The Gonzalez Plaintiffs’ th eory of liability g enerally arises from Google’s recom m endations of content to users. These recom m endations are b a s ed u pon the content and “what is known about the v iewer.” Specifically, t he

com plaint alleges Google u ses com puter algorithm s to m atch and suggest content to users based upon their viewing history. The Go n z a lez Plaintiffs allege that, in this way, Google has “recom m ended ISIS vi d eos to users” and enabled users to “locate other v ideos and accounts related to ISIS,” and that by doing so, Google assists ISIS in spreading its m essage. The Gonza l e z P laintiffs’ theory is that YouTube is “useful[] in facilitating social n etworking am ong jihadists” because itG O N Z A L E Z V

. G O O G L E

14

provides “[t]he ability to exchange comme n ts about videos and to send private m essages to other users.”

The com plaint also asserts that Google p airs videos with advertisem ents and that it targets advertisem en ts based on inform ation about the advertisem ent, the user, and the posted video. The com

plaint alleges tha t b y d oing so, G oogle

ex ercises control over w h ich ad v ertisem en ts are m atch ed w ith videos posted by ISIS on YouTube, c r e ating new unique

content for viewers “by choosing which advertisem ent to

com bine w ith the posted video with k n owledge about the viewer.”

The Gonzalez Pl a i ntiffs’ com p laint also alleges that Google’s prac ti c e is to share a percentage of the revenue it generates from these ads with the users who post the videos. Specifically, th e co m

p lain t a lleg e s that G oogle “reviewed and approved ISIS videos, including videos posted by ISISaffiliated users, for m onetizatio n t h r ough” its placem ent o f ads on those videos, thereby agreeing to share revenue with ISIS and ISIS-affiliated users.

According to the Gonzalez P laintiffs, Google is aware o f ISIS’s presence on YouTube, h as received com plaints about ISIS content, has the ability to r emove ISIS content from YouTube, and has “sus p ended o r b locked selected ISISrelated accounts at various tim es.” The com plaint asserts that in spite of Google’s knowledge and control, Google “did not

m ake substantial or sustained efforts t o ensure that ISIS would not re-esta b l i s h the accounts using new identifiers.” Instead, the Gonzalez Plaintiffs allege, Go ogle som etim es declined to rem ove ISIS accounts b ecause the content posted by those accounts did not v iolate Y ouTube’s policies and, on other o c c asions, Google rem oved only a portion o f theG O N Z A L E Z V

. G O O G L E 15

content posted on ISIS-related accounts b ut perm itted the

accounts to rem ain active. 2

Reynaldo Gonzalez, Nohem i’s father, filed an action against Google, Twitter, and Facebook on June 1 4, 2016, and

a Second Am ended Com

plaint (SAC) o n April 21, 2017. The SAC joined additional fam ily m em bers an d nam ed only Google as a defendant. According to the SAC, G oogle aided

a n d abetted international terrorism and provided m ateri al support to international terrorism by a l l o wing ISIS to use You Tu b e

. See 18 U.S.C. § 2333(a), (d). Claim s One and
Two alleged that G oogle is secondarily liable for aiding and abetting acts of international terrorism and for conspiring with ISIS; Claim s T hree and Four alleged that Google is directly liable for providing m aterial support and resources to ISIS. Google m oved to dism iss all o f the Gonzalez Plaintiffs’ claim s on the grounds that they were b ar r ed by § 2 30 of the CDA. See 47 U .S.C. § 230(c). The d istrict court g ranted the

m otion to dism iss, but gave the Gonzale z Pl a intiffs an opportunity to am end.

The Third Am e n d ed Com plaint (TAC) is the o perative

c o mp l aint. In it, the Gonzalez Plaintiffs added additional claim s. T he Plaintiffs allege that Google is secondarily liable for N ohem i’s d eath because Google aided and abetted an act of international terrorism and engaged in a conspiracy with a

2 T he Gonzalez Plaintiffs also allege that “Google has tools by which it can identify, f

lag, review, and rem ove ISIS Yo u Tu b e a ccounts,” but im properly focuses prim arily “o n w hether the content posted violates Google’s own ‘Com

m unity Standards,’ rather than exam in[ing] whether the account is being used by or f or the benefit” of terrorists. They further

a l l e g e t hat “[e]ven when Google occasionally deletes an account f o r violating its Com m

unity S t andards, i t allows these accounts to be quickly regenerated.”G O N Z A L E Z V

. G O O G L E

16

perpetrator o f an act of international terrorism

. T he Gonzalez TAC also alle g e s t hat Google is directly liable under § 2333(a) for providing m aterial support an d r e s ources to ISIS, and for concealing this support, in violation of 18 U.S.C. §§ 2339A, 2339B(a)(1), and 2339C(c). 3

Google m oved to dism iss the entire TAC based o n § 230 im m u nity, and alternatively m oved to dismi s s t h e § 2333(a) d ir e ct liability claim s (Claim s Three through Six) on t h e gro u nd that they f

ailed to plausibly allege G oogle proxim ately caused the Gonzal e z Plaintiffs’ injury. The district court ruled that all of Plaintiffs’ claim s were barred by § 2 3 0, ex c e p t t o the extent Claim s Three and Four were prem ised on a revenue-sharing theory. The court concluded

t h a t Claim s Three through Six f ailed to plausibly all e g e p ro xim ate cause. T he revenue-sharing claim s w ere d ism issed without p rejudice; all the other claim s were dism issed with prejudice. The Gonzalez Plaintiffs did not further am end, but they did tim ely appeal.

B

Nawras Alassaf, a Jordanian citizen, visi t e d Istanbul,
Tu r k e y with h is wife to celebrate the 2017 New Year

. He

was killed o n January 1, 2 017, when Abdul k a d ir Masharipov—an individual affiliated with and trained by ISIS—carried out a shooting m assacre at the Reina nightclub there (the “Reina Attack”). Masharipov arrived at the R eina nightclub shortly after m i dnight and, d uring a seven-m inute

3 Separat e l y , t h e Gonzalez Plaintiffs allege that Google p rovided funds, goods, or services t o or for the benefit of globa l terrorists in violation of Executi ve O rder No. 13224, 31 C.F.R. Part 594, and 50 U.S.C. § 1705.G O N Z A L E Z V

. G O O G L E 17

attack, fired m ore than 120 rounds into the crowd of 700 people, killing 3 9 a n d injuring 69 others. Masharipov escaped the n ightclub and evaded arrest for o ver two we eks but was ultim ately apprehended. On the day after the attack, ISIS issued a statem ent claim ing responsibility for the R eina Attack.

Twitter is a social networking service that allows users to publicly connect with oth e r users and to distribute content publicly by posting “tweets.” The Taam neh Plaintiffs allege that Twitter has the ability to rem ove tweets and accounts, but does not d o so proactively. Instead, Twitter reviews content that is reported by others as violating its rules.

Facebook is also a social networking service that allows

users to com m

unicate with o ther users and to share and distribute con t ent publicly. Facebook has the ability to

rem ove content posted by its users.

The Taam neh P laintiffs are relatives o f Nawras Alassaf. They allege that Google, Twitter, and Facebook were a cri t i c a l part of ISIS’s growth. Much like the Gonzalez

com plaint, the Taamneh com plaint alleges that ISIS uses defendants’ social m edia platform s to recruit m em bers, issue terrorist threats, spread propaga n d a , instill fear, and intimi d a te civilian populations. According to the Taam n eh Plaintiffs, ISIS could not have grown into o n e of the m ost recognizable and feared terrorist organizations without the effective com m

unications platform s p rovided b y defendants free of charge.

The T aam neh Plaintiffs’ com plaint alleges that ISIS and its affiliated entities have used YouTube, Twitter, and Facebook for m any years with “little or no inter fe r e nce.”G O N Z A L E Z V

. G O O G L E

18

“Despite extensive m edia coverage, com pla i n ts, legal warnings, petitio n s , congressional hearings, and other attention f

o r providing [ t h e ir ] online social m edia p latform s and com m

u nications services to ISIS,

. . . Defend ants continued to provide these resources and services to ISIS and its affiliates.” The Taam neh Plaintiffs also all e g e t h at defendants knowingly p erm itted ISIS and its m em bers and affiliates to use their platform s, and reviewed ISIS’s use only in response to third-party com plaints. The com

plaint further alleges that even when defendants received com plaints about ISIS’s use of their platform s, the defendants “have at v arious tim es determ ined that ISIS’s use of [their] [s]ervices did not violate Defendants’ policies,” and therefore “perm itted ISISaffiliated accounts to rem ain act ive, o r rem o ved only a portion of the content posted on an ISIS-related account

. . . . ”

The T aam neh Plaintiffs’ claim s against Google, Twitter, and Facebook all e g e these defendants aided and abetted an act of international terrorism

, conspired with the perpetrator of an act of inter n a t i o nal terrorism

, and provided m aterial support to I SI S, b y allowing ISIS to use their social m edia pl a t form s. Like the Gonzalez Plaintiffs, the Taam n eh Plaintiffs allege that defendants’ actions violated t h e ATA. Specifically, the Taamn e h c o m plaint includes claim s for direct and secondary liability un d e r t he ATA, 18 U.S.C. § 2333(a), (d), and state-law claim s for negligent infliction o f

em otional distress and wrongful death.

In response to defe ndants’ first m otion to dism iss, the
Taam neh Plaintiffs am ended their com p laint once as a m atter of right and added additional claim s. The F irst Am ended Com plaint (FAC) is the o perative com plaint a n d it alleges that Google, Twitter, and Facebook are seconda r il y liable under § 2333(d) for aiding and abetting an act o f internationalG O N Z A L E Z V

. G O O G L E 19

terrorism and for conspiring with a pe r p etrator of an act o f international terrorism

. The Taamneh com plaint also alleges that Google, Twitter, and Facebook are directly liable under § 2333(a) for providing m aterial support and r e s o u rces to ISIS, a n d for concealing this support, in violation o f 18 U.S.C. §§ 2339A, 2339B(a)(1), and 2339C(c). 4

Defendants m oved to dism iss. The district court ruled the dir e c t l iability claim s failed to adequately allege proxim ate

cause, and that the secondary liability claim s failed to state a claim for conspiracy to com m it an a c t o f international terrorism

, or for aidi ng and abetting an act of international terrorism

. The court dism issed the com plaint with prejudice, and the Taam neh Plaintiffs tim ely appealed.

C

Sierra Clayborn, Tin Nguyen, and Nicholas T h alasinos attended an office holiday party at the Inland Regional Center in San Bernardino, Californ i a on Decem ber 2, 2015. Syed Rizwan Farook, a U.S. citizen, and Tashfeen Malik, Farook’s wife, entered th e b uilding dressed in b lack and arm ed with AR-15 sem i-autom atic rifles, a 9 m m h andgun, and assem

bled pipe bom bs. Farook and Malik indiscrim inately fired m ore than 100 rounds into the office gathering (the San Bernardino Attack). A t som e point during the attack, Malik declared on her Facebook page the couples’ allegiance and loyalty to form er ISIS l e a d er, Abu Bakr al-Baghdadi. Clayborn, Ngyuen, and Thalasinos were am o ng the fourteen people

4 The Taam

neh Plaintiffs’ com

plaint also includes an allegation that Google, Twitter, and F acebook provided funds, goods, or services to or f o r the benefit o f global terrorists in violation of Executive Order No. 13224, 31 C.F.R. Part 594, and 50 U.S.C. § 1705.G O N Z A L E Z V

. G O O G L E

20

m urdered in the attack. Twenty-two oth e r s were seriously wounded. A ft e r t h e San Bernardino Attack, Farook and Malik fled the scene and were killed in a police shootout. ISI S i s s u e d a statem ent two days later that “[t]wo followers of Islam ic State attacked several days ago a ce n te r i n San Bernardino in California, we pray to God to a c ce p t t hem as Martyrs.”

The Clayborn Plaintiffs are relatives of Sierra Clayborn, Tin Nguyen, and Nicholas Thalasinos. Plaintiffs allege that Twitter, Facebook, and Google aided and abetted international terrorism and p rovide d m aterial support to internat i on a l terrorists in violation of the ATA, by allowing ISIS to use their platform s. The C layborn Plaintiffs allege Farook and Malik were radicalized by ISIS’s use of social

m edia. This com

plaint includes direct and secondary liability claim s against all three defendants pursuant to 18 U.S.C. §§ 2333(a) and (d), 2339A, 2 3 39B, and 2 339C, and state-law claim s for n egligent infliction of em otional d istress and wrongful death.

Defendants m oved to dism iss. The district court granted

t h e m otion and dism issed the Clayborn Plaintiffs’ o perative

com plaint o n the grounds that the direct liability claim s failed

to adequately allege proxim ate cause, and that the secondary liab ility claim s f

ailed to p l au sib ly alleg e su b stantial assistance

or that ISIS com

m itted, planned, o r authori z e d t he San Bernardino Attack. The Clayborn Plaintiffs only appeal the distric t c o u r t ’s ruling that they failed to adequately plead a secondary liability claim for aiding and abetting international terrorism under 18 U.S.C. § 2333(d).G O N Z A L E Z V

. G O O G L E 21

II

W e review de novo a district co u r t ’s order granting a

m otion to dism iss pursuant to Federal Rule of Civil Procedure 12(b)(6), accepting all factual allegations as true and construing them in the light m ost favorable to the nonm oving party. Fields v. Twitter, Inc.

, 8 8 1 F .3d 739, 743 (9th Cir. 2018).

These appeals concern claim s for civil liability under the ATA. The civil rem edies section of the ATA perm its United States nationals to recover dam ages for injuries suffered “by

reason of an act of international terrori s m. ” 18 U.S.C. § 2333(a). The ATA contains crim inal provisions, the violation of which can give rise to a cause of a c tion under § 2 3 33(a) p rovided o ther conditions are m et. Fields

, 881 F .3d at 743. Speci fically, 1 8 U .S.C. §§ 2339A, 2339B,

a n d 2339C crim inalize providing m aterial support fo r terrorism

, providing m at e ri a l support for foreign terrorist organizations, and financing terrorism , respectively. 5

5 Section 2339A(a) p rohibits the provision of “m ate ri al support o r resources” by a n y o n e “k nowing or intending that they are to be used in preparation for, or in carrying out” any of sev e ra l e num erated crim es o f terrorism

. 18 U

. S.C. § 2339A(a). S ection 2339B(a)(1) prohibits the knowing provision of “m aterial support or resources to a foreign terrorist organization.” I

d. § 2339B(a)(1). Section 2339C(c) prohibits the kn owing “conceal[m ent] or disguise[] [of] the n ature, location, source, ownership, or control” of any support, resources, or funds, knowing that such “support or resources are to be provided, o r

. . . w ere provided, in violation of section 2339B.” Id. § 2 339C(c). Executive Order No. 13224, 31 C.F.R. Part 594 and 5 0 U.S.C. § 1705 generally prohibit p ro viding funds, goods, or services to or for the benefi t o f d e signated global terrorists.G O N Z A L E Z V

. G O O G L E

22

“[I]nternational terrorism ” is defin e d i n 18 U.S.C. § 2331(1). Acts of international terrorism “involve violent

acts or acts dangerous to hum an life that are a violation of the

c r i mi n al laws of the United States or o f any State, or t h at would be a cri mi n a l violation if com m itted within the jurisdiction of the United States or of any State.” 18 U.S.C. § 2331(1)(A). The acts m ust “appear to be i n tended—(i) to intim idate or coerce a civilian population; (ii) to influence the policy of a governm ent by intim idation o r coercion; o r (iii) to affect the conduct of a governm ent by m ass destr u c tion, assassination, or kidnapping.” Id. § 2331(1)(B). Finally, the

acts m ust “occur prim arily o utside the territorial jurisdiction of the United S tates, o r tr an s c end national boundaries

. . . . ” Id. § 2331(1)(C).

In 2016, Congress broadened the scope of ATA liability by enacting th e J u s tice Against Sponsors of Terrorism A ct (JASTA), Pub. L. No. 144-222, 130 Stat. 852 (2016). JASTA

am ended the ATA to include secondary civil liability for “any

person w ho aids and abets, by k nowingly p r ov i ding substantial assistance, or who conspires with the person w ho

co m mi t ted” an act of international terrorism that was “com m itted, planned, or authorized” by a f oreign terrorist organizat i on. Pub. L. 1 14-222, § 2(b), 130 Stat. 852, 854 (2016); 18 U.S.C. § 2333(d). Thu s , a s am ended, the ATA allows claim s for d irect liability for com m itting acts o f international terror pursuant to § 2333(a), or secondary liability pursuant to § 2333( d ) fo r aiding and abetting, o r conspiring to com m it, acts of international terrorism

.

III

These cases share som e com m on issues but took different paths to reach our court. In Gonzalez

, the dis trict courtG O N Z A L E Z V

. G O O G L E 23

prim arily reli e d o n § 230 im m unity to conclude that all but the G onzalez Plaintiffs’ revenue-sharing claim s were barred. Th e di strict court separately concluded the revenue-sharing claim s failed because the TAC did not plausibly allege t h at Google proxim ately caused No hem i’s death. The court allowed the Gonzalez Plaintiffs an opportunity to am end their revenue-sharing claim s, but the plaintiffs declined to do so, and final judgm ent was entered. In Taamneh and Clayborn

, the district courts did not consider § 230 im m unity. Instead,

t h e direct liability claim s were d ism issed for failure to plausibly allege proxim ate cause, and the secondary liability claim s were dism issed for failure to plausibly allege liability for aiding and abetting or conspiracy.

On appeal, the Gonzalez Plaintiffs begin by arguing that § 230 does not apply to their claim s at all. They m ake three

argum ents in support of this contention: (1) § 230 im m unity has no application to extrater r itorial claim s; (2) Congress im pliedly repealed § 230 when it am ended the ATA in 2016; and (3) § 2 30 im m

unity does not apply to ATA claim s based

on crim inal statut e s

. Alternatively, the Gonzalez Plaintiffs

arg u e t h at their claim s, both revenue-sharing and those

u n related to rev en

u e-sh aring, survive the application of § 2 3 0

. Finally, the Gonzalez Plaintiffs argue that the T AC adequately states claim s for direct and seconda r y liability under the ATA. The Taam neh P laintiffs and t h e Clayborn Plaintiffs ar gue their com plaints adequately allege that defendants v iolated the A TA b y aiding and abetting an act o f international terrorism

. 6 W e begin by considering the

6 Th o u gh t

he district court did not address t he application o f § 2 3 0 im m unity to the Taam

neh P laintiffs’ claim s, defendants raise § 230 o n appeal as an alternative basis for affi rm a n c e. W e decline to reach t h is question in the first instance.G O N Z A L E Z V

. G O O G L E

24

application of § 230 im m unity to the G o nzalez Plaintiffs’ claim s.

A

Congress enacted the C om m unicatio n s Decency Act as part of the Teleco mm

unications Act of 1996, Pub. L

. No. 104-104, 1 1 0 Stat. 56. Section 2 30 of the CDA “im m unizes providers of i n t eractive com puter services against liability arising from conten t cr eated by third parties.” F a ir Hous. Council of San Fernando V alley v. Roommates.Com, LLC

, 521 F.3d 1157

, 1162 (9th C ir. 2008) (en banc) (footnote

om itted). C o ngress des igned § 230 “to prom ote the free exchange of inform ation and i d e a s over the Internet and to
encourage voluntary m onitoring for offensiv e or obscene

m aterial.” Barnes v. Yahoo!, Inc.

, 5 70 F.3d 1096, 1099–1100 (9th Cir. 2009) (quoting Carafano v. Metrosplash.com, Inc.

, 339 F.3d 1119, 1122 ( 9th C ir. 2003)). Congress was concerned with “th e ease with which the Internet delivers indecent or offensiv e material, especially to m inors” and sought “to em

power int e r active com

puter service providers

to self-regulate.” Forc e v

. Facebook, Inc.

, 934 F.3d 53, 78–79 (2d Cir. 2019) (Katzm ann, C.J., concurring in part and d issenting in part). To avoid chilling speech, Con g re s s “m ade a policy choice

. . . n ot to deter harm ful online speech through the separate route of im posing tort liability on

com panies that serve as interm edia r i es for other parties’ potentially injurious m essages.” Carafano

, 339 F.3d at 1123 (alteration in o riginal) (quoting Z eran v. Am. Online

, I

nc. , 129 F.3d 327, 330 (4th Cir. 1997)).

The o p erativ e p ro v isio n , § 2 3 0 (c)(1), states “[n]o provider or user of an interactive com p uter service shall be treated as the publisher o r sp eaker of any inform ation provided byG O N Z A L E Z V

. G O O G L E 25

another inform ation content prov i d e r .” 47 U.S.C. § 230(c)(1). W e have said that, “[i]n general, this section

pr o t e c t s websites from liability for m aterial posted on the website by som eone else.” D oe v. Internet Br an d s , Inc.

, 824 F.3d 846, 850 (9th Cir. 2016).

Section 2 30’s use of the p hrase “p u b l i sher or speaker”

wa s prom pted by a New York state-court decision that h e l d

an inte r n e t service p rovider legally responsible for a defam atory m essage posted to one of its m essage b o a rds.
Roommates

, 5 21 F.3d at 1163 (citing Stratton Oakmont, Inc.

v. Prodigy Servs. Co.

, 1 9 9 5 W L 323710 (N.Y. S up. Ct. May 24, 1995) (unpublished)). S tratton Oakmont concluded that the internet servi c e provider “had b ecom e a ‘publisher’ under state law because it voluntarily deleted som e m essages from its m essage boards ‘ o n th e basis of offensiveness and bad taste,’ an d wa s therefore legally responsible for the

content of defam ato r y me ss a ges that it failed to delete.” Id. (em phasis added) (internal quotatio n m ark s o m itted ) (quoting Stratton Oakmont

, 1995 W L 323710, at * 4 )

. The o riginal goal of § 230 w as m odest. B y passing § 2 3 0

, Congress sought to allow interact i v e com puter services “to p erform

som e editing on user-generated content wi t hout thereby becom ing liable for all defam atory o r otherwise unl a wfu l

m essages that they didn’t edit or delete.” Id.

B

The Gonzalez Plaintiffs f irst argue that the presum ption against the extraterritorial application of federal statutes prevents § 230 from applying to their claim s. W e disagree.

The presum

ption against extraterritoriality requires that, “[a]bsent cle a r ly expressed congressional intent to theG O N Z A L E Z V

. G O O G L E

26

contrary, federal laws will be construed to have only dom estic application.” RJR Nabisco, Inc. v. E u ropean Cmty.

, 1 3 6 S. Ct. 2 090, 2100 (2016). The S uprem e Court “has established

a t w o

- s t e p f

r a me wo r k f

o r d e c i d i n g q u e s t i o n s o f extraterritoriality.” WesternGeco LLC v. ION Ge o p hysical Corp.

, 138 S. Ct. 2129, 2136 (2018). “The fi r s t step asks ‘whether the presum

ption against extraterritoriality has been rebutted.’” Id. (quoting RJR Nabisco

, 136 S. Ct. at 2101). The presum

ption is rebutted only when “ t h e text [of the

sta t u t e] provides a ‘clear indication of an extraterritorial application.’” Id. (quoting Morris o n v. Nat’l Austl. Bank Ltd.

, 5 6 1 U.S. 247, 255 (2010)). If the presum ption is not rebutt e d by the statute’s text, “the second step of [the] fram ework asks ‘ whether the case involves a dom estic application of the statute.’” Id. (quoting R JR N abisco

, 136 S. Ct. at 2101). This step requires the cour t to identify the statute’s focus, and ask “whether the conduct relevant to that focus occurred in United States territory.” Id. “If it d id, then the case involves a permi s s ib l e d om estic application of the statute.” Id.

The Gonzalez Plaintiffs argue that RJR Na b i sco recognized an exception to t his two-step fram ework where,

as here, all relevant conduct takes place o u t s ide the United States. To support this proposition, they rely on the Suprem e Court’s statem ent in R

JR Nabisco that “[b]ecause ‘all the relevant cond u ct’ regarding those v iolations ‘took place outside the United States,’ w e did not need to determ ine . . . the statute’s ‘focus.’” 136 S. Ct. at 2101 (citation om itted) (quoting K

iobel v. Royal Dutch Petroleum Co.

, 569 U.S. 108, 124 (2013)). The Gonzalez Plaintiffs m isread RJR Nabisco

. The passage they rely upon explained only that, on the facts of Ki o b el

, an inquiry into the f ocus of the statute wasG O N Z A L E Z V

. G O O G L E 27

unnecessary b ecause all the relevant co n d u ct was foreign. Id. 7

The G onzalez Plaintiffs next argue that eve n if t h e RJR Nabisco fram ework is applied, the fram ework dem o n s tr a tes that the their claim s involve an extraterritorial application o f § 230. Again, we are not persuaded.

RJR Nabisco requires that we begin by asking whether the

statute “gives a clear, affirm ative ind i c a tion that it applies extraterritorially.” 136 S. Ct. at 2 101. Neither party iden t i fi es any indication that C ongress intended § 230 to apply extraterritorially, so we proceed to step two.

At s t e p two, to determ ine whether claim s involve a dom estic application of the statute, w e m ust identify “the

st a tute’s focus.” Id. A statute’s focus is “the object of its solicitude, which can include the conduct it seeks to regulate,

as well as the parties and interests it seeks to protect o r vindicate.” WesternGeco

, 1 3 8 S

. Ct. at 2 137 (internal quotations and alterations om itted). “If the conduct relevant

to the statute’s focus occurred in the United S tates

. . . , then

7 Goog le separately argues that because § 230 does not directly regulate conduct, extraterritoriality p ri nciples are not im plicated at all. The Ninth Circuit addressed a sim ilar situation in a pre- RJR Nabisco case. See Blazevska v. Raytheon Ai r c r a ft Co.

, 5 22 F .3d 9 48 (9th Cir. 2008). There, our court concluded that t he General Aviation Revitalization Act’s

statute of repose did not “im perm issibly regulate conduc t t h a t ha[d] occurred abroad.” Id. at 953. Instead, the statute “m erely elim inate[d] the

power of any pa rt y t o b ri n g a suit for dam ages

. . . after the l i m itation period.” I

d. “Accordingly, the presum pt i o n against extraterritoriality sim ply [was] not i m

p l i cated

. . . .” Id. Because we conclude this case does not involve an im

perm issibly extraterritorial application of l aw under the R JR N a bisco fram ework, we need not decide the applicability o f Blazevska

. See also Force , 934 F.3d at 74.G O N Z A L E Z V

. G O O G L E

28

the case invol v e s a perm issible d om estic application of the statute.” Id. at 2136 (internal quotatio n marks om itted) (quoting RJR Nabisco

, 136 S. Ct. at 2101).

The ob j e ct of § 2 30(c)(1)’s solicitude is to encourage providers of interactive com puter services t o m o n itor their websites by lim iting liability. Fo r c e , 934 F.3d at 74 (concluding § 230’s “prim ary p urpose is lim itin g c ivil liability in A m erican courts”). Secti on 230 “im m unizes

pro v id ers o f interactive com puter services against liability arising f rom c o n t e nt created by third p arties.” Roommates

, 521 F.3d at 1162 (footnote om itted ) ; s e e also B arnes , 570 F .3d at 1100 (observing § 2 30(c)(1) “pr ecludes liability”). This lim itation of liability had the dual purposes of “prom ot[ing] the free ex change o f inform ation and ideas

over the Internet and

. . . encourag[ing] voluntary m onitoring for o ffensive or obscene m aterial.” Carafano

, 339 F.3d 1122. Because the focus o f § 230(c)( 1 ) i s lim iting liability, the conduct relevant to the statute’s focus occurs at the location associated w ith t h e imp o sition of liability. RJR Nabisco

, 136 S. Ct. at 2101.

In other word s, because § 230(c)(1) focuses on lim iting l iability, the relevant conduct occurs where im m uni ty i s im posed, which is where Congress intended the lim itation o f liability to have an effect, rather than the place where the claim s principally arose. As such, th e conduct relevant to § 230’s focus is entirely within the United States—i.e., at the

s it u s o f this litigation. See Force

, 934 F.3d at 74 (“Th e regulated conduct

— the litigation o f civil claim s in federal

courts—occurs entirely dom estically in its ap p l i cation here.”). W e therefore conclude the Gonzale z Plaintiffs’ claim s involve a dom estic application of § 230.G O N Z A L E Z V

. G O O G L E 29

C

The Gonzalez Plaintiffs also ar gu e t hat § 230 im m unity doe s n o t shield liability arising from violations of the ATA because § 2 30 was im

pliedly repealed. Specific a l l y , they contend that when Congress am end e d t h e ATA in 2016 by enacting JASTA, it im

pliedly repealed § 230. In support o f this argum ent, the Gonzalez P laintiffs rely on JASTA’s

statem ent of purp o s e , w hich explains that the aim of the

am endm ent was “to provide civil litigants with the broadest possible basis

, consistent with the C onstitution of the United States, to seek relief” for acts of interna t ional terrorism

. JASTA § 2(b) (em

phasis added). As explain e d, J ASTA altered the A TA by adding, am ong other things, secondary liability. See 1 8 U.S.C. § 2333(d). Despite its broad purpose, JASTA did not im

pliedly repeal § 230.

“[A]bsent a clearly expre s sed congressional intention, repeals by im

plicat i o n are not favored.” Branch v. Smith

, 5 38 U.S. 254, 273 (2003) (internal quotation m arks an d citations om itt e d )

. “An im plied repeal will only be f ound where provisions in two statutes are in ‘irreconcilable conflict,’ or where the latter A ct covers the whole subject o f the earlier one and ‘is clearly intended as a s u b stitute.’” Id. (quoting Posadas v. Nat’l City Bank

, 296 U.S. 4 97, 503 (1936)). “Irreconcilable conflict o ccurs if ‘there is a positive repugnancy’ between com peting provisions o r if those provisions cannot ‘m

utually c o-exist.’” King v. B lue C ross & Blue Shield of Ill.

, 87 1 F. 3d 73 0, 7 40 (9th Cir. 2 017) (quoting Radzanower v. T ouch e Ro s s & Co.

, 426 U.S. 148, 155 ( 1 9 7 6 ) )

. “[W ]hen two statutes are capable of coexistence, it is the d

uty of the courts

. . . to r eg a r d e ach as effective.” Id. (alterations in original) (quoting Radzanower

, 426 U.S. at 155).G O N Z A L E Z V

. G O O G L E

30

To determ ine whether JASTA had an y e ffect on the application o f § 230, we start b y exam ini n g the statutory l a n g u a g e , a n d n o t

— a s t h e G o n z a l e z P l a i n t i f f s urge—JASTA’s statem ent of purpose. Pr e a m b les and

p refatory language are insufficient to alter the substance of the phrases they precede, even when codified. Se e , e .g. , Kingdomware Techs., Inc. v. United S tates
, 136 S. Ct. 1969, 1978 (2016) (observing that t he “clause announc[ing] an objective

. . . [did] not change the plain m eaning of t h e ope r a tive clause”). The Gonzalez Plaintiffs do not identify

any substantive provision of JASTA that conflicts with § 230. As we h av e r ecognized, § 230 protects from liability only a specific class of defendants facing a par t i c u lar type o f claim

— i.e., it protects provi d e r s and u sers of interactive

com puter services from claim s seeking t o t r eat them as publishers or speakers of inform ation provided by others. See
Barnes

, 570 F.3d at 1100–01; see also § 230(c)(1). Thus, by its own term s, § 230 creates “an affirm ative defense to liability under Section 2 333 [ of the A TA] for only the narrow set of defendants and conduct to which Section 230 applies.”
Force

, 9 34 F .3d at 72. There is no provision of JASTA to the contrary. JASTA expanded the scope o f § 2333 liability for

acts of international terrorism

, see 18 U.S.C. § 23 3 3 ( d), but it did not m odify or repeal § 2 30 im m u n ity, Force

, 934 F.3d

at 7 2 ( “JASTA m erely expanded Section 2333’s cause o f action to secondary liability; it p rovides no obstacle

. . . to applying Section 230.”).

Accordingly, JASTA and § 230(c)(1) can bo th b e

e nforced without contradicting the other, or depriving t h e other of “any m eaning at all.” Radzanower

, 4 26 U.S. at 153 (quoting T. Sedgwick, The Interpretation of S t a tutory and Con s t i t utional Law 98 (2d ed. 1874)). Courts have “not hesitated to give effect to two statutes that overlap, so long asG O N Z A L E Z V

. G O O G L E 31

each reaches som e d istinct cases.” J.E.M. Ag Supply, Inc. v. Pioneer H

i-Bred Int’l, Inc.

, 534 U.S. 124, 144 (2001). Under the Gonzalez Plaintiffs’ reading o f JASTA, a n y l iabilityim posing statute enacted after § 230 would have t o be

c o n s trued to lim it § 230 im m u nity. Such a reading ru n s directly contrary to the presum ption agains t finding im

plied

r e p eal. For these reasons, w e conclude JASTA did n o t im pliedly repeal § 230.

D

Finally, the Gonzalez Plaintiffs arg u e th at § 2 3 0 i m m unity

c a n n e ver apply to A TA claim s because the ATA perm it s private civil enforcem ent of counter-terrorism p rovisions that otherwise give rise to crimi n al liability, and § 230(e)(1) includes an exception pro v iding that “[n]othing in this section sh a l l be construed to im

pair the enforcem ent of

. . . any

. . . Federal crim inal stat u t e

. ” 47 U.S.C. § 230(e)(1). G oogle responds that the e x ce p tion in § 230(e)(1) extends only to crim inal prosecutions, n ot to actions for civil dam ages like this one. On this point, Google has the better argum ent.

C o u rts have consistently held that § 230(e)(1)’s lim itation

on § 230 im m

unity extends o nly to crim inal prosecutions, and

not to civil actions based on crim inal statutes. For exam ple, the First Circuit concl u d e d t h at a civil rem edy provision in

t he T rafficking Victim s P rotection Reauthorization Ac t
, which allowed victim s to brin g s u i t against p erpetrators o f

sex trafficking, did not fall within the § 230(e)(1) exception.
Doe v. Backpage.com, LLC

, 817 F .3d 12, 23 (1st Cir. 2016). The court principally relied on t he m eaning of the statutory phrase “enforcem ent of

. . . any

. . . Federal crim inal statute,” which excludes civil statutes, but also reasoned that any

am biguity in the su b s ection’s text was resolved by its title,G O N Z A L E Z V

. G O O G L E

32

“[n]o effect on crim inal law,” id

. ( a lteration in original), because t h is language “indicate[d] that the p rovision [was] lim ited to crim inal prosecutions,” id. The Second Ci rcuit recently agreed with this analysis when it co n s i d e red the application of § 230 to ATA claim s. See Force

, 9 34 F.3d

at 72 (“W e

. . . join the F irst Circuit in concluding that Section 2 30(e)(1) is ‘quite cle a r l y

. . . lim ited to crim inal prosecution s .’ ” (second alteration in original) (quoting Backpage.com

, 817 F.3d at 23)). W e agree with the First and Second Circuits, and ho l d that § 230(e)(1) is lim ited to crim inal p rosecutions. Accordingly, § 230(e)(1) does not preclude the application of § 230(c)(1) im m unity.

E

Having concluded that the Gonzalez Plaintiffs’ claim s are not categorically excluded from the reach of § 230 im m unity,

we next consider t h e application o f § 230 to the Gonzalez TAC. The G onzalez Plaintiffs argue that the im m unity afforded by § 230 do e s n ot bar their claim s because § 230 im m u nizes only those wh o p u b lish content created by third

par t i es , a nd their claim s are directed to content created by Google. Google responds that the content the T AC challenges w as indeed created by third parties—presum ably, ISIS—and that the Gonzalez Plaintiffs’ claim s im perm issibly seek to treat G oogle as a p ublisher of that content. W e affirm the d istrict court’s ruling that § 230 b ars all of the TAC’s claim s except to the extent the TAC presents claim s prem ised

on the allegation that Google shared advertising revenue with ISIS. 8

8 The Gonzalez Plaintiffs also argue that it was im proper for the district court t o consider the application of § 230 on a m otion to dism iss.
W e disagree. “W hether a particular ground for opposing a claim m ay beG O N Z A L E Z V

. G O O G L E 33

Section 230(c)(1) precludes liability f or “(1) a provider o r

user of an interactive c o mp u ter service (2) whom a plaintiff seeks to treat

. . . as a publisher or speaker (3) of inform ation provided by another inform ation content provider.” 9 B arnes , 570 F.3d at 1100–01 (footnote om itted). W e first address the Gonzalez P laintiffs’ theories o f liability that are not directed

t o revenue-sharing, considering each elem ent of § 2 3 0 separately.

1

As to the f

irst elem ent of § 230, the parties do not dispute that Google is an “interactive com puter service” p rovider as defined in 47 U.S.C. § 230(f)(2). W e agree. Ro o mma t e s , 521 F.3d at 1 162 n.6 (“[T]he m ost commo n interactive

comp u ter services are w ebsites.”); see also Kimzey v. Yelp!,
Inc.

, 836 F.3d 1263, 1268 (9th Cir. 2016) (“Yelp is p lainly a provider of an ‘interactive com puter service’

. . . , a term that

the basis for dism issal for f ailure to state a claim depends on whether the

a llegations in the com

plaint suffice to establish that ground.” J o n e s v

. Bock

, 549 U.S. 199, 215 (2007). H ere, “the ‘allegations in the com plaint suffice t o establish’ the defense,” and thus the “ affirm ative defense m ay be considered p roperly.” Sams v. Yahoo! Inc.

, 7 13 F.3d 1175, 1179 (9th Cir. 2013) (quoting Jones

, 549 U.S. at 215); see also Ricci v. Teamsters Union Local 456

, 781 F.3d 2 5, 27 (2d Cir. 2015) (per curiam ) (considering whether § 230 im

m unity barred plaintiffs’ claim s on a 12(b)(6) m otion to dism iss).

9 Barnes lim i ted i ts sum

m ary of § 230(c)(1) eligibility requirem ents
to instances where “plaintiff[s] seeks to treat [the defendant], under a state law cause of action

, a s a publisher or speaker” because that case only concerned s tate l aw claim s. 570 F.3d at 1100 (em phasis added). I n Roommates

, we acknowledged t hat § 230 im

m unity is not lim ited to cases in which plaintiffs assert state law c l a im s. 521 F.3d at 1164; see also
Barnes , 570 F.3d at 1100 n.4.G O N Z A L E Z V

. G O O G L E

34

we interpret expansively under the C DA. ” ( q uotations and alterations om itted)).

2

As to th e second elem ent, the Gonzalez Plaintiffs argue their claim s do not inherently require a court to treat G oogle

as a publisher or speaker. G oogle responds that the thrust o f the Gonzalez Plaintiffs’ claim s is that Google did n o t d o enough to block o r rem ove content, and that such c l a im s necessarily require the court to tre a t Go o g le as a publisher. On this point, we agree with Google.

W hat m atters when we assess this elem ent is “whether the

cause of action inherently re q ui res the court to treat the defendant as the ‘publisher or speaker’ of content provided by another.” Barnes

, 570 F.3d at 1102. This elem ent is satisfied when “the duty tha t t he plaintiff alleges the defendant violated derives from the defendant’s status or conduct as a ‘publisher or speaker.’” Id.

The Gonzalez Plaintiffs arg u e t h a t their claim s do not
treat Google as a publisher, but instead assert a sim ple “duty

not to support terrorists.” They m aintain that just as the ATA prohibits a retailer like W al-Mart “from supplying fertilizer, knives, or even food to ISIS,” the ATA prohibits Google from supplying ISIS with a com m unication platform

. The Gonzalez Plaintiffs’ characterization of their claim as asserting a “duty not t o s u pport terrorists” overlooks that publication itself is the form of support Google allegedly provided to ISIS. See Force

, 934 F.3d at 65 (recognizing that suppl y i ng a platform and com m unication services “falls within the heartland of what it m eans to be the ‘publisher’ o f inform ation under S ection 230(c)(1)”). The Plaintiffs’ non-G O N Z A L E Z V

. G O O G L E 35

revenue sharing claim s seek to im pose liability for the content Google allowed to be posted on its platform

.
Publishing encom passes “any activity that can be boiled d own to d eciding whether to exclude m aterial that thi r d parties seek to post online

. . . .” Roomma t e s , 5 21 F.3d at 1170–71. “[P]ublication involves reviewing, editin g , and deciding w hether to publish o r to withdraw from publication

t h i r d-party content.” Barnes

, 570 F.3d at 1102; see al s o Klayman v. Zuckerberg

, 753 F.3d 1354, 1359 (D.C. Ci r. 2014) (“[T]he very essence of publishing is m aking the decision whether to pr in t o r r etract a given p iece of content

. . . .”). Here, the Gonzalez Plaintiffs assert that Google failed

to p revent ISIS from using its platform

, and thereby allowed ISIS to disse mi n a te its m essage of terror. Because the nonrevenue sharing claim s seek to im pose liability for allowing ISIS to p lace content o n the YouTube platform

, they seek to treat Google as a publisher.

3

The Gonzalez Plaintiffs argue that Google does m ore than

m erely repu bl i sh content created b y third p arties; the T AC alleges that Google “creat[es]” and “develop[s]” the ISIS

content that appears on YouTube, at least in part, and therefore receives no protection under § 230. Again, we disagree. This argum ent is precluded by this court’s § 230 precedents.

The Gonzalez Plaintiffs are co r rect that § 230 im m unity only applies to the extent interactive c om puter service providers do not also provide the challenged inform ation

content. R oommates
, 521 F.3d at 1162–63; see also Carafano

, 339 F.3d at 1123. An “informa t i o n contentG O N Z A L E Z V

. G O O G L E

36

p ro vider” is defined as “any person or entity th a t i s responsible, i n whole or in part, for the creation or development of inform ation provided through the Internet o r any o ther in t eractive com puter service.” 4 7 U.S.C. § 230(f)(3) (em phasis added).

W e have held that a website that “creat[es] or develop[s]”

content “by m aking a m aterial contribution to [its] creation o r developm e n t ” l o ses § 230 im m unity. Kimzey

, 836 F.3d

at 1 269. A “m aterial contribution” does not refer to “m erely

. . . augm enting the content generally, but to m aterially contributing to its alleged u nlawfulness .” Roomma tes
, 521 F. 3 d a t 1167–68 (em phasis added). This test “draw[s] the line at the ‘crucial dist i n c tion b etween, on the one h and, taking actions” to display “ a c tionable content and, o n the

o th er hand, responsibility for what m akes the display e d

content [itse l f] i llegal o r actionable.” Kimzey

, 836 F.3d at 1269 n .4 (internal quotation m arks om itted) (quoting Jones v. Dirty World E

nt. Recordings LLC

, 755 F .3d 3 98, 413–14 (6th Cir. 2014)). Other circuits have adopted this “m aterial contribution” test, acknowledging that m aking a m aterial contribution does not m ean “m erely taki n g a c t ion that is

necessary to the display o f the allegedly illegal content,” but rather, “being responsi b l e for what m akes the displayed

c o n t e nt allegedly unlawful.” Dirty World Ent.

, 755 F.3d

at 410; see also, e.g. , FTC v. LeadClick Media, LLC

, 838 F.3d 158, 176 (2d Cir. 2016); Klayman

, 753 F.3d at 1358; Nemet Chevrolet, Ltd. v. Consumeraf fairs.com, Inc.

, 591 F.3d 250, 257–58 (4th C ir. 2009); FTC v

. A ccusearch Inc.

, 570 F.3d 1187, 1197–1201 (10th C ir. 2 009 )

. Ab sent this sort o f “m aterial contribution,” G oogle does not qual i fy as anG O N Z A L E Z V

. G O O G L E 37

“inform ation content provider,” and m ay be eligible for § 230 im m u nity. See Kimzey

, 836 F.3d at 1269–70. 1 0

Plainly, an interactive com p uter service does not create o r develop content by m erely p roviding the public wi t h access to its platform

. A “website does n ot create or develop content when it m erely provides a neutral means by which third parties can pos t i n form ation o f their own independent choosing online.” Kimzey

, 836 F.3d at 1270 (quoting Klayman

, 753 F.3d at 1358). Thus, in Kimzey

, we concluded that a provid e r d oes not create o r develop content when its website “does ‘absolutely nothing to enhance the defam atory sting of the m essage’ b eyond the w ords offered by the [thirdparty] user.” Id. (quoting Roommates

, 521 F.3d at 1172).

The Gonzalez Plain t i ffs concede that Google did not initially create any ISIS v ideos, but allege that Google creates the “m osaics” by w hich that content is delivered. According

to the Gonzalez TAC, Google m akes a m aterial contribution

to the unlawfulness of ISIS content by pairing it with selected advertising and other videos because “pairing” enhances user

en ga g em ent with the underlying content. O ur case law forecloses the argum ent that this type of pairing vitiates § 230 im m u nity.

10 The G onzalez Plaintiffs argue in passing that the district court erred by “conflat[ing]” the definitions of “ creation” and “dev e l o pm ent” in § 230(f)(3). A ccording to the Gonzalez P laintiffs

, b ecause Roommates described its “m aterial contribution” t est in the c o n text of construing “developm ent,” it “has nothing to do with t h e d efinition of ‘creation.’”
W hatever the distinction b etween creation and developm ent, our case l aw

m akes clear that an entity that does not m aterially contribute to the alleged unlawfulness of the content is neither a creat o r n o r a developer f o r purposes of § 230(f)(3). See Kimzey , 836 F.3d at 1269–70.G O N Z A L E Z V

. G O O G L E

38

In Roommates

, we recognized that a we bsite is not transform ed into a content creator or developer by virtue o f supplying “neutral tools” that deliver content in response to

user inputs. See 521 F.3d at 1171; see also i d

. at 1169; Kimzey

, 836 F.3d at 1270. Roommates relied o n o u r earlier decision in Ca r afano

, which concerned a prankster’s unauthorized creation of a libelous profile im perso n a t i ng actress C hristianne Carafano on an online datin g site.
Roommates

, 521 F. 3 d at 1171; see also Carafano

, 339 F.3d

at 1121–22. Carafano sued the online dating site for invasion of privacy, m isappropriation of the right of publicity, defam ation, and n egligence. Carafano

, 3 39 F.3d at 1121–22.

W e determ ined that the dating w ebsite in C arafano “provided neutral tools specifically d e s igned to m atch

rom antic partners depending on their voluntary inputs .”
Roommates

, 5 21 F.3d at 1172. The website was not transform ed into the creator o r d eveloper of libelous content contained in users’ dating profiles, even though its

m atchm aking function a l ity allowed that content to be m ore effectively dis s e mi nated. See id

. Carafano held that the dating website’s “decision to structure the inform ation provi d ed by users [in order to]

. . . offer additional features, such as ‘m atching’ profiles w ith sim ilar characteristics” was consistent with § 230 im m u nity. 339 F.3d at 1124–25. “[S]o long as a third party w illingly provides the essential p u blished content, the interactive [com p uter] service provider receives full im m unity regardless of the specific e d i t i n g or selection process.” Id. at 1124.

Cr itically, Carafano ’s “neutral tools” were neutral because the website did n o t “encourage the posting o f defam atory content” by m erely providing a m ean s f or users to publish the profiles they created. R o ommates
, 521 F.3d atG O N Z A L E Z V

. G O O G L E 39

1171. “[I]ndeed, the d efam atory posting was contrary to the website’s express policies.” Id.

In contrast, the defend a n t in Roommates operated a website for m atching renters with prospective tenants that did contribute to the alleged illegality. Before users could search listings or post housing opportun i t i es, the website required them to create profiles

. I d. a t 1161. To do so, users were dir e c t e d t hrough a series of questions to disclose their sex, sexual orientation, and whether they had children. Id. They

were also required to describe their p referred renter or tenant with respect to these sam e three criteria, a n d e ncouraged to “provide ‘Additional Com m ents’ describing them selves and their desired room m ate in an open-ended essay.” Id.

The pla i n tiffs in Roommates alleged that the website

op erato r v i o lated f

ed eral a n d state law s b arrin g discrim ination in housing. Id. a t 1 1 6 2. The d efendant website operator argued that it was entit l e d to § 2 30 im m unity. Id. Our en b an c court concluded the website—by requiring users t o disclose their sex, sexual orient a t ion, whether they h ad child ren , and the traits they preferred in their room m ate—was designed to encourage users to post content that violated fair housing laws. Id. at 1 1 6 1

, 1164–66. “By requiring

s ubscribers to provide the inform ation as a condition o f accessing its service,” and requiring subscribers to choose between “a lim ited set of pre-populated answers” the website becam e “m

uch m ore than a passive transm itter,” and instead becam e “the developer, at least in p art, of that inform ation.” Id. at 1166. The Roommates w ebsite did not em p loy “neutral tools”; it required users to inpu t d i scrim inatory content as a p rereq u isite to accessing its tenant-landlord m atch in g serv ice. See id. at 1169. The website therefore lost its § 230 im m unity with respect to the discrim inatory content it prom pted, but itG O N Z A L E Z V

. G O O G L E

40

retained im m

unity for generically askin g u s ers to provide “Additional Com m ents” without t e l l in g them “what kind o f inform ation they should or m ust include.” Id. at 1174.

W e recently revisited the s c ope of § 230 im m u nity in Dyroff v. Ultimate Software Grp., Inc.

, 934 F. 3 d 1093 (9th Cir. 2 01 9 )

. There, an online m essaging board called the Experience Project allowed users to share first

- person experiences, post and answer questions, and interact with other users about v arious topics. Id. at 1094. A user nam ed
W e sl e y Greer posted an inquiry about opportunities to buy heroin, and received a response fr o m another user. Id. at 1095. A day after m eeting up w ith the responder, Greer died because the h eroi n he purchased had been laced with fentanyl. Id. Greer’s m other filed suit a g ai ns t the website operator, an d the website m oved to dism iss based on § 230 im m u nity. Id. at 1095–96.

The plaintiff in Dyroff argued that the website created and developed online content because the website “used features and functions, including algorithm s, to analyze user posts

. . . and recom m end other u ser groups.” Id. at 1098. W e

c o n c l u d e d “ [ t ] h e s e f

u n c t i o n s

— r e c o mme n d a t i o n s a n d

n o t i f i c a t i o n s

— [ w e r e ] t o o l s me a n t t o f a c i l i t a t e t h e

com m unication and content of o t h e r s , ” and “not content in and of them selves.” I

d

. The m essage board in Dyroff

em ployed neutral tools sim ilar to the ones challenged by the Gonzalez Plaintiffs

. Though we accept as true the TAC’s allegation that Google’s algorithm s recom m end ISIS content
t o users, the algorithm s do not treat ISIS-created cont e n t differently t h an any other third-party created content, and thus are entitled to § 230 im m unity. Id. ; see also Roommates

, 521 F.3d at 1171–72; Carafano , 339 F.3d at 1124.G O N Z A L E Z V

. G O O G L E 41

W e conclude the TAC d oes not allege that Google’s YouTube service i s m aterially distinguishable f rom the

m atchm aking w ebsite at issue in C a rafano or the algorithm s

em ployed by the m essage board in D yroff

. It a l l eges that Goo gle recom m ends content—including ISIS videos—to

users based u

pon users’ viewing history and what is known about the users. The Gonzalez Pl aintiffs allege that G oogle sim ilarly targets users fo r advertising b ased on the content they have selected and other inform ation about users. In this

way, a user’s voluntary a ctions inform Google about that user’s p references for the types of videos and advertisem ents the u se r wo ul d like to see. Rather than suggesting m atches for dating, Google m atches what it knows about users based

on their historical act i o ns and sends third-party content to

users that Google anticipates they will p refer. This system is certainly m ore sophisticated than a traditional search engine, which require s u s ers to type in textual queries, but the core principle is the sam e: Google’s algo r i t h m s select the particular content p r ovided to a user based o n that user’s inputs. See Roommates

, 521 F. 3 d at 1175 (observing that search engines are im m

une under § 230 because they provide

con t e nt in response to a user’s q ueries “with no direct

encouragem ent t o perform illegal searches or to p ublish illegal content”).

The Gonzalez com

plaint is devoid of any allegations that Goo g le specifically targeted ISIS content, or designed its website to encourage videos that further the terrorist group’s

m ission. Instead, the Gonzalez Plaintiffs’ allegations suggest that Google provided a neutral platform tha t d i d not specify

or prom pt the type of content to be subm itted, nor determ ine particular types of content its algorithm s w ould prom ote. The Gon z a l ez Plaintiffs concede Google’s policies expressly prohibited the content at issue. S ee id. at 1171. Accordingly,G O N Z A L E Z V

. G O O G L E

42

the type of al g o r ithm challenged here, without m ore, is indistinguishable from the one i n Dyroff and it does not deprive Google of § 230 im m unity.

W e are not alone in reaching this conclusion. In a c a s e involving allegations that Facebook u nlawfully pr o vided a

com m unications platform to Ham as in violation of the ATA, the Second Circuit concluded that Faceboo k wa s entitled to § 230 im m

unity. Force

, 934 F.3d at 64–72. The plaintiffs in
Force

, surviving fam ily m em bers o f victim s allegedly

m urdered by Ham as, sought to treat Facebook as a p ublisher of third-party inform ation, even where “it use[d] tools such as

a l g o r i thm s that [were] designed to m atch that [third-party] inform ation with a consum er’s in t er e sts.” Id. at 66. The Second Circuit recognized that Facebook’s algorithm s ma y have m ade content m ore v isible or available, but h eld this did

not am ount to developing the underlying inform ation. Id. at 70. Force f

urther observed that since the e a r l y days of the Internet, w ebsites “have always d e c ided

. . . where on their

s it e s

. . . p articular third-party content should reside and t o whom it should be shown” but no case l a w d e nies § 230 im m u nity “because of the ‘ma t chm aking’ results of such editorial decisions.” Id. at 66–67. Our precedent r e q uires that we reach the sam e outcom e and we h old, consistent with

our case law, that Google is entitled to § 230 im m unity with respect to the Gonzalez Plaintiffs’ theories of liability that are not directed to revenue-sharing.

Our dissenting colleague argues § 230 sho ul d n ot im m u nize Google from liability for the clai ms r elated to its algorithm s, which the dissent characterizes as am plifying and contributing to ISIS’s originally p osted content. The dissent shares the views expressed by the partial conc u r rence andG O N Z A L E Z V

. G O O G L E 43

dissent in Force. 934 F.3d at 76–89 (Katzm ann, C.J., concurring in part, dissenting in part).

As explained, Force also arose from terrorist attacks. The
Force plaintiffs alleged that “Facebook collect[ed] d etailed inform ation about it s users” and F acebook’s algorithm s “utilize[d] the collected data to suggest fr iends, groups, products, services and local events, and [ to] target ads based

on each user’s input.” Id

. a t 82 (internal q uotation m arks

om itted).

For t wo r e a sons, the partial d issent in Force argued that
F aceb o o k ’s f

rien d - an d co n ten t -su g g estio n alg o rithm s created

new content, and thus Facebook was n ot entitled to § 230 im m u nity. Id. First, the p artial dissent reasoned t h at F a c e b o o k ’ s a l g o r i t h m s c o m m u n i c a t e d t h e i r o w n

m essage—i.e., the algorithm s suggested the user would lik ely be interested in cert ai n additional content. Id. Second,
F aceb o o k ’s f

rien d - an d co n ten t -su g g estio n alg o rithm s created and m aintained “real-world social networks.” Id.

Cit i ng our circuit’s decision in Roommates

, the p artial dissent in Fo r c e reasoned that suggestions generated by Facebook’s algorithm s b ased on users’ shared interest in terrorism “directly related to the alleged illegality of the site,” and therefore Faceboo k went beyond the role of a m ere publisher. Id. at 82–83. R espectfully, this is not a correct read ing of Roommates

. The Roommates website required

users to identify them selves by sex, sexual orientati o n , and whether they had children, then directed users to d escr i b e th eir preferred tenant o r landlord u sin g p re-p o p u lated an sw ers concerning the sam e criteria. 521 F.3d at 1161, 1169–70. In this way, the w ebsite prom pted discrim inatory responses that violated fair housing laws. Id. at 1169–70. Be cause theG O N Z A L E Z V

. G O O G L E

44

website itself g e n erated the options for selecting a tenant o r landlord based on discr iminatory criteria, our en banc court concluded the website m aterially cont r i b uted to the unlawfulness of the posted content. Id.

As we have explained, Google’s algorithm s function like traditional search engines that s e lect particular content for

users based o n user inputs. See Roommates

, 521 F.3d at 1175 (observing search engines are entitled to § 230 im m unity because they provid e content in response to users’ inquires “with no d irect encouragem ent to perform illegal searches o r to publ is h illegal content”). The TAC does not allege that Google’s algorithm s prom pted ISIS to post unlawful content. Nor does the T AC allege that Google’s algorithms treated ISIS-created content differently than any other t h ird-party created content. See id. at 1171–72. C ontrary to the dissent’s assertion, we do not hold that “m achine-learning algorithm s

can never produce c on t ent within the m eaning of Section 230.” W e o nly rei t e r ate that a website’s use of contentneutral algorithm s, without m ore, does not expose it to liability for content posted by a third-party. Under our existing case law, § 230 requires this result.

The dissent concedes algorithm s can be neutral, but it

argues § 230 im m

unity should not apply when the published “m essage itself is the danger.” B ut this is not where Congress drew the line

. At the tim e Congress enacted § 230, m any considered it “ impossible for service p roviders to screen each of their m illions of po stings for possible problem s.” Carafano

, 339 F.3d at 1124 (em phasis added) (quoting
Zeran

, 129 F .3d at 330–31). Against this backdrop, Congress did n ot differentiate dangerous, crim inal, or obscene content from innocuous content when it drafted § 230(c)(1). Instead, it broadly m andated that “[n]o provider . . . of an interactiveG O N Z A L E Z V

. G O O G L E 45

com puter service shall be treated as the p ublisher or speaker of any information provided by another inform ation content provider.” 47 U.S.C. § 230(c)(1) (em phasis added).

W e share the dissent’s co n c e r ns about the breadth o f § 230. As the dissent observes, “there is a r is i n g c horus o f judicial voices cautioning against an o verbroad reading of the

scope of Section 230 im m unity,” and the feasibi l i t y o f screening for dan g e r ous content is being revisited. For

exam p le, w ebsites are leveraging new technologies to detect, flag, and rem ove large volum es of c r i m inal content such as child p ornography. 1 1 In light of the dem onstrated ability to detect and isolate at least som e dangerous content, Congress

m ay well decide that m ore regulat ion is needed. In the

m eantim e, our decision d oes not extend what the dissent rightly describes as § 230’s sweeping scope.

11 According t o the D epartm ent of Justice, “the vast m ajority o f [National Center for Missing & Exploited C hildren (NCMEC)] reports

com e f rom direct m essaging services and are usually generated as a result of platform s’ use of autom ated hashing m easu re s (such as PhotoDNA),

groom ing indicators, artificial i n t e l l igence and other technologies to identify and report child sexual abuse m a t erial.” DOJ Office of Public Affairs, Ac t i n g AG and Five Country Statement on the T emporary Derogation to the ePri va cy D i rect i ve t o Co m b a t Child S exual Exploitation and Ab u s e , U nited States Departm ent of Justice (Jan. 1 2, 2021), https:// w w w

. j us ti c e

. g o v /o p a / p r/ a c t i n g -a g -a n d -fi v e -country-statem ent-te

m porary-derogation-eprivacy-directive-com b at-chil d

. Facebook reports that “[i]n addition to photo-m atc hi n g t echnology, [Facebook i s] using artificial intelligence and m achine learn ing to proactively d etect child nudity and previously u nknown child exploitative content when it’s uploaded” and to r eport it to NCMEC. See Antigo n e D a v i s , New Technology to Fight Child Exp l o i t a tion

, Facebook (Oct. 24, 2018), https://about.fb.com

/news/2018/10/fighting-child-exploitation.G O N Z A L E Z V

. G O O G L E

46

In his partial concurrenc e a n d partial dissent in Force

, Chief Judge Ka tzm ann provided a thorough analysis o f § 2 30’s legislative history. Fo r c e , 934 F.3d at 77–80 (Ka tz ma n n, C.J., concurring in part and dissenting in part). The F orce partial dissent persuasively explains that when it enacted § 230, “Congress was focused squarely on p rotecting

m inors from offensive online m aterial” and sought to “provide[] ‘Good Sam aritan’ protections from civil liability for providers or users of an interactive com puter service.” Id.

at 79–80 (quoting S. Rep. No. 104-230, at 194 (1996) (Conf. Rep

. ) )

. Despite this clear goal, the language Congress adopted in § 230(c)(1) cu ts a m uch wider swath. Id. ( “ Whatever prototypical situation its drafters m ay have h ad in m ind, § 230(c)(1) does not lim it its protection to situations involving ‘obscene m aterial’ p rovided by others, inst ead

u si n g the expansive word ‘inform ation.’”). Chief Jud g e Katzm ann urged his colleagues to conclude § 230(c)(1) need

not be interpreted to im m

unize websites’ friend- and contentsuggestion algorithm s, but as w e explain, Ninth Circuit case law forecloses his argum ent.

In sum , though we agree the Internet has grown into a sophisticated and powerful global engine the drafters of § 230 could n ot have foreseen, the decision we reach is dictated by the fact that w e are not w riting o n a b lank slate. Congress affirm atively im m

u n ized interactive com puter service providers that publish the speech or content of others. 1 2

12 T h e dissent would create a new federal com m on law cause of action treating social m edia com panies as m akers and sellers of products through forced advertisin g, thereby circum venting § 230’s expansive im m unity.
Even if we agree Congress should act to narrow the scope of § 2 3 0 im m unity or regulate the use of neutral algorithm s, w e a re not f r ee to

m anufacture entirely new causes o f a c t i o n m erely because the political branches have not acted. The Suprem e Court has explained “[t]he vestingG O N Z A L E Z V

. G O O G L E 47

F

The Gonzalez Plaintiffs’ revenue-sharing theory is distinct from the other theories of liability raised in the TAC. This theory is prem ised on the allegation that b ec a u s e it shared advertising revenue with ISIS, Google should be held directly liable for providing m aterial support to ISIS p u rsuant
to § 2333(a) and secondarily liable for p roviding substantial assistance to ISIS pursuant to § 2 333(d). The district court’s order excluded the G onzalez Plaintiffs’ revenue- sharing claim s from its application of § 230. On appeal, Google does

not sep a ra tely respond to the Gonzalez Plaintiffs’ revenuesharing claim s. Instea d

, Google lum ps all of the TAC’s theories together for p u rposes of its § 2 30 argum ent. Based

on our review of case law, the ques t i o n wh e ther § 230 im m u niz es an interactive com puter service provider’s revenue-sharing paym ents appears to be one of first im pression for the courts of appeals. W e conclude that § 230 does not im m

unize Goo gl e fr om the claim s prem ised on revenue-sharing.

Plaintiffs allege that Google generates revenue b y selling advertising space through its A dSense program , including advertising space that a ppears on YouTube. Through AdSense, Google sells advertising o pportunities and displays advertisem ents to YouTube viewers accessing other content. Google targets advertisem ents b ase d o n the content of the

of jurisdictio n i n t h e federal courts does not in and of itself give r i se to authority to form

ulate federal com

m on law . . . nor does the existence of congressional authority under A rt

. I m ean that f ederal courts are free to develop a com m on law to govern those areas until Congress acts.” Texas Indus., Inc. v. Radcliff Materials, In c

. , 451 U.S. 630, 640–41 (1981) (internal citation om itted).G O N Z A L E Z V

. G O O G L E

48

advertisem ent s , wh a t is k nown about the viewer, and the

content of the posted video. If a YouTub e u s e r el ects to participate in the AdSense p rogram , Google shares wi t h t h e

user a portion of the revenue generated b y the advertisem ents

on t h e u ser’s videos. For exam p le, suppose a user participating in the AdSense program posts a v i d e o tutorial about proper h ouse-painting techniques. In th i s scenario, viewers of the video tutorial m ight see a dvertisem ents for paint or paintbrushes, and Google w ould s hare a portion o f the resulting ad revenue w ith the user that posted the video tutorial.

The Gonzalez P laintiffs allege that “each YouTube video

m ust b e reviewed a n d a pproved by Google before G oogle will p erm it advertisem ents to be placed with that v ideo,” and that “Google has reviewed and approved ISIS videos” for advertising. The Gonzalez P laintiffs also allege that, because it approved ISIS v i d eo s for the AdSense program , G oogle shared a percentage of r e v enues generated from those advertisem ents with ISIS.

W e have exp lained that § 2 30 grants im m u nity from claim s seeking to hold providers of interacti v e c o m puter services liable as publishers or speakers of third-party

co n ten t

. The Gon zalez P lain tif f s’ rev en u e-sh arin g alleg a tio n s

are not directed to the publication of third-party inform ation. Th e se allegations are p rem ised on Google providing ISI S with m aterial support by giving ISIS money

. Thus, unlike the Gonzalez Plaintiffs’ ot her allegations, the revenue-sharing theory does not depend o n the p articular content ISIS places

on YouTube; this theory is solely directed to Go o gle’s unlawful paym ents of m oney to ISIS.G O N Z A L E Z V

. G O O G L E 49

It is well settled that § 2 30 “bars only liability that treats

a website as a publisher or speaker of content provided by

som ebody else.” Internet Brands

, 824 F.3d at 851. Perhaps the best indication that the Gonzalez Plaintiffs’ r evenuesharing allegations are not directed to any third-party content is that Google’s al l eged violation of the ATA could be

rem edied without changing any of the content posted by YouTube’s u sers. See id. ; see also HomeAway.com, Inc

. v. City of Santa Monica

, 918 F.3d 676, 68 3 ( 9 th Cir. 2 019) (concluding that a city o rdinance that did “not proscribe,

m andate, or even discuss the content of t h e listings that the [plaintiffs] display[ed] on their websites” fe l l outside the

scope of im m

unity provided by § 230). The Go nz alez Plaintiffs’ allegations of revenue-sharing do not seek to hold Google liable for any content provided by a third-party. Accordi n g l y , we conclude that § 230 does not bar the Gonzalez Plaintiffs’ claim s prem ised on sharing reven u e with ISIS. 1 3

IV

Having conc l u de d that § 2 30 only im m unizes G oogle from liability for all of the Gonzalez Plaintiffs’ non-revenue sharing claim s, we next address whether, based on the TAC’s revenue-sharing theory, the Gonzalez Plaintiffs ’ adequately allege claim s for direct liability and secondary liability under the ATA. W e address the direct liability claim s first.

13 The district court dism issed the revenue-sharing claim s without prejudice f

or failure to adequately allege proxim ate cause. The Gonzalez Plaintiffs chose n ot to am end, and a f inal j ud g m e n t wa s subsequently entered on that basis.G O N Z A L E Z V

. G O O G L E

50

The civil rem edies provision of the ATA, 18 U.S.C. § 2333(a), “allows any United States national ‘injured in his

or h er person, property, or business by reason of an act o f i n ternational terrorism

, or his or h er estate, survivor s o r heirs,’ to sue in federal court and recover treble dam ages and attorney’s fees.” Fields

, 881 F.3d at 743 (quoting § 2 333(a)).
It is undisputed that the Gonzale z Plaintiffs, Taam n eh Plaintiffs, an d C layborn Plaintiffs are United States n ationals.

The ATA includes several crim inal provisions, “the violation of which can provide the basis for a cause o f action under § 2333(a).” Id. The Gonza le z Pl a intiffs argue that Google directly comm itted acts of international terrorism by providing m aterial support for terrorism

, providing m aterial support for foreign terrori st organizations, and financing terrorism in v iolation of sections 2339A(a), 2 339B(a)(1), and 2339C(c), respectively. They also alleg e th at G o o g le violated Executive Order N o. 13224, 31 C.F.R. P art 5 9 4, a n d 50 U.S.C. § 1705.

Section 2 3 33(a) is directed to “act[s] of intern at i onal terrorism

. ” “[I]n t e r national terrorism ” is statutorily defined in 18 U.S.C. § 2331(1). See generally Fields

, 8 8 1 F.3d at 743 n.3; Linde v. Arab Bank, PLC

, 8 82 F.3d 314, 326–27 (2d Cir. 2018). A cts constituting international terrorism m ust “appear

to b e intended—(i) to intim idate or coerce a civilian population; (ii) to influence the p o l i cy o f a governm ent by intim idation or coercion; or (iii) to affe c t the conduct o f a gove rn me n t b y ma s s d e s t r u c t i o n , a s s a ss i n at i o n, o r kidnapping.” Id. § 2331(1)(B).

T h e o p erativ e G o n za lez co m plaint contends that G o o g le’s conduct qualified as an act of “international terrorism ,” citing § 2331(1). W e conclude their com plaint fails t o plausiblyG O N Z A L E Z V

. G O O G L E 51

allege that Google directly perpetrated an act of international terrorism as required by § 2331(1)(B).

W hether an act appears to b e intended to inti mi date o r

coerce a civilian p opulation o r to influence or a ffe c t a g o vernm ent, “does not depend on the actor’s beliefs, bu t im poses on the actor an objective sta n d ard to recognize the apparent intentions of a c t ions.” Weiss v. Nat’l Westminster Bank PLC

, 7 68 F.3d 202, 207 n.6 (2d Cir. 2014); see also
Bo i m v. Holy L and Found. for Relief & Dev.

, 549 F.3d 685, 6 94 (7th Cir. 2008) (en b anc) (“[I]t is a m atter of e x t e rn al

appearance rather than subjective intent, which is internal to the intender.”).

The Gonzalez Plaintiffs argue that the knowing provision of resources to a terrorist organization n ecessarily constitutes “international t e rrorism ,” and satisfies the requirem ents identified in § 2 3 3 1(1)(B). W e disagree. Nothing in the statutory schem e suggests that m aterial s u pport always qualifies as international terrorism b ecause such conduct m ay

or m ay not objectively appear to be intended to intim idate o r

coerce

. M edical assistance rendered to known terrorists by
Doctors W it h o ut Borders illustrates this point. See Boim

, 5 4 9 F.3d at 699. Such assistance m ight arguably p r o v i d e

m aterial support t o te r rorists in v iolation of § 2 339B, but it would not appear to be intended t o i n t i m idate or coerce a civilian population, or to affect the conduct of a governm ent. See id

. To qualify as international terrorism

, the defendant’s

acts m u st satisfy each of the criteria contained in § 2331(1), Linde

, 882 F.3d a t 325–26; see also F ields

, 8 81 F.3d at 743

n

. 3, and “the provision o f m aterial support to a terr o r i s t

o rganization does n ot invariably equate to an ac t of international terrorism ,” Linde

, 882 F.3d at 326.G O N Z A L E Z V

. G O O G L E

52

The Gonzalez Plaintiffs rely heavily o n the Seventh Circuit’s en banc decision in Boim

, but that reliance is

m isplaced. The issue in Boim was whe t h e r d efendants w ho had donated m oney to Ham as and Ham as-affiliated charities—knowing that Ham as u sed its resources to f inance the killing of Israeli Jews—could be held liable under the ATA f or H am as’s 1994 m

u rder of an Ame r i c an teenager in Israel. B oim

, 549 F.3d at 688–690. The en banc court stated that a knowing donor’s contributions to Ham as would satisfy the definitional requirem ents of “international terrorism ” set fo r th in § 2331(1). Id. at 690, 694. Boim reasoned that “donations to Ham as

. . . would enable H am as to kill o r wound, or try to kill” m ore people in Israel. Id. at 694. The Seventh Circuit concluded that such d onations would appear to be intende d to intim idate or coerce a civilian population because of the foreseeabili t y o f these consequences. 1 4 Id

. The G onzale z Plaintiffs’ reliance on Boim is m isplaced because the allegations here are n ot at all s i mi l a r to those in
Boim

, which involved voluntary donations specifically and purposefully directed to a foreign terrorist organization.

Taking as true t h e allegation that Google shared advertising revenue with ISIS as p art of its AdSense program , that action does not perm it the inference that Google’s actions objectively appear to have bee n i n tended to intim idate o r

coerce ci v il ians, or to influence o r affect governm ents. The Seventh Circuit’s decision in K emper v. Deutsche Bank A G

, 911 F.3d 383 (7th Cir. 2018), illustrates this point. There, the

court concluded that the pla i n t iff failed to plausibly allege

14 W e express no view on whether Boim w ould be decided the sam e way today. Notably, that d ecision w as issued before C o n g re ss enacted JASTA, thereby creating secondary liab i li t y for aiding and abetting acts of international terrorism

. See 18 U.S.C. § 2333(d)(2).G O N Z A L E Z V

. G O O G L E 53

that Deutsche Bank’s institution of procedures to evade U.S. sanctions and facilitate Iranian banking transactions qualified

as international terrorism

. Id

. at 390. The court reasoned that Deutsche Ban k’s actions did “not appear intended to intim idate or coerce any civilian population or governm ent” because, “[t]o the objective observer, its i n te ractions with Iranian entities were m otivated by econom ics.” Id.

Sim ilarly here, the Gonzalez Plaintiffs did not allege that Google’s actions were m otivated by anything other than

ec o nom ic self-enrichm ent. The TAC alleges that Google is

a com m ercial service in the business of sel l i n g advertising, and that “Google uses the A dSense m onetization program to

earn revenue, and as an incentive to encourage users to post videos o n YouTube. ” These allegations are easily d istinguished from those involving donations to a kno wn terrorist organization. See Boim , 549 F.3d at 690, 694. The Gonzalez P laintiffs did not allege that Google shared ISIS’s vision and objectives, nor t h at Go ogle intended ISIS to succeed in any future acts of terrorism

. Rather, the

com plaint’s allegations suggest that Google split ad revenue with ISIS in furtherance of its own financial best interest.

The TAC fails to allege t hat G oogle’s provision o f

m aterial support appeared to b e intended to intim idate o r

coerce a civilian population, or to influence or affect a governm ent as required b y the ATA. See 18 U.S.C. § 2331(1)(B). For this reason, the Gonzalez com plaint does

not adequately allege the requirem ents necessary to establish direct liability for an act of international terrorism pursuant to § 233 3 (a )

, a n d we need n ot reach whether the Gonzalez Plaintiffs sufficiently alleged that Google’s actions proxim ately caused Nohem i Gonzalez’s death.G O N Z A L E Z V

. G O O G L E

54

V

Turning to the Gonzalez P laintiffs ’ secondary liability claim s based on revenue-sharing, Google argues that the Gonzalez com

plaint fails to state a claim for secondary liability pursuant to 18 U.S.C. § 2333(d)(2). W e agree. 1 5

As originally enacted, the A TA allowed only c l aim s all eging direct liability against the perpetrators of acts o f international terrorism

. Rothstein v. UBS AG

, 708 F.3d 82, 97 (2d Cir. 2013); see a lso Linde

, 8 8 2 F.3d at 319–20. In 2016, Congress am ended the ATA by enacting JASTA, which extends civil liability to pe r s o n s who aid and abet by providing substantial assistance to persons who com m it acts

o f international terrorism

, and to those who conspire t o

c o mmi t such acts. 18 U.S.C. § 2333(d)(2). Secondary li ab i l i ty for aiding or abetting acts o f terrorism applies only when the principal act of international terrorism is “com m itted, planned, or a uthorized by an organization

. . . designated as a foreign terrorist organization.” Id.

The G onzalez Plaintiffs raise claim s for both aiding-andabetting and conspiracy liability. W e address these theories separately.

15 The d istrict court concluded that the Gonzalez Plaintiffs advanced

a revenue-sharing th e ory o nly with r espect to their claim s f o r direct liability. W e find the TAC som ewhat a m b iguous on this point, but

assum e f or purposes of deciding this appeal that the Gonzalez Pl ai n tiffs

ra i s e d a r evenue-sharing t heory with respect to both their direct liabilit y and secondary liability claim s.G O N Z A L E Z V

. G O O G L E 55

A

Under § 2333(d)(2) of the ATA, “liability m ay be asserted

as to any person who aids and abets, by knowingly providing substantial assistance” to “the person who comm itted

. . . an act of international terrorism ” as set forth in § 2333(a). 18 U.S.C. § 2333(d)(2). JASTA sp ecifies that the D.C. Circuit’s decision in H

alber s t a m v. Welch

, 705 F.2d 472 (D.C. Cir. 1983), describes “the proper legal f ram ework” for assessing aiding-and-abetting liability under § 2333(d). Pub.
L. No. 144-222, § 2(a)(5), 130 Stat. at 852; see also Siegel v. HS BC N. Am. Holdings, Inc.

, 933 F .3d 217, 223 (2d Ci r. 2019).

Halberstam addressed the scope of secondary liability for

com m on law causes of action. See 705 F.2d at 474. The plaintiff, Elliott Halberstam

, was the widow of Michael Halberstam

. Id. Michael Halberstam was a physician killed by Bernard W elch during the course of a burglary. Id

. Halberstam ’s widow brought a w rongful death action against Linda Ham ilton, W elch’ s live-in g irlfriend, alleging that
Ham ilton was civilly liable for Michael Halberstam ’s death, both as an aider-abettor and a co-conspirator. Id. at 474–76.
Ham ilton provided assistance to W elch d uring the course of his m ul t i -year cam

paign of burglaries, including preparing letters of sale for stolen goods, falsify i ng tax returns to conceal incom e derived from stolen good s, m aintaining

accounts on W elch’s behalf, and handling fi n ancial transactions. Id

. at 475, 486, 4 88. The D.C. Circuit ultim ately concluded that Ham ilton was c i v i lly liable for Halberstam ’s death, even though W elch killed Halberstam during a robbery and H am ilton w as not presen t

. The court concluded that Ham ilton was liable under a conspiracy theory and also an aiding-and-abetting theory. Id. at 489.G O N Z A L E Z V

. G O O G L E

56

The scenario presented in Halberstam is, to put it m ildly, dissim il a r t o t h e one at issue here. But Congress selected Halberstam as the governing standard for secondary liability ATA claim s b ecause Halberstam “has been wi dely recognized as the leading case regarding Federal civil aiding and abetting

. . . liability.” P ub. L. No. 144-222, § 2 ( a )(5), 130 Stat. at 852.

In H alberstam

, the D.C. Circuit identified three elem ents that a p laintiff m

ust prove in order to establish a iding-andabetting liability: “(1) the party wh o m the defendant aids

m ust perform a wrongful act that c a u s e s an injury; (2) the d efe ndant m

ust be generally aware of his role as part of an overall illegal or tortious activity at the tim e that he p rovides the assistance; [and] (3) t h e defendant m ust knowingly and substantially assist the principal v iolation.” 705 F.2d at 477.

1

The first elem ent of aiding and abetting liability requires

a showing that the party the defendant aided com m itted an act of international terrorism that injured the plaintiff. 18 U.S.C. § 2333(d)(2); Halberstam

, 705 F.2d at 477; see also S iegel

, 933 F.3d at 223. 1 6 The parties di s pute w hether the relevant princip a l actor is the ISIS organization as a whole or the individual terrorists wh o perpetrated the Paris Attacks. W e agree with th e Go n za lez Plaintiffs that ISIS is the relevant

16 As noted, the ATA’s second a ry li a bility provision only applies where a designated “foreign terrorist organization” “com m itted, p lanned,

or authorized” t

he act of international terrorism

. § 2333(d)(2). The parties do n ot dispute that the Paris Attacks w ere an act of international terrorism

, nor do they dispute that the killing of Nohem i Gonzalez during the Paris Attacks was an injury to the Gonzalez Plaintiffs.G O N Z A L E Z V

. G O O G L E 57

“person w ho com

m itted

. . . an act of international terrorism

. ” 18 U.S.C. § 2333(d)(2).

The TAC alleges that coordinated team s of ISIS terrorists planned and carried out the P aris Attacks. Specifica l l y , i t alleges that the café shooters wh o mu rdered N ohem i G o n z a l e z

— A b a a o u d

, A b d e s l a m, a n d A k r o u h

— w e r e

m em bers of ISIS. T he Gonzalez P laintiffs further allege that Abaaoud, the operational leader of the Paris Attacks, traveled

t o Sy r ia to join ISIS in March 2013, joined ISIS while in Syria, and pu blicly declared his affiliation with ISIS. W e accept as true the allegations that, in 201 4

, Ab a aoud posted

a link o n h i s Fa c ebook profile to an ISIS recruiting video in which he described his life and role with ISIS, and, that in 2015, ISIS’s English-language m agazine, Dabiq

, featured an interview with Abaaoud. These allegations distinguish the TAC from the claim s p resented in Crosby v. Twitter, Inc.

, where the Sixth Circuit reject e d the plaintiffs’ A TA claim s b ecau se t h e co m plaint contained “no allegation s th at I S I S w a s involved w ith the P ulse Night Club shooting” perpetrated by Om ar Mateen. 921 F.3d 6 1 7, 626 (6th Cir. 2019); see also 18 U.S.C. § 2 333(d)(2). W e conclude the Gonzalez Plaintiffs satisfied the first elem ent of aiding-and-abetting liability because the TAC plausibly al l e ged that ISIS, a designated terrorist organization, “com m itted, p lanned, or a uthorized” the Paris Attacks.

2

The second elem ent o f aid i ng -and-abetting liability requires a showing that Google was g eneral l y aware of its role in ISIS’s terrorist activities at the tim e it provided assistance to ISIS. 18 U.S.C. § 2333(d)(2); Halb e r s t am ,G O N Z A L E Z V

. G O O G L E

58

705 F. 2 d at 477; see also L inde

, 882 F.3d at 329. The Gonzalez Plaintiffs also satisfied this elem ent.

Just as the Halberstam court conclud e d that L inda
Ham ilton was generally aware o f her role in Bernard W elch’s ongoing b urglary op e r a t ion b ecause she “knew about and acted to support” it, the Gonzale z Pl a i ntiffs m ust plausibly allege that, by sharing revenue w ith ISIS, Google was aware that it was assum ing a role in ISIS’s terrorist a c tivities. See Halberstam

, 705 F.2d at 488; see a lso Linde

, 882 F.3d at 329 (r e quiring a showing that “the bank was ‘generally aware’ that [by p roviding financial services,] it was thereby playing

a ‘role’ in Ham as’s violent or life-endangering activities” (quoting Halberstam

, 70 5 F.2d at 477)). N otably, this elem ent does not require a s h owi n g of “the specific intent dem anded for crim inal aiding and abetting culpability,” i.e.,

an “intent to participate in a crim inal schem e as ‘so me t h i n g that he wishes to bring about and seek by his action to m ake it succeed.’” L ind

e , 8 82 F.3d at 329 (quoting Rosemond v. United States

, 572 U .S. 6 5

, 7 6 (2014)). Nor does it require that Google “knew of the specific attacks at issue.” Id.

The TAC adequately alleges that Google was aware of the role it p layed in ISIS’s terrorist activities. Specifically, the Gonzalez Plaintiffs allege that Go o g le knowingly shared advertising revenue with ISIS and that G oogle did so d espite

num erous reports from news organizations t h at G oogle placed advertisem ents on ISIS videos. Unde r t h e se circum stances, the allegation that Google knowingly gave “fungible d ollars to a terrorist organization” plausibly alleges that G oogle was aware of the role i t pl a y ed in activities that “m ay be ‘dangerous to hum an life.’” Cf. Kemp e r , 911 F.3d

at 390; see also Fields

, 881 F.3d at 748; Boim

, 5 49 F.3d at 693.G O N Z A L E Z V

. G O O G L E 59

W e are m indful that “aiding and a b etting an a ct o f international terrorism requires m ore than the p rovision o f

m aterial support to a designated terrorist organization. ”
Lin d e , 8 8 2 F .3d at 329. Thus, the mens rea required for the

gener al a wareness elem ent of secondary liability under § 2333(d) m ay not be coextensive with the showing required for m aterial support under § 2339B. The latter “requires only knowledge of the organization’s connection to terrorism

, not intent to further its terrorist activities or awareness that one is playing a role in those activities.” See id. at 330 (citing Holder v. Humani t arian L. Project

, 561 U.S. 1

, 16–17) (2010); s e e also, e.g. , Siegel

, 933 F.3d at 224 (concluding plaintiffs failed to plead general awareness with allegations “suggest[ing] that in p r ov iding banking services to [ a Saudi Arabian bank], HSBC had little reason to suspect that it was

assum ing a role in [al-Qa eda in Iraq’s] terrorist activities”).
But h ere, we are satisfied that the allegations indicating Google knowingly contributed m oney to ISIS suffice to show that Google understood it played a role in the violent and lifeendangering activities unde rtaken by ISIS, and therefore establish the second elem ent of aiding-and-abetting liability for purposes of § 2333(d)(2).

3

The third elem ent o f aiding-and-abetting liab ility requires that the plaintiff show the defendant k n owingly and sub s t an tially assisted the act of terrorism that injured the plaintiff. 18 U.S.C. § 23 33 ( d )(2); see also Halberstam

, 705 F.2d at 4 88 (holding the defendant m ust have “knowingly

a n d substantially assist[ed] the principal violation”). This elem ent con ta ins two com p onents: (1) “knowing[]” assistance, and ( 2 ) “ substantial[]” assistance. See Halberstam , 705 F.2d at 477; see also id. at 488 (evaluatingG O N Z A L E Z V

. G O O G L E

60

whether Linda Ham ilton assisted B ern a rd W elch “with knowledge that he h ad en g a ged in illegal acquisition o f goods” separate from considering whether her “assistance was ‘substantial’”).

The Halberstam court identified six factors relevant to assessing w hether the substantial assistance com pon e nt is satisfied: “(1) the nature of the act encouraged, (2) the

am ount of assistance g iven by d efendant, (3) defendant’s

pre s e n c e or absence at the tim e of the tort, (4) defendant’s relation to the principal, (5) defen d a n t’s state of m ind, and (6) the p e r i o d of defendant’s assistance.” Linde

, 882 F.3d at 329 (citing H

alberstam

, 705 F.2d at 483–84). 1 7

The parties dispute whether the relevant “p ri n cipal vio l ation” for analyzing the third elem ent is ISIS’s broader

cam paign o f terrorism or the Paris A ttacks. See Halberstam

, 705 F.2d at 488. But H

alberstam explained that the extent o f l i a b i l i t y u n d e r a i d i n g

- a n d

- a b e t t i n g e n c o m p a s s e s foreseeability, such that a d efe ndant “who assists a tortious

act m ay be li ab le for o ther reasonably foreseeable acts done in con n e c t i on with it.” 705 F.2d at 484. For exam ple, the

com m on law cases Halberstam d rew upon established that a

17 In d icta, the Second Circuit suggested that these factors determ ine “w h e ther the d efendant’s assistance was sufficiently knowing a nd substantial to qualify as aiding and abetting.” Linde

, 882 F.3d at 329 n.10 (em phasis added). H owever, Halberstam appears t o tre a t “knowing” assistance as an inquiry separate from “substanti al ” assistance. See Halberstam

, 7 05 F.2d at 478 ( explaining that the l i s t ed factors aid “in

m aking th[e] determ ination” of “how m uch encouragem ent or assistance is substantial enough ” (em phasis added)). Indeed, the Halberstam court explicitly acknowledged that “the sci en t e r r eq u i r em en t i n t

he third elem ent of aiding-abetting” requires that “an aider-abettor m ust knowingly assist the underlying violation.” See id. at 485 n.14.G O N Z A L E Z V

. G O O G L E 61

thirteen-year

- o l d boy who broke into a church with som e

com panions could be held liable fo r dam age to the church caused b y h is com

p anions’ failure to extinguish torches they used to light their way in the church attic. Id. at 48 2

– 8 3 (citing Am. Family Mut. Ins. Co. v. Grim

, 440 P .2d 621, 625–26 (Kan. 1968)). Because the need for lighti n g c o u ld have b een anticipated, “the boy w ho had not used a torch, nor

even expect ed one to be lighted, could be liable f or the dam age caused by the torches.” Id. at 483. By contrast, the Halberstam court cite d a n exam ple from the R estatem ent (Se c o n d ) o f Torts where liability was not im posed: if A supplies wire cutters to B to allow B to unlawfully enter the l a n d o f C to recapture chattels belonging to B, and B intentio n ally sets fire to C ’s house in the course of his trespass, A is not liable for the d estroying t h e house

. Id. at 483 n.12 (quoting Restatem ent (Second) of Torts § 8 76, cm t. d (1976)).

Halberstam concluded that Linda Ham ilton was liable for
W elch killing Michael H alberstam because of the nature and

extent of her assistance to W elch’s illegal burglary enterprise. Id. at 488. In the court’s view, the killing “was a n atural and foreseeable c o n s e quence of the activity Ham ilton helped
W elch to undertake.” Id. “[W ]hen she assisted him

, it was enough that she knew he was involved in som e type o f personal p ro p erty crim e at n ig h t— w h ether as a f

ence, burglar,

or arm ed robber m ade n o difference—because violence a n d kil l i n g i s a f

oreseeable risk in any of these enterprises.” Id.
W e have little difficul ty c o n cluding that the Paris Attacks

were a foreseeable result o f I SI S’s broader cam paign o f terrorism

. Accordingly, when assessing whether the T AC satisfies the third elem ent of aiding-and-abetting liability, we

con s id e r ISIS’s broader cam p aign of terrorism to be the relevant “principal violation.”G O N Z A L E Z V

. G O O G L E

62

Pursuant to § 2333(d)(2), liability att a c h e s to an aiderabettor who “ knowingly provid[es] substantial assista nc e.” 18 U.S.C. § 2333(d)(2) (em phasis added). Thus, a plaintiff

m ust show that the defendant “knowingly gave ‘substantial

a s s istance’ to som eone who perform ed wrongful conduct .” Halberstam

, 705 F .2d at 478; see also id. at 485 n.14 (noting “the scienter requirem ent in the third elem ent” addresses the issue o f “whether an aider-abettor m ust knowingly assist the underlying violation”).

W e conclude that the Gonzalez Plaintiffs a d equately allege knowing assistance. T he TAC alleges “each YouTube

v id eo m ust be reviewed and approved by Google” befor e advertisem ents are placed wit h t h at video, and “Google has reviewed and approved ISIS videos . . . for ‘m onetization,’” and Google therefore “shared revenue w ith ISIS.” The T AC alleges t h a t
, prior to the Paris Attacks, num erous news

o r g a n i z a t i o n s r e p o r t e d o n G o o g l e ’ s p l a c e me n t o f advertisem ents in or alongside ISIS vid eos, and G oogle responded to these m edia re p o r t s by stating it w orked to prevent ads f

rom appearing on any video once it determ ined the content was not appropriate for advertising partners.

In Halberstam

, the knowledge requirem e n t of the third elem ent was sat isfied because Linda Ham ilton’s actions “were p e r form ed knowingly to assist W elch in h is illicit

tra d e

. ” 705 F.2d at 486; see also id. at 488 (noting that
Ham ilton h a d “assisted W elch with k nowledge that h e h ad engaged in illegal acquisition o f goods”). Here, the Gonzalez Plaintiffs allege that Google r e v i ewed and approved ISIS videos for m o n etizatio n an d th ereb y k n o wingly provided ISIS with financial assistance for its terrorist oper ations. According to the TAC, Google did so despite it s a wa reness that these videos were created by ISIS and poste d by ISISG O N Z A L E Z V

. G O O G L E 63

using known ISIS accounts. Taking these allegations as true, they are sufficient to plausibly allege that Google’s assistance was knowing as required by § 2333(d)(2).

That leaves the question w hether the Gonzalez Plaintiffs sufficiently allege that Google’s assistance was “substan tial.”
Ba s ed on our review of the six Halberstam factors, we conclude the Gonzalez P laintiffs did not allege that Google’s assistance rose to this level. See Linde

, 882 F.3d at 329; see also Halberstam , 705 F.2d at 483–84.

As to the first factor—the nature of the act
encourage d

— H alberstam explained that the nature o f the principa l ’s act “dictates what aid m ight m atter, i.e.

, be substantia l

. ” Ha lberstam

, 705 F.2d at 484. For exam ple, verbal support m ight be of great im port when a “defendant’s

war cry for m o re blood” contributes to an “assaulter’s hysteria,” b ut less im

portant in a case involving a defam ation. See id. H ere, the Go n za l ez Plaintiffs allege that G oogle assisted ISIS’s long-running terrorist cam paign. F inancial support is “indisputab ly im portant” to the operation o f a terrorist o rganization, id. at 488, and any m oney p rovided to the organization m ay aid its u nlawful goals. F ields

, 8 81 F.3d at 748; cf. Siegel , 933 F.3d at 225.

The second factor con s i d ers “the am ount o f assistance given by the defendant.” Halberstam

, 705 F.2d at 478. This factor recognizes that not all assistance is equally im portant,

s ee id. at 484, and the TAC contains n o inform ation about the

am ount of assistance provided by Google. It only alleges that Google shared some advertising revenue with ISIS.

Third, we consider the defendant’s “presence or absence

at the tim e of the tort” to assess whether the d e fe n dant’sG O N Z A L E Z V

. G O O G L E

64

assistance was “substanti a l .” Id. at 478. The Gonzalez Plaintiffs concede that Google was not present at the tim e o f the Paris A ttacks. However, if the relevant tort is v iewed as ISIS’s broad e r cam

paign of terrorism

, including the dissem ination of propaganda on Google’s website before and after the Paris Attacks, Google w as arguably present for at least som e of the terroristic activities th a t c omprise the “principal violation.”

The fourth factor considers the defendant’s “relation” to the principal, recognizing that som e persons—e.g., those in positions of authority, o r m em b ers of a larger group—m ay possess greater powers o f suggestion. Id. at 478, 484. Halberstam also cautioned that courts should be “especially vigilant” in evaluating a spouse’s assistance, “so as not to infuse the norm al activities of a spouse wi t h the aura of a concerted tort.” Id. at 484. Google allowed m em bers of ISIS who posted v ideos on YouTube to opt into AdSense, and by approving ISIS videos fo r m onetization, Google agreed to share som e percen t age of the resulting advertising revenue with those ISIS m em

bers. Thus, the allegations in the T AC describe arm s-length bus i n ess transactions between G oogle and YouTube users who opted into the AdSense program

.

The fifth factor is directed t o the defendant’s “state o f

m ind.” Id. at 478. E vidence o f a defendant’s state of m ind

m ay show that a defendant was “one in spiri t ” wi th the principal actor. Id. at 484. The Gonzale z Pl a i ntiffs do not allege that Google had any intent to fi n a n c e , prom o te, o r carry out ISIS’s terrorist acts. See Siegel

, 933 F. 3d a t 2 2 5. Nor does the TAC suggest that G oogle shared any o f ISIS’s objectives. Instead, the allegations show, a t mo st, that Google intended to profit from the AdSense progra m. Th e TAC incorporates by reference articles that indicate GoogleG O N Z A L E Z V

. G O O G L E 65

took som e steps to prevent ads from appearing on ISIS videos. 1 8

Finally, the sixth factor concerns the “dur a ti o n of the assistance provided.” Halberstam

, 705 F.2d at 484. Halberstam explained that “[t]he length of ti me a n alleged aider

- a b ettor h as been involved w ith a tortfeasor alm ost certainly affe c t s t he quality and extent of their relationship and probably influences the am ount o f aid provided as well; additionally, it m ay afford evidence of the defendant’s state of m ind.” Id. Here, the TAC lacks specific allegations about the length of tim e Google provided assistan c e t o ISIS in the fo r m of revenue-sharing, but it cites several n ews articles from March 2015 and March 2016 d escribing the p lacem ent of advertisem ents on YouTube videos posted by ISIS. The TAC also provides an exam ple from a video published on May 28, 2 015. Thus, at m o st, the Gonzalez Plaintiffs allege that advertisements were placed on ISIS’s YouTube videos during those periods of tim e.

W e conclude that these allegations fall short o f establishing that Google’s a ssistance was sufficiently “substantial” for purposes of § 2 333(d)(2) liability. W hen we review an order granting a 12(b)(6) m otion to dism iss we are required to assess whether the al l eg ations in the com plaint, taken as true, state a c laim of substantial assistance. See Whitaker v. Tesla Motors, Inc.

, 985 F.3d 1173, 1177 (9th Cir.
2 02 1) (“Our case law does not perm it plaintiffs to rely on anticipated discovery to satisfy Rules 8 and 1 2 (b)(6); rather,

18 See, e.g. , Ads Show n Be f ore YouTube ISIS Videos C a t c h C o m p a n i e s O f f

- G u a r d

, N B C N e w s ( M a r

. 1 0

, 2 0 1 5 )

, h t t p : / / w w w

. n b c n e w s

. c o m / s t o r y l i n e / i s i s

- t e r r o r / a d s

- s h o w n

isis-videos-youtube-catchcom panies-guard-n320946.G O N Z A L E Z V

. G O O G L E

66

pleadings m

ust assert w ell-pleaded factual allegations to advance to discovery”).

Although m onetary support is undoubtedly im portan t to ISIS’s terrorism cam

paign, the TAC is devoid of any allegations about how m

uch assistance Google p rovided. A s such, it does not a ll o w the conclusion that Google’s assistance was substantial. N or do the allegations in the T AC suggest that Google intended to assist ISIS. Accordingly, we

con c lude the G onzalez Plaintiffs failed to state a claim for aiding-and-abetting liability under the ATA. W e do not consider whet h e r the identified defects in the Gonzalez Plaintiffs’ revenue-sharing claim s—principally, the absence of any allegation regarding the am ount of the shared revenue—could be cured by further am endm ent because the Gonzalez Plaintiffs were g i v en leave to am end those claim s and decl i n e d to do so. See WMX T echs., Inc. v. Miller

, 104 F.3d 1133, 1136 (9th Cir. 1997) (en banc).

B

Section 2333(d)(2) also perm its cla im s for secondary liability “as to any person

. . . who conspires with the person who com m itted

. . . an act of interna t i o na l t errorism ” as set forth in § 2333(a). As with aiding-and-abettin g liability, JASTA specifies that th e D.C. Circuit’s decision in Halberstam p rovides “the proper legal fram ework” for

ass e s si n g conspiracy liability under § 2333(d). Pub. L. No. 144-222, § 2(a)(5), 130 Stat. at 852. Halberstam concluded that proof of conspiracy requires three e lem ents: (1) “an

agreem ent to do an unlawful act or a lawful act in an unlawful

m anner,” (2) “an overt act in furtherance of the agreem ent by

s om eone participating in it,” and (3) “injury caused by th e act.” 7 05 F.2d at 487. W e conclude the Gonzalez Plaintiffs’G O N Z A L E Z V

. G O O G L E 67

TAC does n ot state an actionable claim for consp i r a cy liability.

The T AC’s al legations are insufficient to plausibly suggest that Google reached an agreem ent with ISIS to carry out the P aris Attacks that caused Nohem i Gonz a l e z’s death. Halberstam requires the overt act c ausing plaintiffs’ injury

m ust b e “done pursuant to and in furtherance of the com m on schem e.” Id. at 4 77. Google’s sharing of revenues with

m em bers of ISIS does not, b y itself, support the inference that Google tacitly agreed to com m it hom icidal terrorist acts with ISIS, where N ohem i Gonzalez’s m urd e r wa s an overt act
p er pe t rated pursuant to, and in furtherance of, that com m on schem e. 1 9 W e therefore conclude that the Gonzalez Plaintiffs fail to state a claim f

o r conspiracy liab i l i t y u n der the ATA, and affirm the district court’s dism issal w ith prejudice of the TAC.

V I

We now turn to the Taamneh appeal. As we h a v e explained, although the com plaints in Gonzalez and Taamneh

are sim ilar, our decision in Taamneh is largely dictated by the path Taamn e h t o o k to reach o ur court. Because the bulk o f the Gon z al e z Plaintiffs’ claim s w ere properly dism issed on the basis of § 230 immu n i t y , our decision in Gonzalez principally f

ocuses on whether the Gonzalez Plaintiffs’

19 The Gonzalez P laintiffs seek to enlist the TAC’s allegation t hat “Google

. . . con s p i red with ISIS, its m em bers[,] and affiliates” to
pro m o t e , p lan, and carry out “ the acts of i nternational terrorism t hat injured the plaintiffs.” But thi s conclusory allegation is insufficient to survive a m otion to dism iss. See Be l l At l

. Corp. v. Twombly

, 5 5 0 U .S. 544, 555–56 (2007).G O N Z A L E Z V

. G O O G L E

68

revenue-sharing theory sufficed to state a claim under t h e ATA. In contrast, the district court in Taamneh did not reach § 230; it only addressed whether t h e Ta am neh Plaintiffs plausibly alleged violations of the A TA for purposes of Rule 12(b)(6). The Taamneh appeal is further lim ited by t he fact that the Taam neh Plaintiffs only appealed the dism issal o f their aiding-and-abetting claim

.

The Taam neh Plaintiffs’ aiding-and-abetting claim stem s from Abdulkadir Masharipov’s m urder of Nawras Alassaf at
t he Reina nightclub on January 1, 2017. Masharipov’s connection to I SI S is not disputed. He film ed his “m arty r d om ” v ideo, w herein he stated that he w as going to
carry out a suicide attack in the nam e of ISIS, and requested that his son grow up to b e a suicide bom b er like him

. 2 0 About

one year before the Reina Attack, ISIS instructed Masharipov

to m ove to Turkey with his fam ily an d a wa i t further orders. ISIS provided Masharipov with an assault rifle, am m unition, and stun grenades, and d irected Masharipov w hen and where

to attack. ISIS also sent Masharipov footage taken inside the Reina n ightcl u b

, and Masharipov viewed it at length to

m em orize the floor plan in preparation for his attack.

1

The Taam neh Plainti ffs’ aiding-and-abetting claim is governed by the standards set forth in Halberstam

. The first Halberstam elem ent requires that “the party whom the

20 T he operativ e com

plaint alleges “[m ]artyrdom videos, shared via
De fe n dants’ w ebsites, are tools o f propaganda frequently used by ISIS. Th e s e v i d eos are used as psychological weapons in ISIS’s attem pt to establish validity for their actions, inspire fear in their enem ies, or spread their ideology for political or religious am bitions.”G O N Z A L E Z V

. G O O G L E 69

defendant aids m

ust perform a wrongful a c t that causes an injury.” 705 F.2d at 477. The p arties do not d ispute that the Reina A ttack was an “act of international terrorism ” that was “com m itted, planned, or authorized” by ISIS. Nor do the parties dispute that the Reina Attack caused the Ta a mn eh Plaintiffs’ injury—the killing of Nawras Alassaf.

2

The second Halberstam elem ent of aiding-abetting liability requires the defendant to be “generally aware of his role as part o f an overall illegal or tortious activity at the tim e that he provides the assistance.” Id

. The Taam neh Plaintiffs also satisfied this elem ent.

Th e Taam neh Plaintiffs allege that, at the tim e o f t h e Reina Attack, defendants were generally aware that ISIS used defendants’ plat fo r ms to recruit, raise funds, and spread propaganda in support of their terrorist activ i t i e s

. The FAC alleges that, des pite “extensive m edia coverage” and legal and governm ental pressure, defendants “continued to provide these resources and services to ISIS and its affiliates, refusing

to actively identify ISIS’s T witter, Facebook, and YouTube accounts, and only revi e wi n g accounts reported by other social m edia users.” These allegations suggest the defendants, after years of m edia coverage a n d legal and

g o v ernm en t p ressu

re co n cern in g IS IS ’ s u se of their platf o rm s,

were generally aware they were p laying an im portant role in ISIS’s terrorism enterpri s e by providing access to their platform s and not taking aggres s i v e m easures to restrict ISIS-affiliated content. See Linde

, 882 F .3d at 329; see also Halberstam , 705 F.2d at 477.G O N Z A L E Z V

. G O O G L E

70

3

The third Halberstam elem ent re quires the plaintiff to allege the defendant knowingly and substantially assisted the

p rincipal v iolation. 7 05 F.2d at 477. W e conclude t h e Taam neh Plaintiffs’ complaint satisfied this elem ent.

The Taam neh Plaintiffs adequately allege that defendants knowingly assisted ISIS. Specifically, the FAC alleges that ISIS depends on Twitter, Facebook, and YouTube to recruit individuals to join ISIS, to prom ote its terrorist agenda, to solicit donations, to threaten and intim idate civ i l ian populations, and to inspire violence and o ther terrorist activities. The Taam neh Plaintiffs’ com plaint alleges that each defendant has been aware of ISIS’s use of their

res pective social m edia platform s for m any years—through

m edia reports, statem ents from U.S. g overnm ent officials, and threatened lawsuits—but h av e ref u sed to take m eaningful steps to prevent that u se. T he FAC further alleges that Google shared revenue with ISIS by reviewing and approving ISIS’s YouTube videos f o r m onetiz a t i o n through the AdSense progr a m

. Taken as true, these allegations

s ufficiently allege that d efendants’ assistance to ISIS was knowing.

W e ne x t c onsider whether the Taam neh Plaintiffs

p lausibly allege that defendants’ assistan ce w as “su b stan t ial, ” applying the six Ha l b er s t am factors. 2 1 First, the act

21 Many of the allegations we discuss in the context of Taamneh were al s o ra ised in a sim ilar form in t he Gonzalez T AC. But because o f the application of § 230 im

m u nity i n Gonzalez

, we did not have occasion to consider t hem in our evaluation of the Gonzalez Plaintiffs’ aiding-andabetting claim

.G O N Z A L E Z V

. G O O G L E 71

encouraged is ISIS’s terrorism cam paign, a n d the FAC alleges that this enter p r i s e was heavily d ependent on social media platform s to recruit m em bers, to raise funds, a n d t o dissem inate propaganda. T he FAC alleges that by providing ISIS with access to robust com m unications platform s f

ree o f charge, defendants facilitated ISIS’s ability t o reach and

engage audiences it could not otherwise reach, and served as

a m atchm aker for people ar ound the globe who were sym pathetic to ISIS’s vision. It also alleges ISIS’ s terrorist enterprise relies o n f

inancial support, as any m oney provided

to the organization m ay aid it s unlawful goals. Fields

, 881 F.3d at 748.

The second factor

— t h e am ount of assistance given by a defendant—is addre s s ed by the T aam neh Plaintiffs’ all egation that the social m edia platform s were essential to ISIS’s g rowth and expansion. The Taam neh Plaintiffs allege that, without the social m edia platform s, ISIS would have no

m eans of radicalizing recruits beyond ISIS’s territorial borders. Before the era of social m edia, ISIS’s predecessors

were lim ited t o r eleasing short, low-quality videos on websites that could handle only lim ited traffic. According to th e F A C

, IS IS recognized the power of defendants’ platf o rm s, which were offered free of charge, and exploited them

. ISIS form ed its own m edia divisions an d p roduction com panies aim ed at producing highly stylized, professional-quality propaganda. The FAC f

u rther alleges that defendants’ social

m edia platform s were instrum ental in allowing ISIS to instill fear and terror in civilian populations. By using defendants’ platform s, the Taam neh Plaintiffs allege that ISIS has expanded its reach and raised its profile beyond that of other

terro r i st g roups. These are plausible allegations that the assistance provided b y defendants’ social m edi a platform sG O N Z A L E Z V

. G O O G L E

72

was integral to ISIS’s expansion, and t o i t s success as a terrorist organization.

The third factor considers the defendant’s presence o r absence at the tim e of the tort. At oral argum en t
, Taam n eh Plaintiffs unam

biguously conceded the act of i n t ernational terrorism they allege is the Re i n a A ttack itself. There is no dispute that defendants were not present during the Re ina Attack.

Fourth, we consider the defendant’s relat i o n t o the prin c ipal actor, ISIS. The FAC indicates that defendants

m ade their p latform s available to m em bers of the public, and that billions of people around the world u se defendants’ platform s. By m aking their platform s generally available to the m ar k e t
, defendants allowed ISIS to exploit their platforms; but like the Gonzalez TAC, these allegations indicate that defendants had, at m os t
, a n arm s-length transactional relationship with ISIS. The alleged relationship

m ay b e even f

urther attenuated than the o nes defendants have with som e of t he ir other users because the FAC alleges defen d an ts reg u larly rem o v ed IS IS c o n ten t an d IS IS -af f iliated

accounts. The Taam n e h Plaintiffs do not dispute that defendants’ policies prohibit posting content that pr o mo tes terrorist activity or other form s of violence.

The fifth fac t or concerns the defendant’s state of m ind. Here, the Taam neh P laintiffs d o not allege that defendants had any intent to further or aid ISIS’s terrorist activities, see Siegel

, 933 F.3d at 225, or that defenda nts shared any o f ISIS’ s o b

j ectives. Indeed, the record indicates that defendants took steps to rem o ve ISIS-affiliated accounts and videos. W ith respect to advertisem ents on ISIS YouTube vid e o s , the articles incorporated into the com plaint suggestG O N Z A L E Z V

. G O O G L E 73

that Google took at least s o m e s teps to p revent ads from appearing on ISIS videos.

The sixth factor addresses the period of th e d e fendant’s assistance. The Taam neh Plaintiffs allege that defendants provided ISIS with an effective o n l i n e com m unications

p latform s for m any years. The F AC alleges that ISI Saffiliated accounts first appeared o n Twitter i n 2010. According to the Taam neh Plaintiffs’ FAC, ISIS used Facebook as early as 2012, and used YouTube as e arly as 2013.

Ta k ing the FAC’s allegations as true, w e conclude th e Taam neh Plaintiffs adequately allege that defendants’ assistance to ISI S was substantial. The FAC alleges that defe n dants provided services that w ere central to ISIS’s growth and expansion, and that this assistance was provided

over m any years.

W e are m indful that a d e fendant’s state of m ind is an im portant factor, and t h a t the FAC alleges the defendants regularly rem oved ISIS-affiliated accounts and content. See Halberstam

, 705 F .2d at 488 (noting that Ham ilton’s state o f

m ind “assum e[d] a special im portance” because her knowing assistance evidenced “a delibe rate long-term intention to participate in an o ngoing illicit enterprise” and an “intent and desire to m ake the v enture succeed”). But t h e Taam n eh Pla i n ti ffs also allege that defendants allowed ISIS accounts and content to rem ain public even after receiving com plaints about ISIS’s use of their platform s.

W e also recog ni z e the need for caution in im puting aiding-and-abetting liability in the context of an arm s-length transactional relationship of the sort defendants ha v e withG O N Z A L E Z V

. G O O G L E

74

users o f thei r p l atform s. Not every transaction with a d esig n ated t erro rist o rg an ization will sufficiently state a claim fo r a i ding-and-abetting liability under the ATA. B ut given the facts alleged here, we conclude the Taam neh Pl a i n tiffs adequately state a claim for aiding-and-abetting liability.

VII

Finally, we turn to Clayborn

. Th e c l a i m s in Clayborn ari s e f rom a fatal shooting in San Bernardino, California in which S ierra Clayborn, T in Nguyen, an d Nicholas T h alasinos lost th e i r l ives. The district court d id not address § 230 im m u nity and the Clayborn Plaintiffs only app e a l the dism issal o f t

h eir A T A claim f

or aiding-and-abetting liability.

The Cl a yborn Plaintiffs allege that Google, Twitter, and
F acebook p rovided key assistance to the two shooters, F aro o k and Malik. To plausibly allege an aiding-and-abetting claim under the ATA, the C layborn P laintiffs m ust allege that ISIS

“ c o mm itted, planned, or authorized” the San Bernardin o Attack. 18 U.S.C. § 2333(d)(2); see a ls o Halberstam

, 705 F.2d at 477. T h e distr i c t c o u rt held the Clayborn Plaintiffs failed to plausibly allege that ISIS com m itted, authorized, or planned the S an Bernardino Attack because the ties between th e attack and ISIS were “insufficient to plausibly plead claim s for indirect liability.” The court interpreted § 2 333(d)(2) to require “evidence that ISIS itself planned o r carried out the attack,” requiring m ore than allegations that I SI S sought to “generally radicalize” individuals and that ISIS prom oted terrorist attacks.

On appeal, the Clayborn Plaintiffs argue three “ c entral

a l l e gations” sufficiently connect ISIS to Farook and Malik : (1) ISIS claim ed responsibility for the San Bernardino AttackG O N Z A L E Z V

. G O O G L E 75

after the fact; (2) Malik pledged alleg i a n ce to then-ISIS leader Abu Bakr al-Baghdadi at som e point during the attack; and (3) “the FBI confirm ed evidence that Farook h ad face to face m eetings a few years prior to the attack with five people the Bureau investigated and labeled [as] having ‘links to terrorism

. ’” F rom these allegations, the Cla y born Plaintiffs

urge us to infer “that ISIS authorized the San Bern ardino shooting som etim e before the attack.”

W e conclude the operative com plaint does not plausibly allege that ISIS “com m itted, planned, or authorized” the San Bernardino Attack

. It is undispu t e d that Farook and Malik planned and carried out the m as s k il ling, but the Clayborn Plaintiffs’ allegations suggest o nly that ISIS approved of the shooting after learning it had occurred, not that it authorized the a t t a ck beforehand. The allegations in the o perative

com plaint indicate som e connection b e tw een th e sh o o ters and ISIS is possible, but mo r e is needed in order to plausibly allege a cognizable claim for aid i ng -and-abetting liability.
Two mbly

, 5 50 U.S. at 555 (“Factual allegations m u st b e enough to raise a right to relief above the speculative level

. . . on t h e assum

ption that all of the com

plaint’s allegations are true . . . .” (internal citation om itted)).

The Sixth Circuit decision in C rosby aligns with our conclusion. In Crosby

, plaintiffs filed claim s against Google, Twitter, and Facebook under the ATA following the m ass shoo t i n g at the Pulse Night C lub in Orlando, Florida. 621 F.3d at 619. The plaintiffs alleg e d “ISIS ‘virtually recruited’ people through o nline content, [the shooter] saw this content at som e point before the shooting, and [the shooter] injured Plaintiffs.” Id. at 626. The C rosby plaintiffs also alleged that ISIS took responsibility for the attack after the fact. Id. at 619. Even taking the a l legations as true, theG O N Z A L E Z V

. G O O G L E

76

S ix th C ircuit concluded the com p laint alleged the shoo ter w a s “self-ra d i c alized” and never h ad any contact with ISIS, and failed to allege that ISIS gave perm ission for the attack. Id. Thus, the Sixth Circuit held “there [were] insufficient f acts to allege that ISIS ‘com m itted, planned, or authorized’ the Pulse Night Club shooting.” Id. at 626.

The dissent wo u ld hold that the Clayborn Plaintiffs adequately st a t ed a claim for aiding and abetting liability. Spe ci fi cally, the dissent relies on the Clayborn Plaintiffs’ allegation that Farook and Malik used a tactic a Departm ent of Just ice report described as “a frequent, well docum ented practice in international terrori s m incidents” that had been outlined in Al Qaeda and ISIS m ag a zi ne s dissem inated on defendants’ platform s. W e disagree. Farook and Malik’s use of well-known terrorist tactics do not give rise to an inference that their attack was “im plicitly authorized” by ISIS

.

The dissent urges us to apply com m on la w p ri n c iples o f agency to conclude that ISIS authorized the San Bernardino Attack b y rat i fy i ng i t after the fact. W e cannot agree this elem en t is adequately alleged. Section 2333(d)(2) requires plain tiffs to dem onstrate the act of international terrorism was “com m itted, planned, or authorized” by a f oreign terrorist

org a n i z ation. The language Congress adopted gives no indication that the “com m itted, planned, or authorized”

e l e m ent is satisfied m erely because a f oreign terrori s t organization praises an act of terrorism

.

Even if Congress intended “auth o rized” to include acts ratified by terrorist organizations after the fact, I SIS’s

statem ent after the San Bernardino Attack fell s h o r t o f

r a t ification. The com

plaint alleges that ISIS stated, “Two followers o f Islam ic State attacked several days ago a centerG O N Z A L E Z V

. G O O G L E 77

in San Bernardino in California, w e pra y t o Go d to accept them as M a rtyrs.” This clearly alleges that ISIS f ound the San Bernardino Attack praisewor t hy, but not that ISIS

a d o p ted Farook’s and Malik’s actions as its own. S e e Restatement (

Third) of Agency

, § 4.01 cm t. b, ( 1 9 3 3) (“The

a c t o f ratification consists of an externally o bservabl e

m anifestation of assent to be bound by the prior act of another person.”).

Because the Cla y born P laintiffs’ allegations do not plausibly allege that ISIS “com m i t t e d

, planned, o r authorized” the Sa n B ernardino Attack, the Clayborn Plaintiffs did not adequately s t a t e a claim for aiding and abetting an act of international terrorism under § 2333(d)(2). See Crosby , 921 F.3d at 626. 2 2

VIII

The p laintiffs in the se three cases suffered d evastating losses from acts of extrem e and senseless brutality, and their claim s highligh t a n area where technology has dram atically outpaced congressional oversight. There is no indication the drafters o f § 230 im agined the level of s o p histication algorithm s have achiev e d

. Nor did they foresee the

22 T he district court did not reach whether the San Bernardino Attack

was an “act of international terrorism .” (em phasis added). This question

appears t o be m

uch closer in Clayborn than either o f the other appeals before us. See 18 U.S.C. § 2331(1)(C). I n Cl a y b o r n , the attack was planned and executed in the United States by a U.S. citizen and his wife. Although the San Bernardino Attack was undoubtably an act of terror, it is less clear whether the com plaint alleged sufficient i nternational activity

to qualify the San Bernardino Attack as an instance of “i n t e rnational terrorism .” Having held t he Clayborn Plaintiffs failed to state a claim f o r secondary liability on other grounds, we do not decide that question.G O N Z A L E Z V

. G O O G L E

78

cir cum stance we n ow f

ace, in which the use of powerful algorithm s by social m edia websites can encourage, support, and expand terrorist networks. At the tim e § 230 was enacted, it was widely considered “ impossible for service providers to screen each of their m illions of po s t ings for possible problem s.” Carafano

, 339 F.3d at 1124 (e m phasis added) (quoting Zeran

, 129 F.3d at 3 30–31). But it is increasingly apparent that a d v ances in m achine-learning

warrant revisiting that assum ption. Indeed, social m ed ia

com panies are reportedly m aking laudable strides to develop tools to identify, fl a g, and rem ove inherently illegal content such as child pornography. 2 3 Section 230’s swe eping im m u nity is likely prem ised on an antiquated understanding of the extent to which it is p o s s i b le to screen content posted by third parties.

There is no q

uestion § 230(c)(1) shelters m ore activity than Congress envisioned it would. W hether social m edia

com panies should co n t i nue to enjoy im m unity for the thirdparty content they publish, and whether the i r us e o f algorithm s o ught to be regulated, are pressing questions that Congress should address.

IX

W ith re s p ect to Gonzalez

, we affirm the district court’s ruling that § 230 im m

unity bars the plaintiffs’ non-revenue sharing claim s. Separately, we conc lu d e the TAC’s direct liability revenue-sharing claim s did not p lausibly allege that Google’s actio ns qualified as acts of international terrorism within the m eaning of § 2331(1), and t h at the secondary liabi l ity revenue-sharing claim s failed to plausibly allege

23 Supra note 11.G O N Z A L E Z V

. G O O G L E 79

either conspiracy or aiding-and-a b e t ting liability under the ATA.

W ith respect to Taamneh

, we reverse the dis t r ict court’s judgm ent that the FAC failed to adequately state a claim for secondary liability under the ATA.

W ith respect to Clayborn

, we affirm the judgm ent of the district court that Clayborn P laintiffs failed to st at e a claim for secondary liability under the ATA. 2 4

The judgm ent in No. 18-16700 is AFFIRMED.

The judgm en t in No. 18-17192 is REVERSED AND REMANDED.

The judgm ent in No. 19-15043 is AFFIRMED.

24 Am icus Electronic Frontier Foundation (EFF) m oves to fil e an

am icus brief in this appeal. W e grant the m otion, and grant t h e Gonzalez Plaintiffs’ m o tion to fi l e a n oversized reply brief in order to respond to EFF. In its am icus brief, E FF r aises several argum ents con c e rn ing the First Am endm ent. W e often “decline to consider” am icus briefs that seek “to raise i ssues not raised or briefed by the parties.” Am. Trucking Ass’ns,
In c

. v

. City of Los Angeles

, 559 F.3d 1046, 1053 n.11 (9th Cir. 2009) (citing Day v. Apoliona

, 496 F.3d 1027, 1035 n.11 (9th Cir. 2007)). H ere, because t he parties did not raise t

he First Am endm ent, the panel declines to consider EFF’s argum ents on this issue.G O N Z A L E Z V

. G O O G L E

80

BERZON, Circuit Judge, concurring:

I concur in the m ajority o pinion in full. I write separately

to explain that, although we are bound by Ninth Circuit precedent com

pelling the outcom e in this case, I join the growing chorus of voices calling for a m ore l i mi t ed reading of the scope of section 230 im m un it y

. Fo r the reasons

com pellingly given by Judge Katzm ann in his p artial dissent in Force v. F

acebook

, 934 F.3d 53 (2d Ci r

. 2019), cert. denied

, 140 S. Ct. 2761 (2020), if not bound by Ci rc ui t precedent I w ould hold that the ter m “ p ublisher” under section 230 reaches only traditional activities of publication and distribution—such as deciding w hether to publish

, withdraw, or alter content—an d do es not include activities that prom o te o r recom m end content o r connect content users to each other. I urge this Court to reconsider our precedent en banc to the extent that it h olds that section 230 extends to the

use of m achine-lea r ning algorithm s to recom m end content and connections to users.

47 U.S.C. § 230(c)(1) provides: “No provider or u s er o f

an i n teractive com

puter service shall be treated as the publisher or speaker of any inform ation provided by another inform ation content pro v i d e r.” “This grant of im m unity applies only if the interactive com p uter service provider is not also an ‘inform ation content provider,’ wh i c h is defined as

som eone who is ‘responsible, in wh o le or in part, for the creation or developm ent of’ the offe n d i n g content.” Fair Hous. Council of S

an Fernando Valley v. Roommates.Com, LLC

, 5 21 F.3d 1157, 1162 (9th Cir. 2008) (en banc) (quoting 47 U.S.C. § 230(f)(3)). Although the statute was enacted in

response to the risk of liability for d efam ation, the language of the statute applies to any cause of action bas e d o n the publication or speaking of inform ation content. See BarnesG O N Z A L E Z V

. G O O G L E 81

v. Yahoo!, Inc

. , 570 F.3d 1 096, 1101 (9th Cir. 2009). This Court has held that im m

unity under sectio n 2 3 0 extends to

“ (1) a provider or user of an interactive com puter s e r v i c e (2) w hom a p laintiff seeks to treat, u nder a state law cause o f action, as a publisher o r speaker (3) of inform ation provided by another inform ation content p rovider.” Dyroff v. Ultimate Software Grp., Inc.

, 934 F .3d 1093, 1097 (9th Cir. 20 1 9 ),

cert. denied

, 1 4 0 S. Ct. 2761 (2020) (quoting B arnes , 570 F.3d at 1100–01).

The key issue as t o the non-revenue-sharing claim s in Gonzalez v. Google is whether Google, through YouTube, is b ei n g t r eated “as a publisher” of videos posted b y ISIS for

purposes of these claim s. W e have previously held that “publication involves reviewing, edit i n g, and d eciding whether to p u b lish or to withdraw f rom publication thirdparty co nt e n t.” Barnes

, 570 F.3d at 1102. A website’s d ec i s i ons to m oderate content, restrict users, or allow thir d parties full freedom to p o s t c ontent and interact with each other all therefore fall squarely w ithin the actions of a publisher shielded from liability under section 230.

But the conduct of the website operators here—like the conduct of m ost social m edia website operators today—goes

very m uch further. The platform s’ algorithm s suggest new connections between people and groups and recom m end long lists of content, targeted at specific users. As Judge Gould’s dissent cogently explains, the com pl aint alleges that the algorithm s used b y You Tu b e d o not m erely publish user content. Instead, they am plify a n d direct such content, i ncluding v iolent ISIS propaganda, to people the algo r i t h m deter mi n es to be interested in or susceptible to those

m essages and thus willing to stay on the platform t o wa t ch

m ore. Dissent at 96–97. Sim ilarly, “Facebook uses theG O N Z A L E Z V

. G O O G L E

82

algorithm s to create and com m unicate its own m essage: that it thinks you, the reader—you, specifically—will like this

content. A nd

. . . Facebook’s suggestions contribute to the creation of real-world social networks.” For c e , 93 4 F.3d

at 82 (Katzm ann, C.J., concurring in part and dis s e n ting in part).

In m y view, these types of targeted recom m endations and affirm ative prom otion of connections and interactions am ong

o therwise independent users are well outside the scope o f traditional publication. Som e sites use their algori t h ms to
connect users to specific content and highlight it as

recom m ended, rather than sim p ly distributing the content to
anyone who chooses to engage wi t h i t. Others suggest that

users com m

unicate with designated other users p reviou s l y unk n own to the recipient of the suggestion. See Dyroff

, 934 F.3d at 1095. Traditional publication has never included selecting the news, opinion pieces, or classified ads to send to each individual reader b ased on guesses as to their preferences and interests, or suggesting that one reader m ight like to exchange m essages with other readers. The actions o f the social network algorithm s—assessing a u ser’s prior posts, friends, or v iewing habits to recom m end new content and connections—are m ore analogo u s t o the actions of a direct

m arketer, m atchm aker, or recruiter than to those of a publisher. Reading the statute without r e g ar d t o our postBar ne s c a s e law, I would hold that a plaintiff asserting a claim based on the wa y that website algorithm s recom m end

content or connections t o u s er s is not seeking to treat the in teractive com

puter service as a “publisher” within an y u su al

m eaning of that term

. Instead, the website is engaging in its

own com m

unications with users, com posing and sending

m essages to users concerning what they m ight like to view o r who they m ight like to interact with.G O N Z A L E Z V

. G O O G L E 83

Nothing in the h istory of s e c t ion 230 supports a reading of the statute so e xpansive as to reach these websitegenerated m essages and functions. Section 230 “provide[d] internet com

panies with im m

unity from certa i n claim s ‘to

prom ote the continued developm ent of the Internet and other interactive com

puter services.’” HomeAway.com, Inc. v. City of Santa Monica

, 918 F.3d 676, 681 (9th Cir. 2 019) (quoting 47 U.S.C. § 2 3 0(b)(1)). But as Judge Katzm ann thoroughly explained in h is dissent in Force

, the aim of section 230 was to avoid governm ent reg ulation of internet content while “em p o w er[ i n g ] in teractive com p uter serv ice p ro v id ers to self

regulate, and

. . . provid[ing] tools for parents to regulate, children’s access to inappropriate m aterial.” Force

, 934 F.3d

at 79 (Katzm ann, C.J., concurring in part and dissenti n g in part). A New York state court had just he l d t h a t an Internet provider tha t h o s ted online bulletin boards could b e held liable fo r d e fa m ation as a publisher because it actively

m onitored and rem oved o ffensive content. See Batzel v. Smith

, 333 F.3d 1018, 10 2 9 (9th Cir. 2003) (citing Stratton Oakmont, Inc. v. Prodigy Servs. Co.

, 1995 W L 323710, at *4 (N.Y. Sup. Ct. May 24, 1995) (unp u b l ished)). So section 230, responding to Stratton Oakmont

, p r e vented providers from being treated as the publisher of third-party content, 47 U.S.C. § 230(c)(1), and elim inated liability for a c tions taken to restrict access to objec t i o n able m aterial, id. § 230(c)(2). A lthough “Congress grabbed a bazooka to swat the Stratton-Oakmont fly,” Force

, 934 F.3d at 80 (Katzm ann, C.J

. , concurring in part and dissenting in part), still, neither the text nor the history of section 230 supports a reading o f “publisher” that extends so far as to re a c h t argeted, affirm ative recom m endations o f conten t o r of contacts by social m edia algorithm s.G O N Z A L E Z V

. G O O G L E

84

BUT: As the m ajority opinion explain s , o ur case law squarely and irrefutably holds otherwise. There is just no getting around that conclusion, as creativ e ly as Ju d ge Gould’s dissent tries to do so.

Dyroff v. Ultimate Software Grp.

, Inc.

, 934 F.3d 1093, involved a social n e t wo rking w ebsite that allowed users anonym ously to share their experiences on any topic and post and answer q

uestions. Im

portantly, the w ebsite, Experience Project, also “recom m ended groups f or u sers to join, based on the content of their posts and other attributes, using m achinelearning algorithm s.” Id. at 1095. One us e r , W esley Greer, posted a question about buying drugs in a h e r o in-related

group, and the website sent him a notification when a nearby d rug dealer p osted in the sam e group. Id. Greer bou g h t heroin laced with fentanyl from the dealer and died from the drug. Id.

Dyroff held that “[b]y recom m ending user grou p s and sending em ail not i fications, [the website] was acting as a p u b l i s h e r o f o t h e rs ’ c o n t e n t

. T h e s e f u n c t i o n s

—

recom m endations and n otifications—a r e tools m eant to facilitate the com m

unication and content of others. They are not content in and of them selves.” Id. at 1098. To m e, those

tw o sen ten c es actually illustrate w h y th e reco m m en d atio n an d

em ail notifications are not actions taken in the role o f

p ub li s h e r

. Th e a c t i vi t i es h i g h l i g h t e d d o i n v o l v e

com m unication by the service provider, and so are activities independent of simp ly providing the public with content supplied by others.

The recom m endations and notifications in Dyroff are not

m eaningfully different than the recom m endations and connections provided by the social m edia com pani e s in theG O N Z A L E Z V

. G O O G L E 85

cases at issue here. Greer’s m o t h er alleged that Experience Project “steered users t o additional g roups dedicated to the sale and use of narcotics” and “sent users alerts t o p os t s within groups that were dedicated to the sale and use o f

n ar cotics,” both actions that relied on algorithm s to am pli fy and direct users to content. Id. at 1095. Lik e the

recom mendations provided b y YouTube, Experience Project’s r e com m endations com m u nicated to each u ser that the w ebsite t h o ught that user would b e interested in certain posts and topics. And, as h ere, the recom m ended connection

was to individuals openly engaged in illegal activity, and the

consequences were fatal. Just as the terrorist g roup’s deadly activities w ere, according to the c o mp laints in these cases, facilitated by re c o mm ending their gruesom e m essage to potential r e c r u i ts, so the drug dealers’ illegal activities in Dyr o ff were directly facilitated by connecting them with potential custom ers. A nd in b oth instances, the consequences of the service provider’s recom m endations were deadly.

The problem in our c ase law goes considerably further back than Dyroff

. Befo r e Dyroff

, Fair Housing Council of San Fernando Valley v. Roommates.co m, LLC

, held that section 230 d i d not im m

unize a website that “induced third parties to express illegal p r e ferences” by including discrim inatory criteria in a required form for people setting

up profi le s , 5 21 F.3d at 1165. Room m ates.com o p erated a website listing rentals and people seeking room s and required subscribers to list inform ation abou t their own and their preferred room m ate s ’ sex, sexuality, and fam ily status. Roommates held that although t he inform ation itself was provided b y third parties, the m andatory nature of the inform ation and the “lim ited set of p re-populated a n s w ers”

m ade Room m ates.com into “m uch m ore than a p assive

transm itter of inform ation provided by others; it becom es theG O N Z A L E Z V

. G O O G L E

86

d ev e l oper, at least in part, of that inform ation.” Id. at 1166.
Roommates distinguished between “ p roviding neutral tools [for users] to carry out what m ay be u n l a wful or illicit searches” or “allow[ing] users to specify whether they w ill o r will not receive em ails b y m eans of user-defined criteria” and operating “in a m an n er that contributes to the alleged illegality.” Id. at 1169.

As the m a

j o r ity discusses, Maj. O p. at 38, Roommates relied on our p rior decision in Carafano v. Metrosplash.com,
Inc.

, 339 F.3d 1119 (9th Cir. 2003), which held that a dating website was n o t l i a ble for an unauthorized and libelous profile cr e a t e d by a third party, see id. at 1122, 1125. The dating website “provided neutral tools specifically designed

to m atch rom antic partners depending on the i r v o luntary inputs.” Roommates

, 521 F.3d at 1172. Ca ra fa no determ ined that the website was being treated as a publisher and that the “additional features, such as ‘m atching’ profiles with sim ilar characteristics” were not sufficient to m ake the website into the “creator” or “devel op e r ” of the content in user profiles

u n der 47 U.S.C. § 2 3 0(f)(3). 339 F.3d at 1125; s e e i d

. at 1124–25. Instead, the Court determ ined that such features

were m ore akin to editing or selection. Id. at 112 4

. A tool

m atching two people who choose to share sim ilar inform ation about them selves is nearly identical to Facebook’s algorithm suggesting possible connections and is sim ilar to algorithm s

recom m ending new videos based o n past user viewing habits, and to the recom m endation and notification functions of the Experience Project at issue in Dyroff

. D yroff concluded that

“ [ t ] h e [ E x p e r i e n c e P r o j e c t ’ s ] r e c o m m e n d a t i o n a n dG O N Z A L E Z V

. G O O G L E 87

notification f

unctions,” like the tools in Carafano

, could not give rise to liability, b ecause they “helped facilitate

. . . userto-user com m

unication, but

. . . d id not m aterially contribute

. . . to the alleged unlawfulness of the content,” 9 34 F. 3 d at 1099.

The partial dissent considers the Gonzalez Plaintiffs’ allegations “m o r e a kin to those in Roommates.com than Dyroff b e c ause of the unique threat p osed by terrorism

com pounded by social m edia.” Dissent at 99. But the subject

m atter of the third-party content does n ot dictate whether an interactive com

puter service is being treated as a publisher o f that content. Nor does the test proposed in the partial dissent, which focuses on “m essage[s] designed to recruit individuals for a crim inal purpose” and m aterial contri b u t i o n “to a centralized cause giving rise to a p robability of grave h arm , ” id. at 100, m eaningfully d istinguish our case law, particularly Dyroff. The sale of h eroin is a crim inal purpose, and m any drug dealers operate as p art of crim inal n etworks. Although the h arm caused b y a t e rrorist attack is im m ense, the harm caused by the sale of fentanyl-la c e d h eroin is certainly “grave”—it le d t o Greer’s d eath in D yroff

. The allegation that the recom m endation to users of illegal terrorist m essages establishes the illegality of Google’s actions under the AntiTerrorism Act (ATA), 18 U .S.C. § 2333, exactly parallels the allegation in Dyroff that the dissem ination of m essages

con n e c t i ng drug dealers to buyers contributed to the harm s Congress intended to com bat by prohibiting drug trafficking.

I therefore concur in f

ull in the m ajority opinion, as we

are bound by this C ourt’s precedent in Dy roff extending im m u nity under section 230 to targeted recom m endations o f

content and connections. But I agree with the di s s e nt and
Ju d g e K atzm an n th at reco m m en d atio n an d so cial c o nn e c t i v ityG O N Z A L E Z V

. G O O G L E

88

algorithm s—as distinct from the neutral search functions discussed i n Roommates —provide a “m essage” from the social m edia platform s to the user about what conte n t t h e y will b e interested in and other people with whom they should

connect. Tran s mi tting these m essages goes beyond the publishers’ role insulated from liability by section 230.

I urge the Court to take this case en banc to reconsider our

c ase law and hold that websites’ u se of m achine-generat e d algorithm s to recom m end content and contacts are not within the p u b l i shing role im m unized under section 2 3 0. These

cases dem onstrate the dangers posed by extending section 230 im m unity to such algorithm ic recom m endations, an extension, in m y view, com pelled by neither the text nor history of the statute. As Judge Gould and Judge K atzm ann both em phasize, algorithm s on social m edia sites do not offer just one or two suggest i o n s ; they operate cum ulatively and dom inate the us er experience. “The cum ulative effect o f

recom m end[ations]

. . . e n v elops the u ser, im m ersing h er in

an entire universe fill e d wi th p eople, ideas, and events she

may never h ave discovered on her own.” Force

, 9 3 4 F. 3 d

at 83 ( Ka tzm ann, C.J., concurring in part and dissenting in part). If viewers start down a path of watching v ideos that the algorithm s link to interest in terrorist content, their im m ersive universe can easily becom e o ne filled with ISIS propaganda and recruitm ent. Even i f the algorithm is based on contentneutral factors, such as recom m ending videos m ost likely to keep the targeted viewers watching lo nger, the platform ’s

recom m endations of wh at to watch send a m essage to the

user. And that m essage—“you m ay be interested in watching

t hese v ideos or connecting to these people”—can radic a l i z e

users in t o extrem ist behavior and contribute to deadly terrorist attacks like these.G O N Z A L E Z V

. G O O G L E 89

I concur—but, for the reasons stated, reluctantly—in the

m ajority opinion.

GOULD, Circuit Judge, concurring in part and dissenting in part:

I

I concur in part in the m ajority opinion in its Parts I and II, Pa r t I II.A through III.D, Part III.F, and Part VI, but

r e s p ectfully dissent in part as to Part III.E, and Parts IV, V, and VII. These cases involve several s h o o t ing or bom bing i n c idents involving ISIS terrorists at far-flung worldwid e locations of Paris, France; Is tanbul, Turkey; and San Bernardino, California, in the United St ates. They also involve claim s that Internet or social m edia com panies such

as Google, YouTube, F acebook, and Twitter contributed to
acts of terrorism because of the o peration of their procedures and platform s. I concur insofar as the m ajority w ould reverse in part the dism issal of revenue-sharing claim s in Gonzalez v. Google

, and insofar as i t would reverse the district court’s judgm ent in T

a amneh v. Twitter that the com plaint f ailed to adequately state a claim for secondar y l iability under the Anti-Terrorism A ct (“ATA”). However, I respectfully dissent as to the m ajority’s dism issal o f the Gonzalez claim s

on grounds of S ection 230 im m unity, and of f

ailure to state a claim for direct or secondary liability under the ATA, because of the m ajority’s m istaken conclusion that there was no act o f international terrorism

, and I also w ould hold that the

com plaint adequately alleged that there was proxim ate cause supporting dam ages on those claim s.G O N Z A L E Z V

. G O O G L E

90

I f urther note that the m ajority here m akes its dism issive rulings solely on the pleadings a n d with no discovery to illum inate Plaintiffs’ well-plead factual contentions. Federal Rule of Civil Procedure 12(b)(6) perm its dism issal of claim s without p ondering evidence in cases where a com plaint fails

to state a claim

. FRCP 12(b)(6) has an im portant role to play in efficiently clearing the court s o f s uits that lack plausible allegations or where a l e ga l barrier like preem ption exists.
Yet in a case that does not warrant such a prom pt dism issal, 1

we do the legal system a d i s service by dism issing a case before considering the evidence that can arise in a p r operly

m onitored d iscovery period. A d e fe nd ant that actually has i m m unity is a good candidate for 12(b)(6) dism issal, b ut if

t he district court’s conception of the scope o f im m uni t y i s incorrect, as I believe it was here, then its dism issal under that rule will be untenable.

I would h old that Se c t i o n 230 of the Com m unications Decency Act (“CDA”) does not bar the G onzalez Plaintiffs’ claim s for direct and secondary liability under the A TA, and
I would allow those claim s to proceed to the district court for

a reas o n a b l e period for discovery. I agree that claim s can proceed in th e Taamneh case, and accordingly agree with reversing a n d rem anding in that case. And on the Clayborn

1 Doubtless the Defendant social m edia com panies w o u ld benefit from 12(b)(6) d ism issal at the outset—in a cas e w h e re they are actually im m une—to avoid expensive and t im e-consum i ng discovery procedures. However, while that relief would be “swift,” it would not necessarily be just. I am rem inded of t he often-quoted o b s e rv a tion by Justice Potter Stewart, when he was a U.S. Circuit Judge and before h is elevation to the Su prem e Court, t

hat: “Swift justice dem ands m o re than just swiftness.” Henderson v. Bannan

, 256 F.2d 363, 385 (6th Cir. 1 958) (Ste w a rt
, J

., dissenting) (capitalization altered). This observation has currency in civil cases as well, and not only in the crim inal justice context.G O N Z A L E Z V

. G O O G L E 91

v. Twitter case, I respectfully d issent because I think that the

m ajority’s conception of an attack authorized by ISIS is inconsistent with the allegations of the operative com plaint and well-established principles o f tort and agency law. Further, on all these claim s, I would perm it ame n dm ent if sought by plaintiffs based on a theory that t h e claim s are supported by specialized federal com m on law that m ay be applied in cases involving a particularly strong nati o nal interest and a gap in applicable statutory law. 2

I further urge that regulatio n of social m edia com panies would b est be handled by the p olitical branches o f ou r governm ent, the Congress and the E x ecutive Branch, but that in the case of sustained inaction by them

, the federal c o u r t s

are able to provide a forum responding to injustices that need

to be addressed by our justice system

. Here, that m eans to m e that the co u r t s should be able to assess whether certain procedures and m ethods of the social m edia com panies have created an unreasonably dangerous social m edia product that proxim ately caused dam ages, and here, the death of m any.

The i s sues here cannot be considered without

contem plating the specific fa cts alleged in the o perative

2 W e do not ordinarily consider an issue t h a t w a s not raised in the dist rict court, e.g. , Am. President Lines Ltd. v. Int’l Longshore & Warehouse Union, Alaska Longshore Div., Unit 60

, 721 F.3d 1147, 1157 (9th Cir. 2013), and sim ilarly do not norm ally consider issues that are not presented to us in th e briefing, e.g. , United States v. Garcia

, 149 F.3d 1008, 1010 (9th Cir. 1998). However, these rules have exceptions that are applied b y us i n extraordinary cases w here perm itting such an issue to be considered is necessary to avoid a m iscarriage of justice. See Hormel v. Helvering

, 3 12 U.S. 552, 557 (1941). Here, the three com plaints involve sufficiently strong interests o f the fam ilies w ith l oved ones lost to the ISIS attacks, so that these cases fall within the exception to the rule.G O N Z A L E Z V

. G O O G L E

92

com plaints. Because the treatm ent by t h e m ajority of the facts of the three case s captioned above is not contested by

m e , I m ention only the briefest thum bnail sketch of what is involved in the three cases that are now on appeal:

Gonzalez v. Google

, 18-16700, involved an ISIS shooting i n Paris on Novem

ber 13, 2015, which took the life of Nohem i Gonzalez, a 2 6-year-old U.S. citizen. This shooting

was one am ong a broader series of ISIS attacks in Paris on the

s am e day, including several suicide b om bings and m a s s shootings.

Taamneh v. Twitter

, 18-17192, concerns the no torious January 1

, 2 0 17 m ass shooting by an ISIS operative at the Reina nigh t c lu b in Istanbul, Turkey, which left 39 people dea d

, 6 9 others injured, and resulted in the death of Nawras Alassaf.

Clayborn v. Twitter

, 19-15043, concerns the Decem b er 2, 2015 attack by ISIS supporters at the Inland Regional Center in San B ernardino, California, which left 14 people dead and 22 others injured.

All of these terrorist incidents involved ISIS’s supporters. In all three cases, Plaintiffs alleged that Goo g le, through YouTube, and Twitter and Facebook, through their features, provided m aterial support to international terro rism and aided and a b etted international terrorism in violation of the ATA,

as am ended in 2016 b y t he Justice A gainst Sponsors o f Terrorism Act (“JASTA”). I would hold that the challenged conduct of the social m edia com panies is not im m

unized by Section 230 and that the com plaints’ allegations are sufficient

to plausibly allege that the Defendant social m edia com panies violated positive statutory law a nd proxim ately causedG O N Z A L E Z V

. G O O G L E 93

dam ages to Plaintiffs. In addition, I would hold that the sam e t y p es of claim s can perm issibly be asserted as a m att e r of federal com m on law upon am endm ent of the com plaints. See infra

, Section V.

II

My colleagues hold that Section 230 im m unizes Google from the Gonzalez Plaintiffs’ cla i ms that the YouTube platform ’s content-generating algorithm s aid and abet international terrorism by repeated l y recom m ending the propaganda videos of ISIS to u sers and by broadly dissem inating violent and radicalizing terrorist m essages. 3 It i s true that: “No provider or user of an interactive co mp ut e r service shall be treated as the publisher or spe a k e r of any

3 The p roblem I challenge is n ot t hat t he social m edia com

panies republish harm ful propaganda from ISIS; the proble m i s t h e algorithm s devised by these com

panies t o keep eyes focused on their webs i t e s. Historian Anne Applebaum

, who has evaluated the stresses o n dem o cracies in several countries in light o f m odern com m unications and technology, n otes th e fo llowing: “ [S]ocial m edia algorithm s them selves encourage false perceptions of the world. People click on the news they

want to h ear; Facebook, Yo u Tube, and Google t hen show them m ore o f whatever it is that they already favor, whether it is a certain brand of soap or a particular form

of politics. The algorithm s radicalize those who use them too. If you click on perfectly l egitim ate anti-im m igration YouTube sites, for e x a m

p l e, these can lead you quickly, in just a few m ore clicks,

to white n ationalist si t es and then to violent x enophobic sites. Because they have been designed to k eep you o nline, the algorithm s also favor

em otions, especially anger and fear. And because the sites are addictive, they affect people in w ays t hey d o n ’t e xpect. Anger b ecom es a habit. Divisiveness becom es norm al. Even if social m edia is not yet the prim ary

news source for all Am ericans, it already helps shape how politicians and journalists interpret the world and portray it. Polarization has m o v ed from the online world into reality.” See A nne A pplebaum

, Tw i l i g h t of Democracy

— The Seductive Lure of Authoritarianism (1st ed. 2020).G O N Z A L E Z V

. G O O G L E

94

inform ation provided by another info r ma tion content provider

. ” 47 U.S.C. § 230(c)(1). But in m y view, Section 230 was n ot intended to im m uniz e, nor does its literal language suggest that it im m unizes, com

pani e s p roviding interactive com

puter services from liability for serious harm s knowingly caused by their conduct. Plaintiffs raise a genuine factual issue of whether Defendants k new th a t I SIS and its supporters w ere inserting propaganda videos int o t h eir platform s, which perm its the inference that these social m ed ia

com panies were aware o f the risks to the public from incipient terrorists who, inflam ed by ISIS v ideos, would wreak havoc u p o n “infidels” who m ight be encountered by them

.

Even if under S ection 230 Google should n o t be considered the publisher or speaker of p ropaganda m essages posted b y ISIS or its sym pathizers, the YouTube platform nonetheless m agnified and am plified those com m

unications, joining them w ith sim ilar m essages, in a way that contributed

to the ISIS terrorists’ m essage beyond what would be done by considering them alone. Because ISIS depended on recruits

to carry out its cam

paign of worldwide hatred and violence, dissem inating its terrorist m essages through its propaganda videos was a p roxim ate cause of the terrorist attacks at issue here. W hen fairly read with notice pleading principles in

m ind, the com

plaints plausibly allege ISIS’s dependence on

rec r u it ment through social m edia’s free publicity and vast network.

I do not believe that Section 2 3 0 w as ever intended to im m u nize such claim s for the reasons stated in Chief J ud ge Katzm ann’s cogent and well-reasoned o pinion concurring in

par t a n d dissenting in part in Force v. Facebook, Inc.

, 934 F.3d 53, 7 6–89 (2d Cir. 2 019). Chief Judge Katzm ann’sG O N Z A L E Z V

. G O O G L E 95

partially dissenting opinion in Force v. Facebook is appended

as Attachm e n t A to this p artial dissent. Although I substantially agree with Judge Katzm a n n ’ s reasoning regarding S ection 230 im m unity, I add som e thoughts o f m y

own. In short, I do not believe that Section 230 wh olly im m u nize s a s ocial m edia com pany’s role as a channel o f

com m unication f

or terrorists in their recru iting cam paigns and as an intensifier of the violent and h atred-filled m essages they convey. The law should not give social m edia p latform s total im m

u nity, and in m y view it does not, beca u s e the conduct plausibly alleged d oes have “som e direct relationship,” Fields v. Twitter, Inc.

, 881 F.3d 739, 744 (9th Cir. 2018), between the asserte d i n j uries of the Plaintiff fam ilies and the Defendant social m edia com panies’ conduct. Further, Pl a i ntiffs plausibly alleged aiding and abetting claim s because providing the channels of com m unication for inflam m atory videos should be considered substantial assistance to the p r i m ary violations of terrorist shootings o r bom bings.

The m ajority splits Plaintiffs’ claim s into two categories: claim s based o n Google’s content-generating algorithm s (the “non-revenue sharing claims ” )

, and claim s based on ISIS’s

use of Google’s advertising program , AdSense (the “revenue sharing claim s”). The m ajorit y u l tim ately concludes that Secti o n 2 30 shields Google from liability for its contentgenerating algorithm s. I d isagree. I w oul d h o ld that Plaintiffs’ claim s do n ot fall within the am bit of Section 230 because Plaintiffs do not se e k to treat Google as a p ublisher

or speaker of the ISIS video propaganda, and the sam e is true

as to the content-generating m eth o d s an d d ev ices o f F aceb o ok and Twitter.G O N Z A L E Z V

. G O O G L E

96

Accepting plausible com plaint allegations as true, as we

m ust, Google, through YouTube, and Facebook and Twitter through their various platform s and program s, acted affi r m atively to am

plify and d irect ISIS content, repeatedly putting it in the eyes and ears o f persons who were susceptible to acti n g upon it. For exam ple, YouTube’s platform d id so by serving up an endless st r e am of violent propaganda content after any u ser showed an inclination to view such m aterial. At the sam e tim e, i t p e r mitted its

p la t fo rm s to be used to convey recruiting inform ation for ISIS-seeking potential terrorists.

Consider how the Google/YouTube algorithm appears to operate: T o illustrate, let’s assum e that a person went to YouTube and asked it to play a favorite song of s o m e artist like Elvis Presley or L inda Ronstadt, or a classical sym phony by Ludwig van Beethoven or W olfgang Am adeus Mozart, o r

a jazz piece by Miles Da v i s o r C h arlie Parker. A fter that requested song played, the vi e we r or listener would see autom atically a q u e ue of sim ilar or related v ideos showing either other so n g s of the requested artist or of som e other artists within sim ilar g enre. Sim ilarl y , if one went to YouTube to see a video about the viewer’s favorite N ational Park, the viewer would soon see a line of videos about other national parks or sim ilar scenery. A nd here’s the difficulty: If a person asked You Tube to play a video showing o ne bloody ISIS m assacre or attack, o ther such ISIS attacks w o u ld be lined u p, o r even starting to play autom atically. T h us, the

seem ingly neutral algorithm instead operates as a force to intensify and m agnify a m ess a g e. That poses no problem when the v ideo shows Elvi s Pr esley or Linda Ronstadt perform ing a m

u sical song, or shows a beaut i fu l N ational Park. But when it shows acts of the m ost brutal t e r r o r is m im aginable, a n d those types of im ages are m agnified andG O N Z A L E Z V

. G O O G L E 97

repeated o ver and o ver again, o ften coupled with incendiary lectures, t h en the benign aspects of Google/YouTube, Facebook and Twitter have been transform ed into a chillingly effective propaganda device, the results of wh ich were effectively realized in this case.

Section 230 of the CDA was aim ed at giv i n g Internet

com panies s o me breathing space to perm it rapid growth o f them and the econom y by providing that wh e n inform ation

was posted on a website, the interactive com puter service hosting that website would not be liable for the substance o f the content posted by the user. Pub. L. 1 0 4-104, § 509, 110 Stat. 56, 56, 137–39; Reno v. ACLU

, 521 U. S. 8 4 4, 857–58 (1997). Our circuit has developed and consistently applied a three-part test, the Barnes factors, for when im m u n ity ap p lies. See D yroff v. U

ltimate Software Grp., Inc.

, 934 F.3d 1093, 1097 (9th Cir. 2019). Un der those factors, a defendant is entitled to Section 230 im m uni t y wh en: (1) the defendant is “a provider or user of an in te ractive com p uter service, (2) whom the plaintiff seeks to treat, under a state law cause of action, as a publisher or speaker,” (3) “of inform ation

pro v id ed by another inform ation content provider.” Id.

at 1097 (quot ing Barnes v. Yahoo!, Inc.

, 570 F .3d 1096, 1100–01 (9th Cir. 2009)).

The factor at issue here is the second. Although Section 230 arguably m eans that Google and YouTube cannot be liable for the m ere content of t h e p o sts m ade by ISIS, that provision in no way p rovides im m unity for other conduct o f Google or YouTube or Facebook or T w itter that goes b eyond

m erely p ublishing the p ost. Here, P lainti ffs allege that Google’s “Services” include not just publishing content, but also “use of Google’s infrastructure, network, applications, tools and f

eatures, com m

unications services,” and otherG O N Z A L E Z V

. G O O G L E

98

specialized tools like “Social Plugins” and “Badges.” S im ilar allegations a r e m ade about other platform s’ tools and procedures. I would affirm in part to th e e x t ent the district

court applied Section 230 im m u nity to YouTube o r o t her platform s sim

ply carrying the p osts from ISIS on its p latform

, but not to the extent that it am plified and in part developed the t e r r orist m essage by encouraging sim ilar views to be given to those already determ in e d to be m ost susceptible to the ISIS cause.

I b el i e v e that m y view is consistent with our decision in Dyroff. The m ajority relies on D yroff for the proposition that Google’s algorithm s, which recom m end ISIS con t e nt to

users, are “neutral too ls” m eant to facilitate com m u nication and the content of others. According to m y colleagues, then, under Section 230, Google does n ot transcend the role o f a publisher by m erely recom m ending terrorism -related content based on past content viewed.

In D yroff

, Plaintiff c h a llenged a social networking website called “Experience Project,” which allowed users to
anonym ously s h a r e their first-person experiences, post and

an sw er q u estio n s, and interact with other users about differen t topics. 9 3 4 F. 3 d at 1094. The website interface “did not lim it o r p ro mote the types of experiences users shared”—instead, it was up to the u ser to use the site’s “blank box” approach to generate content. Id. The site also used

m achine-learning algorithm s to recom m end groups for users to join b ased on the content of their posts. Id. at 1095

. Plaintiff alleged that the site ’s functions, including

recom m endations o f n ew gr o u p s and notifications from

groups of which the use r is a m em ber, facilitated an illegal drug sale that resulted in the death of Plaintiff’s son, W esley Greer. Id. Greer posted on the site asking about wh er e toG O N Z A L E Z V

. G O O G L E 99

find heroin in a particular city, and a fellow u ser re sp o nded and sold fentanyl-laced heroin to Greer. Id. Greer was sent

an em ail n otification w hen the o th e r user posted, which resulted in the drug transaction. Id. W e held that the site was entitled to Section 230 im m unity because Plaintiff sought to
treat the defendant as the publisher of Greer and his dealer’s

content. Id. at 1097.

W e distinguished the facts in Dyroff from Fair Housing Council o f San Fernando Valley v. Roommates.com

, 5 21 F.3d 1157, 1167–69 (9th Cir. 2 008) ( en banc ). In Roommates

, we held that Section 230 did not im m u n i z e a website that

m atched people renting room s with people look i n g for

som ewhere to live from liability under federal and state housing anti-discrim ination laws. Id. at 1 161–62. The
Roo mma tes.com website d esign guided users through required discrim inatory criteria, “inducing third par t i e s to
express illegal preferences,” id. at 1165, and therefore the website itself “di rectly participate[d] in developing the alleged illegality.” D

yroff

, 9 34 F.3d at 1 099. In Dyroff

, then,

we drew a distinction between true m aterial contribution to a third p arty’s content—wh i c h would involve “responsibility for w hat m akes the d i splayed content illegal o r actionable”—and “actions (traditional to publishers) that are necessary to the display of unwelcom e and actiona ble content.” Id. (quoting Ki mz e y v. Yelp! Inc.

, 836 F.3d 1263, 1269 n.4 (9th Cir. 2016) (citation om itted)).

I would h old that the Gonzalez Plaintiffs’ allegations are

m ore akin to those in Roommates.com than Dyroff because o f the u nique threat posed b y terrorism com po u n d ed by social

m edia. ISIS content on YouTube is a pervasive phenom enon. Plaintiffs allege that “[t]he expansion and success of ISIS i s in large p art d

ue t o i ts use of the internet and social m ediaG O N Z A L E Z V

. G O O G L E

100

platform s to prom ote and carry out its terrorist activ i t ie s.” One study by t h e C o unter Extrem ism Project found that between March and June 2 018, 1,348 ISI S videos were uploaded to YouTube, garnering 1 63,391 views

. 4 Though websites using neutral tools like algorithm s are g enerally im m u nized by Section 230, I w oul d hold that w here the

we b site (1) k

nowingly am

plifies a m essage designed t o recruit individuals for a crim inal purpose, and (2) the dissem ination of that m essage m aterially contributes to a centralized cause giving rise to a probability of grave harm

, then the tools can no longer be considered “neutral.” Further,

a la c k o f reasonable review of content posted that can be expect e d t o be harm ful to the public, like ISIS’s violent propaganda videos, also destroys neutrality. 5

4 T he Counter Extrem ism Project, W h ite Paper, The eGl yph
W e b C r a w l e r : I

S I S C o n t e n t o n Y

o u T u b e ( J u l y 2 0 1 8 )

, h t t p s : / /www.counterextrem is m

. c o m / s i t e s / d e fa u lt / fi l e s / e G LY PH _w eb

_ c rawler_white_paper_July_2018.pdf.

5 Google suggests in its briefing t hat it tries to keep ISIS content from YouTube. But the record in this case suggests t h a t i f so, the control h as been ineffective. T h e r eco rd shows that d espite extensive m edia coverage, legal warnings, and congre s s i o n al hearings, social m edia com panies continued to provid e a p l a tform and com m unication services t o ISIS before t he Paris attacks, and these resources and services went heedlessly

to ISIS and its affiliates, as t he social m edia com

panies refused to actively identify ISIS YouTube accounts, and only r eviewed accounts reported b y other YouTube users. If, for exam ple, a social m edia com

p any m ust take down within a reasonable tim e sites identified as infringing copyrights, i t follows with stronger logic that social m edia com panies should take down propaganda sites of ISIS, once identifie d

, w ithin a reasonable tim e to avoid death and de s t ru c t i on t o the public, which m ay be victim ized b y ISIS supporters. Moreover, if social m edia com panies can b an certain

s pe a k e rs who f

lout their rules by conveying l ies or inciting violence, as

was widely r eported in the afterm ath of t w e e t s and posts relating to the

recent “insurrecti o n ” o f J a n uary 6, 2021, then i t is hard to see why suchG O N Z A L E Z V

. G O O G L E 101

In the case of terrorist recruiting, the dissem ination itself “contributes m aterially to the alleged illegality o f the conduct,” Roommates.com

, 5 2 1 F .3d at 1 1 6 8, in a w ay that dissem inating other violent videos would not. There can be

no doubt that ISIS’s u se of violence and threats of violence is part of its program of terrorism

. Contrary t o t h e ma jority’s contention that Google “m erely provid[ed] the publi c wi th

access to its platform ,” Google affirm atively sent a m essage i n s ubstance to users that individuals who enjoy watchi n g I SIS content m ay also be interested in joining its ranks. Much as allowing a room m ate-m atching website to screen candidates by discrim inatory criteria presents the sam e harm

a s d o i ng such screening in person or b y telephone (which is clearly prohibited by statute), a search engine that knowingly

t ransm its recruitm ent m essages to prospective terrori st s presents the exact danger—m aterial support to the terrorist cause—that Congress intended t o com bat with the ATA. Though indeed t h e r e are som e situations where tools like algorithm s can be “neutral,” wher e t h e m essage itself is the danger, the tool n ecessarily contribut e s to the alleged illegality of the conduct. 6

com p anies could not police and prohibit the transm ission of violent ISIS propaganda videos, in the periods preceding a t errorist attack. See Kate Conger & Mike Isaac, Twitter Permanently Bans Trump, C a p p i n g O n l i n e R e v o l t
, N

. Y

. T I M E S ( J a n

. 8

, 2 0 2 1 )

, h t t p s : / / w w w

. n y t i m e s

. c o m / 2 0 2 1 / 0 1 / 0 8 / t e c h n o l o g y / t w i t t e r

- t r u m p

suspended.htm l.

6 The m ajority contends that m y view, whi c h co n s iders the danger inherent in the m essage am plified by Google, i s inconsistent with congressional intent in enacting Section 230(c)(1). Though it is true t hat

an i n te ra ctive com

puter service is i m

m u ne when it is “treated as the publisher or speaker of any i nformation provided by another inform ation

content provider,” 47 U.S.C. § 230(c)(1) (em phasis added), t h e sam e portion of the statute m a kes clear that for Section 230 to apply, theG O N Z A L E Z V

. G O O G L E

102

Plaintiffs’ allegations u nderscore the danger o f

am plifying ISIS’s recruiting m essages. Plaintiffs allege that ISIS has used YouTube “to cultivate and m aintain an im age of brutality, to instill greater fea r and intim idation,” and to distribute videos “m ade in anticipation of the [Paris] attack showing each o f the ISIS terrorists who carried out the attacks telling of their intentions and then executing a captive for the

cam era.” P laintiffs allege that ISIS “not only uses YouTube for recruiting, planning, inciting, and giving instructions for

terror attacks,” but also u ses it “to issue terroristic threats

. . . intim idate an d c o erce civilian populations, take credit for

terror attacks, com m

u nicate its de sired m essages about the

terror attacks

. . . [and] dem and and attem pt to obtain results from the terror attacks.”

I n o te that Chief Judge Katzm ann’s concurrence in pa r t and dissent in part in Force v. Facebook, Inc.

, 934 F. 3d 53, 76–89 (2d Cir. 2019), relied on a reading of R oommates.com that is consistent with m y v iew here

. Chief Judge K atzm ann contended that Faceb o ok is d eveloping content by actively providing friend suggestions between users who h ave expressed simi l a r interests—in other words, the algorithm s provi d e d a “m essage” from F acebook to the user. Id.

plaintiffs m

ust be attem

pting through the i r s u i t to treat the w ebsite as a publisher or speaker. But em phasizing the danger of the terrorist m essage

s h ows that because Google is am plifying I SIS’s recrui t m e n t

m essage— a n d t h us acting as a content generator, not m erely a publisher—the inherent danger o f dissem ination m aterially contributes to the illegality of the conduct. See Room m a t e s , 5 2 1 F.3d at 1168. I f a website is acting as a publisher, then under Section 230 it will be i m m u ne

no m atter what inform ation it publishes from another source. But if, as is the case here, the dangerous n ature of the m essage m akes am plifying t hat

m essage transform what would o therwise be m ere publishing into content developm ent, then the website is no longer im m une under Section 230.G O N Z A L E Z V

. G O O G L E 103

at 82–83. In the sam e w ay, YouTube is “proactively creating

n et works of people,” id. at 83, who are sym pathetic t o t h e ISIS cause, and Google is d elivering the m essage that those YouTube users m ay b e interested in contributing to ISIS in a

m ore tangible way.

Furtherm o re, propagating ISIS m es sages has an

am plification effect that is greater than the sum of e a ch individual connection. See Force

, 934 F.3d at 83 (Katzm ann, J., dissenting in part) (“The cum ulative effect o f

recom m ending several friends, or se v e r a l groups or events, has an im

p act greater than the su m o f e a c h suggestion. It envelops the user, im m ersing her in an entire universe filled with people, ideas, and events she m ay n ever have d iscovered

on her o wn.”). Plaintiffs allege that G oogle does so in part by “us[ing] Y ou Tu b e to direct viewers to other o nline sites, postings, m edia, and other social network m edia.” W hen an ISIS recruitm ent video m a n a g e s to reach one p erson via YouTube that it m ight not otherwise hav e reached, that

person could join the cause by donating their tim e, m oney, o r

even their life. 7 W ith each person that joins its rank s, I SI S

grows in power and resources. It is the fact of recruitm ent to

7 As the Counter Extrem ism Project observes, “there is a clear l i n k between extrem ist v ideos and individuals who have sought t o support o r j o in ISIS. A j oint study from the University o f Chicago’s P roject on Security and Threats and the A ustralian Strategic Policy Institu te ’s Counter-Terrorism Policy Center found t hat 8 3 % of Am ericans who com m itted or w ere charged with ISIS-re l a t e d crim es between March 2014 and A u g ust 2016 watched ISIS

p ropaganda videos.” See W hite Paper, eGlyph at 2 (citing R o b e rt Pa p e , e t al., “The Am erican Face of ISIS,” Australian Strategic Policy Institute (Feb. 2017), https://s3-ap-southeast-2.am azonaws.com /adaspi/im port/ASPI_CPOST_ISIS_Indictees.pdf? 2Tbn8TshXm uj b 1 ft 8f7P IR7sukzyr hka).G O N Z A L E Z V

. G O O G L E

104

a centralized organization with the abili ty to cause disproportionate h arm that distinguishes a terrorist venture from a “norm a l ” crim inal v enture (as in Dyroff ). In Dyroff

, though the website connected G reer with a drug dealer that he

m ight not have o t herwise m et, the singular connection between the two was u nlikely to contribute t o a centralized effort to com m it international a t r ocities. I contend that the ATA codifies a “duty not to p rovide m aterial support to terrorism” p recisely because Congress recognized the exponential im

pact of s u c h c o nduct. See Force

, 934 F.3d

at 83–84 (Katzm ann, J., dissenting in part) (noting that “ATA

torts are atypical” because the Act prem ises liability “not on publishing qua publishing, but rather on Fa c e b ook’s provision of services and personnel to Ham as”).

For the foregoing reasons, I would hold that Section 230 does not i m m

unize Google from liability for its contentgenerating algorithm s insofar as they d evelop a m essage to ISIS-interested u sers. The sam e reason supports lack o f im m u nity for the other Defe n d ant social m edia com panies’

use o f their own algorithm s, procedures, users, friends, o r other m eans to deliver sim ilar content from ISIS to the users of the social m edia. But even if Dyroff cannot be fairly distinguished, then our circuit shoul d t a k e this case en banc

to m odify or clarify the rule that m achine-learning algorithm s

can never produce content within the m eanin g o f Section 230, 8 or the Suprem e Court should take up t he proper

8 The m ajority d ist i n g u i shes Chief Judge Katzm ann’s dissent in Force in part by em

phasizing t hat “ Ninth Circuit case law forecloses h is

argum ent,” though it recognizes that t he Force dissent m a i n tains t hat Section 230(c)(1) “need not be interpreted to im m unize websites’ friendand-content-suggestion algorithm s.”G O N Z A L E Z V

. G O O G L E 105

int e rp r etation of Section 230 and bring its wisdom and learning to bear on this com plex and difficult topic. 9

III

Having determ ined that S ection 230 does not im mu n ize Google for liability for either set of clai ms (non-revenue

I d isagree that our case law m ust b e read to foreclose the Go n z a l e z Plaintiffs’ argum ent, but the m ajority’s apparent recognition that frien dand-content-suggestion algorith m s could fairly be interpreted as outside of Sec ti on 2 3 0’s am

bit lends support t o a potential en b anc call in t h is

case.

9 Recently, Justice Thom as com m ented in connection with the denial of a writ of certiorari in M alwarebytes, Inc. v. Enigma Software Group USA, LLC

, 141 S. Ct. 13 (2020), that the Cou rt w o uld soon find i t appropriate t o take up a c ase interpreting Section 230. Justice Thom as notes that a n ew look at the statute is w arranted because “[w]hen Congress enacted the statute, m ost o f today ’s m a jor I nternet platform s d id not exist.” Id. at 13 (Thom as, J., writing separately). De s p i t e t h is, “ m any courts have construed the law b roadly t o c onfer sweeping im m unity o n

som e of the largest com

panies i n t

he world.” I

d

. Justice Thom as goes o n to explain that courts’ views of Sectio n 2 3 0 have gone from a “m odest understanding” to beyond what plausibly coul d h a ve been i n t ended b y Congress, inc l u ding conferring i m m u nity “even when a com pany distributes content that it knows is illegal. Id. at 15 (em phasis i n original).
In this separate st a t em ent, Justice Thom as m ade clear his view that the

scope of S ection 230 im m unit y s h ould be narrowed in l i ne with congressional i ntent. I agree with Justice Th o m as that Section 2 3 0 h as

m utated beyond the specific legal backdrop from which it developed, and
I cannot j oin a m ajority opinion that see k s t o extend this sweeping im m unity further. W h en one considers th e a n a lysis in the statem ent o f Justice Thom as in Malwarebytes

, the d issent of Chief Judge Katzm ann in
Force v. Fa c e book

, and the concurring opinion of Judge Tym kovich in FTC v. Accusearch

, 570 F.3d 1187 (10th Cir. 2009), I bel i e v e that there is a rising chorus of judicial voices cautioning against an overbroad reading of the scope of Section 230 im m unity.G O N Z A L E Z V

. G O O G L E

106

sharing and revenue sharing), I next consider wh e t h er Plaintiffs properly stated a claim for d irect liability under the ATA.

The m ajority holds that Se c t ion 230 im m unizes G oogle from liability for Plaintiffs’ non-revenue sharing claim s, so it does not address whether Plaintiffs adequately alleged prim ary liability for those claim s. H aving held that Section 230 d oes not p r eclude it f rom considering that issue for the

revenue sharing claim s, however, the m ajority concludes that Plaintiffs still d o not state a claim for prim ary liability under that th eory. Specifically, m y colleagues would decide that Plaintiffs fail to p lausibly allege that Google com m itted an act o f international terrorism

, or that Google’ s a c tions proxim ately caused Nohem i Gonzalez’s death. I ad d ress both bases for the m ajority’s conclusion in turn.

A

I would hold that th e Plaintiffs plausibly stated a claim that Google could be held p rim ari ly l i a b le under the ATA based on both Google’s revenue-shar ing procedure and Google’s content-generating algorithm s. At the m otion to dism iss stage, w e accep t all factual allegations in the

comp l ai n t as true and construe them in the light m ost favorable to the nonm oving party. Campidoglio LLC v. Wells Fargo & C

o. , 8 7 0 F.3d 963, 9 70 (9th Cir. 2017) (citation

om itted).

The civil rem edies provision of the ATA, 18 U.S.C. § 2333(a), allows a United States national who is a victim o f “an act of international terrorism ” to sue for d am ages in federal court. Acts constituting international terrorism “involve violent acts or acts dangerous to hum an life that areG O N Z A L E Z V

. G O O G L E 107

a violation o f the crim inal laws of the United States or of any State

. . . .” 18 U.S.C. § 2 331(1)(A). Such acts m ust “appear

to b e in t ended

. . . (i) to intim idate o r coerce a civilian

p op ulation; (ii) to influence the policy of a governm en t b y i n tim idation or coercion; or (iii) to affect the conduct of a govern me nt b y ma s s d e s t r u ct i o n , a s s a s s i n a t i o n , o r kidnapping.” Id. § 2331(1)(B).

1

The m ajority acknowledges that Section 23 0 d o e s not shield Google from liability on t he revenue sharing claim s becaus e t h e allegations are “prem ised o n Google providing ISIS with m aterial support b y g iving ISIS money .” I c o n cur with that aspect of the opi n i o n , but I would also add that

pro v id i ng m onetary support to a f oreign terrorist organiza t i on, with the constructive knowledge that that

m oney would likely be u sed as part of the terrorist enterprise, qualifies as an “act of internati onal terrorism .” 18 U.S.C. § 2333(a).

I begin with the contours of Plaintiffs ’ revenue sharing claim

. The com

plaint alleges that Google is aware of ISIS’s

presence on Y ouTube because it has received com plaints about ISIS content, and it has “suspended or b locked selected ISIS-related accounts at various tim es.” P laintiffs also allege that Google shares a percentage of t h e r e venue it generates from pairing advertisem ents and videos with the v ideo p o ster. Through Google’s co mme rcial service, AdSense, users can register their accounts for “m onetization.” Plaint i ffs allege that ISIS uses the AdSense m onetizatio n program to earn

reven ue

. Before the YouTube video can be approved for advertisem ents, Google m ust review and approve the video. Google has therefore “reviewed and approved ISIS videos

,G O N Z A L E Z V

. G O O G L E

108

including videos posted b y ISIS-affiliated users , for ‘m onetizati on’ through Google’s placem ent of ads in connection with those videos.” T hrough those approvals, Google gains constructive knowledge of the fa c t that it provided financial support to ISIS and incentiviz e d I SIS to continue to post videos on Y ouTube. Plaint i ffs ’ allegations about G oogle’s knowledge is bolstered by contention s that various news outlets reported on the kind o f ads appearing before ISIS YouTube videos.

The m ajo rity m i stak e n ly concludes that Google’s conduct

cou l d n ot qualify as international terrorism because it is not “intende d t o intim idate or coerce a civilian p opulation or to influence or affect a governm ent.” I disagree. The standard for intent under the ATA is not subjective; rath e r , i t is a “m atter of external appearance.” Boim v. Holy Land Found. for Relie f & De v

. , 549 F.3d 685, 694 (7th Cir. 2008) ( en banc ). I would hold t h at
, o n the facts alleged, a knowing

p rovision of resources to a terrorist organization constitutes aid to international te rrorism because an entity like G oogle

appears to intend the natural and f oreseeable consequences o f its actions. See Restatement (Second) of Torts

, § 8A (1965)

.

The m ajority relies on Linde v. Arab Bank, PLC

, 882 F.3d 3 14 (2d Cir. 2018), to conclude that k nowingly providing material support to a terrorist organization is not “an act of international terrorism ” if it is mo t ivated by econom ics.
Bes id e s t he fact that Linde is a sister circuit d ecision that is

not b inding on our court, it s facts and holding are also distinguishable. In Linde

, the court expressly held that it was

error f or the district court to instru c t th e jury that proof that Arab Bank provided m aterial support to a designated foreign terrorist organization, in violat ion of § 2339B, “necessarily proved the b a n k ’s com m ission of an act of internationalG O N Z A L E Z V

. G O O G L E 109

terrorism

. ” Id. at 325. Thus, t h e Second Circuit held only that vio l a t in g § 2339B does not inherently create an act o f terrorism

. The court’s reasoning continually references the

con t e x t of its decision: whether it could find that the jury instruction error was harm less. Id. at 327 (holding that “the

m ere p rovision of routine b anking services t o o rganizations and individuals said to be affi l i a t ed w ith terrorists does not necessarily establish causation ”) (internal q uotation m arks and citation om itted) (em

phasis added). Indeed, the court did

not even decide w hether Arab Bank’s financial services to Ham as should b e v i e we d as “routine” under the court’s precedent, because that issue raised a question of f act for the jury to decide. Id.

Even accepting that providing m ateria l a i d “does not invariably equate” to an ac t of international terrorism under § 2331(1), Linde

, 883 F.3d at 326, there are clearly situations where providing such aid operates to endanger hum an life and m anifests an appa r e n t i ntent to coerce or intim idate civilians or to influence or affect g overnm ents. The Seventh Circuit’s Bo i m decision represents such a case, despite the

m ajority incorrectly characterizing Plaintiffs’ relian ce on it as “m isplaced.” In Boim

, the court held that a jury cou l d find defendants liable under the ATA when t h e y had donated

m o n ey to H am as an d H am as-affiliated charities, k n o w in g th at Ham as used such m oney to finance violence towards at least

som e Am erican citizens. 549 F.3d at 690. Because donating

m oney to Ham as w as like “giving a loaded gun to a child,” it did not m atter that the act of giving m oney is not a violent act itself because, i n context, it would be “dangerous to hum an life.” Id. (citation om itted). The Seventh Circuit recognized that im posing liability f

or providing m oney to a te r r o r i st
group “m akes good sense as a counterterrorism m easure,” because “[d]am ages are a less effe ctive rem edy againstG O N Z A L E Z V

. G O O G L E

110

terrorists and their organizations than a gainst their financial angels.” Id.

Boim relied on the foreseeability of the consequences o f donating to Ham as to support its sensible holdi n g t h a t the donations would appear to be intended to intim idate or coerce

a civilian population. Id. at 6 94; see also Linde

, 883 F.3d at 327 (discussing B o im ’ s r easo n in g and stating that “given such foreseeable consequences,” the donations m et the statutory definition for an act of terrorism ). The court analogized donating to a terrori s t o r ganization to giving a sm all child a loaded gun because in both cases, t he actor is “doing

som eth ing extrem ely dangerous and without justification.” Id. at 693. “If the actor kn o ws that the consequences are certain, or substantially certain, to result from his act, and still

goes ahead, he i s t r e a ted by the law as if he had in fact desired to pro d u ce the result.” Id. (quoting Restatement (Second) of Torts , § 8A (1965)). The fact that the actor was

not m otivated by a d esire for the child to sho o t a nyone is o f

no m atter to the tort inquiry. Id.

The Gonzalez Plaintiffs allege that Goo g le k n ew IS IS was using its AdSense program , and that therefore Google knew it was providing m aterial support to a terrorist organization. The fa c t t hat Google was not m otivated by a desire to
augm ent ISIS’s efforts to recruit other terrorists is irrelevant. Th e m ajority’s argum ent—that Google’s interactions wi t h
I SIS via revenue sharing are not intended to intim idate o r

coerce civilian populations because Go o g l e was “m otivated by econom ics”—is an arbitrary l i n e divorced from Section 2333’s text and established principles of tort law. Boim

— a decision properly based upon Section 2333’s text and history—does not attem

pt to draw a line based on m otivation.
In fact, it rejects such a line as irrelevant to the question o fG O N Z A L E Z V

. G O O G L E 111

intent because a person intends what he k nows is substantially cer t a i n to result from h is act. 549 F.3d at 693. My colleagues attem

pt to distinguish Boim by noting t h at a donor to Ham as w ould likely share that organization’s vision

a n d ob jectives, but Boim did not rely on that aspect of targeted donation. Instead, the S eventh C ircuit reasoned that

“ [a] knowing donor to Ham as” is “a donor who knew t h e aim s and ac t i v ities of the organization.” Id. at 693–94 (em phasis added). It was the donor’s knowledge of Ham a s’ activities, rather than his approval of it, that ga v e rise to liability.

2

Because am

plifying ISIS’s m essage and creating new

n etw o rk s o f p ro sp ectiv e terro rist recru its f o reseeab ly p ro v id e s

m aterial support to a terrorist organization, I would likewise hold that the com

plaint in Gonzalez v. Google states a claim that Google is prim ar ily liable on a non-revenue sharing theory.

Terrorism is, in part, psychological warfare. The record shows that for ISIS terrorism is a psychologi ca l w eapon. ISIS’s m ost potent and far-reaching weapon is the Int e rnet. The Go nzalez com

plaint alleges that “Google’s YouTube platform has p layed an essential role in t h e r i se of ISIS,” which has becom e o ne of the largest perpetrators o f violence in the world. ISIS uses YouTube to recruit m em bers, plan terrori s t attacks, issue threats, take credit for attacks, and dem and and attem

pt to o btain re s ul t s from the attacks by influencing governm ent policies and conduct. W hile one o f ISIS’s goals is to com m it acts of violence, “ t he physical

a ttack itself and the harm to the individual victim s of th e attack” is just one piece of the puzzle—ISIS also u ses terrorG O N Z A L E Z V

. G O O G L E

112

attacks as a m eans to com m unicate its p o litical m essage and instill fear in those it considers its c o mb a t ants. Thus, the im pact o f ISIS’s terrorism is dependent upon its ability to

com m unicate its m essage and reach its intende d audiences. Id. Plaintiffs allege that “ISIS’s use of v iolence and threats of violence [ are] part of its program of te r r o rism

, designed

. . . to gain attention, instill fear and ‘terror’ in others, send a

m essage, and obtain results.” Because the com m unication o f ISIS v iolence and threats is part o f th e terrorist attack, repeated postings and encouraged viewings of ISIS videos, as effected by Google’s algorithm s, is also part of the attack.

W hen a terrorist group blows up or shoots up or carves up passengers o n an airplane, railroad car o r a subway car, they do not do it m erely to destroy property or injure peo p le involved in those bom

bings, shootings, and k ni fing attacks. Instead, they a i m to create f ear in the public so that people will be afraid to use airplanes or railroad cars or subways or

any general public area to go abo u t t h e ir business as usual. Publicizing the event is just as essential to terrorists’ success

as is the bom

bing, shooting, o r knifing itself. So-called “neutral” algorithm s create d by Facebook, Twitter, and Googl e , are then transform ed into deadly m issiles o f destruction by ISIS, even though they were not init i ally intended to be used tha t wa y

. But once there is a consistent

stream of conduct by ISIS, it should be und er s t o o d that defendants who p assively ignore that conduct can be held to have intended the natural and probable consequences of their actions. See Restatement (Second) of Torts , § 8A (1965).

Just as sharing revenue with ISIS is “dangerous to hum an life,” Boim

, 549 F.3d at 690 (citation om itted), so is

am plifying its m essage and encouraging recruit me n t to its

r a n k s. Perhaps even m ore so because unlike m oney, whic hG O N Z A L E Z V

. G O O G L E 113

is fungible, YouTube has a virtual m onopoly on h osting

extrem ist videos. 1 0 ISIS can get operating funds fr o m a variety of sources, but very few plat fo r ms have the international network and infrastructure to which YouTube has access. Im

posing liability o n social m edia platform s for affirm atively am

plifying ISIS’s m essage can therefore “cut the terrorists’ lifeline.” See id. at 691.

B

Direct liability claim s under the ATA require that plaintiffs show they suffered injury “by re a s o n o f an act o f international terrorism .” 18 U .S.C. § 23 3 3(a). The “by

reason of” phrasi n g h a s been understood to im pose a requirem en t o f proxim ate causation. See, e.g. , Fields v. Twitter

, 881 F.3d 739, 744 (9th Cir. 2018). To m eet this requirem e n t
, “ a plaintiff m ust show at least som e direct

r e lationship between the injuries that he or she suffered an d the defendant’s acts.” Id. at 744.

On m y view of t h e c ase, the proxim ate cause issue m ust be reached, a n d I believe that it is satisfied. The ATA’s

purpose in part is to p rovide a financial rem edy to victim s o f terrorism. Indeed, ATA’s legislative history dem onstrates Congress’s intent to authorize the “im position of liability at
any point along the cau s a l chain of terrorism .” S. Rep. No. 102-342, at 22 (1992) (referencing “ t h e flow of m oney” to terrorist groups).

1 0 S ee, e.g. , Neim a Jahrom i, The Fight f or the Future of YouTube

, N E W Y O R

K E R ( Ju l y 8

, 2 0 19), h ttps://www.newyork er .co m / t ech / a n n al s- o f

technology/the-fight-for-the-future-of-youtube.G O N Z A L E Z V

. G O O G L E

114

My view is consistent with our decision in Fields v. Twitter. In Fields

, we acknowledged that acts of international

te r r o rism are foreseeable consequences of financial support
to a terrorist organization, but we also no t e d t hat such fungibility “does not relieve claim ants of thei r burden to show causation.” Id. at 749. Fields requires that a p la i n tiff plausibly allege a “direct relationship between a defendant’s

act and [a plaintiff’s] injur[ies],” id. at 748, and that elem ent is m et here because there is a sufficient nexus.

Plaintiffs allege that ISIS operatives involved in the Paris Attacks posted links to ISIS YouTub e v i d e o s. The sum o f Plaintiffs’ allegations dem onstrate that the terrorists responsible for Plaintiffs’ injur i e s u sed YouTube as an integral com

ponent of recruiting, and that such recruiting is necessary to carry out attacks at the scale of those in Paris.

Specifically, Pl a intiffs allege that at least two of the twelve ISIS terror i s ts w ho carried out the Paris Attacks, Abaaoud and Laachraoui, u sed online social m edia p latform s to p ost links to ISIS recruitm ent Y ouTube videos and “ jihadi YouTube vid e o s

. ” Plaintiffs allege that Abaaoud, “considered the o perational leader of the Paris Attac k ,” was

a n active user of social m edia, including YouTube. I n a M arch 2014 ISIS Y ouTube video, “Abaaoud g a v e a

m onologue (in French) recruiting jihadi fighters for ISIS.”

Pla i n ti ffs also allege that at the tim e of the attacks these

two ISIS terrorists, who we re “instrum ental in the Paris Attack,” were m em

bers of or at least involved with ISIS networks in Belgium calle d “ The Z erkani Network” and Sharia4Belgium

. The Belgian networks “used and relied on social m edia to b uild and m aintain connections wi th ISIS recruits.” Plaintiffs allege that there was a pervasive networkG O N Z A L E Z V

. G O O G L E 115

of ISIS recruiters in Belgium

, which ha s b een called “the epicente r o f the Islam ic State’s efforts to attack Europe.” Sharia4Belgiu m ma i nt a ined several active YouTube channels, still active at the tim e of the Paris Attacks, “which it used to post serm ons, speeches, news events, a n d other

m aterials to lure, recruit, and indoctrinate young Muslim s to travel to Syria and Iraq to join I SI S.” Plaintiffs allege that there w as s ig n i ficant o verlap and coordination over tim e between Sh aria4Belgium and “The Zerkani Network.” Plaintiffs alle g e that Laachraoui was involved with Sharia4Belgium at the tim e of the Paris Attacks, and his social m edia accounts appear to s h o w t hat he followed ISIS social m edia and posted links to jihadi YouTube videos on his own account.

Though Pl aintiffs do not specifically allege how the perpetrators of the Par i s Attack were radicalized, such an allegation is not necessary to plausibly state their claim

. It is enough that the com

plaint al l e g e d that the perpetrators them selves activel y u s e d YouTube to recruit others to ISIS, gaining resources with which to p lan and im plem ent their

at t ac ks ; absent the participation of the social m edia

c o m panies for their own profit-centered purposes, terrorist

g roups like ISIS would not have these resou r c e s

. Additionally, P laintiffs alleged that “The Zerk a n i Network”

recru ited one of the shooters, Abaaoud, “an active user o f social m edia, including YouTube,” and also alleged that the

netwo r k “ u sed and relied on social m edia” to recruit,

perm itting the inference t h at it is probable Abbaoud was radicalized through social m edia. Viewing these allegations in the light m ost favorable to the nonm ovi n g party, as we

m ust, Campidoglio

, 870 F.3d at 9 70, Plaintiffs have plausibly alleged a sufficient nexus between Google’s conduct and theG O N Z A L E Z V

. G O O G L E

116

Paris Attack v i c t i ms ’ injuries to satisfy a proxim ate cause threshold standard. 1 1

A possible analogy m ay help to illustrate h o w the social

m edia com

panies’ e n h ancem ent and spread of ISIS propaganda prom oting violence and seeking to convert recruits h as a direct relation to the dam ag e s caused here. Let’s assum e that a person on one side of a crowded football stadium fires a high-powered rifle aim e d a t a c rowd on the

opp os i t e s ide of the stadium

, filled with people, though all identities are unclear. W ould the m a j o rity here say that the rifle shot striking an unidentified viewer on the other side o f the stadium had no “direct relati on ” to the shooter and that the shot d id n ot proxim ately cause a resulting death? I think

not. There is d irect relation between shooter and victim there sufficient to satisfy Fields and there is sim ilar direct relation here between the challenged conduct o f the Defendant social

m edia com

panies an d t he victim s of ISIS violence in these

cases to say that the c ha ll enged conduct, if shown to be illegal, was a proxim ate cause of dam ages.

11 It i s worth noting t h at the contrary conclusion, espoused by the

m ajority, would put these and future plaintiffs i n an untenabl e p o sition. If we required plaintiffs t o specify exactly how an individual terrorist becam e radicalized without the benefit of discovery, then it is unlikely t hat

a n y such claim s could go forward. At the m otion to dism iss stage, w i t h notice plead i n g p rinciples in m i nd, the Gonzalez Plaintiffs need only plausibly allege “som e direct r elation” between the terrorist’s actions and the social m edia com

panies’ conduct. See F ields

, 8 8 1 F. 3d at 7 4 9 (citation o m itted). Here, P laintiffs alleged that the perpetrator of the Paris Attack was a m em

b e r o f a p articular network that used social m edia to recruit its m em

b ers, and that t

he perpetrator him self was a regular user o f social m edia. G iven that it is u n l ikely potential terrorists will announce the avenues by which they were radicalized, such i nferences are perm issible.G O N Z A L E Z V

. G O O G L E 117

IV

I next turn to whether P laintiffs have adequately alleged claim s against Google for secondary liability under JASTA. As with prim ary liability, the m ajority addressed only the

revenue sharing claim s in its opinion, but I wo u ld hold that for either set of claim s, Plainti ffs h ave successfully stated a claim for secondary liability.

Congress am ended the ATA by enacting JASTA in 2016, Pub. L. No. 144-222, 1 30 Stat. 8 54 ( Se p t. 28, 2016), which extends liability to persons who aid and abet by providing substantial assistance to p ersons w ho com m it a c ts o f internat i on a l terrorism

, and those who conspire to com m it such acts. 18 U.S.C. § 2333(d)(2). Under § 2333(d)(2) of the ATA, “liabil i t y m ay be asserted as to any person who aids and abets, b y knowingly providing substantial assistance” to “the p erson who co m m itted

. . . an act of international terrorism

. ” Id

. I recognize the p r o pe r legal fram ework for analyzing such claim s as that described in H alberstam v. Welch

, 705 F. 2 d 4 7 2 (D.C. Cir. 1983). Like the m ajority, I first conclude that the first two Halberstam factors h ave been satisfied here: (1) the p arty whom the defendant aids perform ed a wrongful act that caused an injury; and (2 ) the d efendant was “generally aware of his role as part o f an overall il l e g a l or tortious activity at the tim e that he provide[d] the assistance.” H alberstam

, 705 F.2d at 477. For the first elem ent, the com pla i nt plausibly alleges that the Paris Attacks were “com m itted, p lanned

, o r authorized” by ISIS, a designated terrorist organization. See 18 U.S.C. § 2 333(d)(2). For the second elem ent, I agree that Go o g le

was “generally aware of its role in ISIS’s terrorist activities”

at the tim e it used its content-generating algorithm s to send a

m essage to YouTube users and at the tim e it shared revenueG O N Z A L E Z V

. G O O G L E

118

through AdSense. In both cases, Google wa s a wa re that it

assum ed a role in ISIS’s terrorist activities. See Halberstam

, 705 F.2d at 488; see also Linde

, 8 82 F.3d at 329 (noting that the elem ent does not require a showing of “specific intent” as in crim inal aiding and abetting, nor d oes it require that t he defendant “knew of the specific attacks at issue”).

Unlike m y colleagues, however, I also conclude that the final elem ent is m et: the defendant “knowingly and

s u b s t a n t i a l l y a s s i s t e d [ e d ] t h e p r i n c i p a l v i o l a t i o n

. ” Hal be r s t am , 705 F.2d at 488

. The m ajority acknowledges that Go o g l e knowingly assisted the principal violation, but denies that such assistance was “substantial.” 1 2

I would hold that Google’s assistance via its contentgenerating algorithm s and revenue sharing was both knowing and substantia l

. I need not view the non-revenue sharing claim s and revenue sharing claim s in isolation in this portion of m y analysis. Because I conclude that both sets of Plaintiffs

c l aim s are not barred by Section 230, it is the sum of Google’s conduct that m ust be considered when as s e ssing wheth er t h e assistan ce w as substantial. The Halberstam court identified six factors relevant to assessing wh ether the substantial assistance com ponent is satisfied: “(1) the nature of the act encouraged, (2) the am ount of assistance given by

12 It m ay be that what is considered by one person to be “substantial assistance” i s consider ed b y an o t h er m er el y d

e m i n i mis or inconsequential.
But even if that is so, i t would be a better procedure t o leave that decision

to fairly selected jurors w ith proper j u ry instructions explaining the “substantial assistance” elem ent. But to m e it is clear that ISIS could not exist and renew itself without constant recruitm ent of foot soldiers to carry out its violent m issions, often at the cost of their own lives, so I regret t hat
I cannot persuade m

y colleagues here to adopt a m ore perm issive standard for substantial assistance.G O N Z A L E Z V

. G O O G L E 119

defendant, (3) defendant’s presence or absence at the tim e o f the tort, (4) defendant’s relation to the principal, (5) defendant’s state of m ind, an d ( 6) the period o f defendant’s assistance.” Linde

, 882 F.3d a t 3 2 9 (citing Halberstam , 705 F.2d at 483–84).

Under the first fa ct o r, the Halberstam court em phasized that the nature of the principal’s act “dictates w hat aid m ight

m atter, i.e.

, be substantial.” 705 F.2d at 484. T he rem aining factors m ust be viewed through this len s. ISIS’s long-running and far-ranging terrorist cam paign d epends on the continued provision of m oney and recruits. Google provided both. A s the m ajo rity acknowledges, financial support is “indisputably im portant” to operating a terrorism cam paign, and any m oney provided to the organization m ay aid its g oals. See id. at 488; Fields

, 881 F

. 3d at 748. The m ajority also acknowledges, in the context of reversin g t h e district court’s dism issal o f Taamneh

, that YouTu b e videos encourage ISIS’s terrorism

cam paign—an enterprise that is “heavily dependent on social

m edia platform s t o re c r u it m em bers, to raise funds, and to dissem inate propaganda.” Google provided free exposure to

a dangerous organization, thereb y f acilitatin g ISIS’s ability to reach and rouse prospective recruits. The Gonz a l ez

com plaint alleges that ISIS through YouTube exaggerated its territorial expansion by dissem in a ti n g videos with m aps showing ISIS’s claim s that it controlled certain regions where other groups h a d p l edged allegiance to ISIS. The fourth fac t o r a lso weighs in f avor of recognizing substantial

a s s i stance: defendant’s “relation” to the principal—or th e extent to which an entity “m ay possess greater powers o f

sugge s t i o n.” YouTube’s role in cultivating extrem ist behav i o r h as been w idely acknowledged and the platform reaches a virtually unlim ited num ber o f potential recruits d

ue to the ubiquity of the Internet. The sixth factor, “duration o fG O N Z A L E Z V

. G O O G L E

120

the assistance provided,” c o nc erns the length of tim e an alleged aid e r and abettor h as been involved with the tortfeasor. See H

alberstam

, 705 F.2d at 484 (em phasis

om itted). Though the com plaint in Gonzalez lacks s p ecific evidence about the length of tim e Google provided assistance

to ISIS, Plaintiffs allege the p lacem ent of ISIS recruiting videos going b ack at least four years before the Paris Attacks, i n 2014. The com

p laint also alleged through news sou r c e s that advertisem ents were placed on ISIS’s YouTube videos as early as March 2015, three years before the Paris Attacks. I would hold that years of hosting ISIS content and providing it with a percentage o f revenue is sufficient duration. Though Plaintiffs do not allege that G oo g l e shared ISIS’s terrorist goals, Halberstam also directs that under the fifth f a c t o r, defendant’s “state of m ind,” the court can c o nsider the duration factor because it “alm ost certainly affects the quality and extent” of the aid, the am ount o f a id provided, and “it

m ay afford evidence of the defe ndant’s state o f m ind.” Id.
Even considering st ate o f m ind on its own and viewing that fa c t o r in light of “the nature of the act encouraged,” see id.

, providing financial assistance an d exposure to—to put it

m ildly—a dangerous group, is sufficient for state of m ind to weigh again s t Go o g l e. As I see it, the conduct o f Google, Twitter, a nd Facebook as related to the risks o f terrorist attacks by ISIS, absent their m ore active review and policing

o f sites, is either recklessly indifferent o r willfully b l i n d

, a s they enjoy in c reased advertising revenue associated with eyeballs on videos or posts about ISIS attacks.

Taken as true and viewed in t h e light m ost favorable to Plaintiffs, I would hold that the s e allegations establish that Googl e ’s assistance was sufficiently “substantial” for

purposes of § 2333(d)(2). These sam e considerati o n s applyG O N Z A L E Z V

. G O O G L E 121

in all three cases, so in each I would hold ther e was substantial assistance for purposes of § 2333(d)(2).

I add a brief com m ent about the Clayborn v. Twitter case. Th ere the m ajority would u phold dism issal of the clai ms because o f its view that Plaintiffs do n ot plausibly allege that ISIS “com m itted, planned, or authorized” the San Bernardino

a t t a c k, as is required under 18 U.S.C. § 2333(d)(2). Th e

m ajority relies on a Sixth Circuit decision, Crosby v Twitter, Inc

. , 9 21 F.3d 617 (6th Cir. 2019), but its reasoning is not
per s uasive and does n ot bind or even guide our circuit, because there, the com

plaint prod u c e d “no allegations that ISIS was involved with the Pulse Night Club shooting.” Id.

at 626. However, the record here is distinctly and plainly to the contrary: The com

plaint expressly alleges that prior to o r during the attack, one of the perpetrators—Tashfeen Malik—declared on h er Facebook page the two shooters’ allegiance and loyalty to an ISIS leader. Two days after the attack, ISIS issued a st atem ent on a radio station claim ing responsibility for the attack. The FBI confirm ed that one of the shooters, a fe w years before the attack, had face-to-face

m eetings with five people known to have “links to terrorism

. ” Further, Pl ai n tiffs allege that FBI investigators found an explosive d ev i ce placed at the crim e scene that was likely intended to be detonated by the arrival of first responders. A Departm ent of Justice report described th is as “a frequent,

we ll docum ented practice in international terrori s m incidents.” Im

portantly, FBI investigators explained that this “terrorist tactic ha[d] b e en outlined in Al Qaeda’s Inspire Magazine

, as w ell as in ISIS’s Dabiq Magazine .” Plaintiffs allege that these m agazines are dissem inated on Defendants’ platform s. Together, these allegations perm it t h e fair inference tha t the attack which was planned for at least o ne year was inspired by—and im plicitly authorized by—ISIS.G O N Z A L E Z V

. G O O G L E

122

In m y v iew, even if Malik had been “self-radical i z e d” without d ir e c t com m

u nications or m eetings with ISIS operatives, P laintiffs p lausibl y allege that the selfradicalization process included exposure to t he violent recruiting videos of ISIS, along with lectures from incendiary advocates of violence against n on-believers. According to the com plaint, in Senate Judiciary Com m ittee testim ony, then-FBI Director Jam es Com ey described the pair as having “consum [ed] poison on the internet” and been “radicalized to jihadism and to m artyrdom via social m edia p latform s available to them .” Finally, even assum ing the perpe t r ators had l ittle advance connection w ith ISIS, well-established principles of agency law illust r a te that authorization can

occur not only by advance planning, but also by ratification. S ee Restatement (Third) of

Agency

, § 4.01 (1 ) (1 9 3 3 ) (d ef in in g ratification as “the affirm ance of a prior act done by another, whereby the act is given effect as if done by an agent acting with actual aut h o r ity”). 1 3 Because the San Bernardino

13 Contrary t o the m ajority’s contention, there is support for applying

com m on law agency principles to secondary liability for a c t s o f international terrorism

. For one thing, “statutes are presum ed not to disturb the com

m on law, ‘u n l e s s the language of a statute [is] clear and

e x plicit for this purpose.’” State Eng’r of Nev. v. S. Fork Band of TeMoak Tribe of W. Shoshone Indians of Nev. , 339 F.3d 804, 814 (9th Cir.
2 003) (quoting Norfolk Redevelopment & Hous. Auth. v. Chesapeake &
Potomac Tel. Co. of Va.

, 464 U.S. 30, 3 5 ( 1983)). In m y view, nothing in the statute precludes co n si deration of com m o n l aw principles. Second, the Suprem e Court h as stated that apparent authority principles “ha[ve] long been the settled r

ule in t he f ederal sys t em

. ” Am. Soc’y of Mechanical Eng’rs, Inc. v. Hydrolevel Corp.

, 456 U.S. 556, 567 (1982). Section 2333(d)(2) assigns liability for injuries arising from acts of international terrorism

, where that act was authorized by a t errorist o rganization. See 18 U.S.C. § 2333(d)(2 ). In m y view, asking whether a terrorist organization authorized a particular terrorist act is properly viewed underG O N Z A L E Z V

. G O O G L E 123

shooters pledged them selv e s to ISIS before or d uring the attack, and an act is ratifiable “if the actor acted or purported

to a c t as an agent on the person’s behalf,” id. § 4.03, the attack can be considered authorized by ISIS.

For the foregoing reasons, the com plaint in Cla

y b orn

m akes allegations sufficient to state a claim for liability under the ATA.

V

In m y view, the claim s asserted in the three com plaints on appeal should all be sustained and perm itted to go forward in discovery based o n the statutory law standards above discussed. B ut even if I am incorrect in m y view of the governing statutory law, t ho se claim s should be able to go forward wi t h co m

plaint am endm ent based on a still extant specialized federal c o mmo n law in aid of national security against terrorism

. Aft e r the general com m on law regim e o f Swift v. Tyson was overruled by Erie

, a sphere of specialized federal com m on law rem ains and co u l d s upport Plaintiffs’ claim s here. S

e e e

. g

. , 19 Charles Alan W right & Arthur R. Miller, Federal Practice & P rocedure § 4514 (3d ed. 2021). As the W right & Miller treatise explains, “the federal

com m on law that has d eveloped s i n c e Erie differs from the federal general com m on law [rejected in] Swi ft v

. Tyson because it falls within an area of federal o r n ational

com petence.” Id. (footnote om itted); see also 17A Moore’s Federal Practice

, Civil § 124.40 (2020). Many federal court precedents have applied th e s e principles, which are particularly well-suited when claim s i n v o lve an area o f

the com m on-law a gency fram ework as a question of whether the perpetrator was acting as an agent of the terrorist organization.G O N Z A L E Z V

. G O O G L E

124

heightened federal interest, such as international terrorism

, o r when gaps exist in a f

e d e r a l re gulatory schem e. E.g.

, Boyle

v. United Techs. C

orp. , 4 87 U .S. 500, 507–08 (1988); Textile Workers Union of

Am. v. Lincoln Mills of

Ala.

, 353 U.S. 448, 456–57 (1957); King Jewelry, Inc. v. Fed. Express Co r p. , 316 F.3d 961, 964–65 (9th Cir. 2003). 1 4

Also, our court should not ignore other potential areas o f hum an conduct that can b e n egatively im pa cted by an unregulated social m edia regim e, coupled with efforts by

gro u p s hostile to the idea of Am erican dem ocracy to use social m edia in order to divide or terrorize our public. A reas of particular concern include im pacts of s o c i al m edia in realm s such as election law, the laws governing public order and protest, and even insurrection.

W e should not of co ur s e ignore the trem endous, indeed alm ost unquantifi a ble, benefits to the public from social

m edia. Social m edia perm its friends to stay in contact, as for

exam p le with a club o r g ro up from high school or college, lets people ma k e n e w friends, or even lets people see or be exposed to new sights from different pa rts of the world. People m et th r o u g h social m edia, who m ay h ave different interests, p erspectives, and priorities from other social m edia

users, can in m any cases enrich those users’ li v es

. Places visited on the internet, often encouraged or directed through social m edia, can serve the sam e benign function. But at the

14 Contrary to the m ajority’s c o n tention, m y view on when federal

com m on law m ay be created i s narrow. Though the federal courts “are

not free to m anufacture entirely new causes of action m erely because the political branches have not acted,” I believe that we can act w h e re gaps

are present i n an existing federal statutory schem e and the claim involved is one of unique federal concern.G O N Z A L E Z V

. G O O G L E 125

sam e tim e, benefit alone c a n not end the inquiry. Social

m edia activities also carry with them som e risks and detrim ents to the public. F or exam ple, there is no doubt that

m odern pharm aceutical drugs g ive benefits to the public that

were im possible at earlier tim es and are greatl y v a lu ed by those who use them

. But d rugs can also have ha r m f ul im pacts and, accordingly, they are regulated by the Food & Drug Adm inistration. Sim ilarly, m odern aircraft h elp people

m ove from one part of our world to another with great speed and ease, but we regu late airlines through the Federal Aviation Adm inistration. One could go on and on as alm ost
every m ajor activity in the m odern world faces som e type o f federal regulation.

This regulation of the social m edia com panies would best be exam ined by congressional com m ittees with subpoena

power and the ability to create new regulatory laws if needed and desirable. Or the g overnm ent could create a new federal

agency o r Board or add powers or som e supplem ental standards to an existing federal agency, leaving the regulation of social m edia in part to a f ederal executive agency that is

com m itted to bringing its technical expertise and knowledge of an y areas of specialized federal concerns such as international terrorism and threats to dem ocracy to b e a r o n this issue. A specialized federal agency could call witnesses for testim ony, assist m eaningfully in a congressional task to

prepare appropriate legislative g uidance or p ro h ibitions, have investigators to look into areas of concern, establish regulatory standards, and possibly als o i nclude an arm to enforce the law and its standards. See, e.g. , Myers v. United States

, 272 U.S. 52, 129 (1926) (recognizing congress i o n al authority to create federal agencies and define their scope and jurisdiction).G O N Z A L E Z V

. G O O G L E

126

V I

These cases, and others like them pending in the federal courts, try f

or basic justice, but there is a fundam ental question whether the federal courts are best suited to deliver it. I conclude with the following thoughts.

First, it would b e preferable if the poli t i ca l branches o f

governm ent, the legislature or the E xec u tive Branch, would seriously grapple with the issue of unregulated social m edia

power being u s ed to am

plify or to distort views asserted by

user s, and som etim es even by hostile nations using social

m edia to wage asym m etric warfare or to imp a ir dem ocracy. But if Congress continues to sleep at the switch of social

m edia regulation in the face of court s broadening what

appears to h ave been its initial and literal language and expressed intention under Section 230, then it m u st fall to the federal courts to consider rectifying those errors itself by providing rem edies to t h ose who are injured by dangerous and unreasonable conduct.

Second, it wo u l d be p referable if the social m edia

com panies m onitored their o wn activities sufficiently to protect the p ublic, but in m y view, to date they have not done that. It w as o ne thing, at the dawn of the Internet era, to give

p ro t e c tion to Internet com panies to facilitate growth. But i t is q u ite another thing to provide broad im m unity at a tim e such as now when such com panies are rem arkably large and with m assive staffs and perh a ps the best technical abilities.
It is not realistic to anticipate that social m edia com panies will self-police adequately in th e fa c e of their incentives to

m axim ize p rofits by m axim izing advertising revenues, which

m eans increasing the eyeballs d irected to their websites. The large corporations controlling the platform s at issue in theseG O N Z A L E Z V

. G O O G L E 127

appeals can instead be ex pected to act in their own best financial interest, and to m e, it m akes absolutely n o sense to leave such decisions to the self-interested proclam ations o f CEOs or othe r e m

ployees of the various social m edia

com panies. 1 5 Society for centuries has known that it is folly to ask the fox to guard the henhouse.

Third, the problem with a lack of social m edia regulation

goes even b e yond the dreadfully im portant subject o f terrorism

. Indeed, in connection with 21st-centur y p o l itical ele c t ions, som e com m enters have expressed concerns that social m edia has the ability to distort and tribalize public opinion, t o s p re ad falsehoods as well as truth, and to funnel like-m inded news reports to groups in a way that m akes them think there ar e “alternative f acts” or “com

peting realities” that exist, rather than recognize m ore correctly that there are “truth” and “lies.” 1 6

Fourth, to the extent a n y o f our Ninth Circuit precedent stands in the way of a sensible resolution of claim s like those presented o n a ppeal here, where terrorist organizations like ISIS have obviously played Google a n d Yo u Tube like a fiddle, then in m y view we should take these o r other related

cases en banc to give a full review.

15 E.g. , 1 A D A M S M ITH , A N I N Q U IRY IN T O TH E N A TU RE A N D C A U S ES

O F TH E W EA LTH O F N A T IO N S 13 (1776) (“It is n o t from the benevolence of the butcher, the brewer, or the baker that we expect our dinner, but from their regard to their own interest.”).

1 6 See Ross Douthat, Why Do So Many A mericans Thin k

t h e E l e c t i o n W

a s S t o l e n ?

, N

. Y

. T I M E S ( D e c

. 5

, 2 0 2 0 )

, h t t p s : / / w w w

. n y t i me s

. c o m/ 2 0 2 0 / 1 2 / 0 5 / o p i n i o n / s u n d a y / t r u m p

- e l e c t i o n

fraud.htm l? sm id=tw-share.G O N Z A L E Z V

. G O O G L E

128

Fifth, because the issues are d ifficult a n d only the Suprem e Court can speak with authority ultim ately on federal la w, i t would b e desirable for the Suprem e Court to take up the subject o f Section 230 im m unity and perhaps any related First Am endm ent issues, to the extent claim s rela t i n g t o terrorist spe e ch are properly considered under that fram ework. Justice Oliver W endell Holm es, Jr. m ade fam ous and enshrined in our la w th e i dea that: “The life of the law has not b e e n l ogic, it has been experience.” O L I V E R W E N D E L L H O L M E S
, J R ., T H E C O M M O N L A W , Lect ur e I (1881). But when alm ost all c l aim s against social m edia

com panies are di s mi s sed at the outset because of an overbroad view of Section 230 im m unity, how is society to develop the experience that can guide its developm ent o f law in a sensible way that protects people from undue harm ? Justice Holm es also d eveloped the idea that speech should not be constrained ab sen t “clear and present danger,” see Schenck

v. United States

, 249 U. S. 47 (1919). To som e degree this

test still resounds in o u r Fi r s t Am endm ent law. See United States v. Alvarez

, 6 17 F.3d 1198, 1214 (9th Cir. 2010). A variation on this view c ulm inated in Brandenburg v. Ohio

, 395 U.S. 4 44 (1969), where the S uprem e C ourt suggested that im m inent lawless action was necessary before speech should be constrained. But perhaps given the current state of society, and the catastrophic dangers to t h e p u b lic that can be p osed b y terrorist activities, public safety m ay require that speec h be lim ited wh en i t p oses a clear and increasing or gathering d an g er, rather than only “im m inent” danger as reflected in Brandenburg

, whic h I consider the Suprem e Court’s last word on this subject.G O N Z A L E Z V

. G O O G L E 129

I also note that Oliver W endell Holm es, Jr.’s fam ous p en pal and intellectual collaborator, Sir Frederick Pollock, 1 7 in h is beginning prim er of the law of torts, suggested t h a t a principal fo r c e underlying all the varied types of tort cases

was the desire of courts to provide a doctrinal b a s is for

rem edy in the case of injuries f rom harm ful and unreasonable conduct. Pollock suggested that a “tort is an act or om ission (not m erely the breach of a duty arising o u t of a personal relation, or undertaken by contract) which is related to harm suffered by a determ ina t e p erson in one of the following ways.” See S I R F R

E D E R I C K P O L L O C K , T H E L A W O F T O R T S : A
T R E A T I S E O

N T H E P R

I N C I P L E S O F O B L I G

A T I O N S A R

I S I N G F R O M C I V I L W R O N G S I

N T H E C O M M O N L A W 20 (4th ed. 1895)

. Am ong those ways a person can be h ar me d were these two, which are pertinent in assessing whether Plaintiffs’ claim s

can be asserted as part of a federal com m on law: “(c) it m ay be an act o r om ission causing harm w hich the person so acting or so om itting did not intend to cau s e , b u t m ight and should with d

ue diligence have foreseen and prevented,” and “(d) it m ay in special cases consist i n n o t a v oiding o r preventing that w hich the party wa s bound, absolutely o r within l i mi t s t o avoid or prevent.” Id. Here, it could be expected that through federal com m o n law developm ent o r statutory p o si t ive law, the social m edia com panies will be held to som e reasonable standard of conduct when they have

17 See Oliver W endell Holm es Jr. & Sir Frederi ck Po l lock, Holm es–Pollock Letters: The Correspondence of Mr Justice Holm es and Sir Frederick Pollock, 1874–1932 (2d ed. 1961).G O N Z A L E Z V

. G O O G L E

130

failed to regulate their o wn acti ons in the interests of the public. 1 8

As a ma t ter of federal com m o n law, I would hold that when social m edia com

panies in their platform s use system s

o r procedures that are unreasonably dangerous t o t h e public—as in the case where their system s l i n e up repeated

m essages in aid of terrorists like ISIS—or when they om it to
act to avoid harm w hen om itting the ac t i s u nreasonably d an g erous to the public—as in the case where they f ail to review and self-regulate their websites adequately to notice and r e m o ve propaganda videos f rom ISIS that are likely to

cause harm

—then there should b e a federal commo n law claim available against them

. Consider the m ost widely used standard for products liability cases. See Restatement (S e c o n d ) o f Torts

, § 402A (1965). This suggests that

m anufacturers are responsible in tort if they m ake unreasonably da ngerous products that cause individual o r

s oc i al harm

. Section 402A states: “One who sells any product in a defe c t i ve condition unreasonably dangerous to the user or consum er or to his property is subject to liability for physical harm thereby caused” to the user o r a third party. Id. Here and sim ilarly, s o c i a l m edia com panies should be viewed as m aking and “selling” their social m edia p roducts through t h e device of forced advertising under the eyes o f

users. Viewed in this light, they should be tested u n der a federal tort principle with a standard sim ilar to and adapted from this Restatement language under a federal com m on law

18 D eveloping federal com m on law on these issues will require the diligent and com

bined efforts of the federal courts and of legal scholars. See, e.g. , Hon. W ade H. McCree, Jr., The Annual John Randolph Tucker Lecture, Partners in a Process: The Academy and the Courts

, 37 W A S H

. & L EE

. L. R EV . 1041 (1981).G O N Z A L E Z V

. G O O G L E 131

developm ent. If social m edia com panies use “neu tral” algorithm s that cause unreasonably dangerous consequences, under proper standards o f law with lim iting jury instructions, they m ight b e held responsible. D eveloping a fed eral

co m m o n law standard would be superior to m erely d ism issin g all claim s against social m edia com p anies based on an overbroad i nterpretation o f Section 230 delivering a b lanket i mmu n i ty, which in m y view is inconsistent with congressional i n t e n t and detrim ental to the interests of the general public.G O N Z A L E Z V

. G O O G L E

132

A TTACHMENT A1 K ATZMANN , Chief Judge

, concurring in part and dissenting in part:ȱȱ

I agree with much of the reasoning in the excellent majority opinion, and I

join that opinion except for Parts I and II of the Discussion. But I must respectfully

part company with the majority on its treatment of Facebook’s friend Ȭȱ and content

Ȭ

suggestion algorithms under the Communications Decency Act (“CDA”). 1ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ

1 I agree with the majority that the CDA’s exception for enforcement of criminal laws, 47 U.S.C. § 230(e)(1), does not apply to plaintiffs’ claims, see ante
, at 50

Ȭ 54. However,
I find the question to be somewhat closer than the majority does, in part because some of
the statutes enumerated in § 230(e)(1) themselves contain civil remedies. Section 230(e)(1)

states that “[n]othing in [§ 230] shall be construed to impair the enforcement of section
223 or 231 of this title, chapter 71 (relating to obscenity) or 110 (relating to sexual exploitation of children) of title 18, or any other Federal criminal statute.” One of those
enumerated chapters—Chapter 110 of Title 18—includes a civil suit provision for victims
of specific child sex crimes. See 18 U.S.C. § 2255. Meanwhile, 47 U.S.C. § 223—which prohibits obscene or harassing phone calls—specifies that civil fines may be levied “pursuant to civil action by,” or “after appropriate administrative proceedings” of, the
Federal Communications Commission (“FCC”), and it authorizes the Attorney General

to bring civil suits to enjoin practices that violate the statute. 47 U.S.C. § 223(b)(5)(B)

Ȭ (b)(6). If § 230(e)(1) covers “enforcement” of the listed chapters in their entirety, it is
difficult to see how it would not cover other provisions that authorize civil suits for
violations of criminal laws, particularly given that the enumerated list is followed by “or

any other criminal law.”ȱȱ However, as detailed post
, § 230 was designed as a private

Ȭ sector

Ȭ driven
alternative to a Senate plan that would allow the FCC “either civilly or criminally, to punish people” who put objectionable material on the Internet. 141 Cong. Rec. 22,045 (1995) (statement of Rep. Cox); accord id. at 22,045

Ȭ 46 (statement of Rep. Wyden); see Reno

v. ACLU

, 521 U.S. 844, 859 & n.24 (1997). On the House floor, author Christopher Cox disparaged the idea of FCC enforcement and then stated: “Certainly, criminal
enforcement of our obscenity laws as an adjunct is a useful way of punishing the truly2 As to the reasons for my disagreement, consider a hypothetical. Suppose

that you are a published author. One day, an acquaintance calls. “I’ve been reading

over everything you’ve ever published,” he informs you. “I’ve also been looking

at everything you’ve ever said on the Internet. I’ve done the same for this other

author. You two have very similar interests; I think you’d get along.” The

acquaintance then gives you the other author’s contact information and photo,

along with a link to all her published works. He calls back three more times over

the next week with more names of writers you should get to know.ȱȱ

Now, you might say your acquaintance fancies himself a matchmaker. But

would you say he’s acting as the publisher of the other authors’ work?

Facebook and the majority would have us answer this question “yes.” I,

however, cannot do so. For the scenario I have just described is little different from

how Facebook’s algorithms allegedly work. And while those algorithms do end

up showing users profile, group, or event pages written by other users, it strainsȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ

guilty.” 141 Cong. Rec. 22,045 (emphasis added). This history, along with the provision’s title, strongly suggests that § 230(e)(1) was intended as a narrow criminal

Ȭ law exception.
It would be odd, then, to read § 230(e)(1) as allowing for civil enforcement by, among others, the FCC, even if only in aid of criminal law enforcement.3 the English language to say that in targeting and recommending these writings to

users—and thereby forging connections, developing new social networks—

Facebook is acting as “the publisher of

. . . information provided by another

information content provider.” 47 U.S.C. § 230(c)(1) (emphasis added).ȱȱ

It would be one thing if congressional intent compelled us to adopt the

majority’s reading. It does not. Instead, we today extend a provision that was

designed to encourage computer service providers to shield minors from obscene

material so that it now immunizes those same providers for allegedly connecting

terrorists to one another. Neither the impetus for nor the text of § 230(c)(1) requires

such a result. When a plaintiff brings a claim that is based not on the content of the

information shown but rather on the connections Facebook’s algorithms make

between individuals, the CDA does not and should not bar relief.ȱȱ

The Anti

Ȭ Terrorism Act (“ATA”) claims in this case fit this bill. According

to plaintiffs’ Proposed Second Amended Complaint (“PSAC”)—which we must

take as true at this early stage—Facebook has developed “sophisticated

algorithm[s]” for bringing its users together. App’x 347 ¶ 622. After collecting

mountains of data about each user’s activity on and off its platform, Facebook4 unleashes its algorithms to generate friend, group, and event suggestions based

on what it perceives to be the user’s interests. Id. at 345

Ȭ 46 ¶¶ 608

Ȭ 14. If a user posts

about a Hamas attack or searches for information about a Hamas leader, Facebook

may “suggest” that that user become friends with Hamas terrorists on Facebook

or join Hamas

Ȭ related Facebook groups. By “facilitat[ing] [Hamas’s] ability to

reach and engage an audience it could not otherwise reach as effectively,”

plaintiffs allege that Facebook’s algorithms provide material support and

personnel to terrorists. Id. at 347 ¶ 622; see id. at 352

Ȭ 58 ¶¶ 646

Ȭ 77. As applied to the

algorithms, plaintiffs’ claims do not seek to punish Facebook for the content others

post, for deciding whether to publish third parties’ content, or for editing (or

failing to edit) others’ content before publishing it. In short, they do not rely on

treating Facebook as “the publisher” of others’ information. Instead, they would

hold Facebook liable for its affirmative role in bringing terrorists together.

When it comes to Facebook’s algorithms, then, plaintiffs’ causes of action do

not run afoul of the CDA. Because the court below did not pass on the merits of

the ATA claims pressed below, I would send this case back to the district court to

decide the merits in the first instance. The majority, however, cuts off all possibility5 for relief based on algorithms like Facebook’s, even if these or future plaintiffs

could prove a sufficient nexus between those algorithms and their injuries. In light

of today’s decision and other judicial interpretations of the statute that have

generally immunized social media companies—and especially in light of the new

reality that has evolved since the CDA’s passage—Congress may wish to revisit

the CDA to better calibrate the circumstances where such immunization is

appropriate and inappropriate in light of congressional purposes.

**I.**

To see how far we have strayed from the path on which Congress set us out,

we must consider where that path began. What is now 47 U.S.C. § 230 was added

as an amendment to the Telecommunications Act of 1996, a statute designed to

deregulate and encourage innovation in the telecommunications industry. Pub. L.

104

Ȭ 104, § 509, 110 Stat. 56, 56, 137

Ȭ 39; see Reno

, 521 U.S. at 857. Congress devoted

much committee attention to traditional telephone and broadcast media; by

contrast, the Internet was an afterthought, addressed only through floor

amendments or in conference. Reno

, 521 U.S. at 857

Ȭ 58. Of the myriad issues the

emerging Internet implicated, Congress tackled only one: the ease with which the6 Internet delivers indecent or offensive material, especially to minors. See

Telecommunications Act of 1996, tit. V, subtit. A, 110 Stat. at 133

Ȭ 39. And § 230

provided one of two alternative ways of handling this problem.

The action began in the Senate. Senator James J. Exon introduced the CDA

on February 1, 1995. See 141 Cong. Rec. 3,203. He presented a revised bill on June

9, 1995, “[t]he heart and the soul” of which was “its protection for families and

children.” Id. at 15,503 (statement of Sen. Exon). The Exon Amendment sought to

reduce the proliferation of pornography and other obscene material online by

subjecting to civil and criminal penalties those who use interactive computer

services to make, solicit, or transmit offensive material. Id. at 15,505.

The House of Representatives had the same goal—to protect children from

inappropriate online material—but a very different sense of how to achieve it.

Congressmen Christopher Cox (R

Ȭ California) and Ron Wyden (D

Ȭ Oregon)

introduced an amendment to the Telecommunications Act, entitled “Online

Family Empowerment,” about two months after the revised CDA appeared in the

Senate. See id. at 22,044. Making the argument for their amendment during the

House floor debate, Congressman Cox stated:7 We want to make sure that everyone in America has an open invitation and feels welcome to participate in the Internet. But as you know, there is some reason for people to be wary because, as a Time Magazine cover story recently highlighted, there is in this vast world
of computer information, a literal computer library, some offensive material, some things in the bookstore, if you will, that our children ought not to see.

As the parent of two, I want to make sure that my children have access
to this future and that I do not have to worry about what they might
be running into on line. I would like to keep that out of my house and
off my computer.

Id. at 22,044

Ȭ 45. Likewise, Congressman Wyden said: “We are all against smut and

pornography, and, as the parents of two small computer

Ȭ literate children, my wife

and I have seen our kids find their way into these chat rooms that make their

middle

Ȭ aged parents cringe.” Id. at 22,045.ȱȱ

As both sponsors noted, the debate between the House and the Senate was

not over the CDA’s primary purpose but rather over the best means to that shared

end. See id. (statement of Rep. Cox) (“How should we do this?

. . . Mr. Chairman,

what we want are results. We want to make sure we do something that actually

works.”); id. (statement of Rep. Wyden) (“So let us all stipulate right at the outset

the importance of protecting our kids and going to the issue of the best way to do

it.”). While the Exon Amendment would have the FCC regulate online obscene8 materials, the sponsors of the House proposal “believe[d] that parents and families

are better suited to guard the portals of cyberspace and protect our children than

our Government bureaucrats.” Id. at 22,045 (statement of Rep. Wyden). They also

feared the effects the Senate’s approach might have on the Internet itself. See id.

(statement of Rep. Cox) (“[The amendment] will establish as the policy of the

United States that we do not wish to have content regulation by the Federal

Government of what is on the Internet, that we do not wish to have a Federal

Computer Commission with an army of bureaucrats regulating the Internet

. . . .”).

The Cox

Ȭ Wyden Amendment therefore sought to empower interactive computer

service providers to self

Ȭ regulate, and to provide tools for parents to regulate,

children’s access to inappropriate material. See S. Rep. No. 104

Ȭ 230, at 194 (1996)

(Conf. Rep.); 141 Cong. Rec. 22,045 (statement of Rep. Cox).

There was only one problem with this approach, as the House sponsors saw

it. A New York State trial court had recently ruled that the online service Prodigy,

by deciding to remove certain indecent material from its site, had become a

“publisher” and thus was liable for defamation when it failed to remove other

objectionable content. Stratton

Ȭ Oakmont, Inc. v. Prodigy Servs. Co.

, 1995 WL 323710,9 at *4 (N.Y. Sup. Ct. May 24, 1995) (unpublished). The authors of § 230 saw the

Stratton

Ȭ Oakmont decision as indicative of a “legal system [that] provides a massive

disincentive for the people who might best help us control the Internet to do so.”

141 Cong. Rec. 22,045 (statement of Rep. Cox). Cox

Ȭ Wyden was designed, in large

part, to remove that disincentive. See S. Rep. No. 104

Ȭ 230, at 194.

The House having passed the Cox

Ȭ Wyden Amendment and the Senate the

Exon Amendment, the conference committee had before it two alternative visions

for countering the spread of indecent online material to minors. The committee

chose not to choose. Congress instead adopted both amendments as part of a final

Communications Decency Act. See Telecommunications Act of 1996, §§ 502, 509,

110 Stat. at 133

Ȭ 39; Reno

, 521 U.S. at 858 n.24. 2 The Supreme Court promptly struck

down two major provisions of the Exon Amendment as unconstitutionallyȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ

2 It helped that the Cox

Ȭ Wyden Amendment exempted from its deregulatory regime the very provisions that the Exon Amendment strengthened, see Telecommunications Act of 1996, §§ 502, 507

Ȭ 508, 509(d)(1), 110 Stat. at 133

Ȭ 39, and that Congress stripped from the House bill a provision that would have denied jurisdiction

to the FCC to regulate the Internet, compare id. § 509, 110 Stat. at 138 (eliminating original § 509(d)), with 141 Cong. Rec. 22,044 (including original § 509(d)).10 overbroad under the First Amendment, leaving the new § 230 as the dominant

force for securing decency on the Internet. See Reno

, 521 U.S. at 849.

Section 230 overruled Stratton

Ȭ Oakmont through two interlocking

provisions, both of which survived the legislative process unscathed. The first,

which is at issue in this case, states that “[n]o provider or user of an interactive

computer service shall be treated as the publisher or speaker of any information

provided by another information content provider.” 47 U.S.C. § 230(c)(1). The

second provision eliminates liability for interactive computer service providers

and users for “any action voluntarily taken in good faith to restrict access to or

availability of material that the provider or user considers to be

. . . objectionable,”

or “any action taken to enable or make available to

. . . others the technical means

to restrict access to [objectionable] material.” Id. § 230(c)(2). These two subsections

tackle, in overlapping fashion, the two jurisprudential moves of the Stratton

Ȭ

Oakmont court: first, that Prodigy’s decision to screen posts for offensiveness

rendered it “a publisher rather than a distributor,” 1995 WL 323710, at *4; and

second, that by making good

Ȭ faith efforts to remove offensive material Prodigy

became liable for any actionable material it did not remove.11 The legislative history illustrates that in passing § 230 Congress was focused

squarely on protecting minors from offensive online material, and that it sought

to do so by “empowering parents to determine the content of communications

their children receive through interactive computer services.” S. Rep. No. 104

Ȭ 230,

at 194. The “policy” section of § 230’s text reflects this goal. See 47 U.S.C.

§ 230(b)(3)

Ȭ (4). 3 It is not surprising, then, that Congress emphasized the narrow

civil liability shield that became § 230(c)(2), rather than the broad rule of

construction laid out in § 230(c)(1). Indeed, the conference committee summarized

§ 230 by stating that it “provides ‘Good Samaritan’ protections from civil liability

for providers or users of an interactive computer service for actions to restrict orȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ

3 The policy section of the statute also expresses Congress’s desire “to preserve the
vibrant and competitive free market that presently exists for the Internet and other
interactive computer services, unfettered by Federal or State regulation.” 47 U.S.C. § 230(b)(2). It is therefore true that “Section 230 was enacted, in part
, to maintain the
robust nature of Internet communication.” Ricci v. Teamsters Union Local 456

, 781 F.3d 25,
28 (emphasis added) (quoting Zeran v. Am. Online, Inc.

, 129 F.3d 327, 330 (4th Cir. 1997));

see ante
, at 24. As the legislative history laid out in this opinion shows, however, one
cannot fully understand the purpose of § 230 without considering that it was one chamber’s proposal in a disagreement between the two houses of Congress over how best

to shield children from indecent material, and that in that contest the House was principally concerned with two things: (1) overruling Stratton

Ȭ Oakmont and (2) preventing “a Federal Computer Commission with an army of bureaucrats regulating
the Internet.” 141 Cong. Rec. 22,045 (statement of Rep. Cox).12 to enable restriction of access to objectionable online material”—a description that

could just as easily have applied to § 230(c)(2) alone. S. Rep. No. 104

Ȭ 230, at 194.

Congress also titled the entirety of § 230(c) “Protection for ‘Good Samaritan’

blocking and screening of offensive material,” suggesting that the definitional rule

outlined in § 230(c)(1) may have been envisioned as supporting or working in

tandem with the civil liability shield in § 230(c)(2).ȱȱ

None of this is to say that § 230(c)(1) exempts interactive computer service

providers from publisher treatment only when they remove indecent content.

Statutory text cannot be ignored, and Congress grabbed a bazooka to swat the

Stratton

Ȭ Oakmont fly. Whatever prototypical situation its drafters may have had in

mind, § 230(c)(1) does not limit its protection to situations involving “obscene

material” provided by others, instead using the expansive word “information.” 4ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ

4 This point—that Congress chose broader language than may have been necessary
to accomplish its primary goal—should not be confused with the Seventh Circuit’s
rationale for § 230(c)(1)’s general application: that “a law’s scope often differs from its genesis.” See Chi. Lawyers Cmte. for Civil Rights Under Law, Inc. v. Craigslist, Inc.

, 519 F.3d 666, 671 (7th Cir. 2008). True as this axiom might be, it does not apply here—the language
of § 230(c)(1) remained untouched from introduction to passage. Nor is there any evidence from the legislative record that interest groups altered the statutory language.
But cf. id. (“Once the legislative process gets rolling, interest groups seek (and often obtain) other provisions.”). That § 230(c)(1)’s breadth flowed from Congress’s desire to13 Illuminating Congress’s original intent does, however, underscore the extent of

§ 230(c)(1)’s subsequent mission creep. Given how far both Facebook’s suggestion

algorithms and plaintiffs’ terrorism claims swim from the shore of congressional

purpose, caution is warranted before courts extend the CDA’s reach any further.

**II.**

With the CDA’s background in mind, I turn to the text. By its plain terms,

§ 230 does not apply whenever a claim would treat the defendant as “a publisher”

in the abstract, immunizing defendants from liability stemming from any activity

in which one thinks publishing companies commonly engage. Contra ante
, at 30

Ȭ

31, 33

Ȭ 34, 49. It states, more specifically, that “[n]o provider or user of an interactive

computer service shall be treated as the publisher or speaker of any information

provided by another information content provider.” 47 U.S.C. § 230(c)(1) (emphases

added). “Here grammar and usage establish that ‘the’ is a function word

indicating that a following noun or noun equivalent is definite

. . . .” Nielsen v.

Preap

, 139 S. Ct. 954, 965 (2019) (citation and internal quotation marks omitted).ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ

overrule Stratton

Ȭ Oakmont

, rather than from mere interest group protectionism, matters.14 The word “publisher” in this statute is thus inextricably linked to the “information

provided by another.” The question is whether a plaintiff’s claim arises from a

third party’s information, and—crucially—whether to establish the claim the court

must necessarily view the defendant, not as a publisher in the abstract, but rather

as the publisher of that third

Ȭ party information. See FTC v. LeadClick Media, LLC

,

838 F.3d 158, 175 (2d Cir. 2016) (stating inquiry as “whether the cause of action

inherently requires the court to treat the defendant as the ‘publisher or speaker’ of

content provided by another”).

For this reason, § 230(c)(1) does not necessarily immunize defendants from

claims based on promoting content or selling advertising, even if those activities

might be common among publishing companies nowadays. A publisher might

write an email promoting a third

Ȭ party event to its readers, for example, but the

publisher would be the author of the underlying content and therefore not

immune from suit based on that promotion. See 47 U.S.C. § 230(c)(1), (f)(3).

Similarly, the fact that publishers may sell advertising based on user data does not

immunize the publisher if someone brings a claim based on the publisher’s selling

of the data, because the claim would not treat the defendant as the publisher of a15 third party’s content. Cf. Oberdorf v. Amazon.com Inc.

, No. 18

Ȭ 1041, 2019 WL

2849153, at *12 (3d Cir. July 3, 2019) (holding that the CDA does not bar claims

against Amazon.com “to the extent that” they “rely on Amazon’s role as an actor

in the sales process,” including both “selling” and “marketing”). Section 230(c)(1)

limits liability based on the function the defendant performs, not its identity.

Accordingly, our precedent does not grant publishers CDA immunity for

the full range of activities in which they might engage. Rather, it “bars lawsuits

seeking to hold a service provider liable for its exercise of a publisher’s traditional

editorial functions—such as deciding whether to publish, withdraw, postpone or

alter content” provided by another for publication. LeadClick

, 838 F.3d at 174

(citation and internal quotation marks omitted); accord Oberdorf

, 2019 WL 2849153,

at *10; Jane Doe No. 1 v. Backpage.com, LLC

, 817 F.3d 12, 19 (1st Cir. 2016); Jones v.

Dirty World Entm’t Recordings LLC

, 755 F.3d 398, 407 (6th Cir. 2014); Barnes v.

Yahoo!, Inc.

, 570 F.3d 1096, 1102 (9th Cir. 2009); Zeran

, 129 F.3d at 330; see Klayman

v. Zuckerberg

, 753 F.3d 1354, 1359 (D.C. Cir. 2014); Ben Ezra, Weinstein, & Co., Inc. v.

Am. Online Inc.

, 206 F.3d 980, 986 (10th Cir. 2000). For instance, a claim against a

newspaper based on the content of a classified ad (or the decision to publish or16 withdraw that ad) would fail under the CDA not because newspapers traditionally

publish classified ads, but rather because such a claim would necessarily treat the

newspaper as the publisher of the ad

Ȭ maker’s content. Similarly, the newspaper

does not act as an “information content provider”—and thus maintains its CDA

protection—when it decides to run a classified ad because it neither “creates” nor

“develops” the information in the ad. 47 U.S.C. § 230(f)(3).ȱȱ

This case is different. Looking beyond Facebook’s “broad statements of

immunity” and relying “rather on a careful exegesis of the statutory language,”

Barnes

, 570 F.3d at 1100, the CDA does not protect Facebook’s friend Ȭȱ and content

Ȭ

suggestion algorithms. A combination of two factors, in my view, confirms that

claims based on these algorithms do not inherently treat Facebook as the publisher

of third

Ȭ party content. 5 First, Facebook uses the algorithms to create and

communicate its own message: that it thinks you, the reader—you, specifically—

will like this content. And second, Facebook’s suggestions contribute to theȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ

5 Many of Facebook’s algorithms mentioned in the PSAC, such as its third

Ȭ party advertising algorithm, its algorithm that places content in a user’s newsfeed, and (based

on the limited description in the PSAC) its video recommendation algorithm, remain
immune under the analysis I set out here.17 creation of real

Ȭ world social networks. The result of at least some suggestions is

not just that the user consumes a third party’s content. Sometimes, Facebook’s

suggestions allegedly lead the user to become part of a unique global community,

the creation and maintenance of which goes far beyond and differs in kind from

traditional editorial functions.

It is true, as the majority notes, see ante
, at 47, that Facebook’s algorithms

rely on and display users’ content. However, this is not enough to trigger the

protections of § 230(c)(1). The CDA does not mandate “a ‘but

Ȭ for’ test that would

provide immunity

. . . solely because a cause of action would not otherwise have

accrued but for the third

Ȭ party content.” HomeAway.com, Inc. v. City of Santa

Monica

, 918 F.3d 676, 682 (9th Cir. 2019). Rather, to fall within § 230(c)(1)’s radius,

the claim at issue must inherently fault the defendant’s activity as the publisher of

specific third

Ȭ party content. Plaintiffs’ claims about Facebook’s suggestion

algorithms do not do this. The complaint alleges that “Facebook collects detailed

information about its users, including, inter alia, the content they post, type of

content they view or engage with, people they communicate with, groups they

belong to and how they interact with such groups, visits to third party websites,18 apps and Facebook partners.” App’x 345 ¶ 608. Then the algorithms “utilize the

collected data to suggest friends, groups, products, services and local events, and

target ads” based on each user’s input. Id. at 346 ¶ 610.ȱȱ

If a third party got access to Facebook users’ data, analyzed it using a

proprietary algorithm, and sent its own messages to Facebook users suggesting

that people become friends or attend one another’s events, the third party would

not be protected as “the publisher” of the users’ information. Similarly, if Facebook

were to use the algorithms to target its own material to particular users, such that

the resulting posts consisted of “information provided by” Facebook rather than

by “another information content provider,” § 230(c)(1), Facebook clearly would

not be immune for that independent message.

Yet that is ultimately what plaintiffs allege Facebook is doing. The PSAC

alleges that Facebook “actively provides ‘friend suggestions’ between users who

have expressed similar interests,” and that it “actively suggests groups and events

to users.” App’x 346 ¶¶ 612

Ȭ 13. Facebook’s algorithms thus allegedly provide the

user with a message from Facebook. Facebook is telling users—perhaps implicitly,

but clearly—that they would like these people, groups, or events. In this respect,19 Facebook “does not merely provide a framework that could be utilized for proper

or improper purposes; rather, [Facebook’s] work in developing” the algorithm and

suggesting connections to users based on their prior activity on Facebook,

including their shared interest in terrorism, “is directly related to the alleged

illegality of the site.” Fair Housing Council of San Fernando Valley v. Roommates.Com,

LLC

, 521 F.3d 1157, 1171 (9th Cir. 2008) (en banc). The fact that Facebook also

publishes third

Ȭ party content should not cause us to conflate its two separate roles

with respect to its users and their information. Facebook may be immune under

the CDA from plaintiffs’ challenge to its allowance of Hamas accounts, since

Facebook acts solely as the publisher of the Hamas users’ content. That does not

mean, though, that it is also immune when it conducts statistical analyses of that

information and delivers a message based on those analyses.

Moreover, in part through its use of friend, group, and event suggestions,

Facebook is doing more than just publishing content: it is proactively creating

networks of people. Its algorithms forge real

Ȭ world (if digital) connections through

friend and group suggestions, and they attempt to create similar connections in

the physical world through event suggestions. The cumulative effect of20 recommending several friends, or several groups or events, has an impact greater

than the sum of each suggestion. It envelops the user, immersing her in an entire

universe filled with people, ideas, and events she may never have discovered on

her own. According to the allegations in the complaint, Facebook designed its

website for this very purpose. “Facebook has described itself as a provider of

products and services that enable users

. . . to find and connect with other users

. . . .” App’x 250 ¶ 129. CEO Mark Zuckerberg has similarly described Facebook as

“build[ing] tools to help people connect with the people they want,” thereby

“extending people’s capacity to build and maintain relationships.” Id. at 251 ¶ 132.

Of course, Facebook is not the only company that tries to bring people together

this way, and perhaps other publishers try to introduce their readers to one

another. Yet the creation of social networks goes far beyond the traditional

editorial functions that the CDA immunizes.

Another way to consider the CDA immunity question is to “look

. . . to what

the duty at issue actually requires: specifically, whether the duty would

necessarily require an internet company to monitor[, alter, or remove] third

Ȭ party

content.” HomeAway.com

, 918 F.3d at 682. Here, too, the claims regarding the21 algorithms are a poor fit for statutory immunity. The duty not to provide material

support to terrorism, as applied to Facebook’s use of the algorithms, simply

requires that Facebook not actively use that material to determine which of its

users to connect to each other. It could stop using the algorithms altogether, for

instance. Or, short of that, Facebook could modify its algorithms to stop them

introducing terrorists to one another. None of this would change any underlying

content, nor would it necessarily require courts to assess further the difficult

question of whether there is an affirmative obligation to monitor that content.

In reaching this conclusion, I note that ATA torts are atypical. Most of the

common torts that might be pleaded in relation to Facebook’s algorithms “derive

liability from behavior that is identical to publishing or speaking”—for instance,

“publishing defamatory material; publishing material that inflicts emotional

distress; or

. . . attempting to de

Ȭ publish hurtful material but doing it badly.”

Barnes

, 570 F.3d at 1107. The fact that Facebook has figured out how to target

material to people more likely to read it does not matter to a defamation claim, for

instance, because the mere act of publishing in the first place creates liability.22 The ATA works differently. Plaintiffs’ material support and aiding and

abetting claims premise liability, not on publishing qua publishing, but rather on

Facebook’s provision of services and personnel to Hamas. It happens that the way

in which Facebook provides these benefits includes republishing content, but

Facebook’s duties under the ATA arise separately from the republication of

content. Cf. id. (determining that liability on a promissory estoppel theory for

promising to remove content “would come not from Yahoo’s publishing conduct,

but from Yahoo’s manifest intention to be legally obligated to do something, which

happens to be removal of material from publication”). For instance, the operation

of the algorithms is allegedly provision of “expert advice or assistance,” and the

message implied by Facebook’s prodding is allegedly a “service” or an attempt to

provide “personnel.” 18 U.S.C. § 2339A(b).

For these reasons, § 230(c)(1) does not bar plaintiffs’ claims.

**III.**

Even if we sent this case back to the district court, as I believe to be the right

course, these plaintiffs might have proven unable to allege that Facebook’s

matchmaking algorithms played a role in the attacks that harmed them. However,23 assuming arguendo that such might have been the situation here, I do not think we

should foreclose the possibility of relief in future cases if victims can plausibly

allege that a website knowingly brought terrorists together and that an attack

occurred as a direct result of the site’s actions. Though the majority shuts the door

on such claims, today’s decision also illustrates the extensive immunity that the

current formulation of the CDA already extends to social media companies for

activities that were undreamt of in 1996. It therefore may be time for Congress to

reconsider the scope of § 230.

As is so often the case with new technologies, the very qualities that drive

social media’s success—its ease of use, open access, and ability to connect the

world—have also spawned its demons. Plaintiffs’ complaint illustrates how

pervasive and blatant a presence Hamas and its leaders have maintained on

Facebook. Hamas is far from alone—Hezbollah, Boko Haram, the Revolutionary

Armed Forces of Colombia, and many other designated terrorist organizations use

Facebook to recruit and rouse supporters. Vernon Silver & Sarah Frier, Terrorists

Are Still Recruiting on Facebook, Despite Zuckerberg’s Reassurances

, Bloomberg

Businessweek (May 10, 2018), http://www.bloomberg.com/news/articles/2018

Ȭ 05

Ȭ24 10/terrorists

Ȭ creep

Ȭ onto

Ȭ facebook

Ȭ as

Ȭ fast

Ȭ as

Ȭ it

Ȭ can

Ȭ shut

Ȭ them

Ȭ down. Recent news

reports suggest that many social media sites have been slow to remove the

plethora of terrorist and extremist accounts populating their platforms, 6 and that

such efforts, when they occur, are often underinclusive. Twitter, for instance,

banned the Ku Klux Klan in 2018 but allowed David Duke to maintain his account,

see Roose & Conger, supra , while researchers found that Facebook removed fewer

than half the terrorist accounts and posts those researchers identified, see Waters

& Postings, supra , at 8; Desmond Butler & Barbara Ortulay, Facebook Auto

Ȭ Generates

Videos Celebrating Extremist Images

, Assoc. Press (May 9, 2019),

http://apnews.com/f97c24dab4f34bd0b48b36f2988952a4. Those whose accounts

are removed often pop up again under different names or with slightly differentȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ

6 See, e.g. , Gregory Waters & Robert Postings, Spiders of the Caliphate: Mapping the
Islamic State’s Global Support Network on Facebook 8, Counter Extremism Project (May 2018), http://www.counterextremism.com/sites/default/files/Spiders%20of%20the%20 Caliphate%20%28May%202018%29.pdf; Yaacov Benmeleh & Felice Maranz, Israel Warns
Twitter of Legal Action Over Requests to Remove Content

, Bloomberg (Mar. 20, 2018), http://www.bloomberg.com/news/articles/2018

Ȭ 03

Ȭ 20/israel

Ȭ warns

Ȭ twitter

Ȭ of

Ȭ legal

Ȭ steps

Ȭ over

Ȭ incitement

Ȭ to

Ȭ terrorism; Mike Isaac, Twitter Steps Up Efforts to Thwart
Terrorists’ Tweets

, N.Y. Times (Feb. 5, 2016), http://www.nytimes.com/2016/02/06/ technology/twitter

Ȭ account

Ȭ suspensions

Ȭ terrorism.html; Kevin Roose & Kate Conger, YouTube to Remove Thousands of Videos Pushing Extreme Views

, N.Y. Times (June 5, 2019), http://www.nytimes.com/2019/06/05/business/youtube

Ȭ remove

Ȭ extremist

Ȭ videos.html.25 language in their profiles, playing a perverse and deadly game of Whack

Ȭ a

Ȭ Mole

with Silicon Valley. See Isaac, supra ; Silver & Frier, supra

.

Of course, the failure to remove terrorist content, while an important policy

concern, is immunized under § 230 as currently written. Until today, the same

could not have been said for social media’s unsolicited, algorithmic spreading of

terrorism. Shielding internet companies that bring terrorists together using

algorithms could leave dangerous activity unchecked.ȱȱ

Take Facebook.ȱȱ As plaintiffs allege, its friend

Ȭ suggestion algorithm appears

to connect terrorist sympathizers with pinpoint precision. For instance, while two

researchers were studying Islamic State (“IS”) activity on Facebook, one “received

dozens of pro

Ȭ IS accounts as recommended friends after friending just one pro

Ȭ IS

account.” Waters & Postings, supra , at 78. More disturbingly, the other “received

an influx of Philippines

Ȭ based IS supporters and fighters as recommended friends

after liking several non

Ȭ extremist news pages about Marawi and the Philippines

during IS’s capture of the city.” Id. News reports indicate that the friend

Ȭ

suggestion feature has introduced thousands of IS sympathizers to one another.

See Martin Evans, Facebook Accused of Introducing Extremists to One Another Through26 ‘Suggested Friends’ Feature

, The Telegraph (May 5, 2018),

http://www.telegraph.co.uk/news/2018/05/05/facebook

Ȭ accused

Ȭ introducing

Ȭ

extremists

Ȭ one

Ȭ another

Ȭ suggested.

And this is far from the only Facebook algorithm that may steer people

toward terrorism. Another turns users’ declared interests into audience categories

to enable microtargeted advertising. In 2017, acting on a tip, ProPublica sought to

direct an ad at the algorithmically

Ȭ created category “Jew hater”—which turned

out to be real, as were “German Schutzstaffel,” “Nazi Party,” and “Hitler did

nothing wrong.” Julia Angwin et al., Facebook Enabled Advertises to Reach ‘Jew

Haters

, ’ ProPublica (Sept. 14, 2017), https://www.propublica.org/article/facebook

Ȭ

enabled

Ȭ advertisers

Ȭ to

Ȭ reach

Ȭ jew

Ȭ haters. As the “Jew hater” category was too

small for Facebook to run an ad campaign, “Facebook’s automated system

suggested ‘Second Amendment’ as an additional category

. . . presumably because

its system had correlated gun enthusiasts with anti

Ȭ Semites.” Id.ȱȱ

That’s not all. Another Facebook algorithm auto

Ȭ generates business pages

by scraping employment information from users’ profiles; other users can then

“like” these pages, follow their posts, and see who else has liked them. Butler &27 Ortutay, supra

. ProPublica reports that extremist organizations including al

Ȭ Qaida,

al

Ȭ Shabab, and IS have such auto

Ȭ created pages, allowing them to recruit the

pages’ followers. Id. The page for al

Ȭ Qaida in the Arabian Peninsula included the

group’s Wikipedia entry and a propaganda photo of the damaged USS Cole,

which the group had bombed in 2000. Id. Meanwhile, a fourth algorithm integrates

users’ photos and other media to generate videos commemorating their previous

year. Id. Militants get a ready

Ȭ made propaganda clip, complete with a thank

Ȭ you

message from Facebook. Id.

This case, and our CDA analysis, has centered on the use of algorithms to

foment terrorism. Yet the consequences of a CDA

Ȭ driven, hands

Ȭ off approach to

social media extend much further. Social media can be used by foreign

governments to interfere in American elections. For example, Justice Department

prosecutors recently concluded that Russian intelligence agents created false

Facebook groups and accounts in the years leading up to the 2016 election

campaign, bootstrapping Facebook’s algorithm to spew propaganda that reached

between 29 million and 126 million Americans. See 1 Robert S. Mueller III, Special

Counsel, Report on the Investigation Into Russian Interference in the 2016 Presidential28 Election 24

Ȭ 26, U.S. Dep’t of Justice (March 2019), http://www.justice.gov/storage/

report.pdf. Russia also purchased over 3,500 advertisements on Facebook to

publicize their fake Facebook groups, several of which grew to have hundreds of

thousands of followers. Id. at 25

Ȭ 26. On Twitter, Russia developed false accounts

that impersonated American people or groups and issued content designed to

influence the election; it then created thousands of automated “bot” accounts to

amplify the sham Americans’ messages. Id. at 26

Ȭ 28. One fake account received

over six million retweets, the vast majority of which appear to have come from

real Twitter users. See Gillian Cleary, Twitterbots: Anatomy of a Propaganda

Campaign

, Symantec (June 5, 2019), http://www.symantec.com/blogs/threat

Ȭ

intelligence/twitterbots

Ȭ propaganda

Ȭ disinformation. Russian intelligence also

harnessed the reach that social media gave its false identities to organize “dozens

of U.S. rallies,” some of which “drew hundreds” of real

Ȭ world Americans.

Mueller, Report

, supra , at 29. Russia could do all this only because social media is

designed to target messages like Russia’s to the users most susceptible to them.

While Russia’s interference in the 2016 election is the best

Ȭ documented

example of foreign meddling through social media, it is not the only one. Federal29 intelligence agencies expressed concern in the weeks before the 2018 midterm

election “about ongoing campaigns by Russia, China and other foreign actors,

including Iran,” to “influence public sentiment” through means “including using

social media to amplify divisive issues.” Press Release, Office of Dir. of Nat’l

Intelligence, Joint Statement from the ODNI, DOJ, FBI, and DHS: Combatting

Foreign Influence in U.S. Elections, (Oct. 19, 2018), https://www.dni.gov/index.

php/newsroom/press

Ȭ releases/item/1915

Ȭ joint

Ȭ statement

Ȭ from

Ȭ the

Ȭ odni

Ȭ doj

Ȭ fbi

Ȭ

and

Ȭ dhs

Ȭ combating

Ȭ foreign

Ȭ influence

Ȭ in

Ȭ u

Ȭ s

Ȭ elections. News reports also suggest

that China targets state

Ȭ sponsored propaganda to Americans on Facebook and

purchases Facebook ads to amplify its communications. See Paul Mozur, China

Spreads Propaganda to U.S. on Facebook, a Platform It Bans at Home

, N.Y. Times (Nov.

8, 2017), https://www.nytimes.com/2017/11/08/technology/china

Ȭ facebook.html.

Widening the aperture further, malefactors at home and abroad can

manipulate social media to promote extremism. “Behind every Facebook ad,

Twitter feed, and YouTube recommendation is an algorithm that’s designed to

keep users using: It tracks preferences through clicks and hovers, then spits out a

steady stream of content that’s in line with your tastes.” Katherine J. Wu, Radical30 Ideas Spread Through Social Media. Are the Algorithms to Blame?

, PBS (Mar. 28, 2019),

https://www.pbs.org/wgbh/nova/article/radical

Ȭ ideas

Ȭ social

Ȭ media

Ȭ algorithms.

All too often, however, the code itself turns those tastes sour. For example, one

study suggests that manipulation of Facebook’s news feed influences the mood of

its users: place more positive posts on the feed and users get happier; focus on

negative information instead and users get angrier. Adam D. I. Kramer et al.,

Experimental Evidence of Massive

Ȭ Scale Emotional Contagion Through Social Networks

,

111 PNAS 8788, 8789 (2014). This can become a problem, as Facebook’s algorithm

“tends to promote the most provocative content” on the site. Max Fisher, Inside

Facebook’s Secret Rulebook for Global Political Speech

, N.Y. Times (Dec. 27, 2018),

http://www.nytimes.com/2018/12/27/world/facebook

Ȭ moderators.html. Indeed,

“[t]he Facebook News Feed environment brings together, in one place, many of

the influences that have been shown to drive psychological aspects of

polarization.” Jaime E. Settle, Frenemies: How Social Media Polarizes America (2018).

Likewise, YouTube’s video recommendation algorithm—which leads to more

than 70 percent of time people spend on the platform—has been criticized for

shunting visitors toward ever more extreme and divisive videos. Roose & Conger,31 supra ; see Jack Nicas, How YouTube Drives People to the Internet’s Darkest Corners

,

Wall St. J. (Feb. 7, 2018), https://www.wsj.com/articles/how

Ȭ youtube

Ȭ drives

Ȭ

viewers

Ȭ to

Ȭ the

Ȭ internets

Ȭ darkest

Ȭ corners

Ȭ 1518020478. YouTube has fine

Ȭ tuned its

algorithm to recommend videos that recalibrate users’ existing areas of interest

and steadily steer them toward new ones—a modus operandi that has reportedly

proven a real boon for far

Ȭ right extremist content. See Kevin Roose, The Making of

a YouTube Radical

, N.Y. Times (June 8, 2019), http://www.nytimes.com/interactive/

2019/06/08/technology/youtube

Ȭ radical.html.ȱȱ

There is also growing attention to whether social media has played a

significant role in increasing nationwide political polarization. See Andrew

Soergel, Is Social Media to Blame for Political Polarization in America?

, U.S. News &

World Rep. (Mar. 20, 2017), https://www.usnews.com/news/articles/2017

Ȭ 03

Ȭ 20/is

Ȭ

social

Ȭ media

Ȭ to

Ȭ blame

Ȭ for

Ȭ political

Ȭ polarization

Ȭ in

Ȭ america. The concern is that

“web surfers are being nudged in the direction of political or unscientific

propaganda, abusive content, and conspiracy theories.” Wu, Radical Ideas

, supra

.

By surfacing ideas that were previously deemed too radical to take seriously,

social media mainstreams them, which studies show makes people “much more32 open” to those concepts. Max Fisher & Amanda Taub, How Everyday Social Media

Users Become Real

Ȭ World Extremists

, N.Y. Times (Apr. 25, 2018),

http://www.nytimes.com/2018/04/25/world/asia/facebook

Ȭ extremism.html. At its

worst, there is evidence that social media may even be used to push people toward

violence. 7 The sites are not entirely to blame, of course—they would not have such

success without humans willing to generate and to view extreme content.

Providers are also tweaking the algorithms to reduce their pull toward hate speech

and other inflammatory material. See Isaac, supra ; Roose & Conger, supra

. Yet the

dangers of social media, in its current form, are palpable.

While the majority and I disagree about whether § 230 immunizes

interactive computer services from liability for all these activities or only some, itȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ

7 See, e.g. , Sarah Marsh, Social Media Related to Violence by Young People, Say Experts

, The Guardian (Apr. 2, 2018), https://www.theguardian.com/media/2018/apr/02/social

Ȭ media

Ȭ violence

Ȭ young

Ȭ people

Ȭ gangs

Ȭ say

Ȭ experts; Kevin Roose, A Mass Murder of, and for,
the Internet

, N.Y. Times (Mar. 15, 2019), https://www.nytimes.com/2019/03/15/ technology/facebook

Ȭ youtube

Ȭ christchurch

Ȭ shooting.html; Craig Timberg et al., The New Zealand Shooting Shows How TouTube and Facebook Spread Hate and Violent Images—Yet Again

, Wash. Post (Mar. 15, 2019), https://www.washingtonpost.com/technology/ 2019/03/15/facebook

Ȭ youtube

Ȭ twitter

Ȭ amplified

Ȭ video

Ȭ christchurch

Ȭ mosque

Ȭ shooting; Julie Turkewitz & Kevin Roose, Who Is Robert Bowers, the Suspect in the Pittsburgh Synagogue Shooting?

, N.Y. Times (Oct. 27, 2018), https://www.nytimes.com/2018/ 10/27/us/robert

Ȭ bowers

Ȭ pittsburgh

Ȭ synagogue

Ȭ shooter.html.33 is pellucid that Congress did not have any of them in mind when it enacted the

CDA. The text and legislative history of the statute shout to the rafters Congress’s

focus on reducing children’s access to adult material. Congress could not have

anticipated the pernicious spread of hate and violence that the rise of social media

likely has since fomented. Nor could Congress have divined the role that social

media providers themselves would play in this tale. Mounting evidence suggests

that providers designed their algorithms to drive users toward content and people

the users agreed with—and that they have done it too well, nudging susceptible

souls ever further down dark paths. By contrast, when the CDA became law, the

closest extant ancestor to Facebook (and it was still several branches lower on the

evolutionary tree) was the chatroom or message forum, which acted as a digital

bulletin board and did nothing proactive to forge off

Ȭ site connections. 8ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ

8 See Caitlin Dewey, A Complete History of the Rise and Fall—and Reincarnation!—of
the Beloved ‘90s Chatroom

, Wash. Post (Oct. 30, 2014), http://www.washingtonpost.com/ news/the

Ȭ intersect/wp/2014/10/30/a

Ȭ complete

Ȭ history

Ȭ of

Ȭ the

Ȭ rise

Ȭ and

Ȭ fall

Ȭ and

Ȭ reincarnation

Ȭ of

Ȭ the

Ȭ beloved

Ȭ 90s

Ȭ chatroom; see also Then and Now: A History of Social Networking Sites

, CBS News, http://www.cbsnews.com/pictures/then

Ȭ and

Ȭ now

Ȭ a

Ȭ history

Ȭ of

Ȭ social

Ȭ networking

Ȭ sites (last accessed July 9, 2019) (detailing the evolution of social
media sites from Classmates, launched only “as a list of school affiliations” in December 1995; to “the very first social networking site” Six Degrees, which launched in May 1997
but whose networks were limited “due to the lack of people connected to the Internet”;34 Whether, and to what extent, Congress should allow liability for tech

companies that encourage terrorism, propaganda, and extremism is a question for

legislators, not judges. Over the past two decades “the Internet has outgrown its

swaddling clothes,” Roommates.Com

, 521 F.3d at 1175 n.39, and it is fair to ask

whether the rules that governed its infancy should still oversee its adulthood. It is

undeniable that the Internet and social media have had many positive effects

worth preserving and promoting, such as facilitating open communication,

dialogue, and education. At the same time, as outlined above, social media can be

manipulated by evildoers who pose real threats to our democratic society. A

healthy debate has begun both in the legal academy 9 and in the policyȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ

to Friendster, launched in March 2002 and “credited as giving birth to the modern social
media movement”; to Facebook, which was “rolled out to the public in September 2006”).

9 See, e.g. , Danielle Keats Citron & Benjamin Wittes, The Problem Isn ȇ t Just Backpage: Revising Section 230 Immunity

, 2 Geo. L. Tech. Rev. 453, 454

Ȭ 55 (2018); Jeff Kosseff, Defending Section 230: The Value of Intermediary Immunity

, 15 J. Tech. L. & Pol ȇ y 123, 124 (2010); Daniela C. Manzi, Managing the Misinformation Marketplace: The First Amendment
and the Fight Against Fake News

, 87 Fordham L. Rev. 2623, 2642

Ȭ 43 (2019). Much of the enterprising legal scholarship debating the intersection of social media, terrorism, and
the CDA comes from student Notes. See, e.g. , Jaime E. Freilich, Note, Section 230’s Liability
Shield in the Age of Online Terrorist Recruitment

, 83 Brook. L. Rev. 675, 690

Ȭ 91 (2018); Anna
Elisabeth Jayne Goodman, Note and Comment, When You Give a Terrorist a Twitter: Holding Social Media Companies Liable for their Support of Terrorism

, 46 Pepp. L. Rev. 147,
182

Ȭ 86 (2018); Nicole Phe, Note, Social Media Terror: Reevaluating Intermediary Liability35 community 10 about changing the scope of § 230. Perhaps Congress will clarify

what I believe the text of the provision already states: that the creation of social

networks reaches beyond the publishing functions that § 230 protects. Perhaps

Congress will engage in a broader rethinking of the scope of CDA immunity. Or

perhaps Congress will decide that the current regime best balances the interests

involved. In the meantime, however, I cannot join my colleagues’ decision to

immunize Facebook’s friend Ȭȱ and content

Ȭ suggestion algorithms from judicial

scrutiny. I therefore must in part respectfully dissent, as I concur in part.ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ

Under the Communications Decency Act

, 51 Suffolk U. L. Rev. 99, 126

Ȭ 30 (2018).

10 See, e.g. , Tarleton Gillespie, How Social Netowrks Set the Limits of What We Can Say Online

, Wired (June 26, 2018), http://www.wired.com/story/how

Ȭ social

Ȭ networks

Ȭ set

Ȭ the

Ȭ limits

Ȭ of

Ȭ what

Ȭ we

Ȭ can

Ȭ say

Ȭ online; Christiano Lima, How a Widening Political Rift Over
Online Liability Is Splitting Washington

, Politico (July 9, 2019), http://www.politico.com/ story/2019/07/09/online

Ȭ industry

Ȭ immunity

Ȭ section

Ȭ 230

Ȭ 1552241; Mark Sullivan, The
1996 Law That Made the Web Is in the Crosshairs

, Fast Co. (Nov. 29, 2018), http://www.fastcompany.com/90273352/maybe

Ȭ its

Ȭ time

Ȭ to

Ȭ take

Ȭ away

Ȭ the

Ȭ outdated

Ȭ loophole

Ȭ that

Ȭ big

Ȭ tech

Ȭ exploits; cf. Darrell M. West & John R. Allen, How Artificial Intelligence Is Transforming the World

, Brookings (Apr. 24, 2018), http://www.brookings.edu/research/how

Ȭ artificial

Ȭ intelligence

Ȭ is

Ȭ transforming

Ȭ the

Ȭ world (“The malevolent use of AI exposes individuals and organizations to unnecessary risks and undermines the virtues of the emerging technology.”).