“Sisters in Hate” Author Seyward Darby on AI and the Alt-Right
By: James Kotecki
Tags: Technology, Artificial Intelligence, Algorithms, Ai Ethics, YouTube

Audio + Transcript

James Kotecki:

This is Machine Meets World. Infinia ML’s ongoing conversation about artificial intelligence. I’m James Kotecki and today’s interview is going to be different. Usually on this show, we talk AI with business leaders and tech journalists and members of Congress. But today we’re interviewing Seyward Darby, author of the book Sisters in Hate: American Women on the Front Lines of White Nationalism. Now that is a heavy topic and maybe not an obvious one for an AI business show, but so much of what happens in the book takes place on the internet. The hate movement is online and of course algorithms shape how the message spreads. So today we’re going to talk to someone who has been swimming through this algorithmic stew and who has thought deeply about trends and groups that algorithms promote. This is a vital and unique perspective on AI ethics. Here’s my interview with Seyward Darby.

James Kotecki:

Seyward, first of all, congratulations, because your book was a New York Times Editors’ Choice/Staff Pick. Am I right about that?

Seyward Darby:

That’s correct, yeah. Pinch me — it feels pretty good.

James Kotecki:

Before we get into the AI part of this conversation, it’d be great for you to just set up what your book is about. Give us a summary.

Seyward Darby:

My book looks at white nationalism and its ascendance in the Trump era, but connects it to a long history of hate in America. And specifically it looks at gender on the far right. And women are often written out of the history of the hate movement in the same way that they’re often written out of various histories. And it looks at women as vital actors in a variety of different roles in the hate movement. And I tell the story through the lens of three different women who have in the last decade or so, been a part of the hate movement, Two of them are still in it, one of them has since left. I use their stories to interrogate different themes and the history of the far right.

James Kotecki:

I was struck by how much and how often you are digging into online content: forums, blogs, YouTube videos, podcasts. So much of this information is spreading online. And I think there was even a specific part of the book where you talked about going into a bit of an online rabbit hole yourself. When did you start thinking about or becoming aware of how algorithms are shaping some of this stuff?

Seyward Darby:

I started researching this book right after the election in 2016. And it started from a very simple question. I was reading a lot of coverage of the so-called alt-right at the time. And I saw no women quoted, no women mentioned, and I thought that was really odd. And so I went looking for the women of the alt-right and I found them on social media and specifically a lot of influencer types on YouTube. And I did, I mean, I went down rabbit holes, essentially. I would find one person who made videos and had content related to white nationalism, and I would watch all of their videos. There were some people that they had as guests on, and then those people had their own YouTube channels, but then also there were lots of recommendations popping up, obviously. YouTube was definitely a starting point, Twitter to a certain extent, Facebook to a certain extent. And then over time, different social media networks have emerged, whether you were talking about Gab or Telegram and Discord to a certain extent, but YouTube is definitely, it was the starting point and arguably the most interesting from an AI and algorithmic perspective.

James Kotecki:

Describe to me the process of going down one of these rabbit holes, as things get more and more extreme and as different content is served to you in a place like YouTube.

Seyward Darby:

I would choose to watch a video because I was doing this research and it would be about the decline of Western civilization, which is a dog whistle for the far right. And if you looked at your recommendations, it would start to include videos on that topic. And if you clicked on it, it just kind of kept taking you through videos that got more and more overt in what they were talking about, whether you’re talking about the specific language, certain people who just have a more overt approach.

Seyward Darby:

I’ve been reading some of the most recent research about algorithms and basically how might they shape radicalization. There are some researchers who say, “Algorithms have nothing to do with this.” And they base that on data that they’ve been able to pull, but part of the problem with data research is that a lot of the user data, actual user data for individuals like myself, who would go in, make choices — that’s all personalized and companies aren’t giving out that data. And so a lot of the quantitative research is actually based on anonymized data. And so in some ways the people who take a more anecdotal approach often say, “Actually, this is in some ways as instructive because it is personalized. I am the person doing it. And I’m seeing how my preferences lead me to more overt racist content over time.”

James Kotecki:

Did you get a sense that folks in the white nationalist movement are intentionally leveraging this technology and understanding it to the fullest in order to drive their message? Or are they just getting lucky because YouTube would prioritize any degree of extremism, not because YouTube is racist, but because it just wants to keep you on the site longer?

Seyward Darby:

Yeah. It’s such a great question. And I do think there’s some people who, I don’t know if we’d call them whistleblowers exactly, but people who used to work at YouTube and have since spoken out about how, when the platform was being built, as they were thinking through what YouTube could be, user engagement was the key metric, right? And that often means keeping people engaged with content that makes them angry. And so there certainly is evidence that that is what YouTube was going for and continues to go for. What’s interesting about white nationalists is that they’ve always been remarkably tech savvy. White nationalists were some of the very first people to use the proto-internet in the mid 1980s, because they knew that it would allow them to communicate without the prying eyes of law enforcement or scrupulous media, frankly.

Seyward Darby:

And then they were early adopters of the internet, as soon as the internet became something that people had in their individual homes. Certainly they’ve seen how this is an opportunity for them to craft their own message and deliver it repeatedly. Do they understand how algorithms work? I don’t think necessarily any more than a lot of anybody studying algorithms does. But what they are aware of are the low barriers of entry to getting onto one of these sites, monetizing these sites. And even when they get banned, they just create a new site, a new name, a new something, and so you end up playing whack-a-mole with them.

James Kotecki:

Did swimming through all this content that was so deliberately served and calibrated to you specifically by algorithms, did it have an impact on your psyche? Did you find that it was effective in any way for you?

Seyward Darby:

I did not personally feel myself being influenced or affected by the messages other than from the standpoint of my emotional wellbeing. There were times where I was like, “Why didn’t I choose to write about puppies?” That being said, I’ve talked to people, who I would otherwise think would not align themselves with people in this space, will say, “Well, actually that person has a good point,” or, “I had never really thought about it that way. It doesn’t really sound that bad.” And I think this is the other thing to remember about these messages is that yes, they’re consistent, but they’re also revising them and tweaking them over time. And so when they find that certain messages have purchase, they move with that message.

James Kotecki:

Theoretically, you could program a bot or some kind of AI to write this. And you may have seen recently the news out from OpenAI, they have software and technology now that is writing prose that is very, very convincing to folks. So how does that play into where you think all this is going in the future? Could we see a lot of this content just be algorithmically generated in the first place?

Seyward Darby:

I think it’s absolutely possible. Media literacy used to mean one thing, right? And now I think media literacy means — it has layers and layers of meaning, including being able to differentiate between not just a reliable source and an unreliable source, but a human source and a non-human source. That’s something that we’re going to have to figure out as a society, how to get better at explaining to people.

James Kotecki:

Do you have any kind of macro advice for business leaders for how to think about the AI projects that they’re working on so that they don’t do things that end up turning around and biting them or biting society?

Seyward Darby:

Increasingly, we’ve seen tech companies that rely on algorithms really botching responses, frankly, to things, whether it’s the proliferation of QAnon content on their platforms or radical content that people then link back to violent actors and that they had been consuming that content. And to me, those things, it’s all important when you realize that it’s happening to take it down, to figure out how to police it. But there’s not a lot of preemptive thinking happening, frankly. And I think about it as healthcare, a lot of the time, as an analogy, that the more you can think about preventative care, the better health your platform and your users are going to be in over time, as opposed to, do you just treat the problem when it becomes apparent? So thinking preventatively, I think is key. And there are some really interesting academics out there doing research on extremism and on radicalization. And I think that people who work in AI and tech generally would do well to find ways to build bridges with those folks, to come up with protocols and best practices for thinking preventatively about this stuff.

James Kotecki:

That is right on track. We talk a lot about monitoring and auditing this stuff when it’s actually out there in the world to make sure that you actually know what it’s doing, and is it performing the way that you actually even intended it to in the first place, until, as you said, the problem gets so big and you realize it’s so big that you actually have to do something, but it’s too late.

Seyward Darby:

Right.

James Kotecki:

Seyward Darby is the author of Sisters In Hate, thanks so much for being here on Machine Meets World.

Seyward Darby:

Thank you so much for having me.

James Kotecki:

I have been, and I still am, I guess James Kotecki and that is what happens when Machine Meets World.