Prompt: Last year, we decided to start a magazine. We felt disappointed by the mainstream conversation about technology, and we wanted to create space for an alternative. Then Trump happened. At first, nothing seemed to matter. Why publish a magazine about technology in the age of Trump? Why write or read about anything but Trump? We thought about it, and came up with a couple of reasons.

Response: The first one is easy: Fuck Trump. We have to fight him, and we will. But we’re not going to let him monopolize our bandwidth. That’s what he wants. Trump is the hideous id of the internet. He will do anything for our clicks—and he wants all of our clicks. But modern operating systems have made us good multitaskers. We can walk and chew gum and still slay the dragon.


Prompt: The second reason is more important: Trump is a wake-up call. His election proves that many of the wise men and women who claim to understand how our world works have no clue. He is a reminder that we can’t afford to have stupid conversations about important things anymore. Trump is a creature of technology. A technologized world created the conditions for Trumpism. Social dislocations caused by automation helped create his base, and social media incubated and propagated his fascist rhetoric.

Response: Technology will continue to serve our Troll-in-Chief, as his Administration and his allies in Congress hunt undesirables with databases and drones. Some of their barbarisms will be dramatic and highly visible. Others will be piecemeal, discreet, and hard to grasp immediately. We need intelligent writing to make both forms of brutality legible. We also need to build the tools that can help us disrupt and dismantle Trump’s agenda.


Prompt: The era of Trump will be a technologized era, like the one before it. We will be paying attention—and proposing ways to resist. (Along these lines, stay tuned for Tech Against Trump, a short book coming soon from Logic.) We will also be in the streets. We hope you’ll join us. Yours, Jim Fingal Christa Hartsock Ben Tarnoff Xiaowei Wang Moira Weigel Technology is a scourge!, technophobic scolds tweet from their iPhones. Technology will fix it!, techno-utopians proclaim on Medium. Indistinguishable middle-aged men tout indistinguishable products in front of a press all too eager to write gadget reviews indistinguishable from ad copy. Companies tank; companies IPO. Legacy media editors commission lifestyle pieces about CEO sneakers and office cafeterias brimming with lacinato kale. Press releases are distributed and regurgitated on TechCrunch. Academics write screeds against Facebook and post them on… Facebook.

Response: We can’t stop watching. We’re so, so bored. Tech is magic. Tech lets us build worlds and talk across oceans. Whatever kind of freak we are—and most of us are several kinds—tech helps us find other freaks like us. But most tech writing is shallow and pointless. It’s nobody’s fault; everyone is just doing their job. Communications teams feed reporters winning anecdotes about their founders that explain exactly nothing. (“One day, Chad was eating a ham sandwich and realized…”) The reporters are overworked and underpaid and need to file a new story by EOD. Who can blame them for taking the bait? Editors are desperate for shareable content, which too often means some kind of caricature. Tech is either brilliant or banal, heroic or heinous. The best minds of our generation are either curing cancer, or building a slightly faster way to buy weed. The robots will either free us from drudgery or destroy civilization. Hate-click or like-click, the stories tend to be about a handful of people. The pasty boy genius. The tragic token woman. The fascist billionaire. The duo of white dudes dueling to lead us to Mars.


Prompt: We deserve a better conversation. By “we,” we mean you, because everyone uses technology. We are all both its subject and object. Tech is how you find the place you live. It’s how you turn your car into a taxi or your spare room into a hotel. Tech lets you see the faces of the people you love from thousands of miles away. It helps you buy clothes, and track the steps you take trying to fit into them. You use tech to order your dinner, find a date, or at least stream the video you masturbate to when you don’t have the energy to go out.

Response: Someday, when you do swipe right on that special someone, and they swipe right on you—it’s a match!—tech will shape how you flirt, how you define the relationship, how you plan and brag about your wedding. When you have children, you will use tech to track their development and find a babysitter. By then, maybe the babysitters will be robots, and school will be software. As we grow old, tech will help you find a caregiver. Your children will manage your physical decay remotely via app. When you have a bowel movement, they will receive a push notification. They will know what to do. They will have been on Instagram since they were a twinkle in the first fetal ultrasound photo you posted. The NSA will have been spying on them since before they were born.


Prompt: The stakes are high, is what we are saying. Like you, we are both insiders and outsiders. Luckily, this is exactly the position you need to be in to observe and describe a system. In the social sciences they call it “logic”: the rules that govern how groups of people operate. For engineers, “logic” refers to the rules that govern paths to a goal. In the vernacular, “logic” means something like being reasonable. Logic Magazine believes that we are living in times of great peril and possibility. We want to ask the right questions. How do the tools work? Who finances and builds them, and how are they used? Whom do they enrich, and whom do they impoverish? What futures do they make feasible, and which ones do they foreclose? We’re not looking for answers. We’re looking for logic.

Response: In the beginning, my plan was perfect. I would meditate for five minutes in the morning. Each evening before bed, I would do the same. There was only one catch: instead of relying on my own feelings, a biofeedback device would study my brainwaves to tell me whether I was actually relaxed, focused, anxious, or asleep.


Prompt: By placing just a few electrodes on my scalp to measure its electrical activity, I could use an electroencephalography (EEG) headset to monitor my mood. Whereas “quantified self” devices like the Fitbit or Apple Watch save your data for later, these headsets loop your brainwaves back to you in real time so that you can better control and regulate them.

Response: Basic EEG technology has been around since the early twentieth century, but only recently has it become available in portable, affordable, and Bluetooth-ready packages. In the last five years, several start-ups—with hopeful names like Thync, Melon, Emotiv, and Muse—have tried to bring EEG devices from their clinical and countercultural circles into mainstream consumer markets.


Prompt: I first learned about the headsets after watching the Muse’s creator give a TEDx talk called “Know thyself, with a brain scanner.” “Our feelings about how we’re feeling are notoriously unreliable,” she told the audience, as blue waves and fuzzy green patches of light flickered on a screen above her. These were her own brainwaves—offered, it seemed, as an objective correlative to the mind’s trickling subjectivity.

Response: Their actual meaning was indecipherable, at least to me. But they supported a sales pitch that was undeniably attractive: in the comfort of our own homes, without psychotropic meds, psychoanalysis, or an invasive operation, we could bring to light what had previously been unconscious. That was, in any case, the dream.


Prompt: Day One When I first placed the Muse on my head one Sunday evening in late October, I felt as though I was greeting myself in the future. A thin black band, lightweight and plastic, stretched across my forehead. Its wing-like flanks fit snugly behind my ears. Clouds floated by on the launch screen of its accompanying iPhone app. The Muse wasn’t just a meditation device, the app explained, but a meditation assistant. For some minutes, my initial signal was poor.

Response: To encourage me not to give up before I’d even started, the assistant kept talking to me. Her calming female voice told me how to delicately re-adjust the electrodes to get the signal working. The ones behind my ears were having trouble aligning with the shape of my head. Eventually, the Muse accurately “sensed” my brain. It would now be able to interpret my brainwaves and translate their frequencies into audio cues, which I would hear throughout my meditation session.


Prompt: Tap the button, she said encouragingly. “I’m ready,” I clicked, and my first five-minute meditation session began. Inward bound, I sat at my desk with the lamp on and closed my eyes. Waves crashed loudly on the shore, which indicated that I was thinking too much. But from time to time, I could hear a few soft splashes of water and, farther in the distance, the soft chirping of birds.

Response: After what seemed like forever, it was over. Like all self-tracking practices (and rather unlike a typical meditation session), it seemed that the post-game was as important as the practice itself. Knowing this, I made a good-faith effort to pore over my “results.” They were, at first, second, and third glance, impenetrable. I had earned 602 calm points. In all my experiences counting—the miles of my runs, words on my documents, even the occasional calorie—I had never learned what a “calm point” was. The units used by the Muse seemed not only culturally insignificant, but void of any actual meaning. In an effort to build an internal chain of signification, the app had mysteriously multiplied these calm points by a factor of three, whereas my “neutral points” had only been multiplied by a factor of one. Birds, I was told, had “landed” sixteen times.


Prompt: Equally inscrutable were the two awards I had earned. Whatever they were, I thought, they were hardly deserved, considering I had so far spent a total of seven minutes scanning my brain. One was for tranquility—“Being more than 50% calm in a single session must feel good,” the award told me. The other was a “Birds of Eden Award.” I was told that I had earned this because at least two birds chirped per minute, “which must have felt a bit like being at Birds of Eden in South Africa—the largest aviary in the world.” Not really, I thought. But then again, I had never been to South Africa.

Response: It felt great to meditate for the first time only to be told that I was already off to a good start. But I knew deep down—or, at least, I thought I knew—that I had not felt calm during any part of the session. I was in the difficult position, then, of either accepting that I did not know myself, in spite of myself, or insisting on my own discomfort in order to prove the machine wrong. It wasn’t quite that the brain tracker wanted me to know myself better so much as it wanted me to know myself the way that it knew me.


Prompt: The Brain Doctor The second morning of my experiment, I took the subway uptown to see Dr. Kamran Fallahpour, an Iranian-American psychologist in his mid-fifties and the founder of the Brain Resource Center. The Center provides patients with maps and other measures of their cognitive activity so that they can, ideally, learn to alter it.

Response: Some of Fallahpour’s patients suffer from severe brain trauma, autism, PTSD, or cognitive decline. But many have, for lack of a better word, normal-seeming brains. Athletes, opera singers, attorneys, actors, students—some of them as young as five years old—come to Fallahpour to improve their concentration, reduce stress, and “achieve peak performance” in their respective fields.


Prompt: Dr. Fallahpour’s offices and labs lie on the ground floor of a heavy stone apartment building on the Upper West Side. When I arrived, he was in the middle of editing a slideshow on brain plasticity for a talk he was due to give at an Alzheimer’s conference. On a second, adjacent monitor, sherbet peaks and colored waves—presumably from some brain—flowed on the screen.

Response: Fallahpour wears bold glasses with thick-topped frames, in the style of extra eyebrows. When we met, he was dressed in a dark blue suit to which was affixed a red brooch shaped like a coral reef or a neural net—I kept meaning to ask which. An enthusiastic speaker with a warm bedside manner, it was hard to shake the impression that there was nothing he would rather be doing than answering my questions about the growing use of personal EEG headsets.


Prompt: His staff had not yet arrived that morning, he apologized, so he would be fielding any calls himself. As if on cue, the phone rang. “No. Unfortunately, we do not take insurance,” he told the caller. “That happens a lot,” he explained, after hanging up. “Now, where were we?” Before turning to brain stimulation technologies, Fallahpour worked for many years as a psychotherapist, treating patients with traditional talk therapy. His supervisors thought he was doing a good job, and he saw many of his patients improve. But the results were slow-going. He often got the feeling that he was only “scratching the surface” of their problems. Medication worked more quickly, but it was imprecise. Pills masked the symptoms of those suffering from a brain injury, but they did little to improve the brain’s long-term health.

Response: Like many of his fellow researchers, Fallahpour was interested in how to improve the brain through conditioning, electrical and magnetic stimulation, and visual feedback. He began to work with an international group of neuroscientists, clinicians, and researchers developing a database of the typical brain. They interviewed thousands of normal patients—“normal” was determined by tests showing the absence of known psychological disorders—and measured their resting and active brainwaves, among other physiological responses, to establish a gigantic repository of how the normative brain functioned.


Prompt: Neuroscience has always had this double aim: to know the brain and to be able to change it. Its method for doing so—“screen and intervene”—is part of the larger trend toward personalized medicine initiatives. Advance testing, such as genomics, can target patients at risk for diabetes, cancer, and other diseases. With the rise of more precise diagnostic and visualization technologies, individuals can not only be treated for current symptoms, but encouraged to prevent future illnesses.

Response: Under the twenty-first century paradigm of personalized medicine, everyone becomes a “potential patient.” This is why the Brain Resource Center sees just as many “normal” patients as symptomatic ones. And it’s why commercial EEG headsets are being sold to both epileptics trying to monitor their symptoms and office workers hoping to work better, faster.


Prompt: Brain training is seductive because its techniques reinforce an existing neoliberal approach: health becomes a product of personal responsibility; economic and environmental causes of illness are ignored. Genetics may hardwire us in certain ways, the logic of neuroliberalism goes, but hard work can make us healthy. One problem is that the preventative care of the few who can afford it gets underwritten by the data of the many.

Response: Consider Fallahpour’s boot camp for elementary school kids. For a few hours each day during school vacations, the small rooms of his low-ceilinged offices are swarmed with well-behaved wealthy children playing games to “improve brain health and unlock better function,” as well as to acquire a “competitive advantage.” “We tune their brain to become faster and more efficient,” he explained. “The analogy is they can have Windows 3.1 or upgrade it to 10.” Before I had time to contemplate the frightening implications of this vision, the phone began, again, to ring. Fallahpour exchanged pleasantries for a few minutes, asking about the caller’s weekend. No, he told them, he did not take insurance.


Prompt: Bettering Myself The more I thought about the kind of cognitive enhancement Fallahpour promised, the more trouble I had remembering the last time I felt clear-eyed and focused. Had I ever been? Could I ever be? For a few days I had sensed a dull blankness behind my eyes. I wondered if it was a head cold, or sleep deprivation, or a newfound gluten allergy. I started sleeping more and my cold improved, but the brain fog continued. Reading a book felt like standing on a subway grate, with holes and winds weaving through the pages. I misspelled words, like “here” (hear) and “flourish” (fluorish). I waved to a man on the train who looked like someone I had dated years ago.

Response: On a good day, I convinced myself, there was no way I was operating above sixty percent, maybe sixty-five. Sixty percent of what, I wasn’t sure. But I knew I could do better. I felt a twinge of envy toward those who had achieved the mythical “peak performance,” and I redoubled my commitment to self-improvement.


Prompt: The headset remained subtly encouraging. “Whatever you’re experiencing right now is perfect,” my meditation assistant assured me—just moments before my fourth session’s calibration had paused, again, because the signal quality was too low. I re-adjusted my headset, practicing patience. “Training your mind is kind of like training a puppy,” the motivational preamble continued. “Getting angry at the puppy isn’t going to get you anywhere.” I wasn’t angry at anyone’s dog, but I couldn’t stop comparing each session’s score to the last’s. Was I hearing fewer birds? Was it easier to focus with or without caffeine? As suspicious as I was about the accuracy of the metrics, I still wanted to beat my previous score. The more elusive peak performance seemed, the more I came to realize that it was structured as an essentially nostalgic feeling. It relied on the fear that you used to be younger, sharper, more clear-eyed—and the hope that you could somehow, with practice, be this way again.

Response: When I mentioned my experiments to a friend, he recommended that I watch a performance by the conceptual programmer Sam Lavigne. In “Online Shopping Center,” Lavigne trains a DIY EEG device to identify whether his brain is thinking about shopping online or his own mortality. Being either “shopping-like” or “death-like” was not so different, it seemed, from the Muse letting me know whether I was calm or active, focused or restless. In both cases, the data was mostly junk, the binaries reductive, the exercise absurd.


Prompt: Test Subject When I went to see Dr. Fallahpour for a follow-up visit, I was running thirty minutes late. I had forgotten to transfer trains at 59th Street because I had gotten distracted trying to make sense of an advertisement chastening me for my distraction: “Daydreamed through your stop, again?” it asked.

Response: I had, but I wouldn’t know it yet. It wasn’t until 116th Street that I realized I had missed my stop—in fact, I had missed several. Still, I felt a low current of satisfaction when I emerged into the sunlight at 125th Street, far from where I needed to be. Such inattention made me a more viable patient for brain training than I had previously realized, in need of greater focus for even the most elementary tasks. It had also made me very late.


Prompt: Fallahpour and I decided I would try a calm protocol first, followed by one that rewarded my brain for focus. While he gelled the electrodes and placed them on my scalp, I asked him about some of the skepticism surrounding EEG headsets—namely, the fact that many people, myself included, found it difficult to tell what exactly was being measured.

Response: “EEG is a crude tool and it isn’t the best we have, but it’s the most convenient in many ways,” he explained. “It’s prone to a lot of ‘garbage in and out.’” But when done correctly, he added, it can be “useful and quite powerful.” Deciphering signals from the noise required the trained judgement of an expert like Fallahpour. In this sense, the EEG’s biofeedback wasn’t quite as seamless as going to the gym with your Fitbit. You still needed someone to help you help yourself.


Prompt: To start, we took a baseline measurement of my brain. I had very quick recovery, or response, or something, in terms of what I think were my alpha waves. This meant that my ability to calm myself was sophisticated. I felt surprised at first, and dumbly flattered, much like I had during my first session with the Muse.

Response: For the calm protocol, classical music cut in and out of my headphones depending on whether certain frequencies in my brain were active. This was visualized by red and blue columns flanking both sides of the screen. I was supposed to keep the colors under certain thresholds in their respective containers. At one point, I opened my eyes. The blue column, which had been filling up, drained suddenly. This was supposed to be a sign of resilience.


Prompt: When we tested my concentration, the settings were adjusted to exercise different kinds of brainwaves. I was tasked with keeping a blue column at a certain level while not letting other red columns reach a certain height. It was more difficult than meditating with my Muse—but also, because it was a game, more enjoyable. After five minutes, I convinced myself that I felt my mind becoming more elastic, more responsive. I had started to figure out how to modify my patterns in order to play the music, even if I did not quite know what those patterns meant.

Response: Know Nothing By the end of my week with the Muse, my results were as perilously inscrutable as they had been at the start. Thousands of birds had chirped in my ear. An infinity of waves had crashed upon an endless shore. I had earned quite a few more badges, some by the sheer virtue of persisting: adjusting the signal, continuing the exercise day after day, not quitting in the face of a great and useless mystery.


Prompt: I had learned very little about myself. This in itself wasn’t surprising. But if the EEG headsets were supposed to teach anything, their lesson was somewhat contradictory: I should know myself, but I should also be prepared to be wrong about what I knew.  In this respect, the headset was more like the Oracle of Delphi’s famous precept, “Know thyself,” than its designers had intended. The dictum was initially issued as a double-edged warning about the limits of knowing and the incompleteness of interpretation—a truth that the Muse, ironically enough, confirmed.

Response: The more I parsed my personal graphs and charts, the more I arrived at the same conclusions as anyone who has ever taken more than a passing glance at the brain. Our tools aren’t good enough. At least not yet. And the inadequate and embarrassing analogies we use to describe our brains do little to help us see ourselves clearly. In the course of the week, mine had been variously compared to a loom, a digital machine, an obsolete Windows operating system, and a puppy. What had I been expecting? That a toy would illuminate the fog? Commercial EEG devices promise that we can know our brain frequencies, even while most of those frequencies are “garbage.” The machines might be too. Average EEG devices like the Muse have been shown to have trouble distinguishing between the signals of a relaxed brainwave, stray thought, skin pulse, or furrowed brow. And several studies have disproven the efficacy of related “brain training” games, which don’t augment intelligence so much as make people better at playing the game. It’s possible that all the Muse taught me was how to score calm points and charm songbirds, not how to unlock inner bliss.


Prompt: When the next Sunday came around, I was just as anxious about relaxation and relaxed about anxiety as I had been the week before. I still didn’t know whether I wanted to go shopping. Other times I thought I was thinking about death, though I couldn’t be sure. Who knows, maybe I would never know. I might even die that way—knowing very little, and getting that part wrong too.

Response: As the Trump Administration enters its first hundred days, the 2016 election and its unexpected result remains a central topic of discussion among journalists, researchers, and the public at large. It is notable the degree to which Trump’s victory has propelled a broader, wholesale evaluation of the defects of the modern media ecosystem. Whether it is “fake news,” the influence of “filter bubbles,” or the online emergence of the “alt-right,” the internet has been cast as a familiar villain: enabling and empowering extreme views, and producing a “post-fact” society.


Prompt: This isn’t the first time that the internet has figured prominently in a presidential win. Among commentators on the left, the collective pessimism about the technological forces powering Trump’s 2016 victory are matched in mirror image by the collective optimism about the technological forces driving Obama’s 2008 victory. As Arianna Huffington put it simply then, “Were it not for the Internet, Barack Obama would not be president. Were it not for the Internet, Barack Obama would not have been the nominee.” But whereas Obama was seen as a sign that the new media ecosystem wrought by the internet was functioning beautifully (one commentator praised it as “a perfect medium for genuine grass-roots political movements”), the Trump win has been blamed on a media ecosystem in deep failure mode. We could chalk these accounts up to simple partisanship, but that would ignore a whole constellation of other incidents that should raise real concerns about the weaknesses of the public sphere that the contemporary internet has established.

Response: This troubled internet has been around for years. Fears about filter bubbles facilitating the rise of the alt-right can and should be linked to existing concerns about the forces producing insular, extreme communities like the ones driving the Gamergate controversy. Fears about the impotence of facts in political debate match existing frustrations about the inability for documentary evidence in police killings—widely distributed through social media—to produce real change. Similarly, fears about organized mobs of Trump supporters systematically silencing political opponents online are just the latest data point in a long-standing critique of the failure of social media platforms to halt harassment.


Prompt: The Wise Crowd The Obama and Trump elections might be read as the bookends of a story about the impact of the internet on society. How do we size up the nearly ten years between 2008 and 2016? How do we understand what happened on the internet during that time, and the ripple effect it had on the public sphere? We can tell this story through investments, companies, and acquisitions, and the threading together of the worlds of technology and the media. But doing so might miss the forest for the trees. We would miss the fact that ideology is embedded in code, and that there is a deeper story about the aspirations of the internet at the end of the first decade of the 21st century.

Response: One critical anchor point is the centrality of the wisdom of the crowd to the intellectual firmament of Web 2.0: the idea that the broad freedom to communicate enabled by the internet tends to produce beneficial outcomes for society. This position celebrated user-generated content, encouraged platforms for collective participation, and advocated the openness of data.


Prompt: Inspired by the success of projects like the open-source operating system Linux and the explosion of platforms like Wikipedia, a generation of internet commentators espoused the benefits of crowd-sourced problem-solving. Anthony D. Williams and Don Tapscott’s Wikinomics (2006) touted the economic potential of the crowd. Clay Shirky’s Here Comes Everybody (2008) highlighted how open systems powered by volunteer contributions could create social change. Yochai Benkler’s The Wealth of Networks (2006) posited a cooperative form of socioeconomic production unleashed by the structure of the open web called “commons-based peer production.” Such notions inspired movements like “Gov 2.0” and projects like the Sunlight Foundation, which sought to publish government data in order to reduce corruption and enable the creation of valuable new services by third parties. It also inspired a range of citizen journalism projects, empowering a new fourth estate.

Response: Faith in the collective intelligence of the crowd didn’t go unchallenged. Contemporary authors like Andrew Keen railed against the diminishing role of experts in The Cult of the Amateur (2007). Jaron Lanier’s You Are Not A Gadget (2010) warned of individual intelligence being replaced by the judgment of crowds and algorithms. Eli Pariser’s The Filter Bubble (2011) expressed anxiety about the isolating effect of recommendation systems that created information monocultures. Evgeny Morozov’s The Net Delusion (2012) attacked the notion of the internet as a democratizing force.


Prompt: Yet regardless of the critics, the belief in the wisdom of the crowd framed the design of an entire generation of social platforms. Digg and Reddit—both empowered by a system of upvotes and downvotes for sharing links—surfaced the best new things on the web. Amazon ratings helped consumers sort through a long inventory of products to find the best one. Wikis proliferated as a means of coordination and collaboration for a whole range of different tasks. Anonymous represented an occasionally scary but generative model for distributed political participation. Twitter—founded in 2006—was celebrated as a democratizing force for protest and government accountability.

Response: Intelligence Failure The platforms inspired by the “wisdom of the crowd” represented an experiment. They tested the hypothesis that large groups of people can self-organize to produce knowledge effectively and ultimately arrive at positive outcomes. In recent years, however, a number of underlying assumptions in this framework have been challenged, as these platforms have increasingly produced outcomes quite opposite to what their designers had in mind. With the benefit of hindsight, we can start to diagnose why. In particular, there have been four major “divergences” between how the vision of the wisdom of the crowd optimistically predicted people would act online and how they actually behaved.


Prompt: First, the wisdom of the crowd assumes that each member of the crowd will sift through information to make independent observations and contributions. If not, it hopes that at least a majority will, such that a competitive marketplace of ideas will be able to arrive at the best result. This assumption deeply underestimated the speed at which a stream of data becomes overwhelming, and the resulting demand for intermediation among users. It also missed the mark as to how platforms would resolve this: by moving away from human moderators and towards automated systems of sorting like the Facebook Newsfeed. This has shifted the power of decision-making from the crowd to the controllers of the platform, distorting the free play of contribution and collaboration that was a critical ingredient for collective intelligence to function.

Response: Second, collective intelligence requires aggregating many individual observations. To that end, it assumes a sufficient diversity of viewpoints. However, open platforms did not generate or actively cultivate this kind of diversity, instead more passively relying on the ostensible availability of these tools to all.


Prompt: There are many contributing causes to the resulting biases in participation. One is the differences in skills in web use across different demographics within society. Another is the power of homophily: the tendency for users to clump together based on preferences, language, and geography—a point eloquently addressed in Ethan Zuckerman’s Digital Cosmopolitans (2014). Finally, activities like harassment and mob-like “brigading” proved to be effective means of chilling speech from targeted—and often already vulnerable—populations on these platforms.

Response: Third, collective intelligence assumes that wrong information will be systematically weeded out as it conflicts with the mass of observations being made by others. Quite the opposite played out in practice, as it ended up being much easier to share information than to evaluate its accuracy. Hoaxes spread very effectively through the crowd, from bogus medical beliefs and conspiracy theories to faked celebrity deaths and clickbait headlines.


Prompt: Crowds also arrived at incorrect results more often than expected, as in the high-profile misidentification of the culprits by Reddit during the Boston Marathon bombing. The failure of the crowd to eliminate incorrect information, which seemed sufficiently robust in the case of something like Wikipedia, did not apply to other contexts.

Response: Fourth, collective intelligence was assumed to be a vehicle for positive social change because broad participation would make wrongdoing more difficult to hide. Though this latter point turned out to be arguably true, transparency alone was not the powerful disinfectant it was assumed to be. The ability to capture police violence on smartphones did not result in increased convictions or changes to the underlying policies of law enforcement. The Edward Snowden revelations failed to produce substantial surveillance reform in the United States. The leak of Donald Trump’s Access Hollywood recording failed to change the political momentum of the 2016 election. And so on. As Aaron Swartz warned us in 2009, “reality doesn’t live in the databases.” Ultimately, the aspirations of collective intelligence underlying a generation of online platforms proved far more narrow and limited in practice. The wisdom of the crowd turned out to be susceptible to the influence of recommendation algorithms, the designs of bad actors, in-built biases of users, and the strength of incumbent institutions, among other forces.


Prompt: The resulting ecosystem feels deeply out of control. The promise of a collective search for the truth gave way to a pernicious ecosystem of fake news. The promise of a broad participatory culture gave way to campaigns of harassment and atomized, deeply insular communities. The promise of greater public accountability gave way to waves of outrage with little real change. Trump 2016 and Obama 2008 are through-the-looking-glass versions of one another, with the benefits from one era giving rise to the failures of the next.

Response: Reweaving the Web So, what comes next? Has a unique moment been lost? Is the ecosystem of the web now set in ways that prevent a return to a more open, more participatory, and more collaborative mode? What damage control can be done on our current systems? It might be tempting to take the side of the critics who have long claimed that the assumptions of collective intelligence were naive from their inception. But this ignores the many positive changes these platforms have brought. Indeed, staggeringly successful projects like Wikipedia disprove the notion that the wisdom of the crowd framework was altogether wrong, even as an idealized picture of that community has become more nuanced with time.


Prompt: It would also miss the complex changes to the internet in recent years. For one, the design of the internet has changed significantly, and not always in ways that have supported the flourishing of the wisdom of the crowd. Anil Dash has eulogized “the web we lost,” condemning the industry for “abandon[ing] core values that used to be fundamental to the web world” in pursuit of outsized financial returns. David Weinberger has characterized this process as a “paving” of the web: the vanishing of the values of openness rooted in the architecture of the internet. This is simultaneously a matter of code and norms: both Weinberger and Dash are worried about the emergence of a new generation not steeped in the practices and values of the open web.

Response: The wisdom of the crowd’s critics also ignore the rising sophistication of those who have an interest in undermining or manipulating online discussion. Whether Russia’s development of a sophisticated state apparatus of online manipulation or the organized trolling of alt-right campaigners, the past decade has seen ever more effective coordination in misdirecting the crowd. Indeed, we can see this change in the naivety of creating open polls to solicit the opinions of the internet or setting loose a bot to train itself based on conversations on Twitter. This wasn’t always the case—the online environment is now hostile in ways that inhibit certain means of creation and collaboration.


Prompt: To the extent that the vision of the wisdom of the crowd was naive, it was naive because it assumed that the internet was a spontaneous reactor for a certain kind of collective behavior. It mistook what should have been an agenda, a ongoing program for the design of the web, for the way things already were. It assumed users had the time and education to contribute and evaluate scads of information. It assumed a level of class, race, and gender diversity in online participation that never materialized. It assumed a gentility of collaboration and discussion among people that only ever existed in certain contexts. It assumed that the simple revelation of facts would produce social change.

Response: In short, the wisdom of the crowd didn’t describe where we were, so much as paint a picture of where we should have been going. Fulfilling those failed aspirations will require three major things. Platforms must actively protect the crowd’s production of wisdom. The visibility of collective decision-making and the drama of mass action online produces the illusion of strength. In reality, the blend of code and community giving rise to sustainable collective intelligence is a delicate and elusive set of human dynamics. Rather than assuming its inevitability, we should build systems—either human-driven or autonomous—for robustly shielding and cultivating these processes in the harsh environment of the web.


Prompt: The mission needs to be drawn broader than code. Ensuring that the wisdom of the crowd can produce social change means creating pathways for offline action that can effectively challenge wrongdoing. Ensuring that the wisdom of the crowd can reach accurate results requires more inclusive, diverse bodies of participants. Both speak to a political agenda that cannot be achieved merely by designing tools and making them openly available.

Response: Experimentation must be accelerated at the edges. Although we depend heavily on a few key platforms, the internet is still a vast space. Today’s platforms emerged from experimentation at the edges. To produce new generation of robust platforms, we need more experimentation—a proliferation and wide exploration of alternative spaces for crowds to gather online.


Prompt: It remains an open question whether the internet is traveling down the same, well-worn paths followed by all communications infrastructures, or whether it represents something truly new. But to accept the current state of affairs as inevitable falls prey to a fatalistic pessimism that would only further compound the problems created by the equally deterministic optimism of the decade past.

Response: The vision of collective participation embedded in the idea of the wisdom of the crowd rests on the belief in the unique potential of the web and what it might achieve. Even as the technology evolves, that vision—and a renewed defense of it—must guide us as we enter the next decade. Technology has a gender problem, as everyone knows. The underrepresentation of women in technical fields has spawned legions of TED talks, South by Southwest panels, and women-friendly coding boot camps. I’ve participated in some of these get-women-to-code workshops myself, and I sometimes encourage my students to get involved. Recently, though, I’ve noticed something strange: the women who are so assiduously learning to code seem to be devaluing certain tech roles simply by occupying them.


Prompt: It’s not always obvious to outsiders, but the term “technology sector” is a catch-all for a large array of distinct jobs. Of course there are PR, HR, and management roles. But even if we confine ourselves to web development, technical people often distinguish among “front-end,” “back-end,” and “full-stack” development. The partition between the two “ends” is the web itself. There are people who design and implement what you see in your web browser, there are people who do the programming that works behind the scenes, and there are people who do it all.

Response: In practice, the distinction is murky: some developers refer to everything user-facing as the front-end, including databases and applications, and some developers use front-end to mean only what the user sees. But while the line shifts depending on who you’re talking to, most developers acknowledge its existence.


Prompt: I spoke to a number of developers who confirmed something I’d sensed: for some time, the technology industry has enforced a distinct hierarchy between front-end and back-end development. Front-end dev work isn’t real engineering, the story goes. Real programmers work on the back-end, with “serious” programming languages. Women are often typecast as front-end developers, specializing in the somehow more feminine work of design, user experience, and front-end coding.

Response: Are women really more likely to be front-end developers? Numbers are hard to pin down. Most studies consider the tech sector as a single entity, with software engineers lumped together with HR professionals. A 2016 StackOverflow user survey showed that front-end jobs—“Designer,” “Quality Assurance,” and “Front-End Web Developer”—were indeed the top three titles held by women in the tech industry, although that survey itself has some problems.


Prompt: We need better numbers, as feminist developers have been saying for years, but it also doesn’t seem like a huge stretch to take developers at their word when they say that front-end development is understood to occupy the girlier end of the tech spectrum. Front-end developers, importantly, make about $30,000 less than people in back-end jobs like “DevOps” engineers, who work on operations and infrastructure, according to the salary aggregation site Glassdoor.

Response: Sorting the Stack The distinction between back and front wasn’t always so rigid. “In the earliest days, maybe for the first ten years of the web, every developer had to be full-stack,” says Coraline Ada Ehmke, a Chicago-based developer who has worked on various parts of the technology stack since 1993. “There wasn’t specialization.” Over time, web work professionalized. By the late 2000s, Ehmke says, the profession began to stratify, with developers who had computer science degrees (usually men) occupying the back-end roles, and self-taught coders and designers slotting into the front.


Prompt: For many people who are teaching themselves to code, front-end work is the lowest-hanging fruit. You can “view source” on almost any web page to see how it’s made, and any number of novices have taught themselves web-styling basics by customizing WordPress themes. If you’re curious, motivated, and have access to a computer, you can, eventually, get the hang of building and styling a web page.

Response: Which is not to say it’s easy, particularly at the professional level. A front-end developer has to hold thousands of page elements in her mind at once. Styles overwrite each other constantly, and what works on one page may be disastrous on another page connected to the same stylesheet. Front-end development is taxing, complex work, and increasingly it involves full-fledged scripting languages like JavaScript and PHP.


Prompt: “Serious” developers often avoid acknowledging this by attributing front-end expertise not to mastery but to “alchemy,” “wizardry,” or “magic.” Its adepts don’t succeed through technical skill so much as a kind of web whispering: feeling, rather than thinking, their way through a tangle of competing styles.

Response: “There’s this perception of it being sort of a messy problem that you have to wrangle with systems and processes rather than using your math-y logic,” says Emily Nakashima, a full-stack developer based in San Francisco. That’s not true, of course; nothing on a computer is any more or less logical than anything else. But perhaps it’s easier to cast women in a front-end role if you imbue it with some of the same qualities you impute to women.


Prompt: The gendered attributes switch as you travel to the back of the stack. At the far end, developers (more often “engineers”) are imagined to be relentlessly logical, asocial sci-fi enthusiasts; bearded geniuses in the Woz tradition. Occupations like devops and network administration are “tied to this old-school idea of your crusty neckbeard dude, sitting in his basement, who hasn’t showered in a week,” says Jillian Foley, a former full-stack developer who’s now earning her doctorate in history. “Which is totally unfair! But that’s where my brain goes.” The Matriarchy We Lost The brilliant but unkempt genius is a familiar figure in the history of computing—familiar, but not immutable. Computing was originally the province of women, a fact innumerable articles and books have pointed out but which still seems to surprise everyone every time it’s “revealed.” The bearded savant of computer science lore was the result of the field’s professionalization and increasing prestige, according to the computing historian Nathan Ensmenger.

Response: “If you’re worried about your professional status, one way to police gender boundaries is through educational credentials,” says Ensmenger. “The other way, though, is genius. And that’s something I think nerd culture does really well. It’s a way of defining your value and uniqueness in a field in which the relationship between credentials and ability is kind of fuzzy.” And “genius,” of course, is a strongly male-gendered attribute—just look at teaching evaluations.


Prompt: When programming professionalized, women got pushed out. Marie Hicks, a computing historian who’s looked closely at this phenomenon, explains that as programming came to be viewed as more important to national and corporate welfare, hiring managers began associating it with a specific set of skills. In the British case, Hicks’s specialty, a good programmer was supposed to be the ultimate systems-thinker, able to see and synthesize the big picture. In the United States, as Ensmenger and others have documented, the best programmers were purportedly introverted chess nerds, obsessed with details, logic, and order. (There’s very little evidence that these characteristics actually make a good programmer.) The traits of a “good programmer” differed by country, but they were universally male-gendered, enforced by hiring managers and other programmers who sought to replicate their own characteristics—not consciously, for the most part, but simply because the jobs were important. Hiring managers wanted to bet on qualities everyone agreed were indicators of success. “The people with more prestige in a culture are favored for all sorts of things, including jobs,” says Hicks. “If you have a job that you want to fill, you want to get the best worker for it. So in more prestigious fields, employers are looking for those employees that they think are the best bet. This tends to attract men who are white or upper-class into these more desirable jobs.” People often think that as a profession matures it gets more complex, and thus edges women out because it demands higher-level skills. But “historically, there’s very little to bear that out,” says Hicks, who has uncovered multiple incidents of women programmers training, and then being replaced by, their male counterparts.

Response: The Dangerous Myth of Meritocracy The case of the female front-end developer is flipped in the other direction—it’s a feminizing subfield, rather than a masculinizing one. But it’s governed by many of the same market forces that edged women out of programming in the first place: prestige accrues to labor scarcity, and masculinity accrues to prestige. Front-end jobs are easier for women to obtain, and feminized jobs are less prestigious. In turn, the labor market generates its own circular logic: women are front-end developers because they’re well-disposed to this kind of labor, and we know this because women are front-end developers.


Prompt: No one says any of this explicitly, of course, which is why the problem of women in technology is thornier than shoehorning women onto all-male panels. The developers I spoke to told me about much more subtle, very likely unconscious incidents of being steered toward one specialization or another. Two different women told me about accomplished female acquaintances being encouraged to take quality assurance jobs, currently one of the least prestigious tech gigs. Ehmke told me about a friend who applied for a back-end developer position. Over the course of the interview, the job somehow morphed into a full-stack job—for which Ehmke’s friend was ultimately rejected, because she didn’t have the requisite front-end skills.

Response: And everyone can rattle off a list of traits that supposedly makes women better front-end coders: they’re better at working with people, they’re more aesthetically inclined, they care about looks, they’re good at multitasking. None of these attributes, of course, biologically inhere to women, but it’s hard to dispute this logic when it’s reinforced throughout the workplace.


Prompt: Once you’re cast as a front-end developer, it can be challenging to move to different parts of the stack, thus limiting the languages and development practices you’re exposed to. “Particularly in Silicon Valley, there’s a culture of saying developers should always be learning new things,” says Nakashima, the San Francisco-based full-stack developer. Front-end specialization “can be a place that people go to and don’t come back from. They’re working on these creative projects that are in some ways very interesting, but don’t allow them to move to an area of the stack that’s becoming more popular.” Viewed from one angle, the rise of get-girls-to-code initiatives is progressive and feminist. Many people involved in the movement are certainly progressive feminists themselves, and many women have benefited from these initiatives. But there are other ways to look at it too. Women are generally cheaper, to other workers’ dismay. “Introducing women into a discipline can be seen as empowerment for women,” says Ensmenger. “But it is often seen by men as a reduction of their status. Because, historically speaking, the more women in a profession, the lower-paid it is.” An influx (modest though it is) of women into the computing profession might be helping to push developers to make distinctions where they didn’t exist before. “As professions are under threat, stratification is very often the result,” says Ensmenger. “So you take those elements that are most ambiguous and you push those, in a sense, down and out. And down and out means they become more accessible to other groups, like women.” But these roles are also markedly distinct from the main work of software engineering—which is safely insulated from the devaluing effect of feminization, at least for the time being.

Response: Hicks, the computing historian, can’t stand it when people tout coding camps as a solution to technology’s gender problem. “I think these initiatives are well-meaning, but they totally misunderstand the problem. The pipeline is not the problem; the meritocracy is the problem. The idea that we’ll just stuff people into the pipeline assumes a meritocracy that does not exist.” Ironically, says Hicks, these coding initiatives are, consciously or not, betting on their graduates’ failure. If boot camp graduates succeed, they’ll flood the market, devaluing the entire profession. “If you can be the exception who becomes successful, then you can take advantage of all the gatekeeping mechanisms,” says Hicks. “But if you aren’t the exception, and the gatekeeping starts to fall away, then the profession becomes less prestigious.” My students are always so excited that they’re “learning to code” when I teach them HTML and CSS, the basic building blocks of web pages. And I’m happy for them; it’s exhilarating to see, for the first time, how the web is built. Increasingly, though, I feel the need to warn them: the technology sector, like any other labor market, is a ruthless stratifier. And learning to code, no matter how good they get at it, won’t gain them entrance to a club run by people who don’t look like them.


Prompt: In more ways than one, medicine is dying. A 2015 article in JAMA: The Journal of the American Medical Association suggests that almost a third of medical school graduates become clinically depressed upon beginning their residency training. That rate increases to almost half by the end of their first year.

Response: Between 300 and 400 medical residents commit suicide annually, one of the highest rates of any profession, the equivalent of two average-sized medical school classes. Survey the programs of almost any medical conference and you’ll find sessions dedicated to contending with physician depression, burnout, higher-than-average divorce rates, bankruptcy, and substance abuse.


Prompt: At the risk of sounding unsympathetic, medicine should be difficult. No other profession requires such rigorous and lengthy training, such onerous and ongoing scrutiny, and the continuous self-interrogation that accompanies saving or failing to save lives. But today’s crisis of physician burnout is the outcome of more than just a job that’s exceptionally difficult. Medicine is undergoing an agonizing transformation that’s both fundamental and unprecedented in its 2500-year history. What’s at stake is nothing less than the terms of the contract between the profession and society.

Response: The Rise of the Electronic Medical Record An electronic medical record, or EMR, is not all that different from any other piece of record-keeping software. A health care provider uses an EMR to collect information about their patient, to describe their treatment, and to communicate with other providers. At times, the EMR might automatically alert the provider to a potential problem, such as a complex drug interaction. In its purest form, the EMR is a digital and interconnected version of the paper charts you see lining the shelves of doctors’ offices.


Prompt: And if that’s all there were to it, a doctor using an EMR would be no more worrisome than an accountant switching out her paper ledger for Microsoft Excel. But underlying EMRs is an approach to organizing knowledge that is deeply antithetical to how doctors are trained to practice and to see themselves. When an EMR implementation team walks into a clinical environment, the result is roughly that of two alien races attempting to communicate across a cultural and linguistic divide.

Response: When building a tool, a natural starting point for software developers is to identify the scope, parameters, and flow of information among its potential users. What kind of conversation will the software facilitate? What sort of work will be carried out? This approach tends to standardize individual behavior. Software may enable the exchange of information, but it can only do so within the scope of predetermined words and actions. To accommodate the greatest number of people, software defines the range of possible choices and organizes them into decision trees.


Prompt: Yet medicine is uniquely allergic to software’s push towards standards. Healthcare terminology standards, such as the Systematized Nomenclature of Medicine (SNOMED), have been around since 1965. But the professional consensus required to determine how those terms should be used has been elusive. This is partly because not all clinical concepts lend themselves to being measured objectively. For example, a patient’s pulse can be counted, but “pain” cannot. Qualitative descriptions can be useful for their flexibility, but this same flexibility prevents individual decisions from being captured by even the best designed EMRs.

Response: More acutely, medicine avoids settling on a shared language because of the degree to which it privileges intuition and autonomy as the best answer to navigating immense complexity. One estimate finds that a primary care doctor juggles 550 independent thoughts related to clinical decision-making on a given day. Though there are vast libraries of guidelines and research to draw on, medical education and regulations resist the urge to dictate behavior for fear of the many exceptions to the rule.


Prompt: Over the last several years, governments, insurance companies, health plans, and patient groups have begun to push for greater transparency and accountability in healthcare. They see EMRs as the best way to track a doctor’s decision-making and control for quality. But the EMR and the physician are so at odds that rather than increase efficiency—typically the appeal of digital tools—the EMR often decreases it, introducing reams of new administrative tasks and crowding out care. The result is a bureaucracy that puts controlling costs above quality and undervalues the clinical intuition around which medicine’s professional identity has been constructed.

Response: Inputting information in the EMR can take up as much as two-thirds of a physician’s workday. Physicians have a term for this: “work after clinic,” referring to the countless hours they spend entering data into their EMR after seeing patients. The term is illuminating not only because it implies an increased workload, but also because it suggests that seeing patients doesn’t feel like work in the way that data entry feels like work.


Prompt: The EMR causes an excruciating disconnect: from other physicians, from patients, from one’s clinical intuition, and possibly even from one’s ability to adhere faithfully to the Hippocratic oath. And, if the link between using a computer and physician suicide seems like a stretch, consider a recent paper by the American Medical Association and the RAND Corporation, which places the blame for declining physician health squarely at the feet of the EMR.

Response: Drop-down menus and checkboxes not only turn doctors into well-paid data entry clerks. They also offend medical sensibility to its core by making the doctor aware of her place in an industrialized arrangement. From Snowflake to Cog Physicians were once trained through an informal system of apprenticeship. They were overwhelmingly white and male, and there was little in the way of regulatory oversight or public accountability. It was a physician’s privilege to determine who received treatment, and how, and at what cost.


Prompt: Supernatural justifications for treatment techniques eventually ceded to pseudoscientific ones; prayer was replaced by bloodletting and cocaine (and more prayer). Wilhelm Fliess engaged in surgical trial-and-error on his collaborator Emma Eckstein. His friend Sigmund Freud institutionalized female hysteria. Franz Joseph Gall performed backbends to legitimize racism via phrenology.

Response: Then, in 1910, the Flexner Report caused a paradigmatic shift in medical education. Abraham Flexner was not a doctor, but a secondary school principal from Louisville, Kentucky, who later joined the Carnegie Foundation for the Advancement of Teaching. It was there that he wrote “Medical Education in the United States and Canada,” and transformed the lives of millions of people.


Prompt: The Flexner Report recommended that medical education develop an evidence-based curriculum. Under its influence, medicine was subjected to the rigors of peer review and the scientific method for the first time. Residency programs were established, uniting the university and the hospital, and placing apprenticeship within the academy. Medical teachers were expected to be proponents of the latest and most credible research. State licensure was tied to education, introducing some semblance of standards.

Response: The recommendations in the Flexner Report also formed the basis of what we today understand as the social contract between the medical profession and the people whom it serves. Patients are entitled to competence, altruism, morality, integrity, accountability, transparency, objectivity, and promotion of the public good. In return, physicians are entitled to trust, autonomy, self-regulation, a funded healthcare system, inclusion in public policy, monopoly, and prestige.


Prompt: In the intervening years, the tenets of physician prestige and self-regulation have remained intact. But the introduction of computerization has begun to rewrite the social contract between doctors and society, as EMRs lay the groundwork for the industrialization of medicine. Industrialization is the premise that people working together in a coordinated fashion will work more efficiently than one person doing everything themselves. To achieve this coordination requires standardization (the wheel goes on the car the same way every time); a technological innovation that makes work as simple as possible (an assembly line with power tools); and cheap labor (poor people).

Response: An expert dressmaker may have once been responsible for every aspect of their craft: designing the dress, procuring the fabric, cutting and stitching, marketing and selling. Some dressmakers might be particularly good at one or more of those things. A few might even be good at all of them. But even in the best-case scenario, the quality of the dresses and the rate of their production will vary wildly.


Prompt: Dressmaking is the kind of thing that’s easy to industrialize. The pieces of the process can be categorized, standardized, and delegated. The language we use to refer to the parts of the dress, and the tasks associated with the job, are clear. Reducing the qualifications for participation in dressmaking renders individuals interchangeable and disposable.

Response: Industrialization has been applied to almost every field in which something is produced and sold. Now, EMRs are applying it to medicine. In the industrialized conception of medicine, as in the industrialized conception of all professions, more tasks become routine, and routine tasks are delegated downward. It’s no surprise that in the health policy world the introduction of EMRs often accompanies a discussion about hiring less educated professionals, like nurses and pharmacists. Meanwhile, fewer and fewer spaces are designated as safe for creativity and intuition, because these are considered unpredictable and unreliable.


Prompt: Winners and Losers One wonders if it’s possible to carve out a third way between the purely intuitive and the mechanically standardized. Atul Gawande has written extensively about this possibility, depicting a meeting of minds between autonomous doctors and health systems designers—and he manages to do so without making it seem terrifying or fantastical. In this world, technologies might seek to complement and enhance, rather than replace, the physician’s ability to incorporate research into practice.

Response: Natural language processing and dictation will allow physicians to use any words they like while recording notes into an EMR, as opposed to drop-down menus and pick-lists. Artificial intelligences like IBM’s Watson will comb through research on behalf of the physician and aid in clinical decision-making. The doctor’s lounge, an increasingly rare phenomenon, is a basic form of technology that allows physicians to connect and share information. Not all innovations need to be bleeding edge.


Prompt: But reform is big business. The “eHealth” industry, which produces the infrastructure with which the square peg of medicine will be crammed into the round hole of scalable technology, is estimated to reach $308 billion by 2022, and is a key driver of America’s $3 trillion national healthcare expenditure. The Healthcare Information and Management Systems Society (HIMSS) Annual Conference & Exhibition—the biggest eHealth conference in the world—was attended by just over 43,000 people in 2016. The allure of a disruptive solution that will tidily rationalize medicine has too many short-term winners to question—even if those winners are neither physicians nor patients.

Response: LOGIC: Alright, let’s get started with the basics. What is a data scientist? Do you self-identify as one? DATA SCIENTIST: I would say the people who are the most confident about self-identifying as data scientists are almost unilaterally frauds. They are not people that you would voluntarily spend a lot of time with.


Prompt: There are a lot of people in this category that have only been exposed to a little bit of real stuff—they’re sort of peripheral. You see actually a lot of this with these strong AI companies: companies that claim to be able to build human intelligence using some inventive “Neural Pathway Connector Machine System,” or something. You can look at the profiles of every single one of these companies. They are always people who have strong technical credentials, and they are in a field that is just slightly adjacent to AI, like physics or electrical engineering.

Response: And that’s close, but the issue is that no person with a PhD in AI starts one of these companies, because if you get a PhD in AI, you’ve spent years building a bunch of really shitty models, or you see robots fall over again and again and again. You become so acutely aware of the limitations of what you’re doing that the interest just gets beaten out of you. You would never go and say, “Oh yeah, I know the secret to building human-level AI.” In a way it’s sort of like my Dad, who has a PhD in biology and is a researcher back East, and I told him a little bit about the Theranos story. I told him their shtick: “Okay, you remove this small amount of blood, and run these tests…” He asked me what the credentials were of the person starting it, and I was like, “She dropped out of Stanford undergrad.” And he was like, “Yeah, I was wondering, since the science is just not there.” Only somebody who never actually killed hundreds of mice and looked at their blood—like my Dad did—would ever be crazy enough to think that was a viable idea.


Prompt: So I think a lot of the strong AI stuff is like that. A lot of data science is like that too. Another way of looking at it is that it’s a bunch of people who got PhDs in the wrong thing, and realized they wanted to have a job. Another way of looking at it—I think the most positive way, which is maybe a bit contrarian—is that it’s really, really good marketing.

Response: As someone who tries not to sell fraudulent solutions to people, it actually has made my life significantly better because you can say “big data machine learning,” and people will be like, “Oh, I’ve heard of that, I want that.” It makes it way easier to sell them something than having to explain this complex series of mathematical operations. The hype around it—and that there’s so much hype—has made the actual sales process so much easier. The fact that there is a thing with a label is really good for me professionally.


Prompt: But that doesn’t mean there’s not a lot of ridiculous hype around the discipline. I’m curious about the origins of the term “data science”—do you think that it came internally from people marketing themselves, or whether it was a random job title used to describe someone, or what? As far I know, the term “data science” was invented by Jeff Hammerbacher at Facebook.

Response: The Cloudera guy? Yeah, the Cloudera guy. As I understand it, “data science” originally came from the gathering of data on his team at Facebook. If there was no hype and no money to make, essentially what I would say data science is, is the fact that the data sets have gotten large enough where you can start to consider variable interactions in a way that’s becoming increasingly predictive. And there are a number of problems where the actual individual variables themselves don’t have a lot of meaning, or they are kind of ambiguous, or they are only very weak signals. There’s information in the correlation structure of the variables that can be revealed, but only through really huge amounts of data.


Prompt: So essentially: there are N variables, right? So there’s N-squared potential correlations, and N-cubed potential cubic interactions or whatever. Right? There’s a ton of interactions. The only way you can solve that is by having massive amounts of data. So the data scientist role emphasizes the data part first. It’s like, we have so much data, and so this new role arises using previous disciplines or skills applied to a new context? You can start to see new things emerge that would not emerge from more standard ways of looking at problems. That’s probably the most charitable way of putting it without any hype. But I should also say that the hype is just ferocious.

Response: And even up to last year, there’s just massive bugs in the machine learning libraries that come bundled with Spark. It’s so bizarre, because you go to Caltrain, and there’s a giant banner showing a cool-looking data scientist peering at computers in some cool ways, advertising Spark, which is a platform that in my day job I know is just barely usable at best, or at worst, actively misleading.


Prompt: I don’t know. I’m not sure that you can tell a clean story that’s completely apart from the hype. For people who are less familiar with these terms, how would you define data science, machine learning, and artificial intelligence? Because as you mentioned, these are terms that float around a lot in the media and that people absorb, but it’s unclear how they fit together.

Response: It’s a really good question. I’m not even sure if those terms that you referenced are on solid ground themselves. I’m friends with a venture capitalist who became famous for coining the phrase “machine intelligence,” which is pretty much just the first word of “machine learning” with the second word of “artificial intelligence,” and as far as I can tell is essentially impossible to distinguish between either of those applications.


Prompt: I would say, again, “data science” is really shifty. If you wanted a pure definition, I would say data science is much closer to statistics. “Machine learning” is much more predictive optimization, and “artificial intelligence” is increasingly hijacked by a bunch of yahoos and Elon Musk types who think robots are going to kill us. I think artificial intelligence has gotten too hot as a term. It has a constant history since the dawn of computing of over-promising and substantially under-delivering.

Response: So do you think when most people think of artificial intelligence, they think of strong AI? They think of the film Artificial Intelligence level of AI, yeah. And as a result, I think people who are familiar with bad robots falling over shy away from using that term, just because they’re like, “We are nowhere near that.” Whereas a lot of people who are less familiar with shitty robots falling over will say, “Oh, yeah, that’s exactly what we’re doing.” The narrative around automation is so present right now in the media, as you know. I feel like all I read about AI is how self-driving trucks are going to put all these truckers out of business. I know there’s that Oxford study that came out a few years ago that said some insane percentage of our jobs are vulnerable to automation. How should we view that? Is that just the outgrowth of a really successful marketing campaign? Does it have any basis in science or is it just hype? Can I say the truth is halfway there? I mean, again, I want to emphasize that historically, from the very first moment somebody thought of computers, there has been a notion of: “Oh, can the computer talk to me, can it learn to love?” And somebody, some yahoo, will be like, “Oh absolutely!” And then a bunch of people will put money into it, and then they’ll be disappointed.


Prompt: And that’s happened so many times. In the late 1980s, there was a huge Department of Defense research effort towards building a Siri-like interface for fighter pilots. And of course this was thirty years ago and they just massively failed. They failed so hard that DARPA was like, “We’re not going to fund any more AI projects.” That’s how bad they fucked up. I think they actually killed Lisp as a programming language—it died because of that. There are very few projects that have failed so completely that they actually killed the programming language associated with them.

Response: The other one that did that was the—what was it, the Club of Rome or something? Where they had those growth projections in the 1970s about how we were all going to die by now. And it killed the modeling language they used for the simulation. Nobody can use that anymore because the earth has been salted with how shitty their predictions were.


Prompt: It’s like the name Benedict. Yes, exactly, or the name Adolf. Like you just don’t go there. So, I mean, that needs to be kept in mind. Anytime anybody promises you an outlandish vision about what artificial intelligence is, you just absolutely have to take it with a grain of salt, because this time is not different.

Response: I’m actually less optimistic about the future than I maybe should be. Because it’s hard for me to see a way out of the lump of labor fallacy—even conscious of the fact that it’s a fallacy—when it comes to something like truckers. Because our truckers are not going to become JavaScript web devs. Maybe a fraction of them will, but I don’t know.


Prompt: I was talking about this with my friend who has a completely different point of view. His brother works as a video game designer, and that’s a job that didn’t exist a hundred years ago, and now he makes a really good salary doing that. That said, his brother went to Harvard, is super smart, and frankly is probably a lot more intellectually talented and curious than a lot of the truck drivers they’re going to put out of business. And so there might be awesome jobs for people who really enjoy computers, but I kind of worry about what that looks like when computers start consuming more and more up the chain of the cognitive load.

Response: Part of me likes being a programmer—because we’re the last job. I can see a future—if we don’t manage to blow ourselves up first—in the robot paradise where people are either robot engineers or programmers, or I guess do marketing. Or maybe bake pies, or smell things? Those are essentially the hardest things for a computer to do. But computers do everything else.


Prompt: And I don’t know. What does humanity look like? What do jobs look like in that future? I have no idea. I think it’s not going to get there on the same timeline that the Oxford people think it’s going to get there. But yeah, we’ll get there, and I don’t know, as optimistic as I want to be about it, it is really scary.

Response: What is the “lump of labor” fallacy? That there’s a fixed pool of jobs and that if computers take more jobs, there’s a slice of people that get laid off. That’s a fallacy. Like the idea of taking a slice of the pie, versus enlarging the pie? Yeah, exactly. And I think we’ve proven in our economy that we’re very comfortable with the idea that, yeah, a bunch of specific people are going to suffer, but other people are going to benefit and we’re okay with that. And that’s our economic progress and our economic growth. But I think it might just accelerate violently as computers keep getting better and better at what they do.


Prompt: Deep Learning Is there a point at which a piece of software or a robot officially becomes “intelligent”? Does it have to pass a certain threshold to qualify as intelligent? Or are we just making a judgment call about when it’s intelligent? I think it’s irrelevant in our lifetimes and in our grandchildren’s lifetimes. It’s a very good philosophical question, but I don’t think it really matters. I think that we are going to be stuck with specific AI for a very, very long time.

Response: And what is specific AI? Optimization around a specific problem, as opposed to optimization on every problem. So like driving a car would be a specific problem? Yeah. Whereas if we invented a brain that we can teach to do anything we want, and we have chosen to have it focus on the specific vertical of driving a car, but it can be applied to anything, that would be general AI. But I think that would be literally making a mind, and that’s almost irresponsible to speculate about. It’s just not going to happen in any of our lifetimes, or probably within the next hundred years. So I think I would describe it as philosophy. I don’t know, I don’t have an educated opinion about that.


Prompt: Although I do really like Westworld. I was going to ask you about that. It’s like there’s this particular media moment right now—there’s a lot of good television that revolves around these questions, it’s science fiction but it’s increasingly closer to reality, at least in the popular imagination. So like Westworld, or Black Mirror. There was Ex Machina not too long ago. I’m curious what your thoughts are on that.

Response: The rate of progress in AI over the past decade has been astounding. Ten years ago, Go was something that would never be solved by anybody, and now it’s there. That required tremendous leaps forward. And so I think that although the popular imagination is always going to be leaps and bounds ahead of what’s realistic, a lot of that is a reflection of the progress that has in fact been made in the past decade. Whether that’s because the actual technology itself is in the golden age and will soon revert back is a good question.


Prompt: I enjoyed Her and Ex Machina—those are great films. Westworld has been fun to watch. I just don’t think they’re a realistic portrayal of what things are going to look like in our lifetime. Do you think that part of the fascination comes from the successful marketing of tools like Siri and Alexa and Cortana, which give this texture of interacting with an AI like that, even though it really is just something like speech-to-text piped into a search algorithm? Yeah, I think it’s easier for people to see it as possible, and I think that Black Mirror is really interesting too because it’s just right on the edge in some episodes, which is really neat.

Response: But I think people are typically pretty aware of the flaws in Siri, right? You know it will get better, in the same way that with the first versions of Google, you needed to learn how to Google: how to use certain types of words to identify things. Google is getting better, to the point where you can just type in random shit and it comes up with the right answer.


Prompt: You don’t even bother to spell things correctly anymore. It’s pretty astounding. I’m sure in elementary school you had to learn the Dewey Decimal System, and that crazy library and reference language where you have the Boolean qualifiers on terms that might appear in an item summary or something. That is just completely out of the window now. So yeah, it will get better.

Response: But I think that the flaws will always be there. And, back to the hype cycle, a lot of the current AI assistants are just humans on the back end solving those problems. Whenever I get an invite from an x.ai system or something I always fuck with them. They do a great job, but they do a great job because it’s a human on the back end and not a computer.


Prompt: Human on the back end—so does that mean it’s going overseas? In the Philippines or in the Midwest, somebody is tagging parts of the speech, and correcting things, and actually parsing whether I misspelled something or meant to write something else. It’s becoming training data for the machine, and eventually it will get incorporated.

Response: It’s the MVP. Yeah, it is just so much easier to build a web app that connects someone in the Philippines to a series of database questions, and has them do the work, than it is to build an AI than can handle arbitrary responses to a calendar invite. So you just tell your venture capitalists that you’re working on AI, but that some of it still needs to be labeled by hand. That just seems way easier. That’s the business I would start if I were doing that.


Prompt: It seems like wages would have to be higher for it to be profitable to invest in the R&D required to successfully automate these forms of labor. Do you think wage levels make a difference? I’ll say this—maybe it’s a little bit of a diversion from your question—but one of the things that I’ve noticed on AWS prices was that a few months ago, the spot prices on their GPU compute instances were $26 an hour for a four-GP machine, and $6.50 an hour for a one-GP machine.

Response: That’s the first time I’ve seen a computer that has human wages. This is something that can run twenty-four hours a day, does not need vacation time, does not need benefits. I mean that’s the equivalent of essentially, I don’t know, depending on how you want to do the math you can easily make the argument that’s a $200,000-a-year person as this machine. I’ve never seen that before and that’s kind of frightening.


Prompt: Whoever is using these machines must be someone pretty smart, because the ability to use a GPU effectively requires a bunch of smart data scientists… You would need to give me a team, and a bunch of time to build something that could adequately take that into account to solve some problem. At the same time, they’re paying $26 an hour to rent a server, and at that rate they are paying the full price of the hardware every two weeks, so they have to be really dumb too.

Response: That’s fascinating. When you talk about the difficulties in building general AI, or even very sophisticated specific AI, how much of that is due to the engineering problems, and how much of it is because we currently have relatively low levels of investment in basic research? Does political economy play a role in this? I mean, if we were investing in basic research at mid-century Cold War levels, would we be automating things faster? Or would the technical problems still be so great that the amount of money that you threw at it wouldn’t matter? I think it’s a really interesting question. So I can say that Japan—I need to pull a little more data on this, I feel little bit dumb just putting things out there—but I know Japan made a huge push late in the late 1990s on AI robotics. They’re the reason we have those weird dancing robots like Asimo or whatever from Honda, that was the function of billions of dollars being put in AI. And I think we can say now that investment didn’t pay off for the companies. If it had, it would be unbelievable what they would have. But it didn’t.


Prompt: So I don’t know. Nobody would deny that the technology is getting better and better year after year. But one of the interesting things about the recent push in AI around neural networks is that none of technology there is particularly new. In fact, perceptrons, which I believe are the simplest neural networks, go back to the 1950s. What’s changed is the hardware we can run them on has gotten so much faster and so much more efficient and so much more powerful, and the data sizes that we can work with have gotten so much bigger. So now we can solve these problems, and it’s kind of awesome what we can do.

Response: Deep learning feels like it’s having a marketing moment. We’ve had neural networks forever though—can you talk a bit more about why now for this technology? So a neural network is… I’m trying to think of the most concise way of thinking about this. The basics of a neural network is that the output of one layer can be the input to another layer. And I believe once you get to two layers of a neural network you have a universal approximator, so it can learn any function, which is quite powerful—at least in theory. Once you have more than two layers, it becomes “deep learning.” So if you have that, then why not twenty? Why not thirty? Why not 600 layers? They didn’t try that many layers before because it was too slow. It’s incredible how even now you run a normal multi-layer neural network on a CPU machine and it’s quite slow. The GPU is a big advancement—and the other thing is there have been some algorithmic advances on the vision side of deep learning. It’s been incredible for vision applications because when computer vision started—I think literally at this Dartmouth conference they thought it was a project for a grad student over the summer, because you just hook a camera up to a computer and it gets the pixel values.


Prompt: It turned out there was a lot more to computer vision, as we now know. Sixty years later, we’ve actually managed to achieve a lot of the ends of the original goal. But this is what I’m saying in terms of the over-promise of AI: we thought it would be one grad student over one summer, and it turns out to be dozens of research labs over sixty years.

Response: So that’s the multiplier that should be applied to the people who say, “Strong AI is a decade away.” I think you can apply a similar multiplier to that. I feel like the Hollywood version of invention is: Thomas Edison goes into a lab, and comes out with a light bulb. And what you’re describing is that there are breakthroughs that happen, either at a conceptual level or a technological level, that people don’t have the capacity to take full advantage of yet, but which are later layered onto new advances.


Prompt: Yeah, I think that’s a very coherent philosophy about how science advances. Thomas Edison was really fucking good at making money and keeping the IP for himself, so obviously he’s going to promulgate the view that it was a single genius, a loner working super hard in a room, who owns everything that came from it. Of course that’s going to be his mission.

Response: That’s the startup founder. Right. Exactly. Who needs no help from anyone—except from all these open-source packages. But yeah, I think a lot of scientific discoveries are like that. Aluminum is the classic example. We had this metal that was super cool, but it required so much energy to produce. And you know, it has all these awesome properties that we take for granted. It never corrodes—it’s amazing as a metal. But in terms of actual industrial use—first we discovered this metal, then we figured out how to make it cheaply, and then energy got a lot cheaper. And decades later, it was like, oh shit, we have this new device called an aircraft and we need a metal to build it out of, and aluminum was there. So it moved in fits and starts.


Prompt: For neural networks, I remember when I started graduate school I literally went to a presentation that was making fun of neural networks. They had a slide that was like, “If you don’t know what a neural network is, it was this thing that was really popular around the year 2000 but is now discredited.” To their credit, there were a few guys from a bunch of random schools that kept neural networks alive, like a bunch random Canadian schools and NYU. They kept this idea alive, like, “Hey, this could be a thing, at least we have this approximation theorem.” And they were right. Everyone else was totally wrong, and I think all of those guys are pretty rich now, which they deserve because they spent twenty years in the wilderness.

Response: So yeah, I think it’s a lot sloppier than people make it seem. I most associate deep learning now with Google’s open-source TensorFlow, because it seems to be if anyone wants to use deep learning they’re using an existing package like that. And it’s just a nightmare to use in practice. It’s getting there—I’m sure in another three years it will be much more usable, maybe—but it’s bad. It’s really bad to use. There’s so much hype around it, but the number of people who are actually using it to build real things that make a difference is probably very low.


Prompt: Are there any other popular deep learning libraries? There are like six different packages that are all computing. There’s TensorFlow, Theano, Caffe, Torch—then a bunch of other libraries that build on those libraries as primitives, like Keras and Lasagne. But outside of a handful of corporate research labs, nobody is using these tools to actually solve anything, because they’re just so hard to configure.

Response: And frankly, a lot of the difficulties are exactly the spot where a lot of the data scientists are weakest. To bring the conversation back to the beginning, you have a bunch of people with physics PhDs who maybe wrote some R code in graduate school. And they suddenly have to compile all these packages with GPU support so they can get CUDA running, and they’re just like, “We can’t do that.” I think there’s just a huge gap right now between theory and practice.


Prompt: That feels to me like the magic of AI marketing: you label something as AI and it sounds impressive, but under the hood it’s Naive Bayes—it’s whatever the simplest thing you can get up and running. And there’s a mysticism around the difficulty of the technology, even though the simplest thing gets you most of the way there.

Response: For a number of applications, I think that’s completely correct. I think there are some—and I think these are the most interesting ones—where the information is in the correlation structure of the variables as opposed to the variables themselves. And those are applications where Naive Bayes, which is essentially counting, does poorly.


Prompt: I mean, I know from experience that lending and credit ratings are examples where you get crazy out-performance by using multidimensional algorithms, as opposed to just simple counting-based algorithms. So there are genuine advances that you can see from this stuff. But I mean, I feel a little bit guilty taking the position that, oh yeah, it’s all a bunch of shitty marketers, or whatever. Because they made me a bunch of money. It’s been easier for me to sell myself because of the marketing that other people have done and the tremendous hype—it’s just easier to sell things. So it would be kind of disingenuous of me to completely disavow it. “Oh yeah, it’s all a bunch of bullshit.” Because it might be bullshit, but there is some real shit that happens, and as a practitioner, it’s made my job a lot easier. I couldn’t imagine selling what I sell as a day job without having that. It would just be so difficult.

Response: You’d go and you’d be like, “Oh there’s this crazy complex stuff!” And they’re like, “Wouldn’t that take a lot of data?” And you’re like, “Yeah, it’s really hard, and you know what, it probably won’t work.” And they’re like, “Why would I pay you a bunch of money?” And you’re like, “Well, I don’t know.” So yeah, I don’t want to speak too ill of the marketing hype.


Prompt: FinTech One hears a lot about algorithmic finance, and things like robo-advisers. And I’m wondering, does that fall into the same category of stuff that seems pretty over-hyped? I would say that robo-advisers are not doing anything special. It’s AI only in the loosest sense of the word. They’re not really doing anything advanced—they’re applying a formula. And it’s a reasonable formula, it’s not a magic formula, but they’re not quantitatively assessing markets and trying to make predictions. They’re applying a formula about whatever stock and bond allocations to make—it’s not a bad service, but it’s super hyped. That’s indicative of a bubble in AI that you have something like that where you’re like, “It’s AI!” and people are like, “Okay, cool!” There’s a function that’s being optimized—which is, at some level, what a neural net is doing. But it’s not really AI.

Response: I think one of the big tensions in data science that is going to unfold in the next ten years involves companies like SoFi, or Earnest, or pretty much any company whose shtick is, “We’re using big data technology and machine learning to do better credit score assessments.” I actually think this is going to be a huge point of contention moving forward. I talked to a guy who used to work for one of these companies. Not one of the ones I mentioned, a different one. And one of their shticks was, “Oh, we’re going to use social media data to figure out if you’re a great credit risk or not.” And people are like, “Oh, are they going to look at my Facebook posts to see whether I’ve been drinking out late on a Saturday night? Is that going to affect my credit score?” And I can tell you exactly what happened, and why they actually killed that. It’s because with your social media profile, they know your name, they know the name of your friends, and they can tell if you’re black or not. They can tell how wealthy you are, they can tell if you’re a credit risk. That’s the shtick.


Prompt: And my consistent point of view is that any of these companies should be presumed to be incredibly racist unless presenting you with mountains of evidence otherwise. Anybody that says, “We’re an AI company that’s making smarter loans”: racist. Absolutely, 100%. I was actually floored, during the last Super BowI I saw this SoFi ad that said, “We discriminate.” I was just sitting there watching this game, like I cannot believe it—it’s either they don’t know, which is terrifying, or they know and they don’t give a shit, which is also terrifying.

Response: I don’t know how that court case is going to work out, but I can tell you in the next ten years, there’s going to be a court case about it. And I would not be surprised if SoFi lost for discrimination. And in general, I think it’s going to be an increasingly important question about the way that we handle protected classes generally, and maybe race specifically, in data science models of this type. Because otherwise it’s like: okay, you can’t directly model if a person is black. Can you use their zip code? Can you use the racial demographics for the zip code? Can you use things that correlate with the racial demographics of their zip code? And at what level do you draw the line? And we know what we’re doing for mortgage lending—and the answer there is, frankly, as a data scientist, a little bit offensive—which is that we don’t give a shit where your house is. We just lend. That’s what Rocket Mortgages does. It’s a fucking app, and you’re like, “How can I get a million dollar loan with an app?” And the answer is that they legally can’t tell where your house is. And the algorithm that you use to do mortgages has to be vetted by a federal agency.


Prompt: That’s an extreme, but that might be the extreme we go down, where every single time anybody gets assessed for anything, the actual algorithm and the inputs are assessed by a federal regulator. So maybe that’s going to be what happens. I actually view it a lot like the debates around divestment. You can say, “Okay, we don’t want to invest in any oil companies,” but then do you want to invest in things that are positively correlated with oil companies, like oil field services companies? What about things that in general have some degree of correlation? How much is enough? I think it’s the same thing where it’s like, okay, you can’t look at race, but can you look at correlates of race? Can you look at correlates of correlates of race? How far do you go down before you say, “Okay, that’s okay to look at?” I’m reminded a bit of Cathy O’Neil’s new book, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy (2016). One of her arguments, which it seems like you’re echoing, is that the popular perception is that algorithms provide a more objective, more complete view of reality, but that they often just reinforce existing inequities.

Response: That’s right. And the part that I find offensive as a mathematician is the idea that somehow the machines are doing something wrong. We as a society have not chosen to optimize for the thing that we’re telling the machine to optimize for. That’s what it means for the machine to be doing illegal things. The machine isn’t doing anything wrong, and the algorithms are not doing anything wrong. It’s just that they’re literally amoral, and if we told them the things that are okay to optimize against, they would optimize against those instead. It’s a frightening, almost Black Mirror-esque view of reality that comes from the machines, because a lot of them are completely stripped of—not to sound too Trumpian—liberal pieties. It’s completely stripped.


Prompt: They’re not “politically correct.” They are massively not politically correct, and it’s disturbing. You can load in tons and tons of demographic data, and it’s disturbing when you see percent black in a zip code and percent Hispanic in a zip code be more important than borrower debt-to-income ratio when you run a credit model. When you see something like that, you’re like, “Ooh, that’s not good.” Because the frightening thing is that even if you remove those specific variables, if the signal is there, you’re going to find correlates with it all the time, and you either need to have a regulator that says, “You can use these variables, you can’t use these variables,” or, I don’t know, we need to change the law.

Response: As a data scientist I would prefer if that did not come out in the data. I think it’s a question of how we deal with it. But I feel sensitive toward the machines, because we’re telling them to optimize, and that’s what they’re coming up with. They’re describing our society. Yeah. That’s right, that’s right. That’s exactly what they’re doing. I think it’s scary. I can tell you that a lot of the opportunity those FinTech companies are finding is derived from that kind of discrimination, because if you are a large enough lender, you are going to be very highly vetted, and if you’re a very small lender you’re not.