# The Intelligent Social Web

**Epistemic status:** [Fake Framework](http://lesswrong.com/lw/p80/in_praise_of_fake_frameworks/)

  

When you walk into an [improv](https://en.wikipedia.org/wiki/Improvisational_theatre) scene, you usually have no idea what role you’re playing. All you have is some initial prompt — something like:

> _“You three are in a garden. The scene has to involve a stuffed bear somehow. Go!”_

So now you’re looking to the other people there. Then someone jumps forward and adds to the scene: “Oh, _there_ it is! I’m glad we finally found it!” Now you know a _little_ bit about your character, and about the character of the person who spoke, but not enough to fully define anyone’s role.

You can then expand the scene by adding something: “It’s about time! We’re almost late now.” Now you’ve specified more about what’s going on, who you are, and who the other players are. But it’s still the case that _none of you knows what’s going on_.

In fact, if you _think_ you know, you’ll often quickly be proven wrong. Maybe you imagine in that scene you’re an uptight punctual person. And then the third person in the scene says to you, “What do you care, Alex? You’re always late to everything anyway!” Surprise! Now you need to flush who you thought you were from your mind, accept the new frame, and run with it as part of your newly evolving identity. Otherwise the scene sort of crashes.

It would go more smoothly if you didn’t hold any preconceptions about who you are or what’s going on. The scene tends to work better if you stay in the present moment and just jump in with the first thing that comes to mind (as long as [it’s shaped by what has happened so far](https://en.wikipedia.org/wiki/Yes,_and...)). Then the collection of interactions and emerging roles spontaneously guides your behavior, which in turn help guide others’ behavior, all of which recursively defines the “who” and “what” of the scene. Your job as a player isn’t to play a character; it’s to _co-create a scene_.

We can sort of pretend that there’s a “director”: it’s the intelligence that emerges between the players via their interactions. It’s a [distributed system](https://en.wikipedia.org/wiki/Distributed_computing) that computes relationships and context by guiding each node in its network to [act freely within constraints](https://en.wikipedia.org/wiki/Yes,_and...). From this vantage point, the network guides players, and the job of each player is to be guidable but not purely passive (since a passive node is just relaying information rather than aiding in the computation). As long as everyone involved is plugged into and responsive to this network, the scene will usually play out well.

* * *

I suspect that improv works because we’re doing something a lot like it pretty much all the time. The web of social relationships we’re embedded in helps define our roles as it forms and includes us. And that same web, as the distributed “director” of the “scene”, guides us in what we do.

A lot of (but not all) people get a strong hit of this when they go back to visit their family. If you move away and then make new friends and sort of become a new person (!), you might at first think this is just who you are now. But then you visit your parents… and suddenly you feel and act a lot like you did before you moved away. You might even try to hold onto this “new you” with them… and they might respond to what they see as [strange behavior](https://en.wikipedia.org/wiki/Deviance_(sociology)) by trying to nudge you into acting “normal”: ignoring surprising things you say, changing the topic to something familiar, starting an old fight, etc.

In most cases, I don’t think this is malice. It’s just that they _need the scene to work_. They don’t know how to interact with this “new you”, so they tug on their connection with you to pull you back into a role they recognize. If that fails, then they have to redefine who _they_ are in relation to _you_ — which often (but not always) happens eventually.

I’m basically taking as an axiom of this framework that people need the “scene” to work — which is to say, they need to be able to play out their roles in relation to others’ roles within a coherent context. I don’t think _why_ this is the case is relevant for using this framework… but I’ll wave my hands at a vague [just-so story](https://www.lesserwrong.com/posts/9AoDAfwv3mEzZ8zeb/just-so-stories) anyway for the sake of [pumping intuition](https://en.wikipedia.org/wiki/Intuition_pump): human beings’ main survival strategy seems to be based on coordinating in often complex ways in tribes. For the individual, this means that _fitting in_ becomes paramount. For the group, this means _knowing what to expect from each person_ is critical. So a trade becomes possible: the individual can fit into and benefit from the group as long as they’re playing a role that fits well with the collective.

This can result in some pretty strange roles. From this vantage point, a person who repeatedly leaves one abusive relationship only to get into another roughly similar one actually makes a lot of sense: this is a role that this person knows how to play. It’s _horrible_, but it’s still better than not fitting into the social scene. It creates a [coherent relationship](https://en.wikipedia.org/wiki/Karpman_drama_triangle) with someone who’s willing to (or has to) play an “abuser” role, and often with people in “rescuer” roles too. The trap they’re in isn’t (just) that their current abusive partner is [gaslighting](https://en.wikipedia.org/wiki/Gaslighting) or threatening them; it’s that they _don’t have another role they can see how to play_. Unless and until that person finds a different one that fits into the social web, the strands of that web will tug them back into their old role. They don’t have enough [slack in the web around them](https://www.lesserwrong.com/posts/yLLkWMDbC9ZNKbjDG/slack) to change their fate.

The same kind of web/slack dynamics show up in more pleasant-to-play roles too. The [privilege](https://everydayfeminism.com/2014/09/what-is-privilege/) of a middle-class American [white](https://en.wikipedia.org/wiki/White_privilege) [man](https://en.wikipedia.org/wiki/Male_privilege) by default has him playing out some kind of roughly known story-like path (probably involving college and having kids and maybe a divorce) that, in the end, will probably still leave him being [one of the richest people on Earth](https://www.givingwhatwecan.org/get-involved/how-rich-am-i/). And all the while, he might well have no clue that he has other options or even [that he’s on a path](http://sjwiki.org/wiki/Privilege_blindness) — but he’ll still know, somehow, not to step off that path (“I _have to_ go to college; are you _crazy_?”). Never mind that his lack of slack here is [awfully convenient for him](http://lesswrong.com/lw/km/motivated_stopping_and_motivated_continuation/).

I’ve watched religious conversions and deconversions happen via basically the same mechanism. I knew a fellow many years ago (unattached to this community) who was a proud atheist. Then he started dating a Christian girl. Something like a month later, he started quoting the Bible — but “only because they’re handy metaphors” and not because he really _believed_ any of that stuff, you see. It later turned out he’d been going to church with her. He kept offering reasons that seemed _vaguely plausible_ (“It’s a neat group of people, and it matters to her, and I can take the time to read”), but there’s a _pattern_ here that was obvious. A few months later he told me he’d converted. Last I heard they had moved to Utah.

The great part is, I knew this was going to happen when they started dating. Why? Because when I warned him that he might [find himself wanting to believe her religion](http://lesswrong.com/lw/i7/belief_as_attire/) once they started having sex, his reaction was to reassure me by _acting confident that he was immune to this_. That meant he was more focused on managing my perception of him than he was in noticing how the social web was tugging him toward a transition of roles. I didn’t know if they’d stay together, but I was pretty sure that if they did, he’d convert.

I could give literally hundreds of examples like this. From where I’m standing, it looks like one of the great challenges of rationality is that people change their minds about meaningful things mostly only when the web tugs them into a new role. _Actually thinking_ in a way that [for real changes your mind](https://wiki.lesswrong.com/wiki/How_To_Actually_Change_Your_Mind) in ways that defy your web-given role is [socially deviant](http://lesswrong.com/lw/m9/aschs_conformity_experiment/), and therefore [personally dangerous](http://mindingourway.com/newcomblike-problems-are-the-norm/), and therefore something you’re [motivated](http://lesswrong.com/lw/km/motivated_stopping_and_motivated_continuation/) not to learn how to do.

Ah, but if we’re immersed in a culture where status and belonging are tied to changing our minds, and we can [signal](https://wiki.lesswrong.com/wiki/Signaling) that we’re open to updating our beliefs, then we’re good… as long as we know [Goodhart’s Demon](https://en.wikipedia.org/wiki/Goodhart%27s_law) isn’t lurking in the shadows of our minds here. But surely it’s okay, right? After all, we’re smart and we know Bayesian math, and we _care_ about truth! [What could possibly go wrong?](http://lesswrong.com/lw/he/knowing_about_biases_can_hurt_people/)

Another challenge here is that the part of us that feels like it’s thinking and talking is (usually) analogous to a character in an improv scene. The _players_ know they’re in a scene, but the _characters_ they’re playing don’t. The characters also aren’t surprised about who or what they are: the not-knowing of identity and context is something only the players experience, to open themselves up to the guidance of the distributed “director”. This means that (a) the characters are actively wrong about why they do what they do, (b) they are deeply confused about how much sense everything makes, and (c) _[they don’t know they’re confused](http://lesswrong.com/lw/if/your_strength_as_a_rationalist/)_.

I claim that most of us, most of the time, are playing out characters as defined by the surrounding web — and we usually haven’t a clue how to [Look](https://www.lesserwrong.com/posts/tMhEv28KJYWsu6Wdo/kensh) at this fact, much less intentionally use our web slack to change our stories.

I think this is also part of why improv is challenging: you have to set aside the character you would normally play in order to create room for something new.

* * *

The web as a whole wants to know what kind of role you’re playing, and how well you’re going to play it, so that it can know what to expect of you. So, a lot of its distributed resources go into computing a model of you.

One of the more obvious transmission methods is chat — idle gossip, storytelling, speculation, small talk. People sync up their impressions of someone they’ve met, and try to make sense of surprising events in conversation. If a lover brings their partner some flowers and the recipient freaks out and runs off, suddenly there’s a _need to understand_, and the flower-giver might try asking a mutual friend for some help understanding. And even if they do come to understand (“Oh, that’s because their last partner brought them flowers to break up with them”), there’s often an impulse to _share the story with friends_, so that the web as a whole can hold everyone in sensible roles and make the scene work. (“Oh, we had a funny misunderstanding earlier, poor Sam….”)

A lot of this is transmitted more subtly too, in body language and facial expressions and vocal tone and so on. If Bob is “creepy” (i.e., is playing a “creepy” role in the web), then it speaks volumes if everyone who meets Bob then cringes just a tiny bit when he’s later mentioned even if they say only good things about him. This means that someone who has _never met Bob_ can get a “vibe” about him from multiple people in a way that shapes how they interpret what Bob says and does when they finally _do_ meet him.

Sometimes, some people with enough web-savvy weaponize this. It doesn’t _mean_ anything for someone to “be creepy” except that they have a web-like impact on others — which is to say, they have a “creepy” role. In a healthy network, this correlates with something actually meaningfully bad that’s worth tracking. But because [perceived roles shape what people expect of a person](https://en.wikipedia.org/wiki/Confirmation_bias), it’s enough for a _rumor_ to echo through the web in order for someone to be _interpreted_ as “creepy”. So a sufficiently cunning person could actually cause someone to be slowly isolated and distrusted without there being any facts at all to justify this as the social web's stance.

(And yes, I’ve seen this happen. Many times.)

The same kind of thing can happen with “positive” labels, too. What it _means_ for someone to be fit for a leadership role, in the social web's eyes, is that they are seen as compatible with that role. So if someone is tall, attractive, and either vicious or strong depending on how you choose to see it, it might be enough to have the “strong” interpretation echo more powerfully than the “vicious” one in order for the web to [conspire](http://sjwiki.org/wiki/Privilege) to put them in a leadership position.

…which means that even people who are seen as good leaders might not, in fact, be good leaders in the sense of _making good leadership decisions_. But they are by _definition_ good leaders in the sense of _playing the role well_. After all, if the general consensus is that Abraham Lincoln was a great President, then there’s a sense in which _that makes it true_, since that’s what “great” _means_ here. The “explanations” thereafter are often stories to [justify](http://lesswrong.com/lw/kq/fake_justification/) one’s holding of a [popular opinion](http://lesswrong.com/lw/i7/belief_as_attire/).

The same thing holds for when someone seems “rational”. This is one reason to worry deeply when members of subgroups internally agree with each other on who is a top-notch clear thinker or “really a rationalist” but disagree with people in other subgroups. This looks less to me like people seeking truth, and a lot more like groups engaging in a subtle memetic battle over what “rational” gets to mean.

[From where I’m standing](https://www.lesserwrong.com/posts/tMhEv28KJYWsu6Wdo/kensh), it looks to me like we’re all immersed in not-knowing, while our “characters” keep talking as though they know what’s going on, implicitly following some hidden-to-them script.

* * *

The web encodes a lot of its guidance about what we should expect and how to behave via the structure of stories. Or rather, story structures are what expectations about roles and scenes _are_.

The trouble is, a lot of the stories we _talk about_ have the structure of what our characters are [supposed to say](http://lesswrong.com/lw/i6/professing_and_cheering/) rather than of what actually happens. Imagine a [movie](https://screencraft.org/2013/09/13/hollywood-movies-same/) where the new kid at a school gets bullied by the popular kids and then makes friends with quirky outcasts. What happens to the bullies in the end? In real life, bullies often don’t get their comeuppance — but _having this fictional story in our hearts_ lets us play out vivid indignation through our characters in the real-life version. Because the bullies aren’t _supposed_ to get away with it, right? That wouldn’t be _[fair](https://en.wikipedia.org/wiki/Just-world_hypothesis)!_

Some parts of our story-like intuitions are scripts for what should actually happen. Some are things our scripts say we should think or feel or talk about within our given roles. Some are merely incidental details. Sussing out which parts are which is part of the trick of getting this framework to work for you.

For instance, the stereotypical story of the worried nagging wife confronting the emotionally distant husband as he comes home really late from work… is actually a pretty good caricature of [a script that lots of couples play out](https://www.goodtherapy.org/blog/ending-anxious-avoidant-dance-part-1-opposing-attachment-styles-0518174), as long as you know to ignore the gender and class assumptions embedded in it.

But it’s _hard_ to sort this out without just enacting our scripts. The version of you that would be thinking about it is your _character_, which (in this framework) can accurately understand its own role only if it has enough slack to become [genre-savvy](http://tvtropes.org/pmwiki/pmwiki.php/Main/GenreSavvy) _within the web_; otherwise it just keeps playing out its role.

In the husband/wife script mentioned above, there’s a tendency for the “[wife](https://aloftyexistence.wordpress.com/2011/03/23/your-partner-might-be-anxious-if/)” to get excited when “she” learns about the relationship script, because it looks to “her” like it suggests how to _save the relationship_ — which is “her” enacting “her” role. This often aggravates the fears of the “[husband](https://aloftyexistence.wordpress.com/2011/03/02/top-ten-signs-your-partner-is-avoidant/)”, causing “him” to pull away and act dismissive of the script’s relevance (which is “his” role), driving “her” to insist that they just need to _talk_ about this… which is the same pattern they were in before. They _try_ to become genre-savvy, but there (usually) just isn’t enough slack between them. So their effort merely changes the topic while they play out their usual scene.

So if you don't like the story you're in, how do you _really_ change it?

Well, it depends on which "you" is asking the question.

Characters often want change _as part of their role_. And just as importantly, their role often requires that they _can't achieve_ that change. The tension between craving and deprivation gives birth to the character's dramatic _raison d'être_. The "wife" can't be as clingy and anxious if the "husband" opens up, so "she" enacts behavior that "she" _knows_ will make "him" close down. "She" can't really choose to change this because "her" thwarted desire for change _is part of "her" role_.

Intentionally creating real personal change requires the _player_ to decide to shake things up. Characters avoid understanding this clearly for basically the same reason that most works of fiction avoid [breaking the fourth wall](https://www.urbandictionary.com/define.php?term=Breaking%20the%20Fourth%20Wall).

But I claim there's a way to sidestep this and inject meaningful genre-savviness into your character if you (the player) so choose.

The essence of this is to _stop_.

Just stop.

For a little while, pause the incessant activity, the trying to figure out, the jumping into reaction when a feeling or idea bursts into awareness, the fidgeting to dispel social or physical discomfort instead of savoring it.

Just let all avenues for acting out a role come to stillness.

And then in your stillness, listen closely to your experience as though this is the first moment you've ever experienced anything at all.

This whole process is practically _guaranteed_ to make your character flip out. I don't claim you'll _like_ doing this (at least at first), or that it'll make sense to you (at first). Maybe it sounds too much like meditation and you have a storm of thoughts associated with that. Maybe the _idea_ is so atrocious or ill-founded or mystic-flavored to you that you don't want to even _try_ it.

And that's fine! Maybe it makes sense for you to wait until death forces this stillness on you.

But if you choose to try it _anyway_, you can watch as your character does these theatrics…

…and clearly see for yourself that they really are _just theatrics_…

…and you can start to consciously remember who you are beyond all that reactivity.

That reactivity is what takes up slack. When you attend to deep stillness this way, you can directly [see for yourself](https://www.lesswrong.com/posts/tMhEv28KJYWsu6Wdo/kensh) how to create slack, just as clearly as you can feel your tongue in your mouth. And just as clearly, you can watch the ebb and flow of the social web and the ways in which you and everyone else pretends to be bound by its laws.

Then it'll be immensely obvious to you how to create real change in your life.

* * *

This was long, so I’ll try to summarize:

*   You can choose to see social groups at all scales as running a distributed computation across the social web. You can choose to view that process as generating an agent — the intelligent social web — who tries to predict and guide each person’s behavior.
*   The social web offers each person a trade: prioritize making the scene work, and you’ll be included in it. In fact, the web _is_ the aggregate efforts of all the people who have accepted that trade. And basically everyone we know about accepts this trade.
*   Everything about yourself that you have conscious access to is subject to your role as part of the social web. If you try to defy this, then your fate will play out through your defiance.
*   Room for interpretation in your role in the scene means your script has room to change. This is slack in the social web.
*   There’s a way of directly seeing how to change your fate by [Looking](https://www.lesserwrong.com/posts/tMhEv28KJYWsu6Wdo/kensh), if you so choose. This amounts to something like pausing long enough to clearly see the reactions that try to keep you from pausing.

I’ll close this post by noting that there’s a meta-level to track here. In the story [The Emperor’s New Clothes](http://www.online-literature.com/hans_christian_andersen/967/), the child’s utterance wasn’t enough on its own to pop the illusion:

> _"But the Emperor has nothing at all on!" said a little child._

> _"Listen to the voice of innocence!" exclaimed his father; and what the child had said was whispered from one to another._

> _"But he has nothing at all on!" at last cried out all the people. The Emperor was vexed, for he knew that the people were right; but he thought the procession must go on now! And the lords of the bedchamber took greater pains than ever, to appear holding up a train, although, in reality, there was no train to hold._

What if the father had instead responded “No, child, you’re just too foolish to see his fine garments”? He might have, out of fear of what those who were standing nearby might think of him and his kid. Then the child’s simple voice of reason would not be heard.

Or what if the people near the father/child pair had felt too uneasy to pass along what the child had said?

What if the Emperor could have instilled this kind of nervousness in his people ahead of time? He might have thought that there will be innocent children in the parade, and it might have occurred to some part of him that they had best not be taken seriously — [to spare others their embarrassment, of course](http://www.wiki.lesswrong.com/wiki/rationalization). Then, oh then what strange propaganda they all would see.

Some of the scripts the social web assigns work less well if they’re known. Because of this, the web will often move to silence people who threaten to speak those fragile truths. This can show up, for instance, as people trying to dismiss and discredit _the person saying the idea_ rather than just the idea. The arguments usually sound sensible on the surface, but the underlying tone ringing through the strands of the web is “Don’t listen to this one.”

If it’s not clear why I’m mentioning this, then I imagine it’ll become really obvious quite soon.