# The Dark Arts As A Scaffolding Skill For Rationality

Epistemic status: [Exploratory](https://www.lesswrong.com/posts/Hrm59GdN2yDPWbtrd/feature-idea-epistemic-status)

Recently I wrote an essay about [Scaffolding Skills](https://www.lesswrong.com/posts/68iix7PFHPADhdQYF/scaffolding-skills). The short explanation is that some skills aren’t the thing you’re actually trying to get good at, but they help you learn the thing you’re actually trying to get good at. Briefly, the literal scaffolds are the temporary thin platforms you see on the outside of construction projects, which give builders a place to stand when constructing or repairing a more permanent building. 

The two sentence summary of this essay is this: In martial arts dojos your partner will often throw punches at you which they would not use in a real fight, so you can practice parrying in a safer environment. I believe a rationality dojo would have your partner lie to you in ways they would not use in an ideal discussion, so you can practice responding.

### I.

Being able to lie well is not a skill you might imagine the ideal rationalist to have.

If I was to introduce you to someone and say “hey, I want you to meet a colleague of mine, they’re the most rational human being I know,” you might expect they would have certain capabilities. Maybe they’re really well calibrated; when they say they’re 90% sure a project will be done on time it’s actually done on time nine times out of ten. Maybe they’re completely unafraid to admit when they’re wrong, doing so exactly when the evidence turns against their previous position. You might have your own expectations, but I expect you’d be surprised if I continued to say “they’re a *really* good liar.”

And indeed I don’t think lying is a core rationalist skill. Do not interpret this essay as saying lying is a rationalist skill. I am saying it’s a scaffolding skill for learning rationality, which is a different thing.

Let's examine Tai Chi for a moment. Tai Chi is a martial art, which many people learn for self defence. Practical self defence assumes that at some point, someone might throw, say, a right hook or a jab at you. Tai Chi has a fairly well enumerated list of movements, and while there’s some variation between styles I have never seen a Tai Chi movement that looked like a right hook. It’s not an attack that’s really in Tai Chi’s idiom. Tai Chi would (if I can reify the martial art as wanting something) rather disrupt someone’s balance by pushing them over or to catch them in a circle that leads to a joint lock.

(Tai Chi does have punches. They just look different, either being straight gut blows or downward moving strikes to the head and shoulders. Throwing a variety of attacks from a multitude of disciplines, especially ones untrained people gravitate towards, and seeing how the good students handle them is a decent check for whether a martial art style is good.)

My favourite Tai Chi instructor had a bit more of a practical mindset than many, and sometimes threw a boxing hook or uppercut at me while I was practicing. My reflexive response to seeing a hook or just an off centre jab is Cloud Hands (aka Hands Move Like Clouds) which is a decent parry that doesn’t require much accuracy. He had a pretty good right hook, which is why my Cloud Hands are *very* reflexive indeed. 

Throwing a hook or an uppercut is not a Tai Chi skill. Having someone around who throws good uppercuts is very useful for learning Tai Chi. If you’re working in a small dojo you probably pair up with someone else to practice throwing/deflecting attacks. 

Has any rationalist practice ever involved pairing up to practice being lied to and responding to lies?

### II.

I’m using “lying” as a specific example of a great many dark arts techniques. Time pressure, cherry picked statistics or examples, loaded connotations, repetition, various status moves that I don’t have good names for, apophasis, these are all things someone can do to convince you to do or believe something which you should not do or which is not true.

I sometimes worry that the walled garden we have *somewhat* succeeded in creating within the rationalist community has left us vulnerable to the dark arts, the way that the vaccination-driven suppression of Measles leaves the USA vulnerable to the Measles coming back. 

I have noticed in myself the habits and impulses that are useful for noticing subtle statistical errors or patiently providing the principle of charity to viewpoints I initially disagree with, and I have noticed those habits *completely whiff* in the face of people saying in plain English claims which were extremely unlikely to be true. Like, if you’d said “hey, here’s this claim. How likely is it that claim is true?” I would have put it at something like one in a thousand, and yet instead of saying out loud “hey, I think that thing you just said is a motivated lie” I tried to maintain a second hypothesis where the other guy *didn't* have the epistemic standards of a fornicating pope. 

I endorse having this charity a bit! One of the things that’s nicer about the rationalist community than the general internet is people are open to how they might be wrong. But uh, at least for me, 2023!Screwtape was not trained to do the other thing as well as he might have been. I think this is an error in my rationality training. Granted, only one person associated with the community has ever sat down and tried to train me to think better, so this is more like a wishlist for rationality training if we ever get far enough to actually do that.

(Okay, three people if you count Yudkowsky writing the sequences and Sabien writing the CFAR handbook. But I generally don’t count the Gray’s Anatomy book as medical training.)

I’m not arguing that the Dark Arts are useful and good to deploy in normal life. I’m not arguing that they *aren’t.* Those are both separate from my current argument, which is that in order to be good at the skill of sifting out the truth and signal from the misdirection and noise, it would be useful practice for someone to use the Dark Arts on you in a controlled setting.

([Case studies](https://www.lesswrong.com/posts/p7WXmG6Fbo3eaSwm3/defense-against-the-dark-arts-case-study-1) will do in a pinch, but we can do better.)

### III.

Let's talk about the controlled setting of a dojo for a moment, because I think that’s actually a very important qualifier.

If I punch you in the nose while you’re standing at the bus stop or while you’re walking through the halls of your high school, I’m a jerk and this is not how martial arts dojos work. It is true that the bus stop or your high school are the places you will want the self defence skills! You can learn from the school of hard knocks. But there’s a lot of advantage in a controlled setting for learning.

*   In a martial arts dojo, nobody will hit you except during a drill or spar where it is made clear you might be hit.
*   In a martial arts dojo, people will stop hitting you as soon as you tap out or yield.
*   In a martial arts dojo, people will tell you how to deflect or block the strike before/after hitting you. (If not immediately prior then at some point in the lessons.)
*   In a martial arts dojo, things are set up to minimize the odds of lasting injuries like broken noses, but especially the ones that will make you weaker long-term like dislocating your knee.
*   In a martial arts dojo, if someone breaks your nose or dislocates your knee anyway, someone will be around with basic first aid and they will make sure you get proper medical attention.
*   In a martial arts dojo, if someone is being reckless with the safety of others, they will get kicked out.
*   ~~In a martial arts dojo, you get to wear a really cool uniform.~~

(Actually what I mean is “in a *good* martial arts dojo” those things are true. [Problems absolutely exist.](https://acesounderglass.com/2015/06/08/inclusivity-is-a-trade-off/))

“Drills which make you better at dislocating someone’s knees, without actually dislocating anyone’s knees *or* training you to pull the blow at the last second” is a very narrow target. Successfully facilitating those drills is an important job for a martial arts instructor. The skills which make you good at being part of those drills are useful scaffolding skills indeed. It’s not just throwing a right hook that’s useful scaffolding for Tai Chi, it’s nailing the right mix of forewarning and surprise to be both safe and also useful practice.

Also beware of [schools proliferating without evidence](https://www.lesswrong.com/posts/JnKCaGcgZL4Rsep8m/schools-proliferating-without-evidence). The problem with practice drills is it's easy to feel like it's working because it works in the drill when the other bloke is cooperating with you, and stops working immediately when you get in a bar fight or even an MMA ring with different rules of engagement.

Back to rationality. 

I want a place where nobody manipulates me without warning, they stop doing it as soon as I tap out, and someone explains how to spot and neutralize the manipulation ahead of time. I want that place to minimize the amount of lasting damage it causes, and to patch my head up afterwards if I do get messed up.

The best place I’ve found for this is the Live Action Roleplaying (LARP) community. For those unfamiliar with the term, LARPs are a kind of game where everyone takes a different role as though they’re in a play, but there isn’t a script. Instead, there’s some rules for how to pretend to stab people or do magic spells, there’s character goals and motivations that are often at odds with each other, and then they cut you loose. Those can be pretty good at setting me up where someone is going to try and manipulate me, but they stop when the game ends and they’ll sometimes explain my mistakes.

(A few pointers on the kinds of things LARP does well: there’s a well maintained distinction between being in or out of character, where you can’t trust my character but you can still trust me. Before and after games, we debrief and hang out, which gives a place to talk about what worked or what didn’t but also helps come down from the emotional aftereffects of being manipulated. There’s a lot of freedom of motion to attack and lie from different angles, which makes for better training than Mafia or Werewolf’s simpler decisions.)

I’m worried someone is going to take this essay, gaslight people, and start pointing at the essay then say they were just trying to train the victim to be a better rationalist. Don’t do that. If you just start messing with someone’s head and don’t fix what you broke and explain how to guard against it better, you’re not doing this right. The student should opt-in via some obvious way. If anyone points at this essay after manipulating you and says any variation on “look, I was trying to help you grow as a rationalist” and wasn’t at least as good about preventing that kind of thing from happening as the average martial arts dojo is at preventing broken noses, I think they did something wrong.

But I am likely to try setting this up, or joining one if someone else sets one up. I’d be excited at throwing a hundred hours of practice time at this. It’s just going to need some work to make it safe and effective. Plausibly somebody has already done a lot of this work; I wouldn't be surprised if lawyers or judges had some version of this, and I'd expect certain strata of social workers to learn this via a modern apprenticeship approach. If you think you've got a good version of this, I'd love if you let me know about it.

Lying convincingly isn’t a rationalist skill. But someone in the training process should have the skill, or the result is brittle rationality which is not used to being attacked.