trentmkelly/LessWrong-43k huggingface dataset:
Argument, intuition, and recursion
by paulfchristiano — 2018-03-05T01:37:36.120Z

Mathematicians answer clean questions that can be settled with formal argument. Scientists answer empirical questions that can be settled with experimentation.

Collective epistemology is hard in domains where it's hard to settle disputes with either formal argument or experimentation (or a combination), like policy or futurism.

I think that's where rationalists could add value, but first we have to grapple with a basic question: if you can't settle the question with logic, and you can't check your intuitions against reality to see how accurate they are, then what are you even doing?

In this post I'll explain how I think about that question. For those who are paying close attention, it's similar to one or two of my previous posts (e.g. 1 2 3 4 5...).

I. An example

An economist might answer a simple question ("what is the expected employment effect of a steel tariff?") by setting up an econ 101 model and calculating equilibria.

After setting up enough simple models, they can develop intuitions and heuristics that roughly predict the outcome without actually doing the calculation.

These intuitions won't be as accurate as intuitions trained against the real world---if our economist could observe the impact of thousands of real economic interventions, they should do that instead (and in the case of economics, you often can). But the intuition isn't vacuous either: it's a fast approximation of econ 101 models.

Once our economist has built up econ 101 intuitions, they can consider more nuanced arguments that leverage those fast intuitive judgments. For example, they could consider possible modifications to their simple model of steel tariffs (like labor market frictions), use their intuition to quickly evaluate each modification, and see which modifications actually affect the simple model's conclusion.

After going through enough nuanced arguments, they can develop intuitions and heuristics that predict these outcomes. For example, they can learn to predict which assumptions are most important to a simple model's conclusions.

Equipped with these stronger intuitions, our economist can use them to get better answers: they can construct more robust models, explore the most important assumptions, design more effective experiments, and so on.

(Eventually our economist will improve their intuitions further by predicti...
On Defense Mechanisms
by Quaerendo — 2018-03-04T18:53:08.650Z

There is much talk about cognitive heuristics and biases in the Rationality community, whereas psychodynamic/psychoanalytic perspectives of psychology tend to be dismissed as "bad science". Now, I will agree that some Freudian ideas like penis envy and Oedipus complex are silly -- but that doesn't necessarily mean that nothing useful can be salvaged from the wreck. In particular, today I want to highlight the concept of psychological defense mechanisms developed by Sigmund Freud and his daughter Anna, and suggest how they might be of interest to rationalists.[1]

According to Freud's theory, our unconscious minds are host to the continual battles between the "id" (our primal, pleasure-driven aspect), the "ego" (our decision-making aspect, mindful of social reality) and the "superego" (our moral compass). These internal conflicts, when prolonged, result in anxiety and other unpleasant emotions like guilt. In order to get rid of this anxiety, we rely on "defense mechanisms", which are feats of mental gymnastics that work through self-deception. And since self-deception can be an obstacle to epistemic and instrumental rationality, it should be clear why we should at least be aware of defense mechanisms.

Here are some of the most common defense mechanisms, taken from Weiten, Dunn & Hammer (2015).[2]

Repression: keeping distressing thoughts and feelings below your conscious awareness. Examples: Amber forgot the name of someone whom she really hates. Billy doesn't remember the time he nearly got killed while serving in Afghanistan.
Projection: attributing your own thoughts and feelings to someone else. Examples: Claire feels sexual tension with a colleague, which she attributes to the colleague's motive to seduce her. Dennis doesn't like his boss, but tells others that he actually likes the boss -- it's just that the boss doesn't like Dennis.
Displacement: re-directing emotions from their original source to a substitute target. Examples: Emily had a rough day at work, so she unleashes her anger onto her husband and cat. Frank has been disciplined by his parents, and takes his anger out on his little sister.
Reaction formation: behaving in a way that is the opposite of what your true feelings would imply. Examples: Gemma unconsciously resents her child, so she spoils the child with gi

...
Correct Models Are Bad
by sapphire — 2018-03-04T16:51:25.919Z
Link: https://brianlui.dog/2018/03/03/correct-models-are-bad/

Quick Summary:

We may think that correct mental models are always good to have, but the reality is that some correct models are useless or even harmful.

All models are simplifications of the world. Whenever we knowingly use a wrong model, we are aware of what the model doesn’t cover. The danger of using correct models is that we are more likely to apply the model without thinking about what’s been left out.

Some ways a correct model can be harmful:

1) The model is ineffective - Some correct models don’t seem to serve much purpose, so even though they are correct, it’s not really worth keeping them around. (ex: Calories In Calories Out)

2) The model is an obstacle to exploration - Once our brains have decided that a mental model is correct, we are unlikely to question the model. This can stunt our learning and personal growth. (ex: Initial research on overconfidence)

3) The model is socially or politically dangerous - Some correct models may even be harmful. These are especially dangerous for people who value truth and correctness, but are not politically savvy. (ex: Genetics research both today and in Soviet Russia)

4) The model is easy to misuse. (ex: p-values)

5) Consequences of misuse are severe - More bad news: even if a correct model is hard to misuse, it can still lead to bad outcomes! This happens when using the model wrongly is catastrophic. (ex: Immigration)
[Paper] Surviving global risks through the preservation of humanity's data on the Moon
by avturchin — 2018-03-04T07:07:20.808Z

My, with David Denkenberger, article about surviving global risks through the preservation of the data on the Moon has been accepted in Acta Astronautica. Such data preservation is similar to the digital immortality with the hope that next civilization on Earth will return humans to life.

I also call this "plan С" of x-risks prevention, where plan A is stopping global catastrophe and plan B is surviving it in a refuge. The plan B was already covered in another my article about Aquatic Refuges (that is nuclear submarines), published in Futures.

Plan C could be done rather cost-effectively by adding some eternal data carriers to many planned space crafts like Arch Mission is planning to do.

Link:

https://www.sciencedirect.com/science/article/pii/S009457651830119X

The article is behind the paywall but the preprint is here:

https://philpapers.org/rec/TURSGR

Abstract: Many global catastrophic risks are threatening human civilization, and a number of ideas have been suggested for preventing or surviving them. However, if these interventions fail, society could preserve information about the human race and human DNA samples in the hopes that the next civilization on Earth will be able to reconstruct of Homo Sapience and our culture. This requires information preservation of an order of magnitude of 100 million years, a little-explored topic thus far. It is important that a potential future civilization will discover this information as early as possible, thus a beacon should accompany the message in order to increase visibility. The message should ideally contain information about how humanity was destroyed, perhaps including a continuous recording until the end. This could help the potential future civilization to survive. The best place for long-term data storage is under the surface of the Moon, with the beacon constructed as a complex geometric figure drawn by small craters or trenches around a central point. There are several cost-effective options for sending the message as opportunistic payloads on different planned landers.

Keywords: Global catastrophic risks, existential risks, moon, time-capsule, METI

Highlights:

Catastrophic risks could be survived if the next civilization on Earth will reconstruct humanity.
The next non-human civilization may appear on Earth around 100 million years from now.
Time-capsules with DNA and data could help in the rec

...
Beta-Beta – Recent Discussion
by Radmin — 2018-03-04T05:34:23.341Z

Over on our beta-beta (I guess the technical term is "alpha") site, we're experimenting with a new system for Recent Comment, renaming it to "Recent Discussion" and tweaking some features.

The recent comments are organized by post, sorted in the order of "post most recently commented on", showing the three most recent comments for each post.

If you haven't read a post, it also shows a short blurb of the first 300 characters of the post (similar to how the recent comments themselves do).

A couple motivations for this:

If there's one thread with loads of discussion happening, it can easily overwhelm the recent comments section. (For example, as of right now on Lesserwrong.com, all 10 Recent Comments are from the Jorden Peterson thread right now. If you haven't logged on in 4 hours, you'd have missed the discussion happening in Connection Between Math and Philosophy and Subtle Forms of Confirmation Bias. Whereas on LessestWrong it shows you each of those posts (plus a few others) clustered by thread.
High-volume discussions not only makes it harder to see all the active threads, but also produces runaway affects where "only one thread is showing up in Recent Comments, so people are predisposed to comment on that discussion instead of others."
Old posts that get commented on get highlighted slightly more than they were before (in particular, if you haven't read them already, you'll get a bit more context about what the post was about).
I wanted it to be easier to skim the opening text of new posts (empirically I don't actually click the "show highlight" button that I worked so hard on last month. :P). Up till now it's been way easier to peruse random comments with no context than to start reading actual posts, which seems backwards to me.
Posts with negative karma will not have their comments show up in the recent discussion, preventing runaway demon threads from taking over the front page.

This is a fairly major change, so we wanted to collect some feedback before moving forward on it, and it seemed easier to get that feedback if we temporarily connected the live database to the Lessestwrong.com. For the next few days you can log in there normally.

Longterm, there's a few other features that could get rolled into this, such as:

displaying proper threading for recent comment

...
Murphy’s Quest Ch 1: Exposure Therapy
by alkjash — 2018-03-04T04:50:00.918Z

[Author’s note: I noticed I’ve been trying to be too original in my fiction writing. This story is trash and so am I.]

My name is Murphy. I’ve been preparing for this day since I was born.

I just woke up in a new world.

I’m standing in line while a mean-looking dwarf shouted a standard training prep talk at us. Unity, courage, yadda, yadda, for the King!

Everything’s just like I expected. I’m standing in a line with four other boys. We’re covered with basic brown adventurer frocks, with wood clubs and swords hanging at our waists and small packs around our shoulders.

I can see stats floating over my arm like a fuzzy overlay. Basic noob stuff.

NAME: Murphy

RACE: Human

LVL: 1

HP: 10/10

STR: 5

INT: 5

FTH: 5

AGI: 5

EQUIPPED: Wood Club (4 damage), Basic Rags.

My grin spreads from ear to ear. Thousands of hours of jRPGs all for this moment. Prepare to bow to the wiles of Dark Warlock Emperor Murphy, plebs!

The drill sergeant dwarf is sending us on our first training mission: a simple fetch quest.

“ – collect one thousand kobold ears.”

Nani?!

—

There’s no way out of the training grounds until we finish this ridiculous fetch quest. Stone-faced Level 10 soldiers stand guard at every entrance.

After taking a hopeless tour of the camp, I decide to grind out the kobolds. How long could this take?

Kobold Younglings spawn out of caves in the hillside, scurrying out every five minutes. The other boys have already claimed the closest caves to camp. I take over the last remaining one on the far side of the hill.

A Kobold Youngling (HP: 6/6) scrambles out of the cave and makes a run for it. I step forward and smack it on the head. It doesn’t die.

The mean little thing claws me across the forearm. I scramble back, and smack him again. He falls over and disappears in a puff, leaving behind a highlighted green ear. I snap that into my pack.

I look down at my left arm.

Blood!?

I’m no good with real blood!

Uh oh.

—

I wake up that evening in my bunk. One of the other boys had dragged me away from the spawn point before another Kobold appeared.

At least I killed the Kobold before passing out at the sight of blood.

The other boys all have 9 ears each. Kobold Scratch deals 1 damage, and you only heal while sleeping, so we could only farm 9 a day. There’s no way to avoid getting scratched at least once per fight.

I do a quick mental calculation. This is going to be a long 112 days.

On the bright side, missing a day d...
Murphy’s Quest Ch 2: Empiricism
by alkjash — 2018-03-04T04:50:00.861Z

How did the cunning Dark Warlock Emperor Murphy solve the Thousand Kobold Challenge?

I ask my bunkmate in the morning to camp a spawn point together. My thinking: if we can both hit the Kobold before he gets a chance to attack, we won’t take damage. That way we can grind all day.

Of course I’m not ready to grant my bunkmate Named Character status. He’ll have to earn it.

He agrees to give it a try. So we’re squatting together around a Kobold cave, and my thighs cramp but I don’t want to look like a wimp. These other boys must have grown up on manual labor. I wonder if they also have STR: 5?

The first Kobold that spawns takes a hard left when he sees us. We step forward in unison, but my bunkmate is farther away and hits him late. The Kobold scratches me right before he dies.

The next one is the same, and the next one.

No matter how closely we time our attacks, the scrawny little rats always get a Kobold Scratch in right in between.

I hang my head. There’s only one possible conclusion…

Combat is turn-based?!

—

No way is the future Dark Warlock Emperor staying here to smack Kobolds for another hundred days.

I sit around dawdling all afternoon with the other trainees. They seem to be bonding and enjoying themselves. Must feel like a vacation for them.

Suddenly, I have a bright idea. I’ll dual wield clubs! That’ll kill the Kobolds in one hit. My bunkmate lets me borrow his club. I pick up my own in my right hand, and grab his in my left.

It won’t budge.

Really?

If I drop my club, I can pick up his in my left hand. Somehow as soon as I pick one up with one hand, the other becomes too heavy to lift. I feel so much sympathy for video game characters now. To live in a world where physics itself is your enemy…

On the other hand, I’m not the most athletic character, so this may come in handy.

—

It’s embarrassing to admit, but I’m finally saved by a bad habit. When I’m really bored, I drink a lot of water. I probably down 15 mugs of water while listening to the campfire conversation.

That’s why I wake up three times to pee this night. The first two times I’m super sleepy and drop back into bed immediately, so drinking 10 mugs of water wouldn’t have been enough. But since I drank 15 mugs, I wake up for a third time just before dawn and sunlight started streaming in before I fall back in bed. That’s when I notice the HP bar over my hand is full!

Do you ever feel as if you’r...
Is there a Connection Between Greatness in Math and Philosophy?
by Scott Garrabrant — 2018-03-03T23:25:51.206Z

I have nothing that looks like an answer, but I wanted to put the question out to start a discussion. I especially hope that this leads to someone here researching the question well. I also have almost nothing to back up the claims below.

I became curious about this from thinking about how to find people who can make progress in MIRI's Agent Foundations questions. It seems to require very high levels of both math and philosophy, and I want to know how to select for both of these skills simultaneously.

It seems like many of the great mathematicians at least had enough of an interest in philosophy to share their philosophical opinions. Also it appears that the people who substantial progress in philosophy often use math. (There are of course disagreements about what makes good philosophy, and I am biased in favor of the mathematical flavored philosophy.)

I am especially surprised by the fact that the tails seem to come together rather than apart. It seems like mediocre mathematicians and philosophers are much less likely to be interested in the other field.

Some possible theories:

1) I am noticing a pattern that is not there.

2) This is a consequence of the fact that all intelligence is correlated, and you will notice a similar pattern between many pairs of fields.

3) Mathematics and Philosophy are very old fields. Historically, fields did not really exist, and people were more interdisciplinary. If you look at modern mathematicians and philosophers, the trend goes away.

4) The skill to produce great math and skill to produce great philosophy are secretly the same thing. Many people in either field do not have this skill and are not interested in the other field, but the people who shape the fields do.

Thoughts?
Funding for AI alignment research
by paulfchristiano — 2018-03-03T21:52:50.715Z
Link: https://docs.google.com/document/d/1NIg4OnQyhWGR01fMVTcxpz8jDd68JdDIyQb0ZZyB-go/edit?usp=sharing

If you are interested in working on AI alignment, and might do full or part time work given funding, consider submitting a short application to funding@ai-alignment.com.

Submitting an application is intended to be very cheap. In order to keep the evaluations cheap as well, my process is not going to be particularly fair and will focus on stuff that I can understand easily. I may have a follow-up discussion before making a decision, and I'll try not to favor applications that took more effort.

As long as you won't be offended by a cursory rejection, I encourage you to apply.

If there are features of this funding that make it unattractive, but there are other funding structures that could potentially cause you to work on AI alignment, I'm curious about that as well. Feel free to leave a comment or send an email to funding@ai-alignment.com (I probably won't respond, but it may influence my decisions in the future).
The Jordan Peterson Mask
by Jacob Falkovich — 2018-03-03T19:49:20.813Z

This is a cross-post from Putanumonit.com

It seems that most people haven’t had much trouble making up their minds about Jordan Peterson.

The psycho-philosophizing YouTube prophet rose to prominence for refusing to acquiesce to Bill C-16, a Canadian law mandating the use of preferred pronouns for transgender people. The sort of liberal who thinks that this law is a great idea slapped the alt-right transphobe label on Peterson and has been tweeting about how no one should listen to Peterson about anything. The sort of conservative who thinks that C-16 is the end of Western Civilization hailed Peterson as a heroic anti-PC crusader and has been breathlessly retweeting everything he says, with the implied #BooOutgroup.

As the sort of rationalist who googles laws before reacting to them, I assured myself that Peterson got the legal facts wrong: no one is actually getting dragged to jail for refusing to say zir. I’m going to use people’s preferred pronouns regardless, but I’m happy I get to keep doing it in the name of libertarianism and not-being-a-dick, rather than because of state coercion.

With that, I decided to ignore Peterson and go back to my media diet of rationalist blogs, Sam Harris, and EconTalk.

But Jordan Peterson turned out to be a very difficult man to ignore. He showed up on Sam Harris’ podcast, and on EconTalk, and on Joe Rogan and Art of Manliness and James Altucher. He wrote 12 Rules for Life: An Antidote to Chaos, a self-help listicle book inspired by Jesus, Nietzsche, Jung, and Dostoyevsky. [Let's see if I can tie all 12 rules to this essay] And he got rationalists talking about him, which I’ve done for several hours now. As a community, we haven’t quite figured out what to make of him.

Peterson is a social conservative, a Christian who reads truth in the Bible and claims that atheists don’t exist, and a man who sees existence at every level as a conflict between good and evil. The majority of the rationalist community (present company included) are socially liberal and trans-friendly, confident about our atheism, and mistake theorists who see bad equilibria more often than intentional malevolence.

But the most salient aspect of Peterson isn’t his conservatism, or his Christianity, or Manicheanism. It’s his commitment, above all else, to seek the truth and to speak it. [Rule 8: Tell the truth – or, at least, don’t lie] Rationalists can forgive ...
Sacred Cash
by Zvi — 2018-03-03T13:20:00.382Z

Previously: Categories of Sacredness, Eternal, and Hearthstone Economy versus Magic Economy, Slack

Related (Compass Rose): Kidneys, Trade, Sacredness and Space Travel

Remember when money was sacred?

Thanks to The Great Transformation, there is an increasingly free market in money. We spend money continuously, and get paid continuously.

This is relatively new. Or at least, its universality is relatively new.

It wasn’t so long ago that money was sacred at the bank. They had to give you a toaster instead.

My parents grew up in a different world. They taught me money was sacred. You had a job for a fixed number of hours, paying a fixed amount. That was all the money you got. Earning more was very difficult. Side jobs were lousy and rare. You’d spend lots of time to sell unwanted items at yard or sidewalk sales, at severe discounts. Everyone kept a strict budget. The edge was never far away.

Money spent other than on necessities was a special treat, a grand honor or dire emergency.

This cultural attitude is hard to learn. It is also hard to unlearn. It is quite jarring when you first see it violated.

The Profane

When I moved out to Denver to work at Social Games on the Cyperpunk TCG, I met a group that neither had enough money, had ways to earn more money nor treated money as sacred.

They’d think nothing of paying a dollar for a Slurpee. On payday, they’d all get takeout. Far from payday, they’d (literally in one case) search their seat cushions to find change with which to gas up their motorcycle, to get to work.

Anything they had, that could have been reasonably sold, would eventually have been.

I did not understand.

In a start-up with one angel investor who lacked the ability to reinvest, and little hope of raising additional funds, they did not prepare for the inevitable cash crunch while trying to sell enough to stem the bleeding. I don’t know if we could have stabilized if given a few more months, but we had a shot only with those months. We likely fail anyway, but to see it all thrown away infuriated me.

MetaMed’s final blow was similar. My replacement as CEO increased our spend, leaving insufficient runway to do the things necessary to raise additional funds, and failed to realize (despite my repeated warnings) that this meant 0% chance of success. The plan’s timeline didn’t work. It was incoherent. Again, we probably fail anyway, but to do that which inevitably fails?...