trentmkelly/LessWrong-43k huggingface dataset:
STRUCTURE: A Hazardous Guide to Words
by Hazard — 2019-06-20T15:27:45.276Z

This post is part of my Hazardous Guide To Rationality. I don't expect this to be new or exciting to frequent LW people, and I would super appreciate comments and feedback in light of intents for the sequence, as outlined in the above link. Also, note this is a STRUCTURE post, again see the above link for what that means.

Intro

Communication can be hard. Thinking can be hard. Trying to get another person to understand what you mean can be hard. Trying to explain to yourself what you mean can be hard. This sequence draws out in detail some of the things that happen in human brains that leads to these hardships, and proposes a few tactics to make it less hard.

Some Problems

When a tool becomes familiar enough, it can become forgotten. You forget a black box is a black box, and see it as a lever of reality.

What you meant is not what you said, but also, what you meant is not what you said.

I was expressing two distinct thoughts there. It wasn't very obvious. The first sentence meant, "You don't always say words that make other people think of the same things you're thinking about". The second meant, "What you mean is a different type of thing from what words you produce, and they should not be mistaken for each other".

What you meant is not what you said; the language you use is not the same thing as the cognition that produced the language. Apologies if you're getting bored.

In this sequence I'm going to make some claims about how we think and how language works, and then point to a bunch of pit-falls that can lead to confusion, never ending argument, and bad times. I believe these pitfalls have to do with inaccurate models of how language works and subtly forgetting that cognition and language are different.

(aside: what if after reading this sequence you think "Well, it's not really inaccurate models and subtly forgetting that is the cause of these problems, it's XYZ". Oh really? Why would it matter if it was the case?)

Cognition, Classification, and Inference

Your brain's job is to keep you alive. This takes a lot of guess work. Is this edible or poisons? Is that blur a tiger on its way to kill me? The usual. It sure would be useful if the brain could turn sensory input into the exact set of motor movements needed to not die. This is what the brain does. You take the outside world and make categories. Once categorized, you are use various "if... then..." rules

...
Defending points you don't care about
by Hazard — 2019-06-19T20:40:05.152Z

This post is part of my Hazardous Guide To Rationality. I don't expect this to be new or exciting to frequent LW people, and I would appreciate comments and feedback in light of intents for the sequence, as outlined in the above link.

A dialogue:

Nicky: I've been wondering, do you think math was invented or discovered?
Dee: Seems like it must have been discovered. I read about how circles are everywhere in nature, and that you can even find the fibonacci sequence in plants!
Nicky: Yeah, but there aren't actually any numbers in nature. Numbers are just something we made up to describe and talk about these patterns that we see in nature. Numbers themselves don't really exist out in the world.
Dee: Of course numbers are real! Made up constructs don't have the predictive power that math does. They totally exists.
Nicky: Well if numbers exist, where are they? You can't show me where a number is. You can't empirically test for numbers. You can't find them anywhere in the physical world. They're just constructs!
Dee: Sure, they don't a physical location in the world. That's silly. There's no circles floating around out behind the moon. What I'm saying is that they exist outside of space and time. Mathematical existence is it's own sort of domain, separate from the domain of physical existence.
Nicky: Bleh, next you're going to tell me that you believe in cartesian dualism.
Dee: Bleh, next you're going to tell me that math is arbitrary and people can build rockets that work however they feel like.
The two never talked again. Dee, remembering this conversation in great detail, went on the become a commited Platonist and write many articles trying to defend this complex philosophical view

To help draw out the point I'm trying to make, here's an alternative history of this conversation.

Nicky: Hey Dee, you got any views on mathematical platonism?
Dee: What's that?
Nicky: It's the idea that mathematical objects exist in reality, but seperate from physical reality. Physical reality defines what's true about the world we live in, and mathematical reality defines what's true about math objects.
Dee: Hmmmm, I'm not sure. I mean, that seems like it would explain why math is so certain and precise, but it also feels weird to posit a whole new fundamental element of reality, and...
Words Aren't Type Safe
by Hazard — 2019-06-19T20:34:23.699Z

This post is part of my Hazardous Guide To Rationality. I don't expect this to be new or exciting to frequent LW people, and I would appreciate comments and feedback in light of intents for the sequence, as outlined in the above link.

∫x2(a2−x2)1/2dx

Student: Hmmm, let's see if I can remember how to integrate. So I know the x is a variable that I'm trying to integrate over. What's that "a" though? I remember it being called a constant. But what's that?

Tutor: A constant is just some number. It could be anything.

Student: Cool!

student disappears for a while and comes back with the following

∫x2(a2−x2)1/2dx=a

Student: I did it!

Tutor: The fuc... um, how about you walk me through how you got that?

Curious. Here's another tale of confusion:

Once upon a time — specifically, 1976 — there was an AI named TALE-SPIN. This AI told stories by inferring how characters would respond to problems from background knowledge about the characters' traits. One day, TALE-SPIN constructed a most peculiar tale.

Henry Ant was thirsty. He walked over to the river bank where his good friend Bill Bird was sitting. Henry slipped and fell in the river. Gravity drowned.

Since Henry fell in the river near his friend Bill, TALE-SPIN concluded that Bill rescued Henry. But for Henry to fall in the river, gravity must have pulled Henry. Which means gravity must have been in the river. TALE-SPIN had never been told that gravity knows how to swim; and TALE-SPIN had never been told that gravity has any friends. So gravity drowned.

TALE-SPIN had previously been programmed to understand involuntary motion in the case of characters being pulled or carried by other characters — like Bill rescuing Henry. So it was programmed to understand 'character X fell to place Y' as 'gravity moves X to Y', as though gravity were a character in the story.1

For us, the hypothesis 'gravity drowned' has low prior probability because we know gravity isn't the type of thing that swims or breathes or makes friends. We want agents to seriously consider whether the law of gravity pulls down rocks; we don't want agents to seriously consider whether the law of gravity pulls down the law of electromagnetism. We may not want an AI to assign zero probability to 'gravity drowned', but we at least want it to neglect the possibility as Ridic...
Arguing Definitions
by Hazard — 2019-06-19T20:29:44.323Z

This post is part of my Hazardous Guide To Rationality. I don't expect this to be new or exciting to frequent LW people, and I would appreciate comments and feedback in light of intents for the sequence, as outlined in the above link.

Here's a dialogue straight out of A Human's Guide to Words (highly recommended, but then if you'd just take my recommendation and read the sequences, I wouldn't be writing this would I?)

If a tree falls in the forest, and no one hears it, does it make a sound?
Albert: Of course it does. What kind of silly question is that? Every time I’ve listened to a tree fall, it made a sound, so I’ll guess that other trees falling also make sounds. I don’t believe the world changes around when I’m not looking.
Barry: Wait a minute. If no one hears it, how can it be a sound?
Albert: What do you mean, there’s no sound? The tree’s roots snap, the trunk comes crashing down and hits the ground. This generates vibrations that travel through the ground and the air. That’s where the energy of the fall goes, into heat and sound. Are you saying that if people leave the forest, the tree violates Conservation of Energy?
Barry: But no one hears anything. If there are no humans in the forest, or, for the sake of argument, anything else with a complex nervous system capable of ‘hearing,’ then no one hears a sound.
Albert: This is the dumbest argument I’ve ever been in. You’re a niddlewicking fallumphing pickleplumber.
Barry: Yeah? Well, you look like your face caught on fire and someone put it out with a shovel.
Albert: The tree produces acoustic vibrations. By definition, that is a sound.
Barry: No one hears anything. By definition, that is not a sound.
Albert: My computer’s microphone can record a sound without anyone being around to hear it, store it as a file, and it’s called a ‘sound file.’ And what’s stored in the file is the pattern of vibrations in air, not the pattern of neural firings in anyone’s brain. ‘Sound’ means a pattern of vibrations.
Barry: Oh, yeah? Let’s just see if the dictionary agrees with you.
Albert: Hah! Definition 2c in Merriam-Webster: ‘Sound: Mechanical radiant energy that is transmitted by longitudinal pressure waves in a material medium (as air).’
Barry: Hah! Definition 2b in Merriam-Webster: ‘Sound: The sensation perceived by the sense of hearing.’
Albert and Barry, chorus: Consarned dictionary! This doesn’t hel...
LW2.0: Technology Platform for Intellectual Progress
by Ruby — 2019-06-19T20:25:20.228Z

This post presents one lens I (Ruby) use to think about LessWrong 2.0 and what we’re trying to accomplish. While it does not capture everything important, it does capture much and explains a little how our disparate-seeming projects can combine into a single coherent vision.

I describe a complimentary lens in LessWrong 2.0: Community, Culture, & Intellectual Progress.

(While the stated purpose of LessWrong is to be a place to learn and apply rationality, as with any minimally specified goal, we could go about pursuing this goal in multiple ways. In practice, myself and other members of the LessWrong team care about intellectual progress, truth, existential risk, and the far-future and these broader goals drive our visions and choices for LessWrong.)

Our greatest challenges are intellectual problems

It is an understatement to say that we can aspire to the world being better than it is today. There are myriad forms of unnecessary suffering in the world, there are the utopias we could inhabit, and, directly, there is an all too real chance that our civilization might wipe itself out in the next few years.

Whether we do it in a state of terror, excitement, or both - there is work to be done. We can improve the odds of good outcomes. Yet the challenge which faces us isn’t rolling up our sleeves and “putting in hard work” - we’re motivated - it’s that we need to figure out what exactly it is we need to do. Our problems are not of doing, but knowing.

Which interventions for global poverty are most cost-effective? What is the likelihood of a deadly pandemic or nuclear war? How does one influence government? What policies should we want governments to adopt? Is it better for me to earn-to-give or do direct work? How do we have a healthy, functioning community? How do we cooperate from groups small to large? How does one build a safe AGI? Will AGI takeoffs be fast or slow? How do we think and reason better? Which questions are the most important to answer? And so on, and so on, and so on.

One of our greatest challenge is answering the questions before us. One of our greatest needs is to make more intellectual progress: to understand the world, to understand ourselves, to know how to think, and to figure out what is true.

Technologies for intellectual progress

While humans have been improving our understanding the world for hundreds of thousands of years, our rate of prog...
LW2.0: Community, Culture, and Intellectual Progress
by Ruby — 2019-06-19T20:25:08.682Z

This post presents one lens I (Ruby) use to think about LessWrong 2.0 and what we’re trying to accomplish. While it does not capture everything important, it does capture much and explains a little how our disparate-seeming projects can combine into a single coherent vision.

I describe a complimentary lens in LessWrong 2.0: Technology Platform for Intellectual Progress.

(While the stated purpose of LessWrong is to be a place to learn and apply rationality, as with any minimally specified goal, we could go about pursuing this goal in multiple ways. In practice, myself and other members of the LessWrong team care about intellectual progress, truth, existential risk, and the far-future and these broader goals drive our visions and choices for LessWrong.)

A Goal for LessWrong

Here is one goal that it might make sense to me for LessWrong to have:

A goal of LessWrong is to grow and sustain a community of aligned* members who are well-trained and well-equipped with the right tools and community infrastructure to make progress on the biggest problems facing humanity, with a special focus on the intellectual problems.

*sharing our broad values of improving the world, ensuring the long-term future is good, etc.

Things not well-expressed in this goal:

LessWrong’s core focus on rationality and believing true things.
What LessWrong aims to provide users.

(These are better expressed in the About/Welcome page.)

I want to point out that while the above might be a goal of the LessWrong team, that doesn’t mean it has to be a goal of our users. I wholeheartedly welcome users who come to LessWrong for their own purposes such as improving their personal rationality, learning interesting things, getting feedback on their ideas, being entertained by stimulating ideas, or participating socially in a community they like.

My RTC-P Framework

The reason I like the goal expressed above is that provides unity to a wide range of activities we devote resources to. I like to group those activities into four overarching categories.

Recruitment: attracting new aligned and capable members to our community (and creating a funnel into EA orgs and projects)

Training: providing means both new and existing members improve their skills, knowledge, and effectiveness.

Community/Culture: we aim to improve community health and flourishing via means such encouraging good epistemic norms, affordance to interact, e.g. ...
1hr talk: Intro to AGI safety
by Steven Byrnes — 2019-06-18T21:41:29.371Z

This is an hour-long talk (actually ~45 minutes plus questions) that I gave on AGI safety technical research, for an audience of people in STEM but with no prior knowledge of the field. My goal was to cover all the basic questions that someone might have when considering whether to work in the field: motivation, basic principles, a brief survey of the different kinds of things that people in the field are currently working on, and resources for people who want to learn more. I couldn't find an existing talk covering all that, so I wrote my own.

In the interest of having good educational and outreach material available to the community, I’m happy for anyone to copy any or all of this talk. The actual slides are in Powerpoint format here, and below you'll find screenshots of the slides along with the transcript.

Abstract: Sooner or later—no one knows when—we'll be able to make Artificial General Intelligence (AGI) that dramatically outperforms humans at virtually any cognitive task. If, at that point, we are still training AIs the way we do today, our AGIs may accidentally spin out of control, causing catastrophe (even human extinction) for reasons explained by Stuart Russell, Nick Bostrom, and others. We need better paradigms, and a small but growing subfield of CS is working on them. This is an interdisciplinary field involving machine learning, math, cybersecurity, cognitive science, formal verification, logic, interface design, and more. I'll give a summary of motivation (why now?), foundational principles, ongoing work, and how to get involved.

Opening

Thanks for inviting me! I had an easy and uneventful trip to get here, and we all know what that means ... Anyone? ... What it means is that evil robots from the future did not travel back in time to stop me from giving this talk. [Laughter.] And there’s an important lesson here! The lesson is: The Terminator movie is the wrong movie to think about when we talk about safety for advanced AI systems. The right movie is: The Sorcerer’s Apprentice![1] When the machines become more and more intelligent, they don’t spontaneously become self-aware and evil. Instead, the machines do the exact thing that we tell them to do … but the thing that we tell them to do is not what we actually want them to do! I’ll get back to Mickey Mouse later.

Here’s the outline of this talk. I’m going to start with some motivation. Artific

...
Magic Arena Bot Drafting
by Zvi — 2019-06-18T16:00:00.402Z

Epistemic Status: Attempting to be useful and to learn via exploration. Real proposals. Content assumes familiarity with Magic: The Gathering booster drafting, but does not much depend on a knowledge of deep Magic strategy, so it holds potential interest for those interested in game narrow AI and machine learning.

Reflections Partially Brought on By (Eric Moyer @ Channel-Fireball): Drafting Like a Computer Part 1, Part 2

Part 1 – The Problem is Hard

I appreciated Eric Moyer’s articles because they show that building a good bot is hard. They do so by failing to design good bots. Eric’s proposed bots are overly formulaic and predictable, with obvious holes in their systems. If you are feeding one of these bots and know their rankings, you can narrow down their whole path based on possible first pick colors and strengths in highly exploitable ways. The proposed bots will reliably fail to take into account holistic information, they won’t draft playable decks or try to shore up their weaknesses or strengths, and so on, and so on.

That’s not because Eric isn’t making a reasonable first attempt here. It’s because the problem is hard! Programming a bot to draft reasonably is hard.

What we want above all is a bot experience that feels human, where you’re trying to figure out what is going on, cannot ever fully rely on your hunches, and need to consider all the information at your disposal. Where you get to make hero reads, cram decks down the throats of other drafters, sweat our attempts to wheel key cards, and otherwise have a fresh, exciting and new experience more often.

Making this even more difficult is that bots that we know are bots will inherently feel like bots, and that bots will get blamed even more than humans for mistakes. When bots mimic human patterns of doing things that look crazy, the bot often looks crazy rather than realistic or human.

As an extreme example. if a person passes us an Ugin, we know that a human made a horrible mistake. Maybe someone didn’t see it, or misclicked, or is trying something weird. And we can be happy for our good fortune. If a bot passed it, as happened to Gerry Thompson on the first day of War of the Spark drafts, we get deeply confused and frustrated.

Same goes for everything else the bots do. Suddenly you have someone who interacted with the problem, and is therefore responsible for it, as per Copenhagen Interpretation of E...
Is the "business cycle" an actual economic principle?
by Liron — 2019-06-18T14:52:00.348Z

E.g. Investopedia: Business Cycle has a section "Stages of the Business Cycle":

All business cycles are characterized by several different stages, as seen below.

1. Expansion. This is the first stage. When expansion occurs, there is an increase in employment, incomes, production, and sales. People generally pay their debts on time. The economy has a steady flow in the money supply and investment is booming.

2. Peak. The second stage is a peak when the

economy

hits a snag, having reached the maximum level of growth. Prices hit their highest level, and economic indicators stop growing. Many people start to restructure as the economy's growth starts to reverse.

3. Recession: These are periods of contraction. During a recession, unemployment rises, production slows down, sales start to drop because of a decline in demand, and incomes become stagnant or decline.

4. Depression:

Economic growth

continues to drop while unemployment rises and production plummets. Consumers and businesses find it hard to secure credit, trade is reduced, and bankruptcies start to increase. Consumer confidence and investment levels also drop.

5. Trough: This period marks the end of the depression, leading an economy into the next step: recovery.

6. Recovery: In this stage, the economy starts to turn around. Low prices spur an increase in demand, employment and production start to rise, and lenders start to open up their credit coffers. This stage marks the end of one business cycle.

If all you know is that markets are anti-inductive, and therefore anticipate a random walk with an average of 10%/year growth, don't you automatically expect the stock market fluctuations to sometimes look like bull markets and recessions?

So does the "business cycle" add any explanatory power, or is it merely the type of retroactive non-explanation that you get when you read a news article about the 1-day movement of the stock market?

Maybe you can argue that the business cycle is self-reinforcing due to psychological reactions.

But to the degree that the business cycle is a separate understandable phenomenon, can't investors use that understanding to place bets which make them money while dampening the effect?
Reason isn't magic
by Benquo — 2019-06-18T04:04:58.390Z
Link: http://benjaminrosshoffman.com/reason-isnt-magic/

Here's a story some people like to tell about the limits of reason. There's this plant, manioc, that grows easily in some places and has a lot of calories in it, so it was a staple for some indigenous South Americans since before the Europeans showed up. Traditional handling of the manioc involved some elaborate time-consuming steps that had no apparent purpose, so when the Portuguese introduced it to Africa, they didn't bother with those steps - just, grow it, cook it, eat it.

The problem is that manioc's got cyanide in it, so if you eat too much too often over a lifetime, you get sick, in a way that's not easily traceable to the plant. Somehow, over probably hundreds of years, the people living in manioc's original range figured out a way to leach out the poison, without understanding the underlying chemistry - so if you asked them why they did it that way, they wouldn't necessarily have a good answer.

Now a bunch of Africans growing and eating manioc as a staple regularly get cyanide poisoning.

This is offered as a cautionary tale against innovating through reason, since there's a lot of information embedded in your culture (via hundreds of years of selection), even if people can't explain why. The problem with this argument is that it's a nonsense comparison.

First of all, it's not clear things got worse on net, just that a tradeoff was made. How many person-days per year were freed up by less labor-intensive manioc handling? Has anyone bothered to count the hours lost to laborious traditional manioc-processing, to compare them with the burden of consuming too much cyanide? How many of us, knowing that convenience foods probably lower our lifespans relative to slow foods, still eat them because they're ... more convenient?

How many people didn't starve because manioc was available and would grow where and when other things wouldn't?

If this is the best we can do for how poorly reason can perform, reason seems pretty great.

Second, we're not actually comparing reason to tradition - we're comparing changing things to not changing things. Change, as we know, is bad. Sometimes we change things anyway - when we think it's worth the price, or the risk. Sometimes, we're wrong.

Third, the actually existing Portuguese and Africans involved in this experiment weren't committed ...