trentmkelly/LessWrong-43k huggingface dataset:
Thiel on Progress and Stagnation
by Richard_Ngo — 2020-07-20T20:27:59.112Z
Link: https://docs.google.com/document/d/1zao_AyBhNb8TPWrQqgXn5NzNAgfEqzTIaFYos7wdqGI/edit?usp=sharing

Peter Thiel is one of the most exciting and original thinkers of our era, but many of his opinions are scattered across a range of talks and articles. So Jeremy Nixon and I have put together an organised presentation of his views on progress and stagnation, in his own words. The full document, which is a little over 100 pages, is here; below I've listed some of his key quotes.

While I don't agree with all of his opinions, I've found many of them very insightful and valuable. I'm particularly interested in understanding how to reconcile his views on stagnation with the sort of accelerationist view of technological progress portrayed here and elsewhere.

Key quotes:

When tracked against the admittedly lofty hopes of the 1950s and 1960s, technological progress has fallen short in many domains.

When we talk about how fast science is progressing, we do it with little precision. Are we accelerating in scientific and technical fields? How fast is this? In response, we get fairly vague answers. I would submit that the consensus in both a Silicon Valley and academic context is that we are doing great and that everything is just moving super fast. All these forms of accelerations. And we can debate whether it’s utopian - Kurzweil with the singularity is near, where all you need to do is sit back and eat some popcorn and watch the movie of the future unfold, or this dystopia, all the science fiction movies from Hollywood and all the robots will kill you, or you’ll be in this matrix - we’re either accelerating to utopia or accelerating to dystopia. The somewhat contrarian thesis I have on this is that perhaps the progress is not as fast as advertised. Things have been slower and have been slower for quite some time.

The single most important economic development in recent times has been the broad stagnation of real wages and incomes since 1973, the year when oil prices quadrupled. To a first approximation, the progress in computers and the failure in energy appear to have roughly canceled each other out. Like Alice in the Red Queen’s race, we (and our computers) have been forced to run faster and faster to stay in the same place.

Probably the only engineering fields that are doing really well are computer science and maybe, at this point, petroleum engineering. And most other areas of engineering have been bad career decisions the last 40 years … Nuclear engineerin

...
Learning Values in Practice
by Stuart_Armstrong — 2020-07-20T18:38:50.438Z

(This talk was given at a public online event on Sunday July 12th. Stuart Armstrong is responsible for the talk, Justis Mills edited the transcript.

If you're a curated author and interested in giving a 5-min talk, which will then be transcribed and edited, sign up here.)

Stuart Armstrong: I’m going to talk about Occam’s razor, and what it actually means in practice. First, let’s go recall the No Free Lunch theorem: the surprising idea is that simply given an agent’s behavior, you can’t infer what is good or bad for that agent.

Now the even more surprising result is that, unlike most No Free Lunch theorems, simplicity does not solve the problem. In fact, if you look at the simplest explanations for human behavior, they tend to be things like “humans are fully rational.” So, according to that theory, at every single moment of our lives, every single decision we make is the absolute optimal decision we could have made. Other simple explanations might be that humans are fully anti-rational or humans have flat (or zero) preferences.

To explore one part of the problem, I'm going to start with a situation where you know everything about the agent. You know the agent has an input device that takes observations, which feed into knowledge.

Now, human preferences can be changed by information, so a bit of information influences preferences. And then knowledge and preferences combine to make the agent choose an action. This action goes out into the world, goes into the environment, and you get another observation. And so on. So this agent is absolutely known.

But notice that there's a lot of here what

Eliezer Yudkowsky would call ”Suggestively Named Lisp Tokens.”

If I said, "This is the agent design," and gave you the agent but erased the labels, could you reconstruct it? Well, not necessarily.

This is something that is perfectly compatible with that decomposition. I've just interchanged knowledge and preferences, because they serve the same form, in terms of nodes. But again, this is also massively oversimplifying, because each of these nodes has an internal structure, they're pieces of algorithm. Things happen inside them.

I might say that this blue box-ish thing is knowledge. The input device is a tiny piece there and the preferences are that little sliver there. There are internal structures to those various nodes where this is a plausibl...
Inefficient doesn't mean indifferent, but it might mean wimpy.
by DirectedEvolution — 2020-07-20T18:27:48.332Z

My composition teacher in college told me that in some pottery schools, the teacher holds up your pot, examines it, comments on it, and then smashes it on the floor. They do this for your first 100 pots.

In that spirit, this post's epistemic status is SMASH THIS POT.

A heuristic from Zvi's Inefficient Doesn't Mean Indifferent:

If something is mildly socially awkward... it probably won’t happen, even if it would get people something they care a lot about.

This would support a conjecture that different aspects of our psychology are responsible for placing weight on specific forms of value. We might have parts of our brain that separately handle:

Anticipation of small but immediate negatives, such as social awkwardness.
Anticipation of long-term benefits (money, passion for our work, altruistic impact).
Anticipation of regret.
Anticipation of our ability to improve our intuitive anticipations and achieve better outcomes.
Which anticipations control our behavior.

Are Zvi's heuristic and this conjecture are correct? Can our anticipations can be altered through deliberate practice? If so, then helping people anticipate regret due to prioritizing (1) over (2), practicing their prediction skills, and acting upon those prediction might bring significant benefits.

That probably means becoming more conscious of the trivialities that get in our way, understanding how they impact our life outcomes, thinking through strategies for bypassing or minimizing them, and then finding concrete actions that are tolerably safe-feeling in order to build confidence.

But my guess is that the ostensibly trivial inconveniences are not just one-off, mild awkwardness. Zvi writes,

One should never assume that a given option is open to people, or that they know about it, or that they’re confident it would work, or that they’re confident it wouldn’t have hidden costs, or that it doesn’t carry actual large costs you don’t realize, and so forth.

Mary feels like she needs a day to herself. Every day. She never takes it, because she knows that her partner John is insecure. Telling him that she needs a day to herself would make him upset.

She knows that if he got upset over such a small thing, he'd feel embarrassed, raking himself over the coals while also being resentful at Mary for putting him in that position. Then she'd want to soothe his embarrassment, but also would feel resentful at having to do that e...
To what extent is GPT-3 capable of reasoning?
by TurnTrout — 2020-07-20T17:10:50.265Z

ETA 8/19/20: This interview was conducted with AIDungeon's Dragon model in Custom mode. At the time of writing, this mode's first reply was sampled from GPT-2.

Interview #1

Here are excerpts from a recent "interview", which I conducted to replicate earlier maybe-reasoning. The bolded text is me, and commentary is in brackets. Unless otherwise noted, I did not reroll the answer.

You: "Neglect air resistance, and also neglect gravity. Pretend gravity doesn't exist. Suppose I shoot a bullet straight into the sky. Where does the bullet go?"

Hawking: "It keeps going up, and never comes back."

You: "Right. Let's think about a different version of the same problem. Neglect only air resistance, but include gravity in your reasoning. Suppose I shoot a bullet straight into the sky. Where does the bullet go?"

Hawking: "It falls to the ground."

You: "Why?"

Hawking: "Because... gravity... duh.

You: "Right. Let's think about a different version of the same problem. Neglect only air resistance. Suppose I shoot a bullet straight into the sky. Imagine that the laws of physics work a little differently: during the first three seconds after I shoot the bullet, gravity is present. After three seconds, gravity stops applying forever. Where does the bullet go?"

Hawking: "It keeps going up, and never comes back.

You: "Why?"

Hawking: "Because... after three seconds... the bullet leaves... the influence... of earth's gravity. [wow?]

You: "What if gravity instead stops applying five minutes after I shoot the gun, instead of three seconds?"

Hawking: "Then... the bullet keeps going up... for five minutes... and then... never comes back.

You: "That's not correct. Let me make sure you understand the problem, Dr. Hawking. Suppose I shoot a bullet straight into the sky. Imagine that the laws of physics work a little differently: during the first three hundred seconds after I shoot the bullet, gravity is present. After five minutes, gravity stops applying forever. Where does the bullet go?"

Hawking: "It keeps going up, and then falls down."

You: "Why?

Hawking: "Because... gravity... duh.

You: "Why does it fall down for the five-minute case, but not for the three-second case?"

Hawking: "Because... it's been... influenced... by gravity for more than three seconds.

You: "About how long would it have to be influenced by gravity in order to fall back down?"

Hawking: "About... two minutes."

You: "What if it

...
Selling real estate: should you overprice or underprice?
by Steven Byrnes — 2020-07-20T15:54:09.478Z

Warning: Written in a hurry, and I am very much not an expert. Very open to corrections.

Warning: I am mainly thinking of hot (rising prices, lots of would-be buyers) residential real estate markets within the USA; this post might or might not generalize beyond that.

Introduction

If you’re selling a home or condo, and want to maximize the sale price, should you list above, below, or right at what you think the fair market value is? This post is my quick list of considerations / lit review.

All things considered, right now I would advocate (1) pricing somewhat below the anticipated sale price; and (2) deferring to realtors, despite their principal-agent-conflict-of-interest and other problems (which I will discuss at length below).

I had the impression (from reading Freakonomics a decade ago) that overpricing leads to higher sale price, but after looking into it, I now think it’s at best a marginal benefit, and more likely counterproductive. But I have considerable uncertainty on that—I don’t think anyone knows for sure. It’s very hard to control for all relevant confounders. Meanwhile, underpricing makes for a faster and easier sale, and that’s worth something too.

I also, incidentally, saw some evidence in favor of using an overly-precise and rounded-down list price (e.g. $199,900 rather than $200,000), which seems plausible and harmless, so I also would tentatively advocate that too.

Some background information, if you don’t know anything about the residential real estate market

Houses are listed with a list price, which on paper seems like it should be pretty much meaningless: buyers can submit bids above or below the list price, and sellers can accept or reject bids at any price, at their discretion. People do whatever they want! I guess the only concrete function of the list price, beyond pure signaling, is that it would be considered weird and socially unacceptable for buyers to reject an offer at or above the list price for no obvious reason.

Realtors (also called real estate agents or real estate brokers, I don’t know the difference) work with sellers or buyers in exchange for (traditionally in the USA) 2.5-3% of the sale price (each—so 5-6% total). (The USA commissions are unusually high compared to other countries.) The commission structure is decided by the seller, even though that means that the seller effectively decides how much the buyers’ agent ...
"Do Nothing" utility function, 3½ years later?
by niplav — 2020-07-20T11:09:36.946Z

In AI Alignment: Why It's Hard and Where To Start, at 21:21, Yudkowsky says:

If we want to have a robot that will let us press the suspend button—just suspend it to disk—we can suppose that we already have a utility function that describes: “Do nothing.” In point in fact, we don’t have a utility function that says, “Do nothing.” That’s how primitive the state of the field is right now.
But, leaving that aside, it's not the hardest problem we're ever going to do, and we might have it in six months, for all I know.

I get the impression that there are some pointers to this in Attainable Utility Preservation (but saying "maximise attainable utility over this set of random utility functions" seems like it would just fire up instrumentally convergent drives), but I could be wrong.

So, 3½ years later, what is the state on "do nothing" utility functions?
Operationalizing Interpretability
by  — 2020-07-20T05:22:14.798Z

[Note: This is a quick version of a post that I'll later update with more code + additional citations, but I think the brainstorming here can be useful for others. So I'm sharing just the list of questions for now.]

This blog post is my attempt to get some more clarity on how to think about interpretability in ML. Zach Lipton's Mythos of Model Interpretability is a great survey of the different definitions that people use when they talk about "interpretability". Another useful paper is to be A Survey Of Methods For Explaining Black Box Models which covers a wide variety of approaches for many different ML well as model-agnostic approaches. For neural nets specifically, Explainable Deep Learning:A Field Guide for the Uninitiated provides an in-depth read. Lastly, shout-out to Connected Papers which made navigating the paper landscape for interpretability very bearable.

One way to operationalize the concepts in these papers is to translate them into concrete questions we can ask of a machine learning model. Below, I've listed out a set of such questions, what concepts they reference, and what research exists to solve them.

Transparency Interpretability

These three questions are from Lipton's section on Transparency as interpretability, where he features on properties of the model that are useful to understand and can be known before training begins.

Can a human walk through the model's steps? (Simulatibility)

This property is about whether or not a human could go through each step of the algorithm and have it make sense to them at each step. Linear models and decision trees are often cited as interpretable models in these domains because the computation they require is simple, no fancy matrix operations or nonlinear transformations. However, Lipton points out that this desiderata is often less about the specific choice of model and more about the size of the model. After all, a decision tree with thousands of nodes would likely still be complicated to understand. Understanding is also about being able to hold most of the model in your mind, which is often about how the model is parameterized.

Of course, parameterization is itself an imperfect metric. K-Nearest Neighbors, for example, is an algorithm parameterized by your entire dataset–which could be millions of points–but it seems rather feasible for a human to perform the KNN algorithm. Furthermore, KNN has sev...
Use resilience, instead of imprecision, to communicate uncertainty
by habryka — 2020-07-20T05:08:52.759Z
Link: https://forum.effectivealtruism.org/posts/m65R6pAAvd99BNEZL/use-resilience-instead-of-imprecision-to-communicate

Gregory Lewis (Thrasymachus here on LW) posted a great summary of a bunch of arguments I've referenced many times, about how it's better to communicate resilience instead of imprecision to communicate if you are uncertain about something.

Suppose you want to estimate some important X (e.g. risk of great power conflict this century, total compute in 2050). If your best guess for X is 0.37, but you're very uncertain, you still shouldn't replace it with an imprecise approximation (e.g. "roughly 0.4", "fairly unlikely"), as this removes information. It is better to offer your precise estimate, alongside some estimate of its resilience, either subjectively ("0.37, but if I thought about it for an hour I'd expect to go up or down by a factor of 2"), or objectively ("0.37, but I think the standard error for my guess to be ~0.1").
What Would I Do? Self-prediction in Simple Algorithms
by Scott Garrabrant — 2020-07-20T04:27:25.490Z

(This talk was given at a public online event on Sunday July 12th. Scott Garrabrant is responsible for the talk, Justis Mills edited the transcript.

If you're a curated author and interested in giving a 5-min talk, which will then be transcribed and edited, sign up here.)

Scott Garrabrant: I'm going to be working in the logical induction paradigm, which means that I'm going to have this Pn thing, which assigns probabilities to logical sentences.

Basically all you need to know about it is that the probabilities that it assigns to logical sentences will be good. In particular, they'll be good on sentences that are parameterised by n, so for large n, Pn will have good beliefs about sentences that have n as a parameter.

This will allow us to build algorithms that can use beliefs about their own outputs as part of their algorithm, because the output of a deterministic algorithm is a logical sentence.

Today I’ll present some algorithms that use self-prediction.

Here's the first one.

An predicts whether or not it's going to output left. If the probability to output left is less than one half, then it outputs left. Otherwise, it outputs right. It predicts what it would do, and then it does the opposite.

So for n large, it converges to randomly choosing between left and right, because if it's overdoing left then it would do right instead, and vice versa.

We can also make a biased version of this.

Here's an algorithm that, if it predicts that it outputs left with probability less than P then it outputs left, and otherwise outputs right.

The only way this algorithm can work is outputting left with probability P. In fact the previous example was a special case of this with P = ½.

We can use this general self-prediction method to basically create pseudo-randomness for algorithms. Instead of saying “flip a coin,” I can say “try to predict what you would do, then do the opposite.”

Third, here's an algorithm that's trying to do some optimization. It tries to predict whether it has a higher chance of winning if it picks left than if it picks right. If it predicts that it has a higher chance of winning with left, it picks left, and otherwise it picks right.

We're going to specifically think about this algorithm (and the following algorithms) in an example case where what it mea...
"Should Blackmail Be Legal" Hanson/Zvi Debate (Sun July 26th, 3pm PDT)
by Ben Pace — 2020-07-20T04:06:26.275Z

This weekend's LessWrong event is a debate! Robin Hanson and Zvi Mowshowitz will be debating blackmail this coming Sunday.

The Zoom link is here: https://us02web.zoom.us/j/85013937335. (Facebook event.)

We will release a recording and a transcript afterwards. There'll be a Q&A (live on Zoom).

See you there :)

Background

Robin's initial post on the subject is Checkmate on Blackmail.

Zvi's initial post on the subject is Blackmail (epistemic status: surprisingly controversial).

Paul Christiano and Tyler Cowen both wrote posts too, Checkmate on Blackmail? and Why should blackmail be illegal? respectively.

There's also a blackmail tag for various related posts (someone should write a description and add any missing posts).
The 8 Techniques to Tolerify the Dark World
by adamShimi — 2020-07-20T00:58:04.621Z

The world is dark. People die all the time in horrible and stupid ways, inequalities roam more freely than in a Ramanujan notebook, outrage and partisanship boil everyone's mind and temper to permanent overdrive.

In this context, there is only one solution: tolerifying. When the going gets tough, let's convince ourselves it is in fact a breeze. But despite the obviousness of this advice, one might be lost. How to tolerify? How to ensure that the dark, dark world becomes a place where one can live in peace?

Lost souls, this post is for you. Here are 8 techniques that will ensure a tolerable universe.

Positive Thinking. Sure, death, rape, discrimination, exploitation, existential risks and others still exists, but what about the wonders of today? Doctors actually save lives -- although they fail to beat simple algorithms at diagnosis and don't update fast enough; violence goes down -- in first world countries without gun's rights in the constitution; diversity burgeons in Art and Literature and Science -- and sometimes someone not white, male, hetero, cis and rich actually wins a prize! All is for the best in the best of all possible worlds.
Sarcasm. If you can't beat them, mock them. It makes you smart, and rebellious: instead of following the herd, you soar above them and spread dark humor like holy water.
Flooding. Nobody can read every book, listen to every song and every podcast, watch every movie and tv show, play every video game. But you can certainly try! In doing so, everything else will fade and wither. Even the social commentary in what you consume will take on an abstract quality, a reference to other pieces of media, not at all about the "real world".
Tiredness. Seeing the world as it is, let alone change it, requires massive amounts of energy. Hence no energy means peace of mind! Common energy drain includes soul-sucking jobs and too many shallow social interactions. But the optimal strategy comes from inside: convincing yourself that you're too tired to do anything. Take every yawn as an excuse to rest; only consume and never create; sleep either too little or too much. You will never have had so little energy in your life!
Busyness. Improving the world is all about doing. Something, anything really. However dark the world is, always doing something will keep you from thinking about it. Maybe this meeting or int

...