trentmkelly/LessWrong-43k huggingface dataset:
Cup-Stacking Skills (or, Reflexive Involuntary Mental Motions)
by Duncan Sabien (Inactive) — 2021-10-11T07:16:45.950Z

This essay will require you to watch three short Youtube videos, totaling less than two minutes.

Naming things is hard. Generally speaking, a thing should be named evocatively, such that people find it memorable and sticky, or precisely, such that people can reconstruct the concept just from its title.

(So, "Moloch," or "trigger-action planning.")

This essay is about "cup-stacking skills." It's a noun that I use in phrases like "I think you're exhibiting a cup-stacking skill right now" or "I'm slowly trying to unravel this cup-stacking skill" or "I think we should consult Dave; he has the relevant cup-stacking skill."

Unfortunately, that's not a great name. Most people, encountering the name, will have to memorize both the concept and the label, rather than having to just memorize the concept and have the label stick, or just memorize the label (and being able to rederive the concept from it).

Sorry. I've made a genuine effort for the past couple of years to find a better name, and failed. Since I've failed, I need you to watch three Youtube videos.

Here's the first video.

This is me, in my kitchen, cup-stacking. It's a fun little game-slash-sport in which you stack and unstack cups in a specific pattern, to see how fast you can go. It's extremely rewarding once you get even a tiny bit good at it; you can feel things going almost-right and the pattern loops onto itself and it's very easy to just chase that feeling of smoothness for hours at a time. I've probably put between 50 and 100 hours into cup stacking over the past ten years, though at the time of filming I hadn't pulled them out much at all in the past two.

Here's the second video.

This is my partner Logan, cup-stacking for the very first time, after having watched me demonstrate the pattern exactly twice. They've got a little card for reference on the table with them, so they know each of the three end-states they're shooting for, but otherwise I told them to not worry about process or technique and just generally do their best to imitate what they'd seen in a low-stress sort of way.

The thing about the literal skill of cup-stacking is that (approximately) "anyone can do it." Even as total beginners, most people can follow the directions and hold the pattern in mind and get the cups to stack up in the right shapes.

There's obviously a big difference between...
Outdoor dancing is likely very safe
by jefftk — 2021-10-11T01:40:02.498Z

After the spontaneous contra dance at

Porchfest, I'm helping organize

another one. I wanted to
get a better sense of how much covid risk an attendee would be taking,
so I ran some numbers on

microcovid. If everyone is
masked and vaccinated, I count ~2.2 microcovids:

~1.7
from your partner. While your partner is not the only person your
head gets close to, you're this close to at most one person at a time,
so for simplicity assume its your current partner.

~0.2
from your neighbors and next/previous neighbors.

~0.2
from your next/previous hands fours.

~0.08
from the hands fours one farther away.

If you have multiple lines close together, you could ~double these
numbers. Other social dances are likely ~half as risky.

This is a very low level of risk: about 1% of a cautious risk budget
of 200 microcovids/week (1% risk of covid/year).

I wish I'd run these numbers sooner: this is probably our last chance
for an outdoor dance in Boston before spring.

An outdoor dance in October 2013

We may end up dancing indoors this winter. Over the next few
months I think our communities are likely to move away from treating
covid as something where we have a duty to make substantial sacrifices
to limit spread. Once everyone is vaccinated who wants to be,
including boosters and approving the vaccine for kids, I think
people will view the tradeoffs very differently.

Comment via: facebook
Book Review: Existential Risk and Growth
by jkim2 — 2021-10-11T01:07:27.320Z

Many questions about existential risk are speculative and hard to answer. Even specific ones, like "How have the odds of nuclear war changed after the United States withdrew from the Iran deal?", are still multi-layered and rather chaotic. Because of this, there haven't been many attempts at answering general questions ones even when they could be more significant. One such question is: what's the effect of economic growth on existential risk?

Leopold Aschenbrenner's paper "Existential Risk and Growth" (2020) begins to answer this. Since it seems like hasn't gotten as much attention as it should, hopefully more people will read this paper after this book review. Since there's this good summary already out there, and already good discussion over this, I'll instead summarise that discussion with additional thoughts that are mine to save time for anyone who's interested. Any errors included here are mine.

Summary

TLDR:

Continued economic growth is needed to better mitigate risks of existential catastrophes. Or as Aschenbrenner states:

It is not safe stagnation and risky growth that we must choose between; rather, it is stagnation that is risky and it is growth that leads to safety.

Longer summary:

Out of existential risks that are human-induced many could be caused by some sort of technological progress. As Nick Bostrom imagines, the process of technological development is like "pulling balls out of a giant urn".

But because technology can outpace safety mechanisms that can regulate it, economic growth could be our best shot to develop the capacity for these safety mechanisms. Examples of dangerous technologies and thier respective safety mechanisms include climate change / carbon capture technology, artificial intelligence / successful alignment.

The paper argues that if humanity is currently in a "time of perils", which is when we are advanced enough to have developed the means for our destruction but not advanced enough to care about safety, stagnation doesn't solve the problem because we would just stagnate at the same level of risk. Eventually a nuclear war would doom humanity. Intead, faster economic growth could initially increase risk, but would also help us get past the time of perils more quickly.

As economic growth makes people richer, they will invest more in safety, protecting against existential catastrophes. This is because the value of safety increases as...
Secure homes for digital people
by paulfchristiano — 2021-10-10T15:50:02.697Z

Being a “digital person” could be scary—if I don’t have control over the hardware I’m running on, then someone else could get my code and run tons of copies in horrible conditions. (See also: qntm’s Lena.)

It would be great to guarantee digital people some control over their situation: 1. to control their local environment and sensations, 2. to avoid unauthorized rewinding or duplicating.

I’ll describe how you could modify the code of a digital person so that they retain this control even if an adversary has access to their source code. This would be very expensive with current cryptography. I think the overhead will eventually become cheap enough that it’s possible to do for some digital people, though it will likely remain expensive enough that it is never applied to most digital people (and with luck most digital people will be able to feel secure for other reasons).

Part 1: the right to control my environment

My ideal

I live in a comfortable virtual home. I control all of the details of that world.
When people communicate with me, I can choose how/whether to hear them, and how/whether to update my home based on what they say (e.g. to render an avatar for them)
Sometimes I may occupy a virtual world where a foreign server determines what I see, feel, or hear. But even then I can place boundaries on my experiences and have the ability to quickly retreat to my home.
I have as much control as feasible over my own mental state and simulated body. No one else can tamper directly with them.
I can choose to pause myself for as long as I want (or permanently).
My local environment is private, and I have access to plenty of tamper-proof storage. I can do whatever I want with computers in my home, including e.g. verifying signatures or carrying on encrypted conversations.

Implementation

First we write a simple environment that reflects all my desiderata (the “home”).
Then I apply indistinguishability obfuscation to (me + home), so that the house becomes private and tamper-proof. (This is an extremely expensive operation, more on that later.)
I distribute the obfuscated home and hopefully destroy any unprotected copies of myself.

One conceptual difficulty is that indistinguishability obfuscation applies to circuits whereas I would like to obfuscate a long-running program. But this can be handled straightforwardly, as discussed in Appendix A.

The home could...
The evaluation function of an AI is not its aim
by Yair Halberstadt — 2021-10-10T14:52:01.374Z

I think this is a relatively straightforward point, but it's one that I think has definitely led to confused thinking on my part about AI alignment, and presumably on others. After writing this post Ruby pointed out there's an entire sequence on this. However I think this post still provides benefit as being shorter and less formal.

The way most machine learning systems learn involves performing a task using some algorithm (e.g. translate this text into Japanese), evaluating the outcome using some evaluation function (e.g. how close is this to the human translation), and then adapting the original algorithm so that it would do better on that task according to the given evaluation function.

I think there's a tendency to conflate this evaluation function with the aims of the AI itself, which leads to some invalid conclusions.

I started thinking about this when I thought about GPT3. The task GPT3 performs is to predict the continuation of a piece of text. It is evaluated on how close that is to the actual continuation. It then uses backpropogation to adjust the parameters of the model to do better next time.

I asked myself whether if we scaled up GPT3 until it has superhuman performance on this task, might we end up with an alignment problem?

My first thought was yes - GPT3 might work out it was being fed articles from the internet for example, and hack the internet so that all articles have identical text, giving it 100% accuracy on it's evaluation function.

I think that the chances of this happening are actually quite slim. It's easier to realize this if we compare machine learning to evolution, where we have the benefit of hindsight to see what actually occurred.

Evolution works similarly to machine learning. It performs a task (reproduce), and is evaluated based on how well it replicates itself. It uses a hill climbing algorithm (randomly mutate + natural selection) to improve at the original task.

It has often been wondered why humans don't intrinsically want to reproduce. For example, why do humans want to have sex even when birth control is being used? Eliezer has a great sequence on that, but the key point is that humans are adaptation executers not fitness maximizers. The pressure of evolution causes adaptations to persist which have proven useful for maximizing reproduction in the past. It doesn't make lifeforms actually care about reproduction.

Similarly the ...
Book Review: Open Borders
by Sam Enright — 2021-10-10T12:27:11.554Z

Edit: I now have a comment response post for this post on my personal website.

This is a (significantly updated)

cross

-

post

from my personal website. All sources are from the book unless otherwise mentioned.

Bryan Caplan is an economist at George Mason University known for his wacky libertarian views about various social and political issues (especially

education

).

Open Borders: The Science and Ethics of Immigration

is his latest book, which is in the format of a graphic novel illustrated by Zach Weinersmith of

SMBC

fame. The graphic novel format works really well. The art style is cute and non-fiction graphic novels are grossly underrated (some favourites of mine are

Maus

and

Logicomix

). Realistically, most people are not going to read a regular book about the economics of immigration. But this way Caplan can lure us in with fun cartoons!

Caplan really does believe that there should be no restrictions on immigration whatsoever, and that’s exactly what his cartoon representation argues for in this book. The basic argument goes like this: people should, in general, be allowed to make decisions that they think will improve their lives, assuming they’re not hurting anyone. Moving to a new country is exactly such a decision. Since immigrants often move in search of work, moving is associated with a massive increase in economic prosperity. By moving to the US and receiving no additional training or education, the average citizen of a developing country can expect their income to increase fivefold; and for countries like Nigeria, tenfold. This is because developed countries are safer and have better quality institutions, so immigrants are more productive in them. Also, the opportunity cost of workers in rich countries is higher, hence why service workers like gardeners are so much more productive there. The gains are so vast that a standard estimate is that open borders would double world GDP. Yet rich countries continue to restrict immigration, sometimes through formal caps, and sometimes through complicated bureaucracy and paperwork which at best dissuades people from entering and at worst makes it literally impossible (like rejecting you for not filling out the middle name section on a form

when you don’t have a middle name

). Some of the arguments against immigration are xenophobic or racist, but many are legitimate concerns brought up in good faith. Most of...
Without a phone for 10 days
by Benjamin Spiegel — 2021-10-10T05:19:31.486Z

I woke up this morning to a bricked Google Pixel 4. After taking it to a local repair shop for a diagnosis, I was told that a fuse had been blown on the motherboard. A board-level repair would cost half as much as a brand new phone, and I was thinking about upgrading to the new Pixel 6 once it came out later this month. After spending a few hours sorting out account details and learning about replacement options with my carrier I learned that it would cost me $150 to get a replacement by this coming Monday. What good would it do to pay $150 for a phone that I would only have for a week until upgrading?

And that’s when I realized I had stumbled into a very unique moment in which I had every reason to attempt something I had been hesitantly curious to try: living without a phone. After all, the Pixel 6 was rumored to launch only ten days from now on October 19th. And if I decide at the end that life is better with a smartphone, then I’ll get one.

Okay so there are a few things I’m a bit worried about. The most obvious one is that I’ll be unreachable to close family and friends during this time. Ten days isn’t a ton of time, so I decided to email those closest to me to tell them about this experiment. A less obvious problem is that I’ll be unable to do typical two-factor authentication, which my university and some other services periodically require. The good news is that I have backup codes saved on my laptop, but it’s kind of a hassle.

I’m very curious to see how this will turn out. I’m hoping that I’ll appreciate the disconnection so much that I won’t want to go back to smartphones. I’ll likely still want the basic call and text functionality, so maybe I’ll go with a simpler phone. I had heard of the lightphone before and loved the idea, but was afraid of giving up apps like Uber for emergencies. Today I looked into some other “feature phones” and discovered the Nokia 6300, the Punkt. MP02, and the Mudita Pure. Anyway, I’ll probably write at least one more post on this experiment when the ten days are up.

Has anyone else made the switch away from their smartphone? Would love to hear about it below.
The Extrapolation Problem
by lsusr — 2021-10-10T05:11:03.072Z

In a previous post I predicted that machine learning is limited by the algorithms it uses (as opposed to hardware or data). This has two big implications:

Scaling up existing systems will not render human thinking obsolete.

Replacing the multilayer perceptron with a fundamentally different algorithm could disrupt[1] the entire AI industry.

Today's artificial neural networks (ANNs) require far more data to learn a pattern than human beings. This is called the sample efficiency problem. Some people think the sample efficiency problem can be solved by pouring more data and compute into existing architectures. Other people (myself included) believe the sample efficiency problem must be solved algorithmically. In particular, I think the sample efficiency problem is related to the extrapolation problem

Extrapolation and Interpolation

Machine learning models are trained from a dataset. We can think of this dataset as a set of points in vector space. Our training data is bounded which means it's confined to a finite region of vector space. This region is called the interpolation zone. The space outside the interpolation zone is called the extrapolation zone.

Consider an illustraion from this paper. Ground truth is black. Training data is green. An ANN's output trained on the training data is in blue.

A human being could look at the green training data and instantly deduce the pattern in black. The ANN performs great in the interpolation zone and completely breaks down in the extrapolation zone. The ANNs we use today can't generalize to the extrapolation zone.

One way around this problem is to paper over it with big data. I agree that big data is a practical solution to many real-world problem. However, I do not believe it is a practical solution to all real-world problems. An algorithm that can't extrapolate is an algorithm that can't extrapolate. Period. No amount of data or compute you feed it will get it to extrapolate. Adding data just widens the interpolaton zone. If a problem requires extrapolation then an ANN can't do it―at least if you're using today's methods.

Under this definition of extrapolation, GPT-3 is bad at extrapolation. An example of extrapolation would be if GPT-n could invent technology in advance of training data (including prompts) from before the technology was invented. For example, if GPT-n could produce schematics for a transistor using only...
The S&P 500 Will Drop Below 3029 Before July 16 (65 percent confidence)
by FCCC — 2021-10-10T04:02:36.671Z

If someone knows the best way for an Australian to buy US Put options, please let me know.

This post is somewhat unfinished, unedited, and much less detailed than I wanted it to be due me changing jobs.

Empirical prediction

My credence that the S&P 500 index will drop below 3029 at some point before this time next year, i.e. over a 30 percent decrease from current levels (at the time of writing), is much higher compared to both the market’s likely estimate (because rational investors would sell their stocks if they thought the index was overvalued at the current level, which at the time of writing is 4327) and compared to my base rate of such large crashes. If I had to estimate my current credence of a 30 percent devaluation happening within the next year (i.e. before July 16, 2022), I’d put it at around 65 percent.

Caveats

As you might see from the dates being behind the date at which this was posted, I’ve been writing this for a while. I’ve delayed posting this because I can’t fully justify the probabilities I’m assigning. I take methodology quite seriously (i.e. how do I know that my line of reasoning reliably leads to true conclusions), so I hate filling out my argument with weakly justified back-of-the-napkin calculations and gut feelings. But that’s all I’ve got here. So take everything I’ve written with a grain of salt.

The market actively fights any prediction. If an indicator of a future market movement is found, it will tend to become less effective over time, as other people learn to adjust their bets to accommodate that information. This means that all of my justifications based on historical data could be very wrong. I’ll be careful to, in each of these instances, give you reasons why a particular historical indicator might not indicate what it once did (note: ran out of time to do this).

To be clear, this is not financial advice. This is an empirical prediction with a justification that might have critical errors. I’m not a permabear, but I think the market conditions are heavily weighting the probability distribution of returns on the negative side of graph. To give you an idea of my confidence in my prediction, I sold all my stocks on 12 July 2021; the week after, I explained my position to some of my loved ones, and they sold their retirement stocks and held cash soon after; currently, I am deciding how to safely execute a large short posit...
Bayeswatch 12: The Singularity War
by lsusr — 2021-10-10T01:04:38.476Z

The Singularity Cyberwar took 6 minutes. Vanilla human beings never again led an organization larger than a million people.

The missile exchange took 6 hours. It destroyed all significant semiconductor fabricators. Computronium became a nonrenewable resource.

The world's aircraft carriers and Gauss battleships lasted 6 days.

It took 6 weeks to shoot down the last F-15 and Chengdu J-20.

Analog radios were being mass-produced 6 months after that.

Cheap analog radios are often staticy. It's not always obvious who's talking, or where they're coming from.

"We're taking heavy casualties on the Southern front."

"I've never seen androids like this."

"The Baltic AI says the Transsiberian AI has gone rogue but the Transsiberian AI said the Baltic AI has gone rogue. What's going on?"

"I tried to radio Bayeswatch HQ but we've lost our entire chain of command. Who's in charge over there?"

An assistant handed Vi the shortwave radio. "I am," she said.

Vi established her HQ in the Natanz fortress, a several-decades-old network of tunnels built inside of a mountain designed to withstand a nuclear bunker-buster. Her hivemind mainframe was stored next to their C&CAI (command-and-control artificial intelligence) in a artificial cave originally built to house plutonium and centrifuges.

She kept the perimeter lightly staffed. The best defense was a dynamic counterattack. The outermost perimeter was just a barbed wire fence.

"I have a report from our security guard," said Trinity. She technically didn't need to talk to Vi but it kept the collective sane. The vanilla human habits helped keep the officers at ease too.

Vi looked up from the war room's C&CAI diagnostic readout. "Is it a diplomat?" she said.

"No. Miriam's back," said Trinity.

Vi brought Miriam into her private office/bedroom. Vi sat in her chair. Miriam sat on the bed with her legs dangling off. There were no windows or viewscreens this deep underground. Just a verdant watercolor painted from memory.

"Can I have a coffee?" said Miriam.

Vi pressed the intercom. "One coffee please. Real, if possible. The best stuff we have. Black. No cream or sugar," said Vi into the intercom.

A private entered through the airlock, silently gave Miriam her coffee and departed.

Vi took off the hazmat suit she wore in the war room. Miriam took off the breathing apparatus she had worn to survive outside.

"This is so much better. I haven't been abl...