trentmkelly/LessWrong-43k huggingface dataset:
Empirical Findings Generalize Surprisingly Far
by jsteinhardt — 2022-02-01T22:30:19.142Z

Previously, I argued that emergent phenomena in machine learning mean that we can't rely on current trends to predict what the future of ML will be like. In this post, I will argue that despite this, empirical findings often do generalize very far, including across "phase transitions" caused by emergent behavior.

This might seem like a contradiction, but actually I think divergence from current trends and empirical generalization are consistent. Findings do often generalize, but you need to think to determine the right generalization, and also about what might stop any given generalization from holding.

I don't think many people would contest the claim that empirical investigation can uncover deep and generalizable truths. This is one of the big lessons of physics, and while some might attribute physics' success to math instead of empiricism, I think it's clear that you need empirical data to point to the right mathematics.

However, just invoking physics isn't a good argument, because physical laws have fundamental symmetries that we shouldn't expect in machine learning. Moreover, we care specifically about findings that continue to hold up after some sort of emergent behavior (such as few-shot learning in the case of ML). So, to make my case, I'll start by considering examples in deep learning that have held up in this way. Since "modern" deep learning hasn't been around that long, I'll also look at examples from biology, a field that has been around for a relatively long time and where More Is Different is ubiquitous (see Appendix: More Is Different In Other Domains).

Empirical Generalization in Deep Learning

I'll consider three examples in deep learning: adversarial examples, data efficiency, and out-of-distribution generalization.

Adversarial examples. Adversarial examples were first discovered in 2013, a year after the AlexNet paper (which arguably marked the start of "modern" deep learning). Since then, there have been at least two qualitative changes in deep networks---pretraining to provide better inductive bias, and the emergence of few-shot learning---plus some smaller changes in architecture. As far as I know, adversarial examples affect every neural network model that exists. Moreover, the main (partial) remedy, adversarial training, is the same in every architecture and domain.

Data efficiency. Starting around 2016, there were papers showing that lea...
Is there a better choice than "Stop After Win" (or does double-checking lead to failure loops)?
by Nicole Dieker — 2022-02-01T20:33:05.504Z

Crossposted.

Here we go.

To recap:

Last week I wrote about

what happened when I applied metrics to my piano practice

, in which I outlined the “secret to learning” in eight steps:

Define win condition.
Define action you are going to take to achieve win condition.
Take defined action.
Evaluate action both against its original definition (that is, did you do what you said you were going to do or did you do something else) and against the win condition.
Ask yourself what is keeping you from achieving your win condition. Describe it as specifically as possible.
Define action you are going to take to solve/address/eliminate obstacle preventing win condition.
Repeat 2-7 until win condition is achieved.
STOP.

Step #8, “Stop after win,” has prompted significant discussion both in various subsets of the internet and in various rooms of our home. It goes against the standard practice (pun intended) of repeating a learned skill multiple times after the first accurate recall, otherwise known as “playing it five times correctly.”

Is it better to stop after an accurate recall, or to immediately repeat?

If I’ve got it wrong, that’s good news. I’ll say “WOO-HOO-HOO, I LEARNED SOMETHING NEW” and update my practice strategy accordingly.

But let’s do some more investigating — and share the initial results of a randomized experiment.

There are two arguments in favor of immediate repetition:

Immediate repetition allows you to confirm that you know something.

I have no counter-argument for this. I agree.

Immediate repetition increases the win/fail ratio. If you fail at a passage four times and win once, what is your brain likely to remember the next day — the four fails or the one win? If you can generate four fails followed by five wins, your brain should be more likely to remember the win condition.

My first counter-argument is that this only works if you can actually generate five consecutive wins in a row. What happens if it looks more like FFFFWFWWFW? What’s your brain going to remember then?
I’m also going to counter-argue — and my entire post was going to be on this particular topic, when I originally set out to write it — that accurately describing why you failed and what you need to do differently next time does as much to strengthen your neural network as an accurate pass. Conversely, you can weaken your neural network by playing the passage over and over without stopping after each pass to

...
2022 ACX Predictions: Buy/Sell/Hold
by Zvi — 2022-02-01T20:30:02.348Z

Previously: 2021 ACX Predictions B/S/H, 2021 ACX Results Analysis

It’s that time again. Scott’s predictions are live and I am here for it. I also have a response in the works for a different ACX post, but that requires editing, and this doesn’t.

I will play by Scott’s rules, since that’s only fair. For each question, there’s a five minute time limit, and checking prediction markets isn’t allowed until the prediction is locked. Checking financial markets is fine.

1. Biden approval rating (as per 538) is greater than fifty percent: 40%

Sell to 30%. Biden is highly unlikely to have any great accomplishments in 2022 other than confirming a new supreme court justice. Presidents get smashed in midterms, which is indicative of generally low popularity around that time. Also with the way popularity works now 50% would be substantially above par. As we’ve seen, many Democrats disapprove of Biden when he doesn’t get things done the way they’d like, and the Republicans sure aren’t approving, also Fed is going to be raising interest rates to control inflation and inflation isn’t going to fully go away within a year, so getting to 50% in time seems really tough.

2. At least $250 million in damage from a single round of mass protests in US: 10%

Buy to 15%. This is not that high a bar, as the Floyd protests seem to have been around $2 billion on a very quick Google search, the Rodney King riots over $800 million. Note that this does not say it has to be BLM protests, it can be caused by anything. That includes anti-Covid-restriction protests, inflation protests, and a number of other plausible candidates. Cutting this down so dramatically from previous estimates while also expanding what counts seems like an overreaction, especially with so much more to protest about coming soon, and I’m going to buy.

3. PredictIt thinks Joe Biden is most likely 2024 Dem nominee: 80%

Hold. For there to be a particular other more likely candidate, a lot has to go wrong or Biden’s health needs to be failing rather badly. Biden simply saying he won’t run might be enough if he is explicitly endorsing Harris but I don’t sense that Harris is earning that nod and isn’t proving to be a strong politician or ally. There’s no one in the wings other than Harris who looks like they could draw together the field enough to be an outright favorite if Biden is in similar health to today. But a year is a lo...
Revisiting Brave New World Revisited (Chapter 3)
by Justin Bullock — 2022-02-01T17:17:39.923Z

We begin with a visit to Huxley's Brave New World, where one of the world controllers, Mustapha Mond, is having a conversation with John The Savage and his two companions Bernard Marx and Helmholtz Watson.

Brave New World: The Controller and The Savage

“But why is it prohibited?" asked the Savage. In the excitement of meeting a man who had read Shakespeare he had momentarily forgotten everything else.

The Controller shrugged his shoulders. "Because it's old; that's the chief reason. We haven't any use for old things here."

"Even when they're beautiful?"

"Particularly when they're beautiful. Beauty's attractive, and we don't want people to be attracted by old things. We want them to like the new ones."

"But the new ones are so stupid and horrible. Those plays where there's nothing but helicopters flying about and you feel the people kissing." He made a grimace. "Goat and monkeys!" Only in Othello's words could he find an adequate vehicle for his contempt and hatred.

"Nice tame animals, anyhow," the Controller murmured parenthetically.

"Why don't you let them see Othello instead?"

"I've told you; it's old. Besides, they couldn't understand it."

Yes, that was true. He remembered how Helmholtz had laughed at Romeo and Juliet. "Well then," he said, after a pause, "something new that's like Othello, and that they could understand."

"That's what we've all been wanting to write," said Helmholtz, breaking a long silence.

"And it's what you never will write," said the Controller. "Because, if it were really like Othello nobody could understand it, however new it might be. And if it were new, it couldn't possibly be like Othello."

"Why not?"

"Yes, why not?" Helmholtz repeated. He too was forgetting the unpleasant realities of the situation. Green with anxiety and apprehension, only Bernard remembered them; the others ignored him. "Why not?"

"Because our world is not the same as Othello's world. You can't make flivvers without steel-and you can't make tragedies without social instability. The world's stable now. People are happy; they get what they want, and they never want what they can't get. They're well off; they're safe; they're never ill; they're not afraid of death; they're blissfully ignorant of passion and old age; they're plagued with no mothers or fathers; they've got no wives, or children, or lovers to feel strongly about; they're so conditioned that they pract...
Before Colour TV, People Dreamed in Black and White
by Sam Enright — 2022-02-01T12:50:30.151Z

On an episode of Julia Galef’s podcast, the philosopher Eric Schwitzgebel said the following:

For [dreams], there was actually a literature that's very interesting where people in the '50 in the United States and the '40s thought that dreams just generally were black and white. I don't think that they thought it was just dreams in the United States, as influenced by media. I think they just thought dreams are a black and white kind of thing. Most people thought that in the 1950s. It's related to the presence of media in the culture, so if you look pre-20th century, very few people will say that dreams are black and white. If you look 21st century, very few people will say that dreams are black and white. You look at the arc of it and it relates to the dominance of black and white film media in the culture.

And we got some cross-cultural evidence for this. This guy emailed me and said, "We should try this in China," because this was about the year 2000. He said, "Well, in rural China, most people are exposed to black and white media, their TVs are black and white, whereas in urban China, most people -- especially the wealthier people -- are exposed to mostly colour media." So we asked about their dreams and we found rural people in China in the early 2000s tended to say that their dreams were black and white, and urban people tended to say their dreams were coloured.

That became the paper Schwitzgebel, Huang and Zhou 2006. If true, this is one of the most bonkers things I have ever learned.

The thing is, it’s extremely unlikely that black and white TV actually changed the contents of people’s dreams. There’s no plausible way that the small proportion of time people spent watching visual media could radically change dreams about things we see in colour every day. Rather, people don’t know whether they dream in colour. Dreams may not even have associated colours one way or the other! Indeed, when I asked a few friends and family whether they dreamed in colour, a surprising number of them answered “I don’t know”. When the dominant culture has a reference of visual media in black and white, you think you dream in black and white. And when your culture has a reference of visual media in colour, you think you dream in colour.

This relates to a generally underappreciated aspect of consciousness: vagueness. Your conscious experience of the world is...
SIA becomes SSA in the multiverse
by avturchin — 2022-02-01T11:31:33.453Z

TL;DR: SIA works only in a finite universe, as an argument for a larger universe. In the infinite universe, we can compare concentrations of observers in different regions, but it is SSA.

SIA: All other things equal, an observer should reason as if they are randomly selected from the set of all possible observers. Note that "randomly selected" is weighted by the probability of the observers existing: under SIA you are still unlikely to be an unlikely observer, unless there are a lot of them.

SSA: All other things equal, an observer should reason as if they are randomly selected from the set of all actually existent observers (past, present and future) in their reference class.

Main thesis: SIA becomes SSA in the infinite universe, and the reference class in that case is the class of my exact copies.

Proof: In the infinite multiverse, there are no “possible observers”, as all possible observers actually exist somewhere, but maybe in small concentrations. (UPDATE: I discuss how infinite size of the universe generates all possible observers in comments.) Therefore, the idea of selecting from possible observers is meaningless. The selection is happening from actually existing observers. Thus, SIA turns into SSA in the multiverse.

The main difference between SIA and SSA is what they are looking at: SIA looks only at the fact that I exist at all, and SSA looks at some observables like a colour of the room in which I am located. In the infinite universe, the fact of my existence becomes uninformative: everybody exists.

Thought experiments like “God toss a coin and created one observer” are meaningless, as there is no God and there are all possible outcomes of tossing a coin. In the infinite universe, there are many of my exact copies, so we can discuss how they are distributed.

Now examples.

According to SIA, if there are two alternatives: (1) there are trillion observers in the universe, and (2) there are trillions of trillions of observers in the universe, – then the second hypothesis is overwhelmingly more probable. This is known as a Presumptuous Philosopher (PP) thought experiment and it is often used as a counterargument to SIA, as such predictive capability seems counterintuitive. Why does it look counterintuitive? Because it looks like a free lunch into the nature of the universe. (...
Immune, In a Nutshell
by Jsevillamol — 2022-01-31T23:07:38.352Z

Immune: A journey into the mysterious system that keeps you alive, by Philipp Dettmer, is a cross between a high-school biology textbook and an epic two-part saga. At its core there are two stories, describing a bacterial and a viral infection respectively. Both battles introduce us to our immune system, how it recognizes pathogens, and how it dispatches them.

I enjoyed the book a lot. It’s become my late night lecture for the last week, as my body fought off its own battle against COVID-19. Even fatigued and hazy, the book manages to explain and entertain.[1]

Through the book I learned many useful and fascinating concepts: how the immune system has weaponized evolution to produce antibodies, how proteins come in matching pairs that are used to recognize pathogens, and how the body prevents the immune system from attacking itself.

While simplified, I felt the book did a good job at explaining the reasons why the immune system works the way it does, instead of leaving the reader with just surface metaphors as many others do.[2][3]

One of the many gorgeous illustrations in the book. It represents the immune system fighting a war on two fronts: a bacterial rush and a viral takeover.

Should you read this book? Possibly!

If you want to learn more about immunology, but you are unsure of whether the book is right for you, I would first watch the relevant Kurzgesagt videos:

How the Immune System ACTUALLY Works

You Are Immune Against Every Disease

Tiny Bombs in your Blood - The Complement System

The videos have a very similar style and cover the same material as some chapters of the book. If they seem interesting and novel, you will definitely enjoy the book!

You should also read this book if you want to learn good communication. The clever use of metaphors and illustrations, interwoven with technical details, make the topic approachable and memorable. I will be thinking from now on of MHC class II molecules as hot dog buns, and I will be wiser because of it.

The only part of the book I am hesitant to recommend is the fourth one. The messages about the importance of vaccinations, the dangers of smoking, and the cruelty of nature fell a bit flat to me. The digression on why autoimmune diseases are on the rise in first world countries, how allergies might have evolved to fight parasitic worms, and the COVID pandemic felt vague and handwavy.

Overall, I am glad that...
Meaning of Words - An Exercise for Active Thinking
by aysajan — 2022-01-31T20:18:38.040Z

An example of fancy math jargon

What comes to your mind when you first hear the phrase “abstract nonsense”? For me, the first picture in my head was some obvious falsehood, like 1+1=3. I was not so sure about the “abstract” part, thinking it might not be something as concrete as 1+1=3. So I decided to look it up and here was what I found (from

Wikipedia

):

“In mathematics, abstract nonsense, general abstract nonsense, generalized abstract nonsense, and general nonsense are terms used by mathematicians to describe abstract methods related to category theory and homological algebra.”

What a surprise! Not being a mathematician, I found this explanation not very helpful. But before looking any further, I decided to do some guessing about what category theory and homological algebra refer to. First, what might be category theory? For instance, in mathematics we have different categories of numbers, such as natural numbers, real numbers, complex numbers, etc. But why and how are these different number categories created? Maybe category theory refers to the study of such categorizations. For homological algebra, I pictured two things in my head. First is homo, which usually means “the same thing/category”. Next is logical, which usually is “some kind of reasonable argument”. For instance, if A>5 and B>7, then we can derive A+B>12 by logic. Since homological algebra is placed together with category theory, I guessed that they were related to each other in some way.

I might be completely wrong with those pictures in my head (Indeed I was wrong for the most part, except the close connection between

category theory

and

homological algebra

), but the point is the active thinking process. The focus is an exercise where, when we come across some unknown/unfamiliar statements/claims, we try to build a picture. Now the question is: Why might this try-to-build-up-a-picture-in-my-head thing be useful? While I try to build up a picture in my head, I am actually actively thinking, instead of looking for answers externally right away. In this process, I dig into my brain and look for examples to interpret what I am reading. It also might illuminate new insights that haven’t been considered by the writer. This happens when the picture in my head turns out to be very different from what the author intends to present. Another benefit of this exercise is that it might help identi...
On infinite ethics
by Joe Carlsmith — 2022-01-31T07:04:44.244Z

(Cross-posted from Hands and Cities)

And for all this, nature is never spent…

— Gerard Manley Hopkins

Summary:

Infinite ethics (i.e., ethical theory that considers infinite worlds) is important – both in theory and in practice.
Infinite ethics puts serious pressure on various otherwise-plausible ethical principles (including some that underlie common arguments for “longtermism”). We know, from impossibility results, that some of these will have to go.
A willingness to be “fanatical” about infinities doesn’t help very much. The hard problem is figuring out how to value different infinite outcomes – and in particular, lotteries over infinite outcomes.
Proposals for how to do this tend to be some combination of: silent about tons of choices; in conflict with principles like “if you can help an infinity of people and harm no one, do it”; sensitive to arbitrary and/or intuitively irrelevant things; and otherwise unattractive/horrifying.
Also, the discourse thus far has focused almost entirely on countable infinities. If we have to deal with larger infinities, they seem likely to break whatever principles we settle on for the countable case.
I think infinite ethics punctures the dream of a simple, bullet-biting utilitarianism. But ultimately, it’s everyone’s problem.
My current guess is that the best thing to do from an infinite ethics perspective is to make sure that our civilization reaches a wise and technologically mature future – one of superior theoretical and empirical understanding, and superior ability to put that understanding into practice.
But reflection on infinite ethics can also inform our sense of how strange such a future’s ethical priorities might be.

Thanks to Leopold Aschenbrenner, Amanda Askell, Paul Christiano, Katja Grace, Cate Hall, Evan Hubinger, Ketan Ramakrishnan, Carl Shulman, and Hayden Wilkinson for discussion. And thanks to Cate Hall for some poetry suggestions.

I. The importance of the infinite

Most of ethics ignores infinities. They’re confusing. They break stuff. Hopefully, they’re irrelevant. And anyway, finite ethics is hard enough.

Infinite ethics is just ethics without these blinders. And ditching the blinders is good. We have to deal with infinites in practice. And they are deeply revealing in theory.

Why do we have to deal with infinities in practice? Because maybe we can do infinite things.

More specifically, we might be a...