trentmkelly/LessWrong-43k huggingface dataset:
The Parable of the King and the Random Process
by moridinamael — 2023-03-01T22:18:59.734Z

~ A Parable of Forecasting Under Model Uncertainty ~

You, the monarch, need to know when the rainy season will begin, in order to properly time the planting of the crops. You have two advisors, Pronto and Eternidad, who you trust exactly equally.

You ask them both: "When will the next heavy rain occur?"

Pronto says, "Three weeks from today."

Eternidad says, "Ten years from today."

"Good," you say. "I will begin planting the crops in a little bit over five years, the average of your two predictions."

Pronto clears his throat. "If I may, Your Grace. If I am right, we should start preparing for the planting immediately. If Eternidad is right, we should expect an extreme drought, and will instead need to use the crown's resources to begin buying up food from our neighbors, for storage. These two predictions reflect totally different underlying world models, and demand two totally different and non-overlapping responses. Beginning the planting in five years is the wrong choice under either model, and guarantees that the nation will starve regardless of which of us is right."

Eternidad adds: "Indeed, Your Grace. From Pronto's point of view, waiting five years to prepare is just as bad as waiting ten years – the rains will be long passed, by his model. From my perspective, likewise, we should take action now to prepare for drought. We must allocate resources today, one way or the other. What you face is not so much a problem of prediction but a decision problem with an important component of probability. Absolutely do not view our predictions as two point estimates to be averaged and aggregated – view them instead as two distinct and mutually exclusive futures that must be weighed separately to determine the best allocation of resources. Unfortunately, given the unrectifiable disagreement between Pronto and myself, the best course of action is that we do our best to make reasonable preparations for both possibilities. We should spend some fraction of our treasury on planting grain now, in case the rains arrive soon, and the remainder on purchasing food for long-term storage, in the case of prolonged drought."

You, the monarch, ponder this. You do not want to have to split your resources. Surely there must be some way of avoiding that? Finally you say: "It seems like what I need from you two is a probability distributi...
To MIRI-style folk, you can't simulate the universe from the beginning
by the gears to ascension — 2023-03-01T21:38:26.506Z

To those whose formalisms still depend on simulating the universe from the beginning (those who don't, ignore this post, I'm not trying to correct connectionism here):

It doesn't matter how smart a being is; Even temporarily assuming superdeterminism[1], Laplace's demon[2] still cannot fit in the universe it's attempting to simulate - it'd be even more extreme than Maxwell's demon[3], which is also forbidden even in a classical deterministic universe, I think. There's cryptographic amounts of chaos-induced entropy in the way. Impressively cryptographic amounts, actually.

There aren't simplifications that let you get around this, you have to simulate probabilistically to deal with the fact that you can never exactly guess what chaos did in the blurred-out details. Everything has sensitive dependence on initial conditions - the amount you can get around that by being smart is significant, but bumping a butterfly to intentionally change the course of history in a specific, intentional way ain't ever happening without the bump transferring nanites, because the atmosphere is far too chaotic. You need active control to make small energy transfers accumulate into large ones when trying to do open-loop control through a system which is in a chaotic fluid regime, eg the atmosphere; you can't pre-model it and bump it in ways that accumulate into perfection, there's too much chaos and the slightest amount of noise in your sim will result in accumulation into error.

A key point here: it really doesn't matter how smart you are. You can do all sorts of crazy stuff, don't get me wrong, intelligence lets you simplify systems where there isn't chaos. But you can't simulate 100 copies of the universe in your head using 100 watts. or 100 gigawatts. or 100exawatts, and expect to get something that exactly matches. It cannot be done, no matter how big your brain is. There's too much chaos.

Though there might be larger scale structures in things like life, since life is in fact built out of resistance to error accumulation. But it's still noisy, and the maximum you can constrain claims about it is still not favorable towards formalisms that simulate from the beginning.

If this changes your plans at all

I'd suggest looking into whether you can generalize your formalism so it works on arbitrarily small or large patches of spacetime with boundary conditions, and then focus on how to ...
OpenAI introduce ChatGPT API at 1/10th the previous $/token
by Arthur Conmy — 2023-03-01T20:48:51.636Z
Link: https://openai.com/blog/introducing-chatgpt-and-whisper-apis

OpenAI add gpt-3.5-turbo to their API, charging $0.002 per 1k tokens. They cite "a series of system-wide optimizations" for 90% cost reduction.

Another example of the dizzying speed of language model progress.
Progress links and tweets, 2023-03-01
by jasoncrawford — 2023-03-01T20:33:23.267Z
Link: https://rootsofprogress.org/links-and-tweets-2023-03-01

The Progress Forum

Anton Howes on what the Dutch did better than the English
AMA with Gale Pooley & Marian Tupy, authors of Superabundance
Ben Reinhardt AMA has concluded

Opportunities

Lex Fridman wants to meet people, fill out this form to get coffee with him
Essay contest: “What does a perfect research institute look like?” (via @akuataya)

News & announcements

Works in Progress Issue 10 (thread from @s8mb)
OpenAI announces its long-term strategy and principles
BioGPT, an LLM trained on biomedical research literature (via @tunguz)
Constitutional AI: training LLMs with behavioral principles (from @AnthropicAI)
UAE turned on its third nuclear reactor in 3 years (@BrianGitt)

Articles & essays

“How can anyone stop being fascinated for long enough to be angry?” Scott Aaronson on GPT
Jerusalem Demsas on “permission-slip culture” in America (via @atrembath)
“Cyborgism” as a strategy for using LLMs

Queries

Can anyone intro Dwarkesh to Robert Caro? (@dwarkesh_sp)
What’s the best book on Taylorism? (@davidtlang)
What are the best books about insurance? (@ByrneHobart)
Best writing to illustrate to the layman where we’re at with AGI? (@PatrickFinley_)

Quotes

Everything has to be invented, including stop signs and numbered highways
The great equalizer: indoor plumbing
When your boat gets in an accident and works better afterward
The restless motivation of Paul Ehrlich (the German microbiologist)
A good metaphor for breakthroughs
In the 19th century this was considered a sick burn

Tweets & threads

Institute for Progress one-year anniversary retrospective (@calebwatney)
All solutions reveal new problems. But to be solutions they must be better problems
Virtually everything about spacecraft was figured out by a Russian eccentric decades before rocketry
“The technology we have can do X. Therefore, it will always be limited to X”
An easy way to trick ChatGPT
Can China lead on AI if free speech is literally a feature the technology?
Listen to people when you’re impressed by how they think, not when you agree with what they think (@AdamMGrant channeling @waitbutwhy)
The invention of the modern pictogram

Charts

Workers needed at S&P 500 companies to generate $1M in revenue (@tobi)

UPDATE: Adjusted for inflation. Less dramatic but same trend (h/t @Zac Hatfield-Dodds below)
Taboo "compute overhang"
by Zach Stein-Perlman — 2023-03-01T19:15:02.515Z

There's no consensus definition of "compute overhang" or "hardware overhang." See e.g.

1

2

3

4

5

, and I've recently seen researchers use several different definitions. And I asked some friends what "hardware overhang" means and they had different responses (a plurality said it means sufficient hardware for human-level AI already exists, which is not a useful concept). If you say "compute overhang" without clarification, many people will misunderstand you.

Instead of tabooing it, we could declare a canonical definition. I think the best candidate is something like: there is a compute overhang to the extent that the largest training runs could quickly be scaled up. But I think it's probably better for people to avoid the term or define it whenever they use it.

P.S. You should also define "takeoff speed," "transformative AI," and "warning shot" whenever you're using them in a precise way; their usage varies a lot too. (Using those terms to vaguely gesture at "the speed of progress around human-level AI," "powerful AI," and "legible scary AI event," respectively, seems ok).
Call for Cruxes by Rhyme, a Longtermist History Consultancy
by Lara — 2023-03-01T18:39:04.358Z
Link: https://forum.effectivealtruism.org/posts/hfXy8EbyNTuBixjJf/call-for-cruxes-by-rhyme-a-longtermist-history-consultancy

(I'm looking to research historical examples with implications for AI Risk specifically. If you want think this could be useful to you, consider filling out this form)

TLDR; This post announces the trial period of

Rhyme

, a history consultancy for longtermists. It seems like longtermists can profit from historical insights and the distillation of the current state of historical literature on a particular question, both because of its use as an intuition pump and for information about the historical context of their work. So, if you work on an AI Governance project (research or policy) and are interested in augmenting it with a historical perspective, consider registering your interest and the cruxes of your research

here.

During this trial period of three to six months, the service is free.

"History doesn’t repeat, but it rhymes." - Mark Twain

What Problem is Rhyme trying to solve?

When we try to answer a complicated question like “how would a Chinese regime change influence the international AI Landscape”, it can be hard to know where to start. We need to come up with a hypothesis, a scenario. But what should we base this hypothesis, this scenario on? How would we know which hypotheses are most plausible? Game theoretical analysis provides one possible inspiration. But we don't just need to know what a rational actor would do, given particular incentives. We also need intuitions for how actors would act irrationally, given specific circumstances.

Would we have thought of considering the influence of close familial ties between European leaders when trying to predict the beginning of the first world war? (Clark, 2014)
Would we have considered Lyndon B. Johnson's training as a tutor for disadvantaged children as a student when trying to predict his success in convincing congresspeople effectively? (Caro, 1982)
Would we have considered the Merino-Wool-Business of a certain diplomat from Geneva when trying to predict whether Switzerland would be annexed by its neighbouring empires in 1815? (E. Pictet, 1892).

In summary: A lot of pivotal actions and developments depend on circumstances we wouldn’t expect them to. Not because we’d think them to be implausible, but because we wouldn’t think of considering them. We need inspiration and orientation in this huge space of possible hypotheses to avoid missing out on the ones which are actually true.

I...
Fighting without hope
by Orpheus16 — 2023-03-01T18:15:05.188Z

We've given up hope, but not the fight. -- Nate Soares, probably

1

Many people I know are working on projects that they believe have a low (or very low) probability of being helpful. Even when they think diving into their work is the “right move” or “best option”, they find it hard to stay motivated or avoid burnout.

It may be hard to feel motivated about continuing to fight, since doubling our chances of survival will only take them from 0% to 0%. (from Death with Dignity)

I’ve been finding it helpful to distinguish between three concepts:

Epistemic hope: Thinking your plan is likely to work and likely to have a meaningful impact.
Emotional hope: Feeling good about your plan (regardless of your underlying forecasts or probability distributions)
Effort: The amount of time, energy, and resources you devote to your plan.

2

The three of these tend to go together. Alice thinks her plan is 80% likely to work, so she feels good when she thinks about her plan, and she puts in a lot of effort.

But they don’t have to be.

Consider Bob: he thinks his plan is <10% likely to work. But he still feels good when he thinks about his plan. He knows he’s taking on extremely hard problems, and he’s proud of himself and his colleagues for working on the plan despite its grim forecast. He also puts in a lot of effort.

I know people like Bob, and sometimes I feel like Bob. I might feel excited about a plan (despite its low probability of success), or grateful toward others for persevering and fighting (even if I have grim forecasts about the value of their work).

But not always. Sometimes I feel more like Carol.

Carol thinks her plan is <10% likely to work. She feels badly about this. She feels frustrated and disappointed with herself, perhaps even her colleagues, perhaps even her entire community or civilization. She feels hopeless. She also puts in a lot of effort.

Carol doesn’t put in effort because she has epistemic hope (she doesn’t think her efforts are likely to lead to success) or emotional hope (she doesn’t feel some sense of energy or gratitude or “fighting against the odds” spirit).

Carol tries because… well, she just does. Maybe she tries because she thinks fighting is

dignified

. Maybe she thinks fighting makes sense under epistemic uncertainty, or she tries out of habit, or to impress her friends, or because she identifies as a p...
Sunlight is yellow parallel rays plus blue isotropic light
by Thomas Kehrenberg — 2023-03-01T17:58:02.706Z

When you look up the color temperature of daylight, most sources will say 6500K, but if you buy an LED with that color temperature, it will not look like the sun in the sky. It will seem bluer (or, less yellow-y). Yet, 6500K is arguably the correct number. What is going on?

The answer is Rayleigh scattering. What we perceive as sun rays on the Earth’s surface has traveled through a lot of atmosphere, whereby a lot of the blue light has been scattered away, such that the sun rays look a bit yellowish. But the blue light isn’t all lost! A lot of it does arrive on the surface eventually, but it does so as diffuse light – light that seems to come from the whole sky. Which is of course why the sky looks blue (if it isn’t blocked by clouds).

The result is that the parallel rays that come from the bright spot in the sky (aka, the sun) are (slightly) yellow, but as they mix with the diffuse blue light from the sky, they form white light with a color temperature of about 6500K. More specifically, the daylight standard D65 is defined as

the average midday light in Western Europe, comprising both direct sunlight and the light diffused by a clear sky.

And that’s where we get the 6500K color temperature.

(The sun as seen from space actually has a color temperature of 5800K to 5900K. I haven’t looked into why these numbers are different but I assume it’s because of absorption in the atmosphere.)

But most of the time, not everything is illuminated evenly. The parallel rays from the sun produce shadows and they are blue:

(Source. This picture was not taken at midday, so the sun is extra yellow, but this allows us to clearly see the blue shadows.)

So, in order to imitate daylight more faithfully indoors, I think you should separate the components of the light as our atmosphere does. So, use spotlights of yellowish light together with diffuse color-of-the-sky blue light that seems to come from everywhere, to produce blueish shadows.

Here is a video of someone doing this the hard way with actual Rayleigh scattering – to very impressive results – but I think you could just buy yellowish LED spotlights (with color temperature of 3500–4500K or so? I tried to find the color temperature of just the sun rays without the skylight, but couldn’t find anything) and blue LED strips mounted on all walls close to the ceiling. And of course, the lights should be very very bright in order to imitate ...
Timeline: The proximal origin of SARS-CoV-2
by ChristianKl — 2023-03-01T17:02:45.699Z
Link: https://usrtk.org/covid-19-origins/timeline-the-proximal-origin-of-sars-cov-2/

Jeremy Farrar, on Februar 2, 2020 (linking to his book):

On a spectrum if 0 is nature and 100 is release – I am honestly at 50! My guess is that this will remain grey, unless there is access to the Wuhan lab – and I suspect that is unlikely!

Jeremy Farrar, on Februar 2, 2020 in a email summarizing a phone conversation to the head of the WHO:

Critical that responsible, respected scientists and agencies get ahead of the science and the narrative of this and are not reacting to reports which could be very damaging.

Ralph Baric, on Februar 6, 2020:

Jim Hughes, Linda Saif, Hume Field, and I believe Rita Cowell will sign it, then I'll send it round some other key people tonight. We'll then put it out in a way that doesn't link it back to our collaboration so we maximize an independent voice.

While the full post is long, it gives many details about how they collaborated to suppress the lab leak theory.
Some Variants of Sleeping Beauty
by SMK — 2023-03-01T16:51:58.318Z

The Sleeping Beauty problem is a classic conundrum in the philosophy of self-locating uncertainty. From

Elga (2000)

:

Sleeping Beauty. Some researchers are going to put you to sleep. During the two days that your sleep will last, they will briefly wake you up either once or twice, depending on the toss of a fair coin (Heads: once; Tails: twice). After each waking, they will put you back to sleep with a drug that makes you forget that waking. When you are first awakened, to what degree ought you believe that the outcome of the coin toss is Heads?

Here are some variants of the problem, not to be taken all too seriously.

Sleeping Logic

Sleeping Logic-1. Some researchers are going to put you to sleep. During the two days that your sleep will last, they will briefly wake you up either once or twice, depending on the parity of the 1,000th digit of π (Even: once; Odd: twice). After each waking, they will put you back to sleep with a drug that makes you forget that waking. When you are awakened, to what degree ought you believe that the 1,000th digit of π is Even?

Sleeping Logic-2. Some researchers are going to put you to sleep. During the two days that your sleep will last, they will briefly wake you up either once or twice, depending on the parity of the third digit of π (Even: once; Odd: twice). After each waking, they will put you back to sleep with a drug that makes you forget that waking. When you are awakened, to what degree ought you believe that the third digit of π is Even?

Sleeping Riemann. A group of researchers (some of whom are mathematicians from an extremely advanced civilization) are going to put you to sleep. During the two days that your sleep will last, they will briefly wake you up either once or twice, depending on the truth of the

Riemann hypothesis

(True: once; False: twice). After each waking, they will put you back to sleep with a drug that makes you forget that waking. When you are awakened, to what degree ought you believe that the Riemann hypothesis is true?

Sleeping Counterfact

Sleeping Counterfact. A group of researchers, spearheaded by a perfect predictor, are going to put you to sleep. During the two days that your sleep will last, they will briefly wake you up either once or twice, depending on the toss of a fair coin (Heads: once; Tails: twice). During each waking in the Tai

...
Dealing with infinite entropy
by Alex_Altair — 2023-03-01T15:01:40.400Z

A major concession of the introduction post was limiting our treatment to finite sets of states. These are easier to reason about, and the math is usually cleaner, so it's a good place to start when trying to define and understand a concept.

But infinities can be important. It seems quite plausible that our universe has infinitely many states, and it is frequently most convenient for even the simplest models to have continuous parameters. So if the concept of entropy as the number of bits you need to uniquely distinguish a state is going to be useful, we'd better figure out how to apply it to continuous state spaces.[1]

For simplicity, we'll just consider the case where the state space is a single real-valued parameter x in some interval. Because x is real-valued, its set of possible values (the "states") is infinite. Extending the results to more complex sets of states is the domain of other fields of mathematics.

The problem

If you were to play the Guess Who? game with an increasing number of cards, you would need an increasing number of questions to nail down each card, regardless of what strategy you were using. The number of questions doesn't go up very fast; after all, each new question lets you double the number of cards you can distinguish, and exponents grow very fast. But, in the limit of infinite cards, you would still need infinitely many questions. So that means that if we use the yes/no questions method to assign binary string labels to the cards, we would end up with each card being represented by infinitely long strings, and therefore having infinite entropy. It ends up being equivalent to playing "guess my number" instead, where your friend could have chosen any real number between, say, 0 and 1.

To map the Guess Who? cards to a continuous parameter: 1) Line them up under the relevant interval of the x-axis. 2) Come up with a set of yes/no questions that can uniquely identify each card. 3) Come up with a new question for which all the current cards would be a "no". 4) Double the number of cards and modify all the new cards to be a "yes" answer to the new question. 5) Iterating this process to infinity will eventually "fill up" the interval, and will consequently generate infinitely many bits per card.

This doesn't seem very useful. But it still feels like we should somehow be able to use the concept of entropy (that is, th...