trentmkelly/LessWrong-43k huggingface dataset:
Covid 4/28/22: Take My Paxlovid, Please
by Zvi — 2022-04-28T15:20:01.378Z

Many have made the observation that if you had told people two years ago that we would have a cure (not a vaccine, but a cure!) for Covid-19 that was free, safe and effective, but that no one wanted to take it, and Congress wasn’t willing to fund further purchases, people would not have believed you. And yet here we are.

The White House is trying to fix the situation. I’ll cover the details below.

We haven’t learned much new about China. When we do, it will go into its own post. I did get private supporting evidence that the situation in Shanghai is being characterized roughly correctly.

Executive Summary

Paxlovid is available, no one wants it.
Case counts up now that we’re past Easter.
Cases hit local peaks including in NYC, indicating BA.2 wave won’t be too bad.

Also, did you hear? Elon Musk is buying Twitter.

Let’s run the numbers.

The Numbers

Predictions

Prediction from last week: 277,800 cases (+20%) and 4,100 deaths (+71%!?)

Results: 328,762 cases (+41%) and 2,485 deaths (+2%!)

Prediction for next week: 400,000 cases (+22%) and 2,720 deaths (+10%?)

On deaths this is one of the biggest misses I’ve had, and it’s not one of the ones I look back on and instantly feel was simply stupid – often it’s obvious in hindsight why I messed up, and mostly this isn’t one of those cases

There was a huge drop for Easter, and it was sustained even as cases popped back up even more than I expected. I don’t understand it, but the distributions involved look real. I can only interpret this as ‘we had inflated death numbers for a while that weren’t real, and now we have less of that’ because I can’t think of any other way to make sense of this. I guess I’m predicting a small increase next week here, but I mostly notice I am confused.

I believe a lot of this jump in cases is related to Easter, and the details suggest that we are going to be reaching another local peak soon especially in the northeast.

Deaths

That is not many deaths. I don’t know how to interpret the drop being sustained in a way that makes the drop ‘real’ which is why I’m assuming this means that the death counts were previously artificially high.

This in turn makes me wonder if death counts have been artificially high at other times as well.

Cases

New York City cases have plateaued and are falling. This is a big deal because New York was ‘ahead of the curve’ on the...
3-bit filters
by iivonen — 2022-04-28T11:55:46.403Z

I've come to appreciate my gut feeling for how things in my everyday life feel over time.

To distill the process of checking in with your actual felt sense of things into a neat information-theoretic package, I've recently been using a 3-bit filter: Three yes-or-no questions you can ask yourself that will give you a rough sense of where you're at with something.

Part of the reason I like this is because there is significant nuance in how you actually interpret those 3 bits. You might just sum them up, which is how I currently work with the life satisfaction filter:

Do you like who you're with?

Do you like what you're doing?

Do you like who you are?

On this I currently rank a |111| = 3. Life is fantastic! But I've also experienced living life at a 2, and even a 1 - and I've noticed a qualitative jump in my baseline contentment any time I manage to flip one of these 3 bits.

But you can also interpret them similar to a conditional flow, similar to the Maybe monad. My 3-bit filter on deciding what to study next starts with a random topic R popping into my head, and then me subjecting it to the next 3 questions:

Does R appear easy to learn, given what I already know?

Does R feel useful to learn about in depth, not just interesting?

Does R still appear interesting, even if I think it's useful?

Order matters in this interpretation. Question A needs an answer of "Yes" or "No" before you even consider Question B - which makes sense, because it's very often the case in real life too that an answer of "No, that's not true" closes off vast trees of questions we would otherwise have much more reason to pursue. The value of asking whether aether gets dragged along by matter has to be downgraded by how likely you think it is that aether actually exists. So really you can have 0, 10, 110, and 111 - and only 111 suggests I should actually set aside the time to study the thing in depth right now.

It's best to stick with your gut feeling about things before you investigate things in depth. But the fun part of the conditional flow is that it suggests further questions to ask yourself a few days or weeks later that might cause your gut feeling to shift like quicksand underneath the experiential floor of conscious noticing.

If R doesn't appear easy to learn, given what I already know, what other R′ might I learn that makes it easi_er_? And might that R′ then pass the 3-bit filte

...
Jaan Tallinn's 2021 Philanthropy Overview
by jaan — 2022-04-28T09:55:50.789Z
Link: https://jaan.online/philanthropy/#2021-results-success

to follow up my philantropic pledge from 2020, i've updated my philanthropy page with 2021 results.

in 2021 i made $22M worth of endpoint grants — exceeding my commitment of $14.4M (20k times $718.11 — the minimum price of ETH in 2021). notes:

this number includes $1.9M to orgs that do re-granting (LTFF, EAIF, impetus grants, and PPF) — so it's likely that some of that $1.9M should not be included in the "endpoint grants in 2021" total. regardless, i'm comfortably above my commitment level for that not to matter;
i have an ongoing substantial charitable project that's not reflected in the 2021 numbers — it's possible (and likely if ETH price holds, as SFF's s-process alone can't handle such amount) that i will report it retroactively next year or in 2024.
Doom sooner
by Flaglandbase — 2022-04-28T07:24:10.276Z

It took me a long time to understand because I think software as it currently exists is fantastically weak and infuriatingly defective, in fact criminally so - but after reading a lot of stuff here, AI research suddenly seems a lot more dangerous than before.
The basic truth of any complex system is that it's always more complex than it seems. Most projects take longer than planned. However, if you repeat a process enough it can be done faster.
Multicellular life took eons to evolve. Animals took millions of centuries to develop intelligence. Primitive humans were stuck in the stone age for thousands of centuries.
Right now, society is about as dumb and inefficient as it can get away with. The most powerful force in the world is the implacable consensus everywhere. There are too few geniuses to overcome the sometimes monstrously deliberate inefficiencies of life.
For those reasons, it seems probable that developing something as complex as Artificial Superintelligence will take several decades at least, and only with a great deal of effort. By which I mean that completely unexpected delays will arise that will keep slowing things down.
Yet it's the only thing that might possibly save us, the closest thing to a magic genie.
The posts on this site make a powerful case that when the first AI does develop superintelligence, it will likely not be "well rounded", but hyper-focused on some inadequately defined goal. Having less general intelligence will not make it less dangerous. The threat range may be "smaller" but no less deadly.
What is the simplest way a brute-force AI could run amuck? All it would take is one super clever idea, like the easiest self-replicating nanobot, DNA rewriting meta-viruses, or even social memes to manipulate personalities. We vastly underestimate how badly things could go wrong. Just dropping a test tube with bat guano can crash the world economy for three years.
Open-ended software entities running on sufficiently powerful hardware are likely to be controlled by nations or large corporations. Due to their extreme cost, and thanks to fears generated on this website and others, it may be possible to impose worldwide restrictions on such projects. For example, they could only be allowed to run on a shared global network, with many "kill switches".
The real danger comes from smaller AI projects using cobbled together superc...
How Might an Alignment Attractor Look like?
by Shmi — 2022-04-28T06:46:11.139Z

It is a generally accepted view in MIRI and MIRI-adjacent circles that the odds are high that an eventual self-improving AGI will not be nice to humans, to put it mildly. It is actually becoming a bit of a zeitgeist, gaining more and more widespread support. There are plenty of arguments supporting this view, including some observational data (like the infamous Boeing 737 MAX MCAS). It is easy to fall in with the prevailing wisdom in this particular bubble, given that really smart people give really persuasive reasons for it. So I would like to ask people to imagine alternatives: a state of the world where a self-improving general AI is naturally aligned with humanity.

Now, what does it mean to be aligned? We sort of have an intuitive understanding of this: a strawberry-picking robot should not rip off people's noses by accident, an AI that finds humans no longer worth their attention would disengage and leave, a planetary defense mechanism should not try to exterminate humans even if they want it shut down, though it can plausibly resist getting shut down, by non-violent means. Where, given some slack instead of a relentless drive to optimize something at any price, it would choose actions that are human-compatible, rather than treating humanity like any other collection of atoms. Where it would help us feel better without wireheading those who don't want to be wireheaded, and would be careful doing it to those who want to. Without manipulating our feeble easily hackable minds into believing or acting in a way that we would not consent to beforehand. There are plenty of other reasonably intuitive examples, as well.

Humans are not a great example of an animal-aligned intelligence, of course. Our influence on other lifeforms is so far a huge net-negative, with the diversity of life on Earth plummeting badly. On the other hand, there are plenty of examples of symbiotic relationships between organisms of widely different intelligence levels, so maybe that is a possibility. For example, maybe at some point in the AI development it will be a logical step for it to harness some capabilities that humans possess and form a cyborg of sorts, or a collective consciousness, or uploaded human minds...

It seems that, when one actually tries to think up potential non-catastrophic possibilities, the space of them is rather large, and it is not inconceivable, given ...
Virtue signaling is sometimes the best or the only metric we have
by Holly_Elmore — 2022-04-28T04:52:53.884Z

Subtitle: Costly virtue signaling is an irreplaceable source of empirical information about character.
The following is cross-posted from my blog, which is written for a more general audience, but I think the topic is most important to discuss here on LW.

We all hate virtue signaling, right? Even “virtue” itself has taken on a negative connotation. When we’re too preoccupied with how we appear to others, or even too preoccupied with being virtuous, it makes us inflexible and puts us out of touch with our real values and goals.

But I believe the pendulum has swung too far against virtue signaling. A quality virtue signal shows that a person follows through with their best understanding of the right thing to do, and is still one of the only insights we have into others’ characters and our own. I don’t care to defend empty “

cheap talk

” signals, but the best virtue signals offer some proof of their claim by being

difficult to fake

. Maybe, like being vegan, they take a great deal of forethought, awareness, and require regular social sacrifices. Being vegan proves dedication to a cause like animal rights or environmentalism proportional to the level of sacrifice required. The virtuous sacrifice of being vegan isn’t what makes veganism good for the animals or the environment, but it is a costly signal of character traits associated with the ability to make such a sacrifice. So the virtue signal of veganism doesn’t mean you are necessarily having a positive impact or that veganism is the best choice, but it does show that you as a person are committed, conscientious, gentle, or deeply bought into the cause such that the sacrifice becomes easier for you than it would be for other people. It shows character and acting out your values. Out of your commitment to doing the most good possible, you may notice that you start to think

veganism

isn’t actually

the best way to help animals for a lot of people.

1

I believe this represents a step forward for helping animals, but one problem is that now it’s much easier to hide lack of virtuous character traits from measurement.

2

It’s harder to know where the lines are or how to track the character of the people you may one day have to decide to trust or not to trust, it’s harder to support virtuous norms that make it easier for the community to act out its values, and it’s harder to be accountable to yourself.
Many will think th...
The Gospel of Martin Luther
by lsusr — 2022-04-28T04:29:58.601Z

Martin Luther: "The Catholic church is corrupt."

Johann von Staupitz: "You cannot say the Catholic church is corrupt."

Luther: "I just did."

Staupitz: "I mean you are not allowed to say it."

Luther: "According to whose authority? Certainly not God's, for He has yet to strike me down."

Staupitz: "Upon Pope Leo X's authority."

Luther: "Pope Leo X is far away, in Rome."

Staupitz: "Upon my authority, then."

Luther: "I notice that the truth value of my claims do not enter your calculus."

Staupitz: "To the contrary, the truth value of your claims very much do enter my calculus. Were you to say something obviously wrong, like 'space is non-Euclidean' then I need not persecute you. I could simply say 'Martin Luther is wrong'. False claims are self-defeating. Heresy is scary because it might be true."

Luther: "What will the Pope do to you if I speak blasphemy?"

Staupitz: "I don't know. I have never allowed blasphemy within the Order of Saint Augustine."

Luther: "Let me get this straight. You're forbidding me from speaking the truth because you're afraid that someone else will punish you for being associated with me for speaking the truth."

Staupitz: "Yes. That's because I'm one of the good guys."

Luther: "The good guys suppress free speech?"

Staupitz: "Yes. The bad guys burn their enemies at the stake."

Luther: "Heretics must be flourishing if the Catholic church burns them at the stake on a regular basis."

Staupitz: "To the contrary, nobody has ever been burned at the stake for declaring the Catholic church corrupt and translating the Bible into German."

Luther: "Let me get this straight. You're suppressing free speech because you're afraid the Catholic church might suppress free speech."

Staupitz: "Precisely!"

Luther: "What. The. Sard."
Letter to my Squire
by lsusr — 2022-04-28T04:16:38.905Z

Hello, my Squire,

Several weeks ago you pointed out that my blog posts contain lots of spelling errors and you offered to proofread them for me. This was a good offer. I accepted it. My blog posts have fewer typos than they used to.

Later, you asked if you could pour me drinks and carry my dirty shoes. This was a good offer too. I accepted it. When I needed my backpack you ran to fetch it for me. When I was weary from hours of nonstop debate, you ensured I returned home safely.

I cannot ask anyone to work for free. That would be unethical. But when a talented young man (metaphorically) beats down my door with an offer to work for free—well, it would be silly for me to refuse.

Me: How did I ever survive without someone to pour me drinks and fetch my dirty shoes?

[Redacted]: You got used to having minions scary fast.

―April was weird

The best way to learn to program is to start your own software projects. The best way to learn to write is to write your own blog. Or maybe not. Perhaps I am wrong. If you believe the best way for you to find your way in the world is to assist me then so be it.

I will make no attempt to teach you. I cannot promise the work I delegate will be worthwhile to you. But I can promise it will be worthwhile to me. You can have the boring, tedious work I don't want to do myself. Perhaps you will learn something by accident.

If I choose to honor you, I shall call you my sandalbearer.

Sincerely,

Lsusr
Slides: Potential Risks From Advanced AI
by Aryeh Englander — 2022-04-28T02:15:20.040Z

I just gave a guest lecture on potential risks from very advanced AI. I didn't end up having much time to put the slides together, so quality varies greatly between sections - some I pulled from more polished talks I had already given, some I pulled from other sources, and some I wrote up quickly and probably missed half the points I should have included.

I probably should polish it up more before presenting it here, but if I wait to do that I will likely not publish it here at all. So I'm putting up what I have as is, warts and all. Maybe somebody will find it useful.

Feel free to borrow any of it for your own purposes as you see fit. Comments, suggestions, and critiques greatly appreciated!

https://docs.google.com/presentation/d/1UGnjtQaZXEEYuFfSw8qnQ2D-yoBiDCgb/edit
Naive comments on AGIlignment
by Ericf — 2022-04-28T01:08:02.507Z

Disclaimer: everything I know about AI I learned from reading stuff on the internet, mostly on this site.

Any self-modifying general intelligence cannot be bound by a utility function. The very nature of being able to self-modify means that the utility function is open to modification or being ignored in favor of a new function.

A self-improving process cannot know which variation is better without external feedback. For humans, that is the physics of the universe, and the behavior of other humans. For a program, it can't get the feedback to know what "improvement" is without affecting the same.

A superintelligence will have different ethics. But we have different ethics than humans 1000 years ago. Or even other humans alive right now. Should we be seeking to impose our values on something that has a superior grasp of reality?

An AI can only destroy everything by acting on the physical world. Which means telling humans to do things. And usually paying them.
4a. Simple AI safety step: ban all crypto
4b. Coordinating people is a hard problem with unique solutions each time. There is no corpus of training data for it. No-one wr9te down exactly what every foreman and manager said to get the Tokyo Olympics to happen, or any other large scale project.
4c. See #3 above. People have different values and priorities from each-other. There is literally nothing an AI could attempt to do that would not be in direct opposition to someone's deeply held beliefs.
AI Alternative Futures: Scenario Mapping Artificial Intelligence Risk - Request for Participation (*Closed*) 
by Kakili — 2022-04-27T22:07:57.906Z

Overview: AI Futures Risk Model

Summary: I am a graduate student researching artificial intelligence (AI) risk and typology classifications that could result from variations in technological transitions. I am developing a scenario modeling tool that captures many of the more complex non-quantifiable dynamics inherent in complex systems. The project aims to map structural forces, trends, technologies, and degree of risk to potential AI futures through an exploratory scenario modeling framework.

As part of the data collection, I am eliciting community perspectives on AI paths through a survey (options: worksheet or Google form) to rank key characteristics that can influence AI progress, risks, and futures.

I appreciate any assistance. My data collection window is closing soon, and I figured it was past time to request assistance from the broader community if able. I am behind the curve on this request (and understand how demanding these are) but any help however minimal is greatly appreciated. None of the questions are required so if you are unsure or are short on time feel free to skip questions or sections.

The model I'm developing requires both plausibility rankings and impact to populate a likelihood/impact matrix for the model's foundation (details below). Since the full version is relatively long, I've broken it up into two shorter versions which I'll post below (under Survey Overview); I have no preference for either at this time, any data helps. The short versions should take less than five minutes each and are completely anonymous.

Survey Overview: Plausibility & Impact

This project is an exercise in exploratory scenario development only. Scenario modeling is not about prediction, but rather a rigorous and methodical way to consider the full scope of imagined future situations, or contexts and model the broad outlines of what could potentially happen rather than what will happen (Figure 2).

Generating scenarios typically involves identifying a set of influential “drivers” which in combination create a range of plausible future states. For each driver or dimension, there are a set of conditions to rank.

Similarly, I developed a range of AI dimensions rather than drivers (key aspects – e.g., AI paradigm) and added a subset of key uncertainties which I call conditions (scenario paths – e.g., current paradigm, new paradigm...
The Speed + Simplicity Prior is probably anti-deceptive
by  — 2022-04-27T19:30:20.173Z

Thanks to Evan Hubinger for the extensive conversations that this post is based on, and for reviewing a draft.

This post is going to assume familiarity with mesa-optimization - for a good primer, check out Does SGD Produce Deceptive Misalignment by Mark Xu.

Deceptive inner misalignment is the situation where the agent learns a misaligned mesaobjective (different from the base objective we humans wanted) and is sufficiently "situationally aware" to know that unless it deceives the training process by pretending to be aligned, gradient descent may alter its mesaobjective.

There are two different reasons that an AI model could become a deceptive mesaoptimizer:

During early training (before Situational Awareness), the agent learns a mesaobjective that will generalize poorly on the later-training/validation distribution. Once the mesaoptimizer becomes Situationally Aware, it will seek to actively avoid changes to whatever mesaobjective it had at that moment.

I'll call this argument "path dependence".

Alternatively, it may be that mesaoptimizer is misaligned even on the training distribution. Given sufficient optimization pressure, the learning process may favor a NN that is a mesaoptimizer with the simplest possible objective (which would fail to get any reward in the real environment), and that a misaligned objective of this sort can persist through deception alone.

I'll call this argument "malign priors".

In this post, I'll focus on the "malign priors" argument, and why I think a well-tuned speed prior can largely prevent it.

Why does this matter? Well, if deceptive inner misalignment primarily occurs due to path dependence, that implies that ensuring inner alignment can be reduced to the problem of ensuring early-training inner alignment - which seems a lot more tractable, since this is before the model enters the "potentially-deceptive" regime.

First, why would anyone think (2) was actually likely enough to justify studying it? I think the best reason is that by studying these pressures in the limit, we can learn lessons about the pressures that exist on the margin. For example, say we have an objective B that is perfectly-aligned on the training data, and there's a very-slightly-simpler objective B′ that is slightly worse on the training distribution. We might ask the question: is SGD likely to push B to become B′, and compensa...