trentmkelly/LessWrong-43k huggingface dataset:
Anthropics: different probabilities, different questions
by Stuart_Armstrong — 2021-05-06T13:14:06.827Z

I've written before that different theories of anthropic probability are really answers to different questions. In this post I'll try to be as clear as possible on what that means, and explore the implications.

Introduction

One of Nick Bostrom's early anthropic examples involved different numbers of cars in different lanes. Here is a modification of that example:

You're driving along, when you turn into a dark tunnel and are automatically shunted into the left or the right lane. You can't see whether there are any other cars in your dark lane, but the car radio announces "there are 99 cars in the right lane and 1 in the left lane".

Given that, what is your probability of being in the left lane?

That probability is obviously 1%. More interesting than that answer, is that there are multiple ways of reaching it. And each of these ways corresponds to answering a slightly different question. And this leads to my ultimate answer about anthropic probability:

Each theory of anthropic probability corresponds to answering a specific, different question about proportions. These questions are equivalent in non-anthropic setting, so each of them feels potentially like a "true" extension of probability to anthropics. Paradoxes and confusion in anthropics results from confusing one question with another.

So if I'm asked "what's the 'real' anthropic probability of X?", my answer is: tell me what you mean by probability, and I'll tell you what the answer is.

0. The questions

If X is a feature that you might or might not have (like being in a left lane), here are several questions that might encode the probability of X:

What proportion of potential observers have X?

What proportion of potential observers exactly like you have X?

What is the average proportion of potential observers with X?

What is the average proportion of potential observers exactly like you with X?

We'll look at each of these questions in turn[1], and see what they say imply in anthropic and non-anthropic situations.

1. Proportion of potential observers: SIA

We're trying to answer "Given that, what is your probability of being in the left lane?" The "that" is means being in the tunnel in the above situations, so we're actually looking for a conditional probability, best expressed as:

What proportion of the potential observers, who are in the tunnel in the situation above, are also in the left lane

...
Let's Go Back To Normal
by Connor_Flexman — 2021-05-06T03:10:02.447Z

(Personal opinion; not a doctor; edited because I accidentally published the draft)

In a few months, the US should stop being careful around COVID. For the vaccinated, most of you should stop being careful now.

We’re in a new era of COVID. We have some amazingly effective vaccines that highly reduce both transmission and severity, so the emergency is effectively over for the nation (after another month or two for everyone to get a chance to take the vaccine, which they certainly all should). Cases will linger, but the point of the lockdowns and precautions of March 2020 was to buy time—to try to get a vaccine, and determine if long-term side effects were crippling—and we’ve done those things.

Now there is debate about whether we should go back to normal—both personally and in government policy. I’ve been dismayed to see headlines like “Planning on opening up? Not so fast…” Even my acquaintances sometimes make comments about whether it’s safe for me to visit a friend, despite that I’m vaccinated. I called up all my friends in March 2020 to tell them to lock down before the government did—but the people now arguing that we or the nation should stay careful are, in my estimation, missing the bigger picture.

Whether or not we take more precautionary measures, the people still susceptible to COVID are going to get it *eventually*. (Both those that didn’t take the vaccine and those that didn’t have a strong enough immune response.) We aren’t going to magically save them by continuing to be careful. We’ll eventually hit herd immunity one way or another—the only question is how much we flatten the curve. We have to flatten it enough to maintain hospital supplies, as before, but I don’t think we should do more.

And for individuals, the people who have gotten vaccinated aren't really going to get any more safe anytime soon. Rates of COVID might go down a little, but it could be years before the remaining un-immunized population hits herd immunity.

Now, there are some benefits to being careful. It won’t be *you* who gave someone COVID. If people get long COVID or die from it, it will be months later than it otherwise would have happened. But the benefits are very small in relative terms if most of the susceptible people will get it anyways, and absurdly small if the careful person is vaccinated.

The benefits are small in relative terms because the costs of c...
Thoughts on Ad Blocking
by jefftk — 2021-05-06T02:50:04.606Z

Disclosure: I work on ads at Google; this is a personal post.

In the discussion of why I work on
ads people asked whether I use an ad blocker (no) and what I think
of them (it's complicated). So, what about ad blockers?

It should be up to you what you see. If you don't want your computer
displaying ads, or any other sort of content, you shouldn't have to.
At the same time, most sites are offering a trade: you're welcome
to our content if you also view our ads.

These are in conflict, but I feel like the resolution could be simple:

You are free to block any ads you want.

Sites can know when ads are blocked.

Sites could choose to respond to ad blocking by showing a message
explaining that ads are what fund the site and requiring users to
either subscribe or allow ads if they want to proceed. Or not: the
marginal cost of serving a page is trivial and perhaps some visitors
will share articles they enjoy. Still others might implement
something like the first-n-free approach you see with paywalls, or
progressively more obnoxious nagging.

This isn't what we have today:

Some sites (ex: Facebook) try
to disguise their ads to get them past blockers. A big site that runs
their own ads might scramble the names of resources on every page
view, while a smaller site might hire an ad-tech company to proxy
their site and stitch in ads. When successful, users are
seeing content they specifically said they didn't want.

Some blockers (ex: uBlock
Origin but not AdBlock)
hide "please disable your ad blocker or subscribe" messages. For
example, 37%
of uBlock Origin issues are people pointing out anti-adblock banners
it misses (ex: #9005,
#9006,
#9007).
When successful, sites are serving content to users
they specifically said they didn't want to serve.

I don't have any sort of proposal here; I'm not proposing a browser
feature or government regulation. But in thinking about how future
decisions might affect ads, I'm going to be most excited about ones
that support (1) and (2).
Decontextualizing Morality
by ACrackedPot — 2021-05-05T20:25:23.568Z

I have a fundamental objection to the way a lot of people do utilitarian morality, which is that most people's morality is context-dependent; copy and paste somebody from one context to another, and the morality of their lives and choices can become entirely different.

Somebody who lives an ordinary life in a utopia becomes a moral monster if you copy-paste them into a dystopia, even if their subjective experience is the same; in the utopia, they walked by a field of flowers without really thinking about them, where in the dystopia, they walk by rivers full of drowning children with the same lack of awareness or concern.

The Copenhagen Interpretation of Ethics discusses this kind of problem in general, but I think most people's moral intuitions, even after adjusting for this, still sees the person who does nothing to improve the world in a utopia as morally neutral, where the person who does nothing to improve the world in a dystopia is at least a little morally repugnant. (Question: Do you think somebody who doesn't save a drowning child is evil?)

In a sense, this is capturing our intuitions that a good person should do something if there is a problem, but that our morality shouldn't be punitive if there isn't a problem to fix - the person in the utopia gets a pass because we can't distinguish between the person who would help, if there was something to help with, and the person who wouldn't.

However, it seems a bit wonky to me, if our metaethics punish somebody for what basically amounts to luck. And if it seems odd that I use the word punish - how can it be punishment to assign moral values to things, that's just a value statement - then I think maybe you don't experience moral blameworthiness as pain. (I think this might be a central active ingredient in a particular kind of moral conscientiousness.)

But also, I think "Feeling like a good person" is something that should at least be available to people who are doing their best, and an ethical system that takes that away from somebody for moral luck is itself at least a little unethical.

Moral Context

Moral context is a very big thing; I'm not going to be able to do it justice. So I'm going to limit the context I deal with to the environmental context; there are other kinds of context which we may want to preserve. For instance, to return to the question of whether someone who walks by a ...
What are your favorite examples of adults in and around this community publicly changing their minds? 
by nim — 2021-05-05T19:26:55.433Z

Search turns up a few threads on self-reported personal changes of mind. I'm curious here about the third-person perspective: When have you noticed and remembered peers or colleagues changing their minds [about important things]? I'm particularly interested in examples from the group of people who seem to regard a book on how to change your mind as a cultural touchstone, but stories about those whom such people work with and look up to seem likely to also be relevant.
[AN #149]: The newsletter's editorial policy
by Rohin Shah — 2021-05-05T17:10:03.189Z

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

Please note that while I work at DeepMind, this newsletter represents my personal views and not those of my employer.

HIGHLIGHTS

In the survey I ran about a month ago, a couple of people suggested that I should clarify my editorial policy, especially since it has drifted since the newsletter was created. Note that I don’t view what I’m writing here as a policy that I am committing to. This is more like a description of how I currently make editorial decisions in practice, and it may change in the future.

I generally try to only summarize “high quality” articles. Here, "high quality" means that the article presents some conceptually new thing not previously sent in the newsletter and there is decent evidence convincing me that this new thing is true / useful / worth considering. (Yes, novelty is one of my criteria. I could imagine sending e.g. a replication of some result if I wasn’t that confident of the original result, but I usually wouldn’t.)

Throughout the history of the newsletter, when deciding whether or not to summarize an article, I have also looked for some plausible pathway by which the new knowledge might be useful to an alignment researcher. Initially, there was a pretty small set of subfields that seemed particularly relevant (especially reward learning) and I tried to cover most high-quality work within those areas. (I cover progress in ML because it seems like a good model of ML / AGI development should be very useful for alignment research.)

However, over time as I learned more, I became more excited about a large variety of subfields. There’s basically no hope for me to keep up with all of the subfields, so now I rely a lot more on quick intuitive judgments about how exciting I expect a particular paper to be, and many high quality articles that are relevant to AI alignment never get summarized. I currently still try to cover almost every new high quality paper or post that directly talks about AI alignment (as opposed to just being relevant).

Highlights are dif...
Fractal Conversations vs Holistic Response
by abramdemski — 2021-05-05T15:04:40.314Z

A kind of conversation which I very much enjoy is one where everyone has time to formulate long thoughtful responses, and most of the conversational points are remembered and addressed later.

For example, a long email thread between Alice and Bob, where Alice and Bob write their next emails as point-by-point replies. In caricature:

(email 1, from Alice:)

How are you doing? I am well. I wanted to ask about your book collection.

(Email 2, from Bob:)

How are you doing?

I am doing poorly. Lately I cannot stop thinking about Arcturus. This makes it difficult to focus on my work.

I am well.

I am glad to hear it! Your knee has been OK?

I wanted to ask about your book collection.

It continues to grow exponentially for the time being. Soon I will need to worry about the weight causing earthquakes in the local geology. I am considering moving it off-planet. What brought it to mind?

(Email 3, from Alice:)

How are you doing?

I am doing poorly. Lately I cannot stop thinking about Arcturus.

That makes sense. To be honest I had forgotten! I know it's important to you. What are you thinking about, though? So far as I understand, there is nothing to be done.

This makes it difficult to focus on my work.

Have you considered taking stimulants? Or would that make the obsessive thoughts worse?

I am well.

I am glad to hear it! Your knee has been OK?

Oh, no. I actually got it amputated! It wasn't much use anyway, so that's literally taken a load off. Dead weight, as they say. But one adjusts. I've noticed I'm happier these days, though I am not sure why.

I wanted to ask about your book collection.

It continues to grow exponentially for the time being. Soon I will need to worry about the weight causing earthquakes in the local geology.

I can see how that would be a problem. Things are delicate down there (at least when you get to these massive scales!)

I am considering moving it off-planet.

Better do it soon, then, if it's exponential growth! You know, rocket equation and all that.

What brought it to mind?

Ah, right. Well, I had an idea for an experiment...

And so on, with Alice and Bob continuing to expand sub-points like that, managing an increasing number of threads in parallel. (Not strictly increasing! Some will usually be dropped pretty fast as they reach a natural conclusion or don't feel relevant to the interesting parts of the discussion.)

If the ...
[link] If something seems unusually hard for you, see if you're missing a minor insight
by Kaj_Sotala — 2021-05-05T10:23:26.046Z
Link: https://drmaciver.substack.com/p/how-to-do-everything

Related to: procedural knowledge gaps, trivial inconveniences, errors vs. bugs.

Summary of the linked post: the world is full of all kinds of minor insights about how to do something (either easily or at all): how to open stitched bags, how to mop floors, how to use search engines, the right way to look at a ball in order to catch it, what it means to 'ping' somebody, how the skill of playing sudoku shares useful elements with other things like proving math theorems, how to pass as 'normal', etc.

For anything that seems unusually hard for you, there's a chance that there's some simple insight that you happen to be missing that would make it easier. You could put a lot of effort into trying to laboriously level up the skill, or you could see if you could acquire that insight by relatively little work.

I particularly like the post for having an extended list of examples of the thing, hopefully making it easier to notice potential applications of this principle when they come up in your own life. (The example that came to my own mind was finding it unexpectedly aversive to vacuum in my current home, when it had felt fine in the previous one, and then only eventually realizing that my former housemate's vacuum cleaner that I'd been using in my previous home was much more pleasant to use than my own.)
Bayeswatch 2: Puppy Muffins
by lsusr — 2021-05-05T05:42:47.279Z

A green humvee arrived at Jieyang Chaoshan International Airport. Vi got in the back with Molly Miriam who handed her clipboard to Vi.

"健重制造公司. A no-name Chinese factory that makes barbells and similar equipment. It's not even fully-automated," read Vi.

"They are registered to use a low-intelligence AGI," said Miriam.

"What are we even doing here? Neither the product nor the AI poses a threat to civilization," said Vi.

"Something must have gone badly wrong," said Miriam.

The road to the factory was blockaded by the People's Liberation Army (PLA). The soldier at the checkpoint scanned the Bayeswatch agents' badges. A young officer—barely out of high school—escorted them inside the perimeter to Colonel Qiang.

"We could probably handle this on our own," said Colonel Qiang, "But protocol is protocol."

"So it is," said Miriam.

There were no lights on in the factory. No sound emanated from it. Fifty soldiers behind sandbags surrounded the factory, along with two armored personnel carriers and a spider tank.

"The police responded first. The sent a SWAT team in. Nobody came back. Then we were called. We would like to just burn the whole thing down. But this incident could be part of a wider threat. We cut power and Internet. Nothing has entered or left the building since our arrival. Rogue AIs can be unpredictable. We wanted your assessment of the situation before continuing," said Colonel Qiang.

"You did the right thing. This is probably an isolated incident. If so then the best solution is to rescue who we can and then level the building. Unfortunately, there is a chance this is not an isolated incident. Therefore our top priority is to recover the AI's hard drives for analysis," said Miriam.

"We will assault the building," said Colonel Qiang.

"You may have our sanction in writing. Assume humans are friendly and robots are hostile," said Miriam.

"Yes sir," said Colonel Qiang.

Miriam and Vi were quartered in a nearby building that had been comandeered by the PLA. They watched the assault from a video monitor.

"In training they taught us to never go full cyber against an AI threat," said Vi.

"That is correct," said Miriam.

"Which is why every assault force is no less than ten percent biological," said Vi.

Miriam nodded.

"Standard operating procedure is you go ninety percent robotic to minimize loss of life," said Vi.

"Ninety percent robotic does tend to minimize loss of life without ...
What are the greatest near-future risks or dangers to you as an individual?
by nim — 2021-05-05T04:02:52.861Z

What factors do you expect have the highest likelihood of severely compromising your own quality and/or duration of life, within the next 1, 5, or 10 years? How do these risks change your behavior compared to how you expect you'd act if they were less relevant to you?

If those greatest personal risks are not things you categorize as existential risks to all of humanity, how do you divide your risk mitigation efforts between the personal-and-near-term and the global-and-long-term ones?

(I ask because I keep catching myself trying to generalize about the personal risk profiles of the kinds of people who concern themselves with x-risks, and I'd rather generalize from self-reports of humans in a community which attracts that demographic than from wild guesses inferred by reading a bunch of blogs)
Simulation theology: practical aspect.
by Valentin2026 — 2021-05-05T02:20:59.684Z

In this post, I elaborate a little bit on the simulation argument by Nick Bostrom and discuss what it may lead to from practical, observable purpose. The paper is excellent, but in case you haven’t read it I write everything in a self-consistent manner so you should be able to get the point without it.

Consider highly advanced technological civilization (call it parent civilization) that can create other civilizations (child civilizations). This can be done in multiple ways. Bostrom focuses only on the computer simulation of other civilizations. This is most likely the easiest way to do it. It is also possible to terraform a planet and populate it with humans (let’s call agents in the parent civilization humans for simplicity) effectively putting it to the period just before written history. Finally, one may consider an intermediate solution, something like “The Matrix” movie, with all the people perceiving not actual reality, but artificial. Parent civilization itself may have a parent. If civilization does not have a parent, let’s call it orphan.

In principle, parent civilization can create child civilizations, similar to its own earlier stages of development. Then one may ask, how is it possible to understand, whether you are a child civilization or orphan? For the case of the computer simulation, it is possible to argue that simulated beings have qualitatively different perceptions, or don’t have perception at all. However, for the other two types of child civilization (other planet and Matrix) humans are the same as in the parent civilization and should have the same type of perception. Thus, in the earlier stages of history, it might be not possible to distinguish between being a child and an orphan if the parent aims to not show its existence.

However, with technological development, some progress in this question can be achieved. I am not talking about distinguishing “fake reality” from the “real reality” – if the parent is much more technologically advanced, it should be able to make it indistinguishable for a child. I am rather talking about reaching the stage of becoming a parent themselves. Then, one can create many children, copies of the earlier stages of development, and see, what fraction can reach the stage of parent themselves (so civilization exploring the question becomes a grandparent). If this fraction is not infini...
What do the reported levels of protection offered by various vaccines mean?
by David Scott Krueger (formerly: capybaralet) — 2021-05-04T22:06:23.758Z

Pfizer is supposed to be "95% effective". Does this mean:
1. There is a 95% reduction in your odds of getting COVID (as measured by the gold standard, which I believe is aserological test?)
2. There is a 95% reduction in your odds of testing positive for COVID
3. There is a 95% reduction in your odds of getting symptomatic COVID
or
4. Something else?