trentmkelly/LessWrong-43k huggingface dataset:
Delta Strain: Fact Dump and Some Policy Takeaways
by Connor_Flexman — 2021-07-28T03:38:34.455Z

Curation notice by Raemon: Curated. It seems really important to figure out how to deal with the Delta variant, and how to interface with covid variants longterm. I appreciated this post for:

Striking a good balance of presenting current facts on the ground
Noting where assumptions lie
Being clear about it's epistemic status, while
Aiming to be useful in the immediate term.

(Not a doctor; merely extrapolating lines of reasoning where they will go; all policy “recommendations” are entirely hypothetical and not actual recommendations; I rushed to finish this while it's still useful and relevant, so errors will exist and I apologize for the style)

Summary

As many of you know, the Delta strain of COVID is basically upon us. In the Bay, as of a few days ago, about .4% of people had it (based on .1% confirmed cases).

Currently it's doubling every week or so, but it's surprisingly difficult to tell if it will be a small blip or a huge wave. The main reason it's hard to tell is that it's entirely dependent on human behavior, which could change quickly if people become scared or if the government institutes rules again. The second reason it's hard to tell is because of mixed evidence on the difficulty of reining it in—while different studies makes this look more or less difficult, India and the UK both appear to have successfully done this without obviously draconian behavioral interventions. Berkeley might easily peak at 1% and drop back down, but for this post I'll be talking about policies for if we get a large wave.

We also don't know very much about vaccine efficacy against Delta. We know that the good vaccines reduce your risk by about 60-90% compared to without the vaccine, but the error bars are extremely large on all these studies. In part this is because it's so hard to disentangle effects from different strains, behavioral change in the vaccinated, social bubble effects, etc. Further, it doesn't answer the important question of whether this means ~80% of people are totally safe, or if it means that standing a few feet closer to people than you do now will cause the 5x difference that moots your protection. (This also plays into the above issue of predicting whether we will have a large increase or not.)

That being said, we can still roughly estimate risk from definitely having Delta. A healthy 30yo probably has about 4x (3x-10x) less risk than before, due to v...
How much do variations in diet quality determine individual productivity?
by Natália — 2021-07-28T03:05:07.574Z

A few days ago Jim Babcock stated

I care about nutrition because I believe it has a very large, very underappreciated impact on individual productivity. Low quality diets make people tired and depressed, so they don't get anything done.

There's obviously a sense in which that is trivially true. If you start consuming no more than zero calories per day, you will get very tired, maybe depressed as well, and you will eventually not be able to get anything done. And getting adequate iodine and other nutrients is very important for children to properly develop their cognition. But Jim Babcock is probably making a stronger claim. I think what he is claiming is something like "going from the 10th percentile in diet quality among Americans to the 90th percentile would have very large, very underappreciated impact on an adult's individual productivity (taking into account that diet quality is almost certainly at least partially dependent on individual factors)."

I'd like to know what evidence we have for that claim. The strongest piece of evidence I can find is Rae et al., 2003, which showed that creatine supplementation increased the average IQ of their sample of vegetarians by 12 points, but that hasn't been replicated[1], and it seems extremely hard to substantially improve the cognition of adults. And, when it comes to depression, people have been trying really hard to show that omega-3 supplementation has substantial effects on it, but it's dubious that it does. L-methylfolate is a nutrient that is apparently effective enough that someone convinced the FDA to approve it to treat depression (as an add-on to treatment to another antidepressant), but only in quantities that far exceed those that anyone gets from their diet.

So I have a fairly low credence that his claim (as I formulated it) is true. But I was wondering if there were any major pieces of evidence I have completely missed.

Update from the future (1/16/2023): iron deficiencies are really bad, worse than I thought when I wrote this.

[1] Apparently some rationalists planned to try to replicate this, but there seemingly hasn't been an update to that in three years.
A reasonable debate about Ivermectin
by CraigMichael — 2021-07-27T22:31:46.160Z

Rebel Wisdom put together a good podcast, "Ivermectin, For and Against, with Tess Lawrie, Graham Walker & Gideon Meyerowitz-Katz." Tess is for sure pro-Ivermectin and is the second author on a favorable Ivermectin meta-analysis. Graham and Gideon are skeptical of Ivermectin. Graham is an ER doctor. Gideon is an epidemiologist.

Either way, want I wanted to hear was bonafide advocates and detractors talk with each other, and it accomplished that.

FWIW If this was an Intelligence Squared debate, I came in to it hoping for a good performance from the pro-Ivermectin side. I felt, and still feel, like that "side" is being treated somewhat unfairly. I was disappointed, however, in that regard and am ow more skeptical of Ivermectin as a treatment and prophylaxis after listening.

Tess, while bright in many ways, may just not be the best advocate for Ivermectin.

At around 20:00, Tess is asked "What would convince you that Ivermectin doesn't work?" and she responds (paraphrasing from memory): "Ivermectin works."

It gave me a flashback to the Billy Nye and Ken Ham debate when they were asked what would change their minds. Ken said “You know what? I’m a Christian, and I know God’s word is true, and there’s nothing he could say that will cast doubt on that.” While Bill said "We would need just one piece of evidence" and then gave several examples.

Then somewhere around 55:00 in Tess is asked if there's anything that should be censored on social media, and the very real example of ”bleach therapy" is offered (that’s where parents are encouraged to give children bleach to prevent autism, not Trump telling you to drink bleach). Tess is unmoved by the example and still categorically against censorship. I wonder what she'd say about PhotoDNA?

I worry that someone like Tess is too biased here. It seems to me that the pro-Ivermectin side has a lot of PR problems.

EDIT: Rebel Wisdom produced a well-researched background document on the topic. There is also a debate between Dr. Pierre Kory (pro-Ivermectin) and Dr. Luis Garegnani (skeptic) from June that is more congenial all around.
Black ravens and red herrings
by Matthew Barnett — 2021-07-27T17:46:03.640Z

The raven paradox is a dilemma in inductive logic posed by Carl Gustav Hempel. It starts by noting that the statement "All ravens are black" can be expressed in the form of an implication: "If something is a raven, then it is black." This statement is logically equivalent to its contrapositive: "If something is not black, then it is not a raven."

We then consider that observing a black raven would typically be considered evidence for the statement "All ravens are black." The paradox comes from asking whether the same holds for the contrapositive; to put it another way, if we observe a non-black non-raven, such as a green apple, does that constitute evidence that all ravens are black?

The standard Bayesian solution, given by I. J. Good, goes as follows. Suppose there are N objects in the universe, of which r of them are ravens, and b of them are black, and we have a probability of 1/N of seeing any given object. Let H_i be the hypothesis that there are i non-black ravens, and let us assume we have some sensible prior over our hypotheses. Then upon observing a non-black non-raven, our probability of H_0 increases, albeit only very slightly when N is large.

I. J. Good's solution reveals a distinct way of thinking about evidence among Bayesians. Most logicians view logical fallacies as examples of poor reasoning, but the Bayesian interpretation is slightly different. Many fallacies, when stated in their appropriate inductive form, are actually valid in a Bayesian sense, though the evidence they provide is usually weak.

Which is to say that if you criticize a Bayesian for using a "red herring" they may not see the issue. Observing a red herring is indeed Bayesian evidence for the statement "All ravens are black."
DeepMind: Generally capable agents emerge from open-ended play
by Daniel Kokotajlo — 2021-07-27T14:19:13.782Z
Link: https://deepmind.com/blog/article/generally-capable-agents-emerge-from-open-ended-play

EDIT: Also see paper and results compilation video!

Today, we published "Open-Ended Learning Leads to Generally Capable Agents," a preprint detailing our first steps to train an agent capable of playing many different games without needing human interaction data. ... The result is an agent with the ability to succeed at a wide spectrum of tasks — from simple object-finding problems to complex games like hide and seek and capture the flag, which were not encountered during training. We find the agent exhibits general, heuristic behaviours such as experimentation, behaviours that are widely applicable to many tasks rather than specialised to an individual task.

...

The neural network architecture we use provides an attention mechanism over the agent’s internal recurrent state — helping guide the agent’s attention with estimates of subgoals unique to the game the agent is playing. We’ve found this goal-attentive agent (GOAT) learns more generally capable policies.

...

Playing roughly 700,000 unique games in 4,000 unique worlds within XLand, each agent in the final generation experienced 200 billion training steps as a result of 3.4 million unique tasks. At this time, our agents have been able to participate in every procedurally generated evaluation task except for a handful that were impossible even for a human. And the results we’re seeing clearly exhibit general, zero-shot behaviour across the task space — with the frontier of normalised score percentiles continually improving.

Looking qualitatively at our agents, we often see general, heuristic behaviours emerge — rather than highly optimised, specific behaviours for individual tasks. Instead of agents knowing exactly the “best thing” to do in a new situation, we see evidence of agents experimenting and changing the state of the world until they’ve achieved a rewarding state. We also see agents rely on the use of other tools, including objects to occlude visibility, to create ramps, and to retrieve other objects. Because the environment is multiplayer, we can examine the progression of agent behaviours while training on held-out social dilemmas, such as in a game of “chicken”. As training progresses, our agents appear to exhibit more cooperative behaviour when playing with a copy of themselves. Given the nature of the environment, it is difficult to pinpoint intentionality — the behaviours we see often appear

...
In Praise (and Defence) of The People
by Olitness — 2021-07-27T07:13:26.366Z
Link: https://olitness.wordpress.com/2021/07/23/in-praise-and-defence-of-the-people/

The People are most significant phenomena in the universe. But lately I noticed some people saying how people are bad, evil, insignificant, unimportant and how we cause harm to everything around us. I have been a sucker for this idea, too. Probably most embarrassing belief from my past. I even remember saying things like “I hate people”. I’m introvert but, like seriously, WTF? I am one of them! Did I hate myself? I don’t think so, I think I just wasn’t thinking (just like all other people thinking this, but I don’t want to insult you in case you are still one of them).
Closer inspection reveals how stupid and wrong this idea is. In fact, it is not only wrong and stupid, it is evil. People turn out to be the most awesome and significant phenomena in the universe, not just for us, but for the whole of the universe!

Saying That Anything Would be Better Without People is Evil:

Some say that planet would be better off without humans. Ehm, so my dear human saying this, why you haven’t killed yourself yet?
Saying that universe would be better off without people or that they are bad or insignificant is evil. It encourages negativity, pessimism, stagnation and promotes and even justifies further evils.
People aren’t good nor bad, we are just very powerful and significant. We can cause lots of good and lots of evil, precisely because of that. We have abilities never seen before in whole history of evolution, and we are learning how to use them (which involves occasional errors). Our potential is much greater than we can imagine, and as I will explain, our rate of progress seems to be evidence of that.
By promoting idea that we are bad or not special we are just hindering and discouraging progress and improvements. The progress that would save your grandma from dying from cancer. Yes, people who broadcast such discouraging messages like “people are bad” or “people cause harm” indirectly killed your grandma (and mine, too, unfortunately).
Who wants to work on improving our wellbeing and the world, or anything for that matter, if we are so bad? Do we deserve any improvement if we are so bad? Such messages imply that: no. And so talented Jenny in high school resides to drugs and nihilism instead of working on cure for cancer, because what difference can one person make if everyone else is so bad and why should one good person care about rest that are so bad?
It seems to me ...
More Useful Way of Answering "Mind vs. Body" Problem
by Olitness — 2021-07-27T07:13:26.300Z
Link: https://olitness.wordpress.com/2021/07/26/useful-way-of-answering-mind-vs-body-problem/

Are you your brain, body, mind or consciousness? If so, where does your "free will" come from if everything is governed by laws of physics? Thorny as questions about minds, consciousness and physics are, answer to what person and free will are, and whether we are "brain" or "body" or "mind", can have simple interpretation based on what we currently known and can be used for all our practical purposes.
You are not your brain, body, DNA, mind, consciousness. Nor are you governed by laws of physics. You are specific part of all those things. You are your body, mind, DNA, consciousness and laws of physics (and perhaps other phenomena, too) all combined. You are interplay between all those things. You are what emerges when all those things combine in a unique way.
You are just like a table. Table is not its legs. Nor its desk. It is what happens when 4 legs and 1 desk combine in specific way that table emerges.
This leads me to determined laws of physics and free will. Let's assume that everything is predetermined based on laws of physics. Then where is your free will? It is in the laws of physics themselves! We are not separate from laws of physics. We are not governed by laws of physics. We are simply part of laws of physics. And if we are part of it and laws of physics are what "determines" everything, then we are the ones doing determining.
It is not my mind, DNA or laws of physics that govern what I do, because I'm those things! So in a sense you can say that my actions are determined by those things, but that just translates into my actions are determined by me.Think about it, as far as we know, if you take away my DNA, mind, body or laws of physics etc. then what remains? Nothing, there is nothing else that could be governed by those things!
Therefore People and Free Will are specific abstract phenomena that emerges from fundamental phenomena like DNA, laws of physics etc.
We shouldn't postulate that we are separate from "our" DNA, body, mind or laws of physics because if you take those things away, there is no longer any "us". It is those things that form us when they are joined in unique and specific way.
Is this perfectly accurate or true way of looking at the world and such a puzzling questions like free will and what people are? Unlikely, but to me it seems like most useful and valid way of looking at things until we find better explanations. Denying free wi...
A conversation about cooking, science, and creativity
by DirectedEvolution — 2021-07-27T05:00:30.236Z

My girlfriend is a creative cook, and worked in the Portland food industry - renowned for its innovation by small kitchens - for ten years. I'm interested in creativity in scientific research. Where does it come from?

Furthermore, what is the relationship between scientific creativity and scholarship - the immense amount of background material that scientists-in-training are expected to absorb? Over the last year, I have spent a lot of time focused on learning how to do effective scholarship. I wanted to know how to learn the theory of differential equations, immunology, biochemistry, and computer science.

In our conversation about culinary creativity, we started off with a simple question. What is the difference between a competent chef and a creative one?

She answered with an example. A competent chef can follow a recipe. A creative chef can predict what the recipe will taste like, and knows in advance how to modify it to suit her own taste. She can tell from the recipe whether the soup will be thick or thin, or have a good flavor, and knows how to adjust the broth or the spices. Some of that happens while she cooks, by tasting and adjusting, but some of it happens in her imagination, during the planning phase.

Let's break this down into components, and see if we can draw an analogy between culinary and scientific creativity. We have:

A recipe
A cook, who has:

Her own preferences, needs and goals
Some imagination and technical knowledge for how to transform the recipe to achieve those goals
The resources to turn her cooking plans into reality

What's the analogy with science?

The recipe is like the prior research in the lab or subfield
The cook is the scientist, who has:

A research problem, with its own requirements and goals
Some imagination and technical knowledge for how to transform the prior research to move closer to achieving those goals
The resources to turn her scientific plan into an experiment

This helps illuminate my intuition that scholarship and creativity feel like related, but separate tasks, and that scholarship doesn't directly lead to creativity. If scholarship is akin to the technical knowledge that helps a chef modify a recipe to her taste, then it's what helps a scientist build on prior research to advance closer to the goals of the research program. You need to have "scientific taste," a sense of where you're trying to go, and a vision for how to get...
Neuroscience things that confuse me right now
by Steven Byrnes — 2021-07-26T21:01:19.109Z

(Quick poorly-written update, probably only of interest to neuroscientists.)

(Update 5 months later: For most of these things, I feel like I've graduated from "I'm confused" to "I think I understand this, but my ideas might be wrong, especially since I haven't written them up & solicited feedback". So I'm still happy for any thoughts!)

It’s not that I’m totally stumped on these; mostly these are things I haven’t looked into much yet. Still, I’d be very happy and grateful for any pointers and ideas.

Most of these are motivated by one or more of my longer-term neuroscience research interests: (1)

“What is the “API” of the telencephalon learning algorithm?”

(relevant to AGI safety because maybe we’ll build similar learning algorithms and we’ll want to understand all our options for “steering” them towards trying to do the things we want them to try to do), (2) “How do social instincts work, in sufficient detail that we could write the code to implement them ourselves?” (relevant to AGI safety for a couple reasons discussed

here

), (3) I’d also like to eventually have good answers to the

“meta-problem of consciousness” and “meta-problem of suffering”

, but maybe that’s getting ahead of myself.

1. Layout of autonomic reactions / “assessments” in amygdala, mPFC, ACC, etc.

In

Big Picture of Phasic Dopamine

, I talked about these areas under “Dopamine category #3: supervised learning”. In the shorter follow-up

A Model of Decision-making in the Brain

I talked about them as “Step 2”, the step where possible plans are "assessed" in dozens of genetically-hardcoded categories like "If I do this plan, would it be a good idea to raise my cortisol levels?".

Anyway, at a zoomed-out level, I think I have a good story that explains a lot. At a zoomed-in level, I'm pretty unclear on exactly what's happening where.

What I have so far is:

I think the outputs in question are coming from (1) agranular prefrontal cortex, (2) agranular anterior cingulate cortex, (3) there’s a little piece of insular cortex which is also agranular; it’s right next to PFC (more specifically OFC) and for all intents and purposes we should lump it in with agranular PFC (see

Wise 2017

—"Although the traditional anatomical literature often treats the orbitofrontal and insular cortex as distinct entities, a detailed analysis of their architectonics, connections, and topology revealed that the agranular insular area

...
Refactoring Alignment (attempt #2)
by abramdemski — 2021-07-26T20:12:15.196Z

I've been poking at Evan's Clarifying Inner Alignment Terminology. His post gives two separate pictures (the objective-focused approach, which he focuses on, and the generalization-focused approach, which he mentions at the end). We can consolidate those pictures into one and-or graph as follows:

And-or graphs make explicit which subgoals are jointly sufficient, by drawing an arc between those subgoal lines. So, for example, this claims that intent alignment + capability robustness would be sufficient for impact alignment, but alternatively, outer alignment + robustness would also be sufficient. I've also added colors to make it a bit clearer.

The red represents what belongs entirely to the generalization-focused path. The yellow represents what belongs entirely to the objective-focused path. The blue represents everything else. (In this diagram, all the blue is on both paths, but that will not be the case in my next diagram.)

Note, in particular, that both paths seek outer alignment + objective robustness + capability robustness. According to the above picture, the disagreement between the two paths is only one of which of these sub-goals are better grouped together.

But this doesn't seem to actually be true. Objective Robustness and Inner Alignment Terminology points out that, really, the two approaches want to define some of the terminology differently. My previous post on the subject suggests even more differences. Putting these things together, and with some other revisions, I suggest this revised joint graph:

The and-or graph here has been supplemented with double-headed arrows, which indicate a looser relationship of pseudo-equivalence (more on this later).

Definitions:

Behavioral Alignment: This is just another way to say "impact alignment" that's more consistent with the rest of the terminology. Behavioral alignment means alignment in terms of what the system actually does. I don't want to delve into the definition of the term "alignment" itself in this post, so, that's about all I can say.
Inner Robustness: This means that the mesa-objective is efficiently pursued under a wide range of circumstances (ie, including distributional shift). In other words: whatever the mesa-optimizer wants, it is broadly capable of achieving it.
On-Distribution Alignment: Objective Robustness and Inner Alignment pointed out that the generalization-focused path re-defines "outer

...