trentmkelly/LessWrong-43k huggingface dataset:
Prerequisite Skills
by lsusr — 2021-12-24T10:11:47.773Z

Learning is a process of trial and error. You try something. It fails. You identify what went wrong. You try something else.

Learning progresses quickly when you can reliably identify what went wrong. The best way to guarantee you'll know what you did wrong is to isolate a single variable. Start with a process that works. Change exactly one thing. If the new process works better you'll know exactly why. If the new process fails you'll know exactly why.

The hard part is "start with a process that works". Suppose you want to be a magician. "Start with a process that works" means "perform a magic trick guaranteed to fool your audience". But if you're not a magician yet then you cannot perform a magic trick guaranteed to fool your audience. You cannot learn misdirection from books alone. You must practice on real people.

You cajole your friends into watching you. You perform a magic trick you read about in The Amateur Magician's Handbook. It falls flat. Why? You don't know. Maybe you goofed up the mechanics of the slight. Maybe your misdirection is mistimed. Maybe you are boring.

Magic tricks are finicky Rube Goldberg machines. The whole deception shatters if you get just one critical component wrong. There is no guarantee you did just one thing wrong. Novice magicians often do everything wrong simultaneously.

A better approach is to learn acting first. Acting is simpler than magic. Actors have to entertain an audience and direct the audience's attention but actors don't have to do it while palming objects and messing with invisible string. Acting involves fewer variables than magic. If you learn acting first then when you learn magic you'll know you're not failing to entertain your audience or direct their attention. If the trick falls flat you'll know it's because you did something wrong that's specific to magic tricks, like goofing up a sleight.

Acting is prerequisite skill to magic tricks. Magicians are a subset of actors; all magicians are actors but not all actors are magicians. It is possible to learn magic and acting simultaneously (I did) but it is more efficient to learn acting first and then magic.
Mechanistic Interpretability for the MLP Layers (rough early thoughts)
by MadHatter — 2021-12-24T07:24:38.699Z
Link: https://www.youtube.com/watch?v=8Pjj_SNIyN4

This post is a link to a video I just made in response to the new work coming out of Anthropic described here and here and discussed on LessWrong here.

In the video I try to puzzle through how best to think about the MLP layers of a transformer in the same spirit as Anthropic is thinking through the self-attention layers.
Risks from AI persuasion
by Beth Barnes — 2021-12-24T01:48:17.231Z

A case for why persuasive AI might pose risks somewhat distinct from the normal power-seeking alignment failure scenarios.

Where I'm currently at: I feel moderately confident that powerful persuasion is useful to think about for understanding AI x-risk, but unsure whether it's best regarded as its own threat, as a particular example of alignment difficulty, or just as a factor in how the world might change over the next decade or two. I think this doc is too focused on whether we'll get dangerous persuasion before strategic misaligned AI, whereas the bigger risks from persuasive technology may be situations where we solve 'alignment' according to a narrow definition, but we still aren't 'philosophically competent' enough to avoid persuasive capabilities having bad effects on our reflection procedure.

This doc is based heavily on ideas from Carl Shulman, but doesn’t necessarily represent his views. Thanks to Richard Ngo for lots of help also. Others have written great things on this topic, e.g. here.

Introduction

Persuasion and manipulation is a natural, profitable, easy-to-train-for application of hard-to-align ML models. The impacts of existing social-media based persuasion are probably overblown, and an evolutionary argument tells us that there shouldn’t be easy ways for a human to be manipulated by an untrusted party. However, it’s plausible that pre-AGI ML progress in things like text and video generation could dramatically improve the efficacy of short-interaction persuasion. It’s also plausible that people will spend significant amounts of time interacting with AI companions and assistants, creating new avenues for effective manipulation. In the worst case, highly effective persuasion could lead to very high-fidelity transmission of ideologies, and more robust selection pressure for expansionary ideologies. This could lead to stable authoritarianism, or isolated ideological clades with poor ability to cooperate. Even in the best case, if we try to carefully ensure truthfulness, it will be hard to do this without locking in our existing biases and assumptions.

2-page summary

Feasibility

The evidence for the efficacy of existing persuasion techniques is mixed. There aren’t clear examples of easy and scalable ways to influence people. It’s not clear whether social media makes people more right-wing or left-wing - there’s evidence in both directions. Based on ...
Prioritizing Information
by jsteinhardt — 2021-12-24T00:00:22.448Z

So far, we've focused on individual skills involved in a forecast: extrapolating trends, employing reference classes, how to combine estimates, and thinking about alternatives. Next I want to turn to integrating these, and other skills, when making complex forecasts. We'll consider the following question (worked through on Dec. 21st):

When will the UK’s pre-March 2022 peak 7-day moving average of current COVID hospitalizations occur?

This question has many considerations and moving parts, and we'll need more than one lecture to get to all of them. For this lecture, I'll be focusing on the following skill: listing key considerations, ranking them by their importance and uncertainty, and following up to resolve the uncertainty. From this we'll end up with a rough point estimate, but not a full probability distribution. In the next lecture I'll focus on how to combine all the considerations together into a full probabilistic forecast.

As usual, I'll include comments from Misha Yagudin (a top-ranked forecaster) in italics.

And for reference, here are case counts in the UK and South Africa as of Dec. 21st:

Step 1: Listing Key Considerations

When approaching a complex forecasting question, it's useful to start by listing as many relevant considerations as you can think of. Here is the list I came up with for the UK Omicron question (ordered by when I thought of them, not by importance):

Hospitalizations lag cases by ~9 days

The peak of the 7-day moving average will occur 3 days later than the single-day peak

Holidays or other mass gatherings could affect the peak

We can forecast by guessing max # of people infected together with Omicron doubling time and current case numbers.

Max # of people infected: Will the UK adopt dramatic NPIs to curb omicron, or do nothing? Or will people's individual precautions slow the curve? (Note: I generated the 3rd option by thinking about "other" options.)

NPIs: max # infected roughly equal to hospital capacity

Do nothing: max # infected roughly equal to herd immunity

Slow the curve: trickier, something in between

Omicron doubling time: ~4 days?

Current case numbers: no clue, can probably estimate from known data

Seasonality effects probably important

What are UK leader's current stance towards NPIs?

"Other" option brainstorming: What if hospitals are at capacity for an extended period of time and so there's no clear "peak"?

Br...
Omicron Post #9
by Zvi — 2021-12-23T21:50:10.466Z

For non-Omicron Covid news (and some additional Omicron news) this week, see the weekly post, the post on testing, and the post on the CDC nowcast.

First off, best possible personal news: Me and my family are feeling much better. The illness has passed. The rapid tests we did all came back negative, so it’s not quite best possible news of getting super mild infections and recovering, but I will very much take it. Thank you to everyone who sent their well-wishes, and to everyone who is sick now, I wish a speedy recovery to you as well.

Now, back to the rest of the Omicron wave.

There are three major fronts. There’s the question of what is happening and how many cases we have, there’s the question of how people are reacting and will react to it, and there’s the question of severity.

On the case front, it looked like the UK might have peaked, which would be big, but it now looks like that was premature. Still, that this could even look like it might have happened is a big piece of the puzzle. My best guess is that the generation time of Omicron is lower than we thought, which means R0 is lower, which means that it’s easier (but still hard, and long term still probably not feasible or advisable) to take enough action to slow it down.

On the severity front, we have a lot more data. As far as I can tell, it mostly tells a consistent story of modestly reduced severity for infection in a given individual (after controlling for everything) in the 30%-60% range, which will help a lot but is no match for exponential growth. I’d love to be conclusive, but it’s all super complex and convoluted. That’s what the bulk of this post is about.

On the reaction front, there have been a lot of extreme reactions to the Omicron situation, by individuals. I’m hearing and noticing a lot of folks deciding to lock it down for themselves. Many of them are doing so in order to ensure they can still have their family Christmas, which seems like an excellent reason even if one is not personally worried.

Reactions

The reaction has been so big that things looked like they might have halted or turned around somehow in the United Kingdom, although that’s looking less likely now. More on that in the UK section.

Individuals reacting is great, because they make much better judgment calls about what’s worthwhile and not worthwhile than broad regulatory rules, and also because of that other prob...
Reply to Eliezer on Biological Anchors
by HoldenKarnofsky — 2021-12-23T16:15:43.508Z

The

"biological anchors" method for forecasting transformative AI

is the biggest non-

trust-based

input into my thinking about likely timelines for transformative AI. While I'm sympathetic to parts of

Eliezer Yudkowsky's recent post on it

, I overall disagree with the post, and think it's easy to get a misimpression of the "biological anchors" report (which I'll abbreviate as "Bio Anchors") - and Open Philanthropy's take on it - by reading it.

This post has three sections:

Most of Eliezer's critique seems directed at assumptions the report explicitly does not make about how transformative AI will be developed, and more broadly, about the connection between its (the report's) compute estimates and all-things-considered AI timelines. One way of putting this is that most of Eliezer's critique doesn't apply to the "bounding-based" interpretation of the report discussed in

this post

(which is my best explanation for skeptics of why I find the framework valuable; I will also give quotes below from the original report showing that its intended interpretation is along the same lines as mine).
Much of Eliezer's critique is some form of "Look at the reference class you're in," invoking "Platt's Law" and comparing the report to past attempts at biological anchoring. Based on my understanding of the forecasts he's comparing it to and the salient alternatives, I don't think this does much to undermine the report.
I also make a few minor points.

A few notes before I continue:

I think the comments on the post are generally excellent and interesting, and I recommend them. (I will mostly not be repeating things from the comments here.)
I generally view Bio Anchors as a tool for informing AI timelines rather than as a comprehensive generator of all-things-considered AI timelines, and will be discussing it as such. Bio Anchors also presents itself this way - see section

Translating into views on TAI timelines

.
Something like half of this post is blockquotes. I've often been surprised by the degree to which people (including people I respect a lot, such as Eliezer in this case) seem to

mischaracterize

specific pieces they critique , and I try to avoid this for myself by quoting extensively from a piece when critiquing it. (This still leaves the possibility that I'm quoting out of context; readers may want to spot-check that.)
This post doesn't address what some have referred to as t

...
Get Set, Also Go
by Zvi — 2021-12-23T15:00:01.722Z

Paxlovid no longer illegal. Excellent. Alas, production will take a while to scale, and right now there are other more pressing concerns.

I thought this week’s headline would be divided over two weeks, but life comes at you fast when cases double every two days instead of every three. I’ve already explained why the CDC Nowcast clearly got ahead of itself and outputted obvious nonsense, which is important for putting together a model, but Omicron is still crossing 50% about now and is well past that in major urban areas.

From this point, predictions get a lot harder, our information sources potentially get a lot less accurate as we run out of testing capacity and (in at least some places) hospital capacity, and reactions become bigger unknowns as well.

I hope to have the Omicron post up by end of day, which will go into my current views on severity (very confusing but looking more promising so far) and spread (which looks less unstoppable thanks to confirming a much lower generation time, I think?) but I haven’t yet had time to sort all of that out and different sources continue to tell different stories.

The immediate, on-the-ground situation is a large jump in cases in many locations, ahead of schedule. This next week is when things potentially get quite bad, and also the first time there’s some chance the graphs mysteriously stop going up for no clear reason, as they did in South Africa and maybe did somewhat in the UK.

I also split off my coverage of the quite bad testing situation into its own post.

Omicron Post #9 will happen later today or failing that tomorrow, but there’s a bunch of developing information and I’m still trying to put it together, so I finished this off first and I’m going to take a break.

Executive Summary

Omicron taking over, cases rising dramatically.
Testing is overloaded, at-homes are hard to find, PCRs are hard to get and slow.
Paxlovid approved but in very short supply for now, Remdesivir study says it too is highly effective.

Let’s run the numbers.

The Numbers

Predictions

Prediction from last week: 862k cases (+15%) and 9,050 deaths (+3%).

Results: 1.03mm cases (+37%) and 9,203 deaths (+4%).

Prediction for next week: 1.85mm cases (+80%) and 10,200 deaths (+10%).

This would be a new record for weekly cases.

Note that my prediction includes substantial discounting for the Christmas break, and for testing overload an...
2021 AI Alignment Literature Review and Charity Comparison
by Larks — 2021-12-23T14:06:50.721Z

cross-posted to the EA forum here.

Introduction

As in

2016

,

2017

,

2018

,

2019

and 2020 I have attempted to review the research that has been produced by various organisations working on AI safety, to help potential donors gain a better understanding of the landscape. This is a similar role to that which GiveWell performs for global health charities, and somewhat similar to a securities analyst with regards to possible investments.

My aim is basically to judge the output of each organisation in 2021 (technically: 2020-12-01 to 2021-11-30) and compare it to their budget. This should give a sense of the organisations' average cost-effectiveness. We can also compare their financial reserves to their 2021 budgets to get a sense of urgency.

This document aims to be sufficiently broad that someone who has not paid any attention to the space all year could read it (and the linked documents) and be as well-informed to make donation decisions as they could reasonably be without personally interviewing researchers and organisations.

I’d like to apologize in advance to everyone doing useful AI Safety work whose contributions I have overlooked or misconstrued. As ever I am painfully aware of the various corners I have had to cut due to time constraints from my job, as well as being distracted by 1) other projects, 2) the miracle of life and 3) computer games.

This article focuses on AI risk work. If you think other causes are important too, your priorities might differ. This particularly affects GCRI, FHI and CSER, who both do a lot of work on other issues which I attempt to cover but only very cursorily.

How to read this document

This document is fairly extensive, and some parts (particularly the methodology section) are largely the same as last year, so I don’t recommend reading from start to finish. Instead, I recommend navigating to the sections of most interest to you. You should also read the Conflict of Interest Section.

If you are interested in a specific research organisation, you can use the table of contents to navigate to the appropriate section. You might then also want to Ctrl+F for the organisation acronym in case they are mentioned elsewhere as well. Papers listed as ‘X researchers contributed to the following research lead by other organisations’ are included in the section corresponding to their first author and you can Cntrl+F to find them.

If you are intere...
Testing, Testing, Hopefully
by Zvi — 2021-12-23T12:30:00.934Z

ProPublica came out this week with the best write-up so far of the full extent to which the FDA has been fighting to prevent us from having access to Covid-19 testing. Post is highly recommended.

We could have had lots of rapid tests early in the pandemic. Instead, many tests are still awaiting approval (aka they remain illegal) and we are facing massive shortages as people go around panicked trying to find a test.

Maddening non-sensical shifting standards and requirements are the FDA’s primary weapon, but simple inaction and delay are potent as well.

There’s a reminder of how the FDA did a similar thing with HIV testing and genetic testing, supposedly because of what might happen if someone got a positive test ‘without counseling.’

In the 1980s, the FDA banned home tests for HIV on the grounds that people who tested positive might do harm to themselves if they did not receive simultaneous counseling. In the 2010s, the agency cracked down on home genetic testing kits, concerned that people might make rash medical decisions as a result.

As opposed to what would happen if they didn’t get a positive test at all. I’m long past the point where I could consider this a well-intentioned mistake. There Is a War.

This is the conclusion, noting that missing a bar by 1% in FDA-land is also known as ‘missing a bar’ and ‘not better than having no test at all’:

And since rejecting Bosch’s submissions, the FDA has been coming around to her way of thinking. In March, the agency published a template for tests that would be used serially and sold in packages of two or more, allowing the kind of frequent testing she had advocated for. And last month, it published a new template that lowered the sensitivity standard for single-use over-the-counter tests to 80%.

Bosch had tests a year and a half ago that missed that bar by 1%.

For a while, even with the FDA’s attempts to stop it, as long as you were willing to pay about $10/test it was easy to get at-home Covid tests. Now that there’s need of them, that’s no longer the case. Stores across the city are sold out. Some places online still have tests, but Binex tests (which are the good ones) are extremely difficult to get, mostly sold out even at premium prices.

A typical anecdote:

Another:

As a Buzzfeed report, note the contrast with how actual people talk:

Since our younger son’s last Covid test ...
Physics Erotica
by lsusr — 2021-12-23T11:01:06.798Z

In Six Specializations Makes You World-Class I wrote about how combining just a few skills can give you an unassailable competitive advantage. However, not all skills are equally synergistic.

My experience dealing blackjack has basically no synergies, but I learned it in a single afternoon. The same goes for the other low-skill jobs I have had. Low-skill jobs have low skill transference because there isn't much skill to transfer.

Challenging domains require you to learn things. It is hard for me to think of a challenging skill I possess that is without synergies. Magic tricks are useless if you are not a magician, but being a magician taught me business and how to entertain an impromptu crowd. Reading Tarot cards improves my empathy and my fiction writing. I am including only practical skills. Stamp collecting (e.g. "history" and "English literature") does not count.

I was going to write "backpacking doesn't have much transference" but then I realized backpacking is useful for commandos, insurgents, adventurers, writers, spies, tour guides, park rangers, geologists, prospectors, conservationists, hippies, survivalists and biologists. I just don't think about the outdoors much when I'm at my keyboard.

Scientific fields like physics, biology and chemistry are like backpacking. They have broad synergy across a cluster of human experience but they cannot be usefully combined with everything. A degree in physics isn't very useful when you try to combine it with writing erotica.

Wait a minute. I would totally read that.

Susan Lovelace oscillated against the scintillating crystal. It sparkled with the energetic radiance of her beauty.

"My passion for you burns like magnesium," said her principal investigator. He hovered at the dangerously unstable distance where electromagnetic repulsion perfectly balances nuclear attraction. One nanometer closer and they would entangle. One nanometer farther and they would pushed each other away forever.

"We can't," said Susan. She knew it would destroy her if she gave in to Dr. Anti-Lovelace's attraction.

What was I writing about? Oh, right. Skill synergies. Some skills can be combined with basically anything.

Entrepreneurship

Sales

Marketing

Personnel Management

Writing

Videography

Drawing

Teaching & Training

Computer Programming

English language

Chinese language

What these universally synergistic skills have in common is that ...