trentmkelly/LessWrong-43k huggingface dataset:
Covid 7/21/22: Featuring ASPR
by Zvi — 2022-07-21T13:50:01.208Z

The slow transition from ‘almost all about Covid’ to ‘only somewhat about Covid’ continues, with the non-Covid various-things-to-note portion this week reaching something like 25%. My presumption is that this portion will continue rising over time.

What’s happening with Covid? More of the same old stuff, basically, and nothing that it is vital to know short term.

The quiet news is that Biden intends to elevate ASPR to coordinate pandemic response rather than the CDC or FDA. My hunch is this will only make things that much more convoluted and worse.

This week’s other post is a fun little trifle called YouTubeTV and Spoilers. Various more substantial things in the works but all in good time.

Executive Summary

ASPR to coordinate future pandemic response.
FDA to evaluate itself a little, but not with respect to pandemic.
ACX offers criticism of criticism of criticism, good show all around.

Let’s run the numbers.

The Numbers

Predictions

Prediction from last week: 750k cases (+1%) and 2,450 deaths (-15%).

Results: 770k cases (+4%) and 2,453 deaths (-15%).

Prediction for next week: 780k cases (+1%) and 2,450 deaths (+0%).

It is very hot. I expect people to stay home a bit, so although BA.5 has probably not yet peaked I am guessing we will get a quiet week. Deaths shouldn’t move much either, so the null prediction seems fine.

Deaths

Almost exactly on target as the West falls back to previous levels.

Cases

Gradual increase across the board as BA.5 fully takes over.

Physical World Modeling

Post pointing out that scientists don’t know what variants will emerge. I was not aware that we thought that anyone might know this?

BA.5 at 78% of cases, so transition is mostly complete.

Mina points out we should be trialing various potential vaccine options. Well, yes.

Chance of reporting Long Covid after 12-16 weeks given you also are reporting Covid around 4%. Seems broadly compatible with previous estimates.

Perhaps the reason there is are economic problems is… Covid? As in, maybe when lots of people are sick that could be bad? The basic premise that a bunch more sick people is going to be an issue is definitely underappreciated. Yet it can be taken too far. At what point does it become ‘misinformation’? What determines which things are and aren’t under that heading?

It continues, and builds up into cle...
How much to optimize for the short-timelines scenario?
by SoerenMind — 2022-07-21T10:47:50.018Z

Some have argued that one should tend to act as if timelines are short since in that scenario it's possible to have more expected impact. But I haven't seen a thorough analysis of this argument.

Question: Is this argument valid and if yes how strong is it?

The basic argument seems to be: if timelines are short, the field (AI alignment) will be relatively smaller and have made less progress. So there will be more low-hanging fruits and so you have more impact.

The question affects career decisions. For example, if you optimize for long timelines, you can invest more time into yourself and delay your impact.

The question interacts with the following questions in somewhat unclear ways:

How fast do returns to more work diminish (or increase)?

If returns don't diminish, the argument above fails.
If the field will grow very quickly, returns will diminish faster.

Is your work much more effective when it's early?

This may happen because work can be hard to parallelize - ‘9 women can't make a baby in 1 month’. And field-building can be more effective earlier as the field can compound-grow over time. So someone should start early.
If work is most effective earlier, you shouldn’t lose too much time investing in yourself.

Is work much more effective at crunch time?

If yes, you should focus more on investing in yourself (or do field-building for crunch time) instead of doing preparatory research.

If timelines are longer, is this evidence that we'll need a paradigm shift in ML that makes alignment easier/harder?

(This question seems less tractable than the others.)

Is your comparative advantage to optimize for short or long timelines?

For example, young people can contribute more easily given longer timelines and vice versa.

If someone would like to seriously research the overall question, please reach out. The right candidate can get funding.
Is Gas Green?
by ChristianKl — 2022-07-21T10:30:03.864Z

The EU decided to declare gas along with nuclear to be

green

. It's easy to see why one might see nuclear energy as a green energy source, it's harder to see the case for gas.

As I

previously pointed out

, we currently have days in Germany where energy is free and companies can get paid to use up energy. The continuously lowering cost of solar power suggests that those days will increase. Rolling out enough solar and wind to be able to power everything on most days means massive overproduction in electricity on summer days that can be used for applications that need a lot of cheap energy but that are okay with only running when energy production is high. One option for using the energy is to turn

CO2 into Synthetic Natural Gas

(SNG):

The overall efficiency of energy storage and reuse (power-to-power) is quite high. In the case of SNG synthesis, the energy stored in methane accounts for 53.2% of the input energy. The maximum recoverable energy in the form of heat and electricity is 37.3%

If we predict a future where we will use surplus solar energy to produce SNG, it today makes sense to build gas plants to provide us with energy on winter days with little wind. It’s building plants that will actually provide green energy in the future, which justifies seeing them as green in the EU energy plan.

It also suggests that moving a huge sector like shipping to using natural gas is a valid strategy for making the sector completely green.
Why are politicians polarized?
by ErnestScribbler — 2022-07-21T08:17:57.646Z

There has been a lot of attention dedicated to increased political polarization. And a lot of it has focused on the factors causing voters to be more polarized. I want to discuss a different, but related question. Given that voters have become more polarized, why are politicians more polarized? At first blush, you’d think this follows inevitably from the preferences of voters. Politicians try to position themselves around the central view of their voters, so if voters are more polarized, so will be the politicians’ views, right? Not necessarily. Let’s see why.

The puzzle: politicians’ polarization

Let’s describe this in a schematic (modeled on the US general elections), where we assume there are only two political candidates and that political views can be described as a single continuum[1] going from left to right, with each voter voting for the candidate which is closer to their position, and the candidate who gets more votes wins. You can imagine it might look like this:

This seems reasonable, until you consider what would happen if one of the candidates (or parties) decided to make their views more moderate. This could take the form of e.g. adopting one of the policies favored by the other party and less favored by the candidate’s party, to appeal to moderates (e.g. a Democratic candidate declaring that they do not intend to pursue any further gun control measures). They would move closer to the center, and grab the votes of the more moderate voters from the other side, and always win by a wide margin.

The other candidate would have to respond by moving closer to the middle as well, and then the original candidate would become more moderate still, until they converge to the median voter. This is how the Median Voter Theorem works. So even in the case of a highly polarized populace, it still predicts that candidates will be very moderate, with almost identical views.

This is decidedly not what we see in the US (or in any country) - the two parties’ views differ substantially. They might have even become more polarized recently. So something else must be going on. What could it be?

Hypothesis 1: Irrational Politicians, or “the echo chamber”

In our simplified model, we’ve been assuming that politicians position themselves optimally to garner the most votes, but maybe they are not rational, or don’t know which position will get them the most votes. The ec...
[AN #173] Recent language model results from DeepMind
by Rohin Shah — 2022-07-21T02:30:02.115Z

HIGHLIGHTS

Scaling Language Models: Methods, Analysis & Insights from Training Gopher (Jack W. Rae et al) (summarized by Rohin): This paper details the training of the Gopher family of large language models (LLMs), the biggest of which is named Gopher and has 280 billion parameters. The algorithmic details are very similar to the GPT series (AN #102): a Transformer architecture trained on next-word prediction. The models are trained on a new data distribution that still consists of text from the Internet but in different proportions (for example, book data is 27% of Gopher’s training data but only 16% of GPT-3’s training data).

Like other LLM papers, there are tons of evaluations of Gopher on various tasks, only some of which I’m going to cover here. One headline number is that Gopher beat the state of the art (SOTA) at the time on 100 out of 124 evaluation tasks.

The most interesting aspect of the paper (to me) is that the entire Gopher family of models were all trained on the same number of tokens, thus allowing us to study the effect of scaling up model parameters (and thus training compute) while holding data constant. Some of the largest benefits of scale were seen in the Medicine, Science, Technology, Social Sciences, and the Humanities task categories, while scale has not much effect or even a negative effect in the Maths, Logical Reasoning, and Common Sense categories. Surprisingly, we see improved performance on TruthfulQA (AN #165) with scale, even though the TruthfulQA benchmark was designed to show worse performance with increased scale.

We can use Gopher in a dialogue setting by prompting it appropriately. The prompt specifically instructs Gopher to be “respectful, polite, and inclusive”; it turns out that this significantly helps with toxicity. In particular, for the vanilla Gopher model family, with more scale the models produce more toxic continuations given toxic user statements; this no longer happens with Dialogue-Prompted Gopher models, which show slight reductions in toxicity with scale in the same setting. The authors speculate that while increased scale leads to an increased ability to mimic the style of a user statement, this is compensated for by an increased ability to account for the prompt.

Another alternative the authors explore is to finetune Gopher on 5 billion tokens of dialogue to produce Dialogue-Tuned Gopher. Interestingly,...
Don't take the organizational chart literally
by lc — 2022-07-21T00:56:28.561Z

"Do you know how much power I'd have to give up to be President?" - Lex Luthor

The CIA, like all intelligence agencies, spends much of its time managing and creating spies. Doing that is its primary job, and though it's also the clandestine operations arm of the U.S. government, the vast majority of employees at the CIA are assigned to analysis or acquisition of information from "human intelligence" sources.

For field officers in particular, creating your first spy is a very significant career milestone. If a CIA officer is unable to accomplish this after being trained for it and spending a few years in a foreign country, they'll probably have to pick a different job.

Let's imagine that newly trained field officer is you. Who would be your ideal informant? Who would you want to flip, if you could flip anybody in the world?

This is an important question. Most of the time you won't be able to just arrange a series of private meetings with people of real stature, like the head of a foreign military, so you can convince them to betray their country. However, it's not unheard of that an officer turns someone of small stature, say an analyst at the ISF, and that person later gets heavily promoted. Perhaps because they got help from a Certain Intelligence Agency.

A counter-terrorism analyst is privy to a lot of secret information throughout the course of their job. But that analyst's commanding officer manages a bunch of counter-terrorism analysts. So, getting your source that manager job means they're going to be even more helpful. Right?

Well, maybe. It certainly sounds like it'd be impressive on a Wikipedia article or a spy resume. But there are a few problems in practice with this strategy, some specific to intelligence and some more general:

Spies often still have an odd sense of patriotism. If they become promoted really heavily, they sometimes start to feel embarrassed on behalf of their country or at least like they no longer need to talk to you, and demand to call the whole relationship off.
An elevated rank tends to carry more scrutiny. Your spy might now be subject to more intense surveillance by their host country, like a permanent security detail, that may complicate your ability to communicate privately or provide them with "gifts" they can actually use.
Most importantly for this conversation, your agent's boss might nominally command the people pr

...
Personal forecasting retrospective: 2020-2022
by elifland — 2022-07-21T00:07:03.511Z
Link: https://www.foxy-scout.com/retro/

Overview

I’ve been forecasting with some level of activity for over 2 years now, so I’m overdue for a retrospective.[1]

I discuss:

My overall track record on each platform

: My track record is generally strong, though my tails on continuous questions are systematically not fat enough.

Selected tournament performances

: My tournament track record seems pretty mediocre.

A few interesting mistakes I made

: I badly underestimated progress on AI benchmarks and the ferocity of COVID waves after much of the population was vaccinated and/or naturally immune.

General reflections

: Some quick intuitions and hot takes I have based on my time forecasting

Platform track records

Metaculus

I started forecasting fairly actively on Metaculus in 2020 when I got interested due to COVID forecasting, though recently I have mostly just been predicting on particularly interesting questions every once in a while. I’m currently 9th on the

overall leaderboard

, due to me trying to predict on ~every question for about a year.

My binary calibration:

I might be a little underconfident? The blue rectangle in each bin represents the 50% confidence interval, and it looks like about 11 of the 20 rectangles contain the bin.

My continuous calibration:

From Metaculus: “We define the surprisal at the resolution to be the total probability assigned to all possible (in-range) resolution values that were thought to be more likely than the resolution itself. A resolution at the mode (peak) of the distribution has a surprisal of zero, whereas a resolution at the minimum of the distribution is maximally surprising.”

I’m guessing the spike at 0-5% surprisal is mostly due to the cases where the value was effectively known before the question closed.

On the other hand, the spike at 95-100% surprisal is conveying something real: my tails are, on average, not long enough. This is an interesting contrast to my binary predictions where ~all my 0-5% and 95-100% predictions look to have been right. Forming calibrated continuous distributions is tricky; in Metaculus’ interface it might be wise to add high variance low weight components, to generally account for unknown unknowns.

I’m not sure what’s going on with the drop at 5-10%, but otherwise this looks pretty good and the overall average surprisal of 46% seems decent.

My average log score (higher is better) vs. the community predictions are as follows:

My score	Co

...
Defining Optimization in a Deeper Way Part 3
by J Bostock — 2022-07-20T22:06:48.323Z

Last time I got stuck trying to remove the need for uncertainty. I have done it now! In the process I have also removed the need for absolute time.

The Setup

First we need to consider the universe as a causal network. This can be done without any notion of absolute time:

We can imagine dividing the world W into two parts, A and B. We can represent the states of each of these parts with vectors wA and wB. We might want this division to be somewhat "natural" in the sense of John Wentworth's work on natural abstractions and natural divisions of a causal network. But it's not particularly necessary.

Now consider what happens if we make a small alteration to one of the variables in wB. Let's choose one which doesn't depend on any other variables. Everything which is "downstream" of it will be affected, so let's highlight those in red. Let's call this universe X.

Now let's imagine that we take everything in A, and replace it with the version from W. So all of the influence coming out of A will be the same as it is in W. Consider what happens to B. We'll call this universe Y, and label everything which is different from X in green:

Now if any part of the universe is doing a decent job of optimizing, we would expect the influence of changing that node in X to be eliminated over time. Crucially, if A is optimizing B, then that difference should be eliminated in X, but not eliminated as much in Y, since wA has no information about the fact that yB is different to wB.

Example

Imagine our state-continuous thermostat. The causal network looks like this:

The world is divided into the thermostat T and room R. The dynamics of the system are described by the following equations, in which d is the "desired" temperature, and k represents the strength of the thermostat's ability to change the temperature of the room.

wTt=wRt−1−d

wRt=wRt−1−k×wTt−1

Choosing wT0=10, wR0=35, k=0.01, d=25 and with 1000 timesteps, we can plot the course of W like this:

As expected, wR approaches 25, and wR approaches 25. If we choose to alter wR0 by δ=0.001, we can calculate xRt, xTt, and yRt values. There's not much point plotting these ...
Cognitive Risks of Adolescent Binge Drinking
by Elizabeth — 2022-07-20T21:10:01.513Z

The takeaway

Our goal was to quantify the cognitive risks of heavy but not abusive alcohol consumption. This is an inhernetly difficult task: the world is noisy, humans are highly variable, and institutional review boards won’t let us do challenge trials of known poisons. This makes strong inference or quantification of small risks incredibly difficult. We know for a fact that enough alcohol can damage you, and even levels that aren’t inherently dangerous can cause dumb decisions with long term consequences. All that said… when we tried to quantify the level of cognitive damage caused by college level binge drinking, we couldn’t demonstrate an effect. This doesn’t mean there isn’t one (if nothing else, “here, hold my beer” moments are real), just that it is below the threshold detectable with current methods and levels of variation in the population.

Motivation

In discussions with recent college graduates I (Elizabeth) casually mentioned that alcohol is obviously damaging to cognition. They were shocked and dismayed to find their friends were poisoning themselves, and wanted the costs quantified so they could reason with them (I hang around a very specific set of college students). Martin Bernstorff and I set out to research this together. Ultimately, 90-95% of the research was done by him, with me mostly contributing strategic guidance and somewhere between editing and co-writing this post.

I spent an hour getting DALL-E to draw this

Problems with research on drinking during adolescence

Literature on the causal medium- to long-term effects of non-alcoholism-level drinking on cognition is, to our strong surprise, extremely lacking. This isn’t just our poor research skills; in 2019, the Danish Ministry of Health attempted a comprehensive review and concluded that:

“We actually know relatively little about which specific biological consequences a high level of alcohol intake during adolescence will have on youth”.

And it isn’t because scientists are ignoring the problem either. Studying medium- and long-term effects on brain development is difficult because of the myriad of confounders and/or colliders for both cognition and alcohol consumption, and because more mechanist experiments would be very difficult and are institutionally forbidden anyway (“Dear IRB: we would like to violently poison some teenagers for four years, while forbidding the ...
Why AGI Timeline Research/Discourse Might Be Overrated
by Noosphere89 — 2022-07-20T20:26:39.430Z
Link: https://forum.effectivealtruism.org/posts/SEqJoRL5Y8cypFasr/why-agi-timeline-research-discourse-might-be-overrated

I no longer stand by this post, and will preserve it here for history reasons.

TL;DR: Research and discourse on AGI timelines aren't as helpful as they may at first appear, and a lot of the low-hanging fruit (i.e. motivating AGI-this-century as a serious possibility) has already been plucked.

David Collingridge famously posed a dilemma for technology governance—in short, many interventions happen too early (when you lack sufficient information) or too late (when it’s harder to change things). Collingridge’s solution was essentially to take an iterative approach to governance, with reversible policy interventions. But, people in favor of more work on timelines might ask, why don’t we just frontload information gathering as much as possible, and/or take precautionary measures, so that we can have the best of both worlds?

Again, as noted above, I think there’s some merit to this perspective, but it can easily be overstated. In particular, in the context of AI development and deployment, there is only so much value to knowing in advance that capabilities are coming at a certain time in the future (at least, assuming that there are some reasonable upper bounds on how good our forecasts can be, on which more below).

Even when my colleagues and I, for example, believed with a high degree of confidence that language understanding/generation and image generation capabilities would improve a lot between 2020 and 2022 as a result of efforts that we were aware of at our org and others, this didn’t help us prepare that much. There was still a need for various stakeholders to be “in the room” at various points along the way, to perform analysis of particular systems’ capabilities and risks (some of which were not, IMO, possible to anticipate), to coordinate across organizations, to raise awareness of these issues among people who didn’t pay attention to those earlier bullish forecasts/projections (e.g. from scaling laws), etc. Only some of this could or would have gone more smoothly if there had been more and better forecasting of various NLP and image generation benchmarks over the past few years.

I don’t see any reason why AGI will be radically different in this respect. We should frontload some of the information gathering via foresight, for sure, but there will still be tons of contingent details that won’t be possible to anticipate, as well as many cases whe

...