trentmkelly/LessWrong-43k huggingface dataset:
Covid 10/13/22: Just the Facts
by Zvi — 2022-10-13T14:40:01.324Z

As discussed last week, I am going to shift procedures a bit. From this point forward, the Covid posts themselves will only cover Covid-19 and related issues.

There will then be a second post that contains all non-Covid developments that I do not think have reason to want their own posts.

Other content will be moved into its own posts, some of which will be held longer in order to offer better logical organization – e.g. there will not be a weekly post on housing or energy, more like a monthly post, and I’ll be holding housing content until I have enough of it. Feedback on this plan is of course welcome. Alas, this post was still too long for email.

Executive Summary

Weekly update now splitting up by topics.

Rising cases in Germany, no alarming general pattern yet.

More of the usual Long Covid studies fail to enlighten us.

Let’s run the numbers.

The Numbers

Predictions

Prediction from Last Week: 221k cases (-18%) and 2,300 deaths (-17%).

Result: 234k cases (-13%) and 2,460 deaths (-12%).

Prediction for next week: 234k cases (+0%) and 2,500 deaths (+2%).

I overshot a bit. That could be that the holiday had lower than usual impact, or that the underlying rates did not slow down much. That makes this a higher uncertainty week than the last several weeks, although not high by historical standards. Even if we are going to get a winter wave, we still likely have some more declines left first.

Deaths

Cases

Florida’s count came in low enough this week that cutting it in half might have been a mistake. For now I’m choosing to follow procedure. Keeping an eye in case Florida does report somewhat next week.

International Cases and Deaths

I haven’t looked at these numbers in a while. Given that there were claims that waves were descending upon Europe, it seemed like time to look.

Here’s cases for major Western European countries since July 1:

For context, same chart zooming out:

We definitely see an upswing in several countries, caused at least in part by the new variants.

What we do not see in any country other than Germany and perhaps Austria (I also checked most of the smaller European ones) experiencing the kind of scary rapid exponential growth that preceded previous major variant-based waves. Every other country is still below France and Italy’s levels and none of them show a rapid ris...
When should you defer to expertise? A useful heuristic (Crosspost from EA forum)
by Noosphere89 — 2022-10-13T14:14:56.277Z
Link: https://forum.effectivealtruism.org/posts/ngFdWkELusg5gAKdp/when-should-you-defer-to-expertise-a-useful-heuristic

One important aspect of our lives as we search for knowledge is knowing what and who to defer to, as we usually must take a lot of our knowledge on faith in expertise. However, how should you defer on issues, given that certain areas could be vastly wrong?

Well, I'll introduce some heuristics from Chris Hallquist that might help you to defer better.

They can be used in the following ways:

When an EA defers to a non-EA expert, or the movement as a whole defers to non-EA expertise.

When a less-knowledgeable EA defers to a more knowledgeable EA on something.

When someone outside a field defers to an insider expert.

Now, before I begin, I want to list caveats here:

The heuristic only applies to non-moral fields.

The heuristic assumes the field is sound. In an upcoming post, I'll talk about signs a field may have unsound bases, and what to expect there.

It's not a replacement for EV calculations.

If you're in a field or plan to work in a cause area, it's best to replace this heuristic with this post by Emrik: The underappreciated value of original thinking below the frontier.

https://www.lesswrong.com/posts/KmkZriGwkn2vDx8gB/the-underappreciated-value-of-original-thinking-below-the

But let's begin.

Conclusion

When the data show an overwhelming consensus in favor of one view (say, if the number of dissenters is less than the Lizardman's Constant), this almost always ought to swamp any other evidence a non-expert might think they have regarding the issue.

When a strong but not overwhelming majority of experts favor one view, non-experts should take this as strong evidence in favor of that view, but there's a greater chance that evidence could be overcome by other evidence (even from a non-expert's point of view).

When there is only barely a majority view among experts, or no agreement at all, this is much less informative than the previous two conditions. It may indicate agnosticism is the appropriate attitude, but in many cases non-experts needn't hesitate before having their own opinion.

Expert opinion should be discounted when their opinions could be predicted solely from information not relevant to the truth of the claims. This may be the only reliable, easy heuristic a non-expert can use to figure out a particular group of experts should not be trusted.

What about selection bias?

Emrik raised a concern about deferring to experts in that ...
Cataloguing Priors in Theory and Practice
by Paul Bricman — 2022-10-13T12:36:40.477Z

This post is part of my hypothesis subspace sequence, a living collection of proposals I'm exploring at Refine. Preceded by an exploration of Boolean primitives in the context of coupled optimizers.

Thanks Alexander Oldenziel and Paul Colognese for discussions which inspired this post.

Intro

Simplicity prior, speed prior, and stability prior — what do they all have in common? They are all means of tilting an optimization surface towards solutions with certain properties. In other words, they are all heuristics informing the navigation of model space, trainer space, etc. However, what brings them together is also a systematic divide between their theoretical/conceptual/abstract framings and their practical/engineering implementations. All those priors appear to have been used in contemporary ML in one form or another, yet conceptual-heavy researchers are often unaware of those interesting data points, while ML engineers often treat those implementations as mere tricks to improve performance (e.g. generalization).

In the language of coupled optimizers I've explored over the few past posts in the sequence, such heuristics are artifacts of meta-optimization (e.g. a human crafting a trainer by building in certain such tendencies), and often tend to be direct human-made artifacts, rather than the result of downstream optimization. Though this need not be the case, as the simplicity prior happens to itself be quite simple... It might emerge naturally from a trainer which itself is trained to be simple. Similarly, the speed prior happens to itself be quite fast, as penalizing a duration is trivial. It might emerge naturally from a trainer, should it be trained to itself be fast.

Anyway, let's briefly catalog a few popular priors, describe the rationale for employing them, list instances of their use in ML, and finally document possible failure modes associated with blindly following them.

Priors

Some members of the list can be better described as heuristics or biases than priors. However, there are some basic connections between those in that they all cause an optimization process to yield certain outcomes more than others. If you start with a prior of possible ML model parametrizations and use training data to update towards your final distribution, your choice of prior will naturally influence the posterior. This prior can be informed by heuristics, such as "we're more i...
Transformative VR Is Likely Coming Soon
by jimrandomh — 2022-10-13T06:25:38.852Z

Meta (formerly known as Oculus and as Facebook) just had their yearly conference. I think right now is a good time to take a brief break from worrying about AI timelines, and take a few minutes to notice the shortness of VR timelines. VR isn't going to determine whether humanity lives or dies, but I think it's close enough to have immediate-term impact on decisions about where to live, and how to structure organizations.

Concretely: I think we're 6 months from the crossover point where VR is better than in-person for a narrow subset of meetings, specifically, 1:1 meetings without laptops, where both sides are knowledgeable enthusiasts using the latest headset in combination with a high-end desktop PC. I estimate about 2.5 years until the crossover point where VR is better than in-person for most meetings.

I think most people are wildly underestimating how impactful, and how sudden, this is going to be. I think they'll continue underestimating how impactful it's going to be, right up until the second crossover happens.

Basically, everyone tried out Zoom meetings and discovered that they sucked compared to in-person meetings, for reasons that are slightly hard to put your finger on. Some people tried VR meetings, and discovered that they were worse. People then updated towards believing that remote work (including both videochat and VR chat) would not be transformative, but did so without a model of what concretely was going wrong and when/whether it would be fixed. But, zooming in on the technology more closely, I see a fairly short list of known problems that seems to fully explain why people prefer in-person meetings in practice, and I'm watching that list slowly shrink. I currently think every item on the VR-meetings-suck list has line-of-sight to a full solution, and that at that point, VR meetings will be straightforwardly better than in-person ones.

For context: Oculus announced a new headset, the Quest Pro, at their yearly conference yesterday. It ships in two weeks and costs $1500. Consumer sentiment about it is probably going to be negative; I expect a lot of grumbling about the price and the battery life, in particular. But this is going to be misleading. The important thing about it is that Quest Pro is that it contains a first-pass version of all but one item on my why-remote-meetings-suck list. These solutions are going to have problems at first, but...
Glossary Dance Game
by jefftk — 2022-10-13T02:20:01.718Z

Harris recently laser-cut some tokens
for walking through contra dances (presumably in preparation for
calling

BIDA

this
Sunday) and remarked that they looked like game pieces:

At dinner we talked some about trying to make this into a proper game,
and made a bit of progress but didn't get to a full game. Here's an
attempt at fleshing it out.

Each player has a hand of "calls" which they use to manipulate the
contra dance line. The players are cooperating to move the dancers
from:

d L2 R1
o u
w p
n R2 L1

to:

d R1 L2
o u
w p
n L1 R2

in eight calls. One player, however, is randomly selected to be a secret traitor, and wins if the game
fails, but not if people correctly guess that they were the traitor.

On your turn you play a call and then
draw a replacement. There are eight possible positions [1] and so
there are eight possible calls:

Long Lines Forward and Back

Circle Left 3/4

Right and Left Through

Star Left 3/4

Right-hand Chain

Left-hand Chain

Dosido on the side 1 1/2

Pass through across, turn alone

The first four are rotations of 0, 1/4, 1/2, and 3/4. The second
four are mirrors along the four axes. The last is not much of a
contra dance call, but I can't think of a better one (twirl to swap is
much less than eight counts).

After eight cards have been played, if the intended formation was
achieved the good guys win. If it wasn't, there's discussion of who
to accuse as the traitor, and then everyone simultaneously votes for
someone by pointing. The traitor needs to be picked by a plurality
vote for the good guys to win.

Unrestricted table talk is allowed, but showing your cards is not.
With perfect play the good guys should always be able to plan out a
win, but for real people that's just going to be much too hard: there
are too many permutations and options to keep in your head.

I think you can balance the game by changing the hand size. Smaller
hands favor the traitor (harder to win, easier to claim you didn't
have any better options). I'd guess three cards is about right, so
you would only have a 3/8 chance of winning by chance on the last
turn? You could auto-balance by having the hand size increase by one
after two sequential traitor wins, and decrease by one after two
...
Niceness is unnatural
by So8res — 2022-10-13T01:30:02.046Z

When I'm arguing points like orthogonality and fragility of value, I've occasionally come across rejoinders that I'll (perhaps erroneously) summarize:

Superintelligences are not spawned fully-formed; they are created by some training process. And perhaps it is in the nature of training processes, especially training processes that involve multiple agents facing "social" problems or training processes intentionally designed by humans with friendliness in mind, that the inner optimizer winds up embodying the spirit of niceness and compassion.

Like, perhaps there just aren't all that many ways for a young mind to grow successfully in a world full of other agents with their own desires, and in the face of positive reinforcement for playing nicely with those agents, and negative reinforcement for crossing them. And perhaps one of the common ways for such a young mind to grow, is for it to internalize into its core goals the notions of kindness and compassion and respect-for-the-desires-of-others, in a manner broadly similar to humans. And, sure, this isn't guaranteed, but perhaps it's common enough that we can get young AI minds into the right broad basin, if we're explicitly trying to.

One piece of evidence for this view is that there aren't simple tweaks to human psychology that make them significantly more reproductively successful. Sociopathy isn't at fixation. Humans can in fact sniff out cheaters, and can sniff out people who want to make deals but who don't actually really care about you — and those people do less well. Actually caring about people in a readily-verifiable way is robustly useful in the current social equilibrium.

If it turns out to be easy-ish to instill similar sorts of caring into AI, then such an AI might not share human tastes in things like art or humor, but that might be fine, because it might embody broad cosmopolitan virtues — virtues that inspire it to cooperate with us to reach the stars, and not oppose us when we put a sizable portion of the stars toward Fun.

(Or perhaps we'll get even luckier still, and large swaths of human values will turn out to have pretty-wide basins that we can get the AI into if we're trying, so that it does share our sense of humor and laughs alongside us as we travel together to the stars!)

This view is an amalgam of stuff that I tentatively understand Divia Eden, John Wentworth and the shard theory advocates...
A strange twist on the road to AGI
by cveres — 2022-10-12T23:27:09.005Z

By now, many people will have seen my (possibly infamous) posts written for the Future Fund worldview prize, where I argue that AGI won't happen any time soon.

To prove that life is stranger than fiction, this morning I saw an article which could change everything. Researchers in Melbourne, Australia have built a small network of real neurons which can learn the game of Pong through simple reinforcement learning. This configuration might some day be able to overcome the limitations I identify in artificial neural networks. For example, networks of real neurons might be able to learn real symbolic processes.

Of course it is way too early to make predictions about what these networks will be capable of in the future. I'm just very excited to see where this goes.

https://www.smh.com.au/national/scientists-teach-brain-cells-in-a-dish-to-play-pong-opening-potential-path-to-powerful-ai-20221012-p5bp32.html

https://www.cell.com/neuron/fulltext/S0896-6273(22)00806-6?_returnURL=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0896627322008066%3Fshowall%3Dtrue
Help out Redwood Research’s interpretability team by finding heuristics implemented by GPT-2 small
by Haoxing Du — 2022-10-12T21:25:00.459Z

Some of Redwood’s current research involves finding specific behaviors that language models exhibit, and then doing interpretability to explain how the model does these behaviors. One example of this is the indirect object identification (IOI) behavior, investigated in a forthcoming paper of ours: given the input When John and Mary went to the store, Mary gave a flower to, the model completes John instead of Mary. Another example is the acronym generation task: given the input In a statement released by the Big Government Agency (, the model completes BGA).

We are considering scaling up this line of research a bunch, and that means we need a lot more behaviors to investigate! The ideal tasks that we are looking for have the following properties:

The task arises in a subset of the training distribution. Both the IOI and the acronym tasks are abundant in the training corpus of language models. This means that we are less interested in tasks specific to inputs that never appear in the training distribution.

The ideal task can even be expressed as a regular expression that can be run on the training corpus to obtain the exact subset, as is the case for acronyms. The IOI task is less ideal in this sense, since it is harder to identify the exact subset of the training distribution that involves IOI.

There is a simple heuristic for the task. For IOI, the heuristic is “fill in the name that appeared only once so far”. For acronyms, the heuristic is “string together the first letter of each capitalized word, and then close the parentheses”. Note that the heuristic does not have to apply to every instance of the task in the training distribution, e.g. sometimes an acronym is not formed by simply taking the first letter of each word.

The gold standard here is if the heuristic can be implemented in an automated way, e.g. as a Python function, but we would also consider the task if a human is needed to supply the labels.

GPT-2 small implements this heuristic. We are focusing on the smallest model in the GPT-2 family right now, which is a 117M parameter model that is frankly not that good at most things. We are moderately interested in tasks that bigger models can do that GPT-2 small can’t, but the bar is at GPT-2 small right now.

Examples

The following is a list of tasks that we have found so far/are aware of. Induction and acronym gener...
Towards a comprehensive study of potential psychological causes of the ordinary range of variation of affective gender identity in males
by tailcalled — 2022-10-12T21:10:46.440Z

This post is also available at surveyanon.

The title of this post might be quite a mouthful, so let's break it down a bit.

"Affective gender identity" concerns how you feel about being a man vs being a woman. That is, in males, it's about how satisfied one is with being male, and how much one wants to be female. If for instance you are a trans woman, then your affective gender identity is almost certainly such that you are distressed about having male body parts and being seen as male, and that you want to live like a woman.
However, "the ordinary range of variation" means that we aren't looking at trans women, who only make up a very small part of the population, but instead that we are looking at relatively ordinary men. Men have substantial variation in how they feel about being male, as has been discussed by rationalists before under labels such as "cis-by-default". For instance, in surveys of the rationalist community, around 50% of men feel like they wouldn't mind being women, and 50% of men feel like it would be distressing to be women.
When I say I am doing a "comprehensive study of potential psychological causes" of this, what I mean is that I try to ask a large bunch of men an enormous number of questions about their values, personality, sexuality, experiences, and things like that, to try and untangle if any of these factors might be contributing to their affective gender identity. This is inherently a difficult problem, since surveys are kind of crude, and inferring causality at this level of analysis is not straightforward. I think the results here are worthwhile; but I'll let you be the judge of that.

This study is unfortunately probably not the definitive answer, if nothing else then because the key findings aren't replicated yet. I'll try to point out any limitations that I see, though. The following post describes my general analysis; if you would like to see the data to perform your own analysis, or to see the exact surveys used for the analysis, you can download them here.

Why?

One of the main things that we might want to do with a study of affective gender identity is to learn about trans people. However, at a first glance, this might make you conclude that my study is absurd! After all, by studying the ordinary range of variation, I'm excluding trans women, who are the individuals we would extrapolate onto.

I definitely think it would be worth d...
Six (and a half) intuitions for KL divergence
by CallumMcDougall — 2022-10-12T21:07:07.796Z
Link: https://www.perfectlynormal.co.uk/blog-kl-divergence

KL-divergence is a topic which crops up in a ton of different places in information theory and machine learning, so it's important to understand well. Unfortunately, it has some properties which seem confusing at a first pass (e.g. it isn't symmetric like we would expect from most distance measures, and it can be unbounded as we take the limit of probabilities going to zero). There are lots of different ways you can develop good intuitions for it that I've come across in the past. This post is my attempt to collate all these intuitions, and try and identify the underlying commonalities between them. I hope that for everyone reading this, there will be at least one that you haven't come across before and that improves your overall understanding!

One other note - there is some overlap between each of these (some of them can be described as pretty much just rephrasings of others), so you might want to just browse the ones that look interesting to you. Also, I expect a large fraction of the value of this post (maybe >50%) comes from the summary, so you might just want to read that and skip the rest!

Summary

1. Expected surprise

DKL(P||Q)= how much more surprised you expect to be when observing data with distribution P, if you falsely believe the distribution is Q vs if you know the true distribution

2. Hypothesis Testing

DKL(P||Q)= the amount of evidence we expect to get for P over Q in hypothesis testing, if P is true.

3. MLEs

If P is an empirical distribution of data, DKL(P||Q) is minimised (over Q) when Q is the maximum likelihood estimator for P.

4. Suboptimal coding

DKL(P||Q)= the number of bits we're wasting, if we try and compress a data source with distribution P using a code which is actually optimised for Q (i.e. a code which would have minimum expected message length if Q were the true data source distribution).

5A. Gambling games - beating the house

DKL(P||Q)= the amount (in log-space) we can win from a casino game, if we know the true game distribution is P but the house incorrectly believes it to be Q.

5B. Gambling games - gaming the lottery

DKL(P||Q)= the amount (in log-space) we can win from a lottery if we know the winning ticket probabilities P and the distribution of ticket purchases Q.

6...