trentmkelly/LessWrong-43k huggingface dataset:
Evolution is sampling error
by willbradshaw — 2020-05-23T16:21:46.864Z
Link: https://apomorphic.com/2019/12/12/evolution-is-sampling-error

This is a linkpost for a (somewhat old) post on my blog apomorphic.com. I'm posting this and other old posts here to get feedback and in hopeful preparation for some follow-on work[1].

See also: An addendum on effective population size

A common mistake people make about evolution is to think it's all about natural selection and adaptation. In fact, random non-adaptive changes often dominate the evolutionary process.

Today I'm going to lay out a useful framework that I hope makes this fact more intuitive, which might in turn help non-experts build better intuitive models of evolutionary processes. This will come in handy when I try to explain non-adaptive theories of ageing later on.

Sampling error and genetic drift

We can think of evolution as sampling error: deviation in the genetic composition of the offspring in a population relative to their parents. To illustrate this, let's imagine a simple, asexual population, evenly divided between two gene variants (alleles) which produce no difference in fitness[2]:

These individuals will reproduce, giving rise to the next generation. Since all the individuals are genetically identical and have the same chance of reproducing, we can think of these offspring being randomly sampled, with replacement, from the previous generation:

Since there is a great deal of randomness involved in who reproduces successfully and whose offspring survive, not all individuals will produce the same number of offspring in the next generation, even though they all had the same probability of reproducing to begin with. As a result, even in the absence of selection effects, the allele distribution of the new generation is likely to differ from that of the previous generation; this random, unbiased change in allele distribution is known as genetic drift.

As a result of genetic drift, the allele distribution will fluctuate up and down stochastically; sooner or later, one or the other will be eliminated from the population, resulting in fixation:

The time to fixation depends on the population size[3] and some other population parameters; here's an example plot for a population with a carrying capacity of 200 instead of 10:

Fairly dramatic genetic changes, then, can accumulate in a population based purely on genetic drift; there's not necessarily any need to invoke selection to explain why genetic differences between populations accumulate over time.

...
How can Interpretability help Alignment?
by RobertKirk — 2020-05-23T16:16:44.394Z

Introduction

We’ve previously written about what interpretability research might be. In this post we think about how different kinds of interpretability research (even loosely formulated) can help AI alignment research agendas and proposals. It seems that there are meaningful differences in the kind of tools and research different agendas would benefit from, and we aim to make these differences clearer. This is useful in helping prioritise what kinds of interpretability research are likely worth doing.

Framing

In solving the problem of AI Alignment, we’ll need to both answer research questions (e.g Is it optimal to tell the truth in a debate game?) and work out how to complete tasks reliably and quickly (e.g. Given an AI, tell me whether it will allow us to turn it off).

The research questions we only need to answer once (and hence can spend significant research effort on), whereas completing one of the tasks can’t be too intensive to make it feasible to complete it regularly, but will require strong tools or methods.

Interpretability research will be useful for alignment by enabling and enhancing other research proposals and agendas, and so one of the best ways of thinking about what kind of interpretability research to do is to think about other proposals and how specifically interpretability can help them. We see three broad ways interpretability could enable other proposals:

Developing a more formal theory of interpretability and explainability could help with avenues such as open source game-theory and mechanistic transparency, where some sense of Interpretation or Understanding is required in an algorithmic sense.

Theoretical exploration of components or tools, such as exploring viable general interpretation methods and their desiderata, future strong versions, promises & limits. This will help in understanding how we may be able to use interpretability in the future in other proposals, in scenarios we don’t yet encounter or haven’t considered. This side of interpretability is likely valuable long-term and neglected by the current mainstream ML research. In a sense this post is the result of trying to do this kind of research.

Enabling or amplifying the insights gained from experiments performed by researchers (both alignment and mainstream) today, helping them develop answers to their proposals’ research questions.

Different type

...
In Search of Slack
by Martin Sustrik — 2020-05-23T11:20:02.929Z

Original article: Studies on Slack

1

Proposition: If you have enough free resources you can evolve irreducibly complex features.

Imagine a bunch of yeast cells added to a barrel of malt when beer is being brewed. They have an ocean of free resources at their disposal. Surely, the evolutionary pressure will be low and the yeast cells would be able to escape the local minimum and evolve an irreducibly complex eye.

Where this idea fails is not taking the nature of exponential growth into account. Exponential means being fast. It means, in fact, being faster than anything you can visualize. The barrel would be fully fermented in just couple of days.

In other words: No matter how big a whale fall is, it will be exploited fast. The body of the whale will be eaten in months and the last bacterial remains of the ecosystem will survive for maybe 100 years. Nowhere near the time needed for a complex evolutionary change to happen by chance.

It's worth looking at E. coli long-term evolution experiment here. Since 1988 E. coli is cultivated in a uniform substrate and observed to see how it evolves. From our perspective it's interesting that each day 1% of population is transferred to a new flask with the fresh substrate. One can think of it as of a new whale fall every day. Unfortunately, I am not aware of a control experiment where substrate would be added gradually, which would allow us to see whether environment with slack is more conductive to evolution of new features than environment without slack.

However, in one of the populations in the experiment a speciation event was observed. There's one strain of E. coli that has advantage during growth on the substrate (slack) and another strain that has advantage during stationary phase, when the substrate runs out (moloch).

That hints that supposed slack may not be slack at all, just a different environment exploited in competitive manner by different subset of species (ecological opportunists).

2

Proposition: Isolation of small subpopulations can produce slack.

Let's start with some basics of population genetics so that we have tools to understand what's going on.

Let's say no human being is able to roll their tongue. What happens when a mutation arises that gives one that ability?

Well, if rolling your tongue makes you more likely to survive and have more children then the trait will be actively selected for. The frequency of

...
AGIs as collectives
by Richard_Ngo — 2020-05-22T20:36:52.843Z

Note that I originally used the term population AGI, but changed it to collective AGI to match Bostrom's usage in Superintelligence.

I think there’s a reasonably high probability that we will end up training AGI in a multi-agent setting. But in that case, we shouldn’t just be interested in how intelligent each agent produced by this training process is, but also in the combined intellectual capabilities of a large group of agents. If those agents cooperate, they will exceed the capabilities of any one of them - and then it might be useful to think of the whole collective as one AGI. Arguably, on a large-scale view, this is how we should think of humans. Each individual human is generally intelligent in our own right. Yet from the perspective of chimpanzees, the problem was not that any single human was intelligent enough to take over the world, but rather that millions of humans underwent cultural evolution to make the human collective as a whole much more intelligent.

This idea isn’t just relevant to multi-agent training though: even if we train a single AGI, we will have strong incentives to copy it many times to get it to do more useful work. If that work involves generating new knowledge, then putting copies in contact with each other to share that knowledge would also increase efficiency. And so, one way or another, I expect that we’ll eventually end up dealing with a “collective” of AIs. Let’s call the resulting system, composed of many AIs working together, a collective AGI.

We should be clear about the differences between three possibilities which each involve multiple entities working together:

A single AGI composed of multiple modules, trained in an end-to-end way.
The

Comprehensive AI Services

(CAIS) model of a system of interlinked AIs which work together to complete tasks.
A collective AGI as described above, consisting of many individual AIs working together in comparable ways to how a collective of humans might collaborate.

This essay will only discuss the third possibility, which differs from the other two in several ways:

Unlike the modules of a single AGI, the members of a collective AGI are not trained in a centralised way, on a single objective function. Rather, optimisation takes place with respect to the policies of individual members, with cooperation between them emerging (either during training or deployment) because it fits the incentives o

...
Highlights of Comparative and Evolutionary Aging
by johnswentworth — 2020-05-22T17:01:30.158Z

Author's Note: the purpose of this sequence is gears, i.e. physiology and molecular/cellular-level mechanisms, not evolution. This post contains only some bare-minimum background information as context for the rest; check out Will Bradshaw's series for a short intro which does better justice to evolutionary theories of aging in their own right.

Many species do not age:

hydra

,

some

turtles

,

rougheye rockfish

,

naked mole rats

, and probably many others which we just haven’t sat around and watched long enough yet. This does not mean these organisms are immortal - in the wild, they get eaten or infected sooner or later. But their physiology does not change with age; post-development, older organisms look physiologically identical to younger organisms. They don’t get more wrinkles, or weaker muscles, or inflamed joints. In particular, non-aging species are not more likely to die soon as they get older. We call it “negligible senescence”.

Contrast to humans:

This is the

Gompertz-Makeham Law

of mortality: after development, humans’ annual death rate increases exponentially with age (doubling time ~8 years). For naked mole rats and other non-aging species, this curve would be completely flat after childhood. (Side note: that bump around age 20 is the car-accident bump).

This raises an evolutionary puzzle: aging obviously entails loss of functionality and increased death rate. Surely those things can’t be beneficial to organism fitness. But we know it’s physiologically possible for organisms to not age, so if aging isn’t evolutionarily beneficial, why do organisms ever age? Why haven’t all organisms evolved negligible senescence?

Well, once an organism has reproduced, evolution doesn’t really care what happens any more. Sure, slowly breaking down over time won’t be beneficial, but it’s not a significant disadvantage either, as long as the kids are grown. And in nature, the vast majority of organisms will die to predation or disease or starvation pretty quickly anyway. So, whenever there’s an opportunity to gain some early-life advantage in exchange for aging later in life, that’s going to be an evolutionarily advantageous trade-off. This is the “antagonistic pleiotropy” evolutionary theory of aging: most organisms age because there’s little selec...
The Oil Crisis of 1973
by Elizabeth — 2020-05-22T15:50:02.578Z

Last month I investigated commonalities between recessions of the last 50 years or so. But of course this recession will be different, because (among other things) we will simultaneously have a labor shortage and a lot of people out of work. That’s really weird, and there’s almost no historical precedent- the 1918 pandemic took place during a war, and neither 1957 nor 1968 left enough of an impression to have a single book dedicated to them.

So I expanded out from pandemics, and started looking for recessions that were caused by any kind of exogenous shock. The best one I found was the 1973 Oil Crisis. That was kicked off by Arab nations refusing to ship oil to allies who had assisted Israel during the Yom Kippur war- as close as you can get to an economic impact without an economic cause. I started to investigate the 1973 crisis as the one example I could find of a recession caused by a sudden decrease in a basic component of production, for reasons other than economic games.

Spoiler alert: that recession was not caused by a sudden decrease in a basic component of production either.

Why am I so sure of this? Here’s a short list of little things,

The embargo was declared October 17th, but the price of oil did not really spike until January 1st.
The price of food spiked two months before the embargo was declared and plateaued before oil prices went up.
A multiyear stock market crash started in January 1973, 9 months before embargo was declared.
Previous oil embargoes had been attempted in 1956 and 1967, to absolutely no effect.

But here’s the big one: we measure the price of oil in USD. That’s understandable, since oil sales are legally required to be denominated in dollars. But the US dollar underwent a massive overhaul in 1971, when America decided it was tired of some parts of the Bretton Woods Agreement. Previously, the US, Japan, Canada, Australia and many European countries maintained peg (set exchange rate) between all other currencies and USD, which was itself pegged to gold. In 1971 the US decided not to bother with the gold part anymore, causing other countries to break their peg. I’m sure why we did this is also an interesting story, but I haven’t dug into it yet, because what came after 1971 is interesting enough. The currency of several countries appreciated noticeably (Germany, Switzerland, Japan, France, Belgium, Holland, and Sweden)…

(I apo

...
Post-crash market efficiency 1696-2020
by bfinn — 2020-05-22T14:13:50.903Z

WHAT DO past stock market crashes tell us about the weak-form Efficient-Market Hypothesis? In a meltdown, do prices become more predictable?

In theory, they instantly adjust to all public information. Hence technical analysis – predicting future stock prices from patterns in past ones – should not work. In particular, there should be no usable trends – tendency for prices to keep moving up or down – nor predictable reversals.

On the other hand, big crashes are so rare and extreme that they might be exceptions. In a crisis, perhaps enough traders act impulsively, or at least do something unusual, that there's a smidgeon of predictability. Surely greed and fear don’t always exactly cancel out?

So let’s take a look. Do big crashes on average produce any kind of pattern or trend in prices?

I’ll consider all major meltdowns that have ever happened in the US and UK, using their broad indices, the S&P 500 and FTSE All-Share; and defining a crash as a fall of 20% or more within 8 weeks.

UK crashes

Starting with the UK (as its stock market began first), here are all crashes throughout history:

Each coloured line is a crash – starting with a 20% fall – and its aftermath. I’ve even included the Bank of England crisis of 1696, and the still-famous South Sea Bubble of exactly 300 years ago (which coined the term ‘bubble’), though both are omitted from the averages. Thereafter, despite slides and slumps, there were no more actual crashes until World War 2, after which they became more frequent.

The red line is the current crash, from mid-January up to this week. The FTSE fell further, faster, than ever before, then has partly recovered; though this may yet turn into a dead cat bounce.

Superficially, all these crashes have little in common: caused by speculation, oil, program trading, terrorism, disease; all kinds of shapes – L, V, U, W; they fall 20% or 70%, take days or years to hit rock bottom, and up to a decade to recover. Like spaghetti, the lines are all over the place.

But to find an underlying pattern of sorts, we can simply take an average: the solid black line shows the mean of all crashes over the past century. And after the initial drop, it’s basically flat for the rest of the year. On average, the market didn’t under- or over-react: prices neither kept falling, nor bounced or recovered. There is no sign of any trend. So the fact that there has been a crash predicts...
Help wanted: Improving COVID-19 contact-tracing by estimating respiratory droplets
by Steven Byrnes — 2020-05-22T14:05:10.479Z
Link: https://github.com/sbyrnes321/audio-respiratory-droplets

At the end of the day, (1) we need working software, and (2) we need the people in charge of contact-tracing systems to decide to use it. Anything that helps on either of those, directly or indirectly, would be great. This includes making the software, figuring out methods of validating it, resolving uncertainties (e.g. about battery-life consequences), quantifying how much benefit this would provide, outreach / PR, and so on.
Get It Done Now
by Zvi — 2020-05-22T12:20:00.566Z

Epistemic Status: Reference

A while ago, I read the book Getting Things Done. Like most productivity books and systems, it includes detailed advice that approximately no one will follow. Unlike most productivity books and systems, it has two highly valuable key concepts. The second alone justified the time cost of reading the book. That principles are these:

Keep a record of tasks you’ve decided to do.

If you decide to eventually do a task that requires less than two minutes to do, that can efficiently be done right now, do it right now.

This wording is a refinement of the original concept of applying the two-minute rule during ‘processing time’ only. I think it’s much better to use it any time doing the new task can be done efficiently – it’s not waiting on anything, you have the necessary tools, it wouldn’t interfere too much with your state, with a key short-term deadline, or the need to protect a large or important block of time, etc etc.

Having this simple concept in your head – it’s better, once you notice something that you need to do, to just do it now rather than add it to your stack of things to do – has saved me far more trouble than one might expect.

Two minutes is a placeholder. Some people should use a lower or more often higher time threshold. The threshold should be adjusted based on the situation.

The book also contains a detailed method of how to create and maintain the list of tasks. It seemed annoying and overly complex and not suited to the way I think, and I never gave it a real try. The basic principle of ‘have a system that ensures such tasks are not forgotten’ still seems very strong.

The principle remains, and can be usefully extended further, which I plan to do in additional posts. But better to, by its own principles, write and get this posted now, so I can refer back to it.
From self to craving (three characteristics series)
by Kaj_Sotala — 2020-05-22T12:16:42.697Z

Buddhists talk a lot about the self, and also about suffering. They claim that if you come to investigate what the self is really made of, then this will lead to a reduction in suffering. Why would that be?

This post seeks to answer that question. First, let’s recap a few things that we have been talking about before.

The connection between self and craving

In “

a non-mystical explanation of ‘no-self’

”, I talked about the way in which there are two kinds of goals. First, we can manipulate something that does not require a representation of ourselves. For example, we can figure out how to get a truck on a column of blocks.

In that case, we can figure out a sequence of actions that takes the truck from its initial state to its target state. We don’t necessarily need to think about ourselves as we are figuring this out - the actual sequence could just as well be carried out by someone else.

I mentioned that these kinds of tasks seem to allow flow states, in which the sense of self becomes temporarily suspended as unnecessary, and which are typically experienced as highly enjoyable and free from discomfort.

Alternatively, we can think of a goal which intrinsically requires self-reference. For example, I might be feeling sad, and think that I want to feel happy instead. In this case, both the initial state and the target state are defined in terms of what I feel, so in order to measure my progress, I need to track a reference to myself.

In that post, I remarked that changing one’s experience of one’s self may change how emotions are experienced. This does not necessarily require high levels of enlightenment: it is a common mindfulness practice to reframe your emotions as something that is external to you, in which case negative emotions might cease to feel aversive.

I have

also previously discussed

therapy techniques that allow you to create some distance between yourself and your feelings, making them less aversive. For example, one may pay attention to where in their body they feel an emotion, keep their attention on those physical feelings, and then allow a visual image of that emotion to arise. This may then cause the emotion to be experienced as “not me”, in a way which makes it easier to deal with.

The next post in my series was on

craving and suffering

, where I distinguished two different kinds o...
Predictions/questions about conquistadors?
by Daniel Kokotajlo — 2020-05-22T11:43:40.786Z

After my post on conquistadors got loads of attention, I figured I should do more in-depth research. (For the post, all I did was read a bunch of Wikipedia articles). So I'm reading "The Conquest of Mexico and The Conquest of Peru" a 1400-page book written by a blind Harvard historian in the 1830s. It was recommended by Wikipedia as a classic work on the topic, the most thorough narrative of events to be found even today. (Let me know if you know of a better book to read!)

It seems good to register predictions and questions in advance, to help resist the natural human tendency to learn a bunch of facts and then rationalize why they support what you thought all along.

In the answers I'll put some of mine, and I would be very interested to hear yours as well.