trentmkelly/LessWrong-43k huggingface dataset:
Why Selective Breeding is a Bad Way to do Genetic Engineering
by GeneSmith — 2021-03-05T02:30:38.775Z

A Brief Intro

During any conversation about genetic engineering, people inevitably bring up worries about eugenics movements of the past and often use the cruelty, bad science, and objective failure of these efforts as an example of why we shouldn't ever try anything remotely related again. In this short post, I'm going to summarize why I think selective breeding of humans is bad both from a moral perspective and ineffective as a means of improving human genes.

Selective breeding at its core involves taking organisms that score well on some test of desirable traits and enabling them to reproduce at higher rates than organisms that score poorly. Despite its many flaws, this technique has lead to amazing gains in both agriculture and animal husbandry, and allowed domesticated corn crops to undergo this incredible transformation over the last few thousand years.

But despite the amazing performance on crops, there are reasons this technique would not work very well on humans.

Humans are Slow Breeders

Every generation that you selectively breed an organism you get some gain in a particular trait. The faster reproduction happens, the faster you see improvements in the trait(s) under selection. Humans are extremely slow-breeding animals. Though humans are capable of reproducing sometime in early adolescence, most humans today opt to wait until their 20's to 30's to have children. This is a very very long time if you want to do selective breeding.

Selective Breeding Leads to an Undesirable Reduction in Genetic Diversity

Genetic diversity is valuable. Because selective breeding can only work by throwing entire organisms out of the gene pool, it naturally ends up reducing a lot of desirable genetic diversity. Even organisms that don't score well overall will still have many good genes. With selective breeding, there is no way to keep this valuable genetic diversity unless one were to select the best X% from every lineage in a population.

Selective Breeding Creates a Single Point of Failure

The right to reproduce is foundational to all modern societies. Even in societies that do place restrictions on reproduction, such as China with its one-child policy, the restrictions are not total: each couple can still have a single child. In order to make any notable gains in desirable traits from selective breeding, one must necessarily only allow a small portion of the populatio...
Participating in a Covid-19 Vaccine Trial #3: I Hope I feel Worse Tomorrow
by ejacob — 2021-03-05T00:02:50.407Z

Today is four weeks to the day after my first injection, and I received my second at my appointment this morning. The process was very similar to my first visit: I entered and met first with someone who led me through paperwork and asked a series of screening questions, then had a brief medical exam, got my shot, waited 30 minutes in case of any acute reaction, and left. Notable differences this time:

The entire process seemed less repetitive and went by faster (both subjectively and objectively). I was also filling out fewer forms than last time and wasn’t getting any orientation so this makes sense.
I was told that my follow-up meeting “had to” be scheduled for two weeks from now by order of Novavax. This is sooner than was specified in the study design. I suspect they have enough data already to show that it works so they are planning to close the site.
In the middle of my health exam, someone rang a Nurse Call button and the doctor ran out of the room. A few minutes later the Nurse Practitioner from last month came in and finished up. The building we were in is also used a clinic for World Trade Center survivors and first responders (and probably other health care I don’t know about) so it’s possible that this was totally unrelated, but the call would imply that someone had a reaction to the injection.
There were specific instructions to inject the 2nd dose in the opposite arm from the first dose. I am right-handed so I used my left arm last time, which meant I had to use my dominant hand this time. The second round is more likely to have side effects though, so I wish I had known this to begin with. By word-of-mouth from a medical professional (my mom), apparently some people have reported pain in the first injection site after getting their second dose. So maybe they are looking for that.
Doors were mostly kept open by default instead of closed by default. The first person asking me health-related questions even asked me if I wanted to close the door.

When I received my shot the NP mentioned that most people don’t have side effects from the vaccine. He also said the side effects seem to have a similar profile as the Pfizer and Moderna vaccines, namely mild flu-like symptoms for a day or two. This was not exactly supported by the literature I ended up reading.

Did I get the Placebo? Using Bayes' Theorem

I would like to have a better idea of my chances at having...
[AN #140]: Theoretical models that predict scaling laws
by Rohin Shah — 2021-03-04T18:10:08.586Z

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

Please note that while I work at DeepMind, this newsletter represents my personal views and not those of my employer.

HIGHLIGHTS

Explaining Neural Scaling Laws and A Neural Scaling Law from the Dimension of the Data Manifold (Yasaman Bahri, Ethan Dyer, Jared Kaplan, Jaehoon Lee, and Utkarsh Sharma) (summarized by Rohin): We’ve seen lots of empirical work on scaling laws (AN #87), but can we understand theoretically why these arise? This paper suggests two different models for how power-law scaling laws could arise, variance-limited and resolution-limited scaling, and argues that neural nets are typically trained in the resolution-limited setting. In both cases, we have versions that occur when the dataset size D is large and the number of parameters P is low (parameter-limited), and when D is low and P is large (data-limited).

Recall that a scaling law is a power-law equation that predicts the test loss L as a function of P and D. In this paper, we consider cases where only one of the resources is the bottleneck, so that our power laws are of the form L = kP^(-α) or L = kD^(-α), for constants k and α. (For simplicity, we’re assuming that the minimum value of our loss function is zero.)

Resolution-limited scaling happens when either the dataset is too small to “resolve” (capture) the true underlying function, or when the model doesn’t have enough capacity to “resolve” (fit) the training dataset. In this case, we’re going to take the common ML assumption that while our observation space might be high-dimensional, the data itself comes from a low-dimensional manifold with dimension d, called the intrinsic dimension. We’ll model our neural net as transforming the input space into a roughly d-dimensional representation of the manifold, which is then used in further processing by later layers. Thus the output of the network is a simple function over this low-dimensional representation.

Let’s first consider the case where P is sufficiently large, so that we perfectly fit the training data, but D is limited. We can think of the training data as a “net” of points covering the ...
A Semitechnical Introductory Dialogue on Solomonoff Induction
by Eliezer Yudkowsky — 2021-03-04T17:27:35.591Z

(Originally posted in December 2015: A dialogue between Ashley, a computer scientist who's never heard of Solomonoff's theory of inductive inference, and Blaine, who thinks it is the best thing since sliced bread.)

i. Unbounded analysis

ASHLEY: Good evening, Msr. Blaine.

BLAINE: Good evening, Msr. Ashley.

ASHLEY: I've heard there's this thing called "Solomonoff's theory of inductive inference".

BLAINE: The rumors have spread, then.

ASHLEY: Yeah, so, what the heck is that about?

BLAINE: Invented in the 1960s by the mathematician Ray Solomonoff, the key idea in Solomonoff induction is to do sequence prediction by using Bayesian updating on a prior composed of a mixture of all computable probability distributions—

ASHLEY: Wait. Back up a lot. Before you try to explain what Solomonoff induction is, I'd like you to try to tell me what it does, or why people study it in the first place. I find that helps me organize my listening. Right now I don't even know why I should be interested in this.

BLAINE: Um, okay. Let me think for a second...

ASHLEY: Also, while I can imagine things that "sequence prediction" might mean, I haven't yet encountered it in a technical context, so you'd better go a bit further back and start more at the beginning. I do know what "computable" means and what a "probability distribution" is, and I remember the formula for Bayes's Rule although it's been a while.

BLAINE: Okay. So... one way of framing the usual reason why people study this general field in the first place, is that sometimes, by studying certain idealized mathematical questions, we can gain valuable intuitions about epistemology. That's, uh, the field that studies how to reason about factual questions, how to build a map of reality that reflects the territory—

ASHLEY: I have some idea what 'epistemology' is, yes. But I think you might need to start even further back, maybe with some sort of concrete example or something.

BLAINE: Okay. Um. So one anecdote that I sometimes use to frame the value of computer science to the study of epistemology is Edgar Allen Poe's argument in 1833 that chess was uncomputable.

ASHLEY: That doesn't sound like a thing that actually happened.

BLAINE: I know, but it totally did happen and not in a metaphorical sense either! Edgar Allen Poe wrote an essay explaining why no ...
A non-logarithmic argument for Kelly
by Bunthut — 2021-03-04T16:21:13.192Z

This post is a response to abramdemski's post, Kelly *is* (just) about logarithmic utility.

any argument in favor of the Kelly formula has to go through an implication that your utility is logarithmic in money, at some point. If it seems not to, it's either:

mistaken
cleverly hiding the implication
some mind-blowing argument I haven't seen before.

Challenge accepted. This is essentially a version of time-averageing which gets rid of the infinity-problem.

Consider the Kelly-betting game: Each round, you can bet any fraction of your wealth on a fair coinflip, which will be tripled if you win. You play this game for an infinite number of rounds. Your utility is linear in money.

The first thing to note is that this game does not have expected utility maximization recommend betting everything each round. This is true for any finite version of the game, but this version has various infinite payoffs, or no well-defined payoffs at all, since it doesn't end. We will get around this by, instead of computing expectations for strategies and comparing them based on expectation size, comparing them directly.

First, consider the formal specification of expected utility maximisation: smax=argmaxs∈S(E(U(game(s))) for S the set of strategies. Or written slightly unconventionally: smax=argmaxs∈S(limn→∞1/nn∑i=0(U(game(s;r(i))))) with r as a source of randomness. This spells out the expected value as the average of a sample of size going to infinity. We can turn this into a comparison between strategies:

s1≥s2⟺limn→∞[1/nn∑i=0(U(game(s1;r(i))))]≥limm→∞[1/mm∑j=0(U(game(s2;r(j))))]

with the idea of then picking the strategy that is maximal under this order. We then try to pull the comparison inside the limit:

s1≥s2⟺limn→∞[1/nn∑i=0(U(game(s1;r(i))))≥1/nn∑j=0(U(game(s2;r(j))))]

but this doesn't quite work, because we have a truth value inside the limit. Replace that with a propability (and dropping the normalizers, since they dont matter):

s1≥s2⟺limn→∞[P[n∑i=0(U(game(s1;r(i))))≥n∑j=0(U(game(s2;r(j))))]]>0

and for the games where classic utility maximization was well-defined this should give the same results.

Now we can properly define our infinite game: game(s;r) gets a third parameter indicating the number of rounds played: game(s,r,t) stands for playing the kelly-game for t rounds instead of infinitely long. The full game is then the limit of t...
Connecting the good regulator theorem with semantics and symbol grounding
by Stuart_Armstrong — 2021-03-04T14:35:40.214Z

EDIT: For a great reappraisal of the good regulator theorem using modern terminology, see here.

I've been writing quite a bit about syntax, semantics, and symbol grounding.

Recently, I've discovered the "good regulator theorem" in systems science[1]. With a bunch of mathematical caveats, the good regulator theorem says:

Every good regulator of a system must be a model of that system.

Basically if anything is attempting to control a system (making it a "regulator"), then it must model that system. The definition of "good" includes minimum complexity (that's why the regulator "is" a model of the system: it includes nothing else that would be extraneous), but we can informally extend that to a rougher theorem:

Every decent regulator of a system must include a model of that system.

Models, semantics, intelligence, and power

I initially defined grounded symbols by saying that there was mutual information between the symbols in the agent's head and features of the world.

I illustrated this with a series of increasingly powerful agents dedicated to detecting intruders in a greenhouse; each of them had internal variables Xi, with the more powerful agent's internal variables corresponding more closely to the presence of a intruder (denoted by the variable x).

For the simplest agent, a circuit-breaking alarm, the symbol Xa just checked whether the circuit was broken or not. It had the most trivial model, simply mapping the Boolean of "circuit broken: yes/no" to that of "sound alarm: yes/no".

It could be outwitted, and it could go off in many circumstances where there was no intruder in the greenhouse. This is hardly a surprise, since the alarm does not model the greenhouse or the intruders at all: it models the break in the circuit, with the physical setup linking that circuit breaking with (some cases of) intruders. Thus the correlation between Xa and x is weak.

The most powerful agent, a resourceful superintelligent robot dedicated to intruder-detection, has internal variable Xr. In order to be powerful, this agent must, by the good regulator theorem, be able to model many different contingencies and situations, having a good grasp of all the ways intruders might try to fool it, and have ways of detecting each of those ways. It has a strong model of the (relevant parts of the) world, and Xr is very closely tied to x.

A more powerful agent could still fool it. If that a...
Covid 3/4: Declare Victory and Leave Home
by Zvi — 2021-03-04T13:20:01.046Z

Health officials look on in horror as individuals both vaccinated and unvaccinated, and state and local governments, realize life exists and people can choose to live it.

This is exactly what I was worried about back in December when I wrote We’re F***ed, It’s Over. The control system would react to the good news in time to set us up to get slammed by the new strains, and a lot of damage can get done before there is a readjustment. The baseline scenario from two months ago is playing out.

The good news, in addition to the positive test percentages continuing to drop for now, is that we have three approved vaccines rapidly scaling up and are well ahead of the vaccine schedule I anticipated, having fully recovered from last week’s dip, and it looks like the new strains are more infectious but not on the high end of the plausible range for that.

The J&J vaccine was approved this week, after a completely pointless three week delay during which no information was found and (for at least the first two-thirds of it) no distribution plan formed. Anything I put at 98%+ on a prediction website isn’t fully news, but the other 2% would have been quite terrible. Supply will initially be limited, but will expand rapidly, including with the help of Merck.

Meanwhile, now that we were provided a sufficiently urgent excuse that we were able to show that mRNA vaccines work, we’ve adopted them to create a vaccine for Malaria. Still very early but I consider this a favorite to end up working in some form within (regulatory burden) number of years. It’s plausible that the Covid-19 pandemic could end up net massively saving lives, and a lot of Effective Altruists (and anyone looking to actually help people) have some updating to do. It’s also worth saying that 409k people died of malaria in 2020 around the world, despite a lot of mitigation efforts, so can we please please please do some challenge trials and ramp up production in advance and otherwise give this the urgency it deserves? And speed up the approval process at least as much as we did for Covid? And fund the hell out of both testing this and doing research to create more mRNA vaccines? There’s also mRNA vaccines in the works for HIV, influenza and certain types of heart disease and cancer. These things having been around for a long time doesn’t make them not a crisis when we have the chance to fix them. And y...
Grabby aliens and Zoo hypothesis 
by avturchin — 2021-03-04T13:03:17.277Z

Robin Hanson created a model of grabby aliens. In this model, we live before the arrival of an alien colonisation wave, because such a wave will prevent the appearance of the new civilizations. Thus, we could find ourselves only before the arrival of the aliens if any exists in our Universe.

However, at least some of the colonisators will preserve a fraction of habitable planets for different reasons: ethics, science, tourism, neglect. Let’s assume that it will be 0.01 of the total colonized volume. The numbers could vary, but it still looks like that in a densely packed universe the total volume of colonized space-time is significantly larger than the space-time for habitable planets before colonization arrival, and thus even a fraction of this volume could be larger than the volume of the virgin habitable space. This is because the colonized space will exist almost forever until the end of the universe.

Moreover, any small effort from the alien civilization to seed life (artificial panspermia) or to protect habitable planets from catastrophes like asteroid impacts will significantly increase the number of habitable planets inside the colonization zone. Hanson’s model also assumes that the probability of civilization appearance for any given planet is growing with time, so later regions will have a higher density of habitable planets, as more planets will reach this stage.

Given all this, our civilization has higher chances to appear after the colonization wave has passed us and thus aliens need to be somewhere nearby, but hidden, which is known as the Zoo Hypothesis. In other words, we live inside the sphere of influence of Kardashev 3 civilization which either helped our appearance via artificial panspermia etc or at least do not prevent our existence.

In this formulation, the idea starts to look like a variant of the simulation argument as here it is assumed that an advance civilization could create many non-advance civilizations.
Book review: "A Thousand Brains" by Jeff Hawkins
by Steven Byrnes — 2021-03-04T05:10:44.929Z

Jeff Hawkins gets full credit for getting me first interested in the idea that neuroscience might lead to artificial general intelligence—an idea which gradually turned into an all-consuming hobby, and more recently a new job. I'm not alone in finding him inspiring. Andrew Ng claimed

here

that Hawkins helped convince him, as a young professor, that a simple scaled-up learning algorithm could reach Artificial General Intelligence (AGI). (Ironically, Hawkins scoffs at the deep neural nets built by Ng and others—Hawkins would say: "Yes yes, a simple scaled-up learning algorithm can reach AGI, but not that learning algorithm!!")

Hawkins's last book was On Intelligence in 2004. What's he been up to since then? Well, if you don't want to spend the time reading his journal articles or watching his research meetings on YouTube, good news for you—his new book, A Thousand Brains, is out! There’s a lot of fascinating stuff here. I'm going to pick and choose a couple topics that I find especially interesting and important, but do read the book for much more that I'm not mentioning.

A grand vision of how the brain works

Many expert neuroscientists think that the brain is horrifically complicated, and we are centuries away from understanding it well enough to build AGI (i.e., computer systems that have the same kind of common-sense and flexible understanding of the world and ability to solve problems that humans do). Not Jeff Hawkins! He thinks we can understand the brain well enough to copy its principles into an AGI. And he doesn't think that goal is centuries away. He thinks we're most of the way there! In

an interview last year

he guessed that we’re within 20 years of finishing the job.

The people arguing that the brain is horrifically complicated seem at first glance to have a strong case. The brain has a whopping 1011 neurons with 1014 synapses, packed full of intricate structure.

One study

found 180 distinct areas within the cerebral cortex. Neuroscience students pour over huge stacks of flashcards with terms like “striatum”, “habenula”, “stria medullaris”, “fregula”, and "interpeduncular nucleus". (Quiz: Which of those are real brain regions, and which are types of pasta?) Every year we get another 50,000 or so new neuroscience papers dumped into our ever-deepening ocean of knowledge about the brain, with no end in sight.

So the brain is indeed horr...