trentmkelly/LessWrong-43k huggingface dataset:
Survival and Flourishing Fund grant applications open until October 4th ($1MM-$2MM planned for dispersal)
by habryka — 2019-09-09T04:12:09.240Z

The Survival and Flourishing Fund is a new fund that (as I understand it) is taking over a significant chunk of the Berkeley Existential Risk Initiative's (BERI) grantmaking, and is just opening a new grant round that I am helping to disburse:

http://survivalandflourishing.org/#2019-q4-grant-round

The plan is to make a total of $1MM-$2MM in grants to organizations working on the long term flourishing and survival of humanity.

At this point in time SFF can only make grants to charities and not individuals, so if you have a project or organization that you want to get funding for, you will have to either already be part of an established charity, found one, or be sponsored by an existing charity.

Also happy to answer any questions you have in the comments, to the best of my knowledge.
Looking for answers about quantum immortality.
by Fivehundred — 2019-09-09T02:16:03.435Z

I've been recently been obsessing over the risks of quantum torment, and in the course of my research downloaded this article: https://philpapers.org/rec/TURFAA-3

Here's a quote:

"4.3 Long-term inescapable suffering is possible

If death is impossible, someone could be locked into a very bad situation where she can’t die, but also can’t become healthy again. It is unlikely that such an improbable state of mind will exist for too long a period, like millennia, as when the probability of survival becomes very small, strange survival scenarios will dominate (called “low measure marginalization” by (Almond 2010). One such scenario might be aliens arriving with a cure for the illness, but more likely, the suffering person will find herself in a simulation or resurrected by superintelligence in our world, perhaps following the use of cryonics.

Aranyosi summarized the problem: “David Lewis’s point that there is a terrifying corollary to the argument, namely, that we should expect to live forever in a crippled, more and more damaged state, that barely sustains life. This is the prospect of eternal quantum torment” (Aranyosi 2012; Lewis 2004). The idea of outcomes infinitely worse than death for the whole of humanity was explored by Daniel (2017), who called them “s-risks”. If MI is true and there is no high-tech escape on the horizon, everyone will experience his own personal hell.

Aranyosi suggested a comforting corollary (Aranyosi 2012), based on the idea that multiverse immortality requires not remaining in the “alive state”, but remaining in the conscious state, and thus damage to the brain should not be very high. It means, according to Aranyosi, that being in the nearest vicinity of death is less probable than being in just “the vicinity of the vicinity”: the difference is akin to the difference between constant agony and short-term health improvement. However, it is well known that very chronic states of health exist which don’t affect consciousness are possible, e.g. cancer, whole-body paralysis, depression, and lock-in syndrome. However, these bad outcomes become less probable for people living in the 21st century, as developments in medical technology increase the number of possible futures in which any disease can be cured, or where a person will be put in cryostasis, or wake up in the next level of a nested simulation. Aranyosi suggested severa...
Stories About Progress
by Ben Pace — 2019-09-08T23:07:10.443Z

This is the 5th post of 5 containing the transcript of a podcast hosted by Eric Weinstein interviewing Peter Thiel.

Interview

Girard’s Mimetic Theories

Eric Weinstein: So let me return back to the line of inquiry. I mean, sorry, just enjoying so much hearing what you have to say. Some of it's new to me. The theories that might be portals into a different way of looking at the world.

Eric Weinstein: One of them that you brought into my, I've never heard of before was Girard's various theories. And I wonder if you might say, you've often credited success in business to how you understood and you applied Girard. I mean obviously he didn't have this kind of level of business success. So can you talk a little bit about your personal relationship to Rene Girard's theories as a portal into a different way of seeing the world?

Peter Thiel: Well let's say a little bit about the theory. So it's, it was sort of this theory of human psychology as deeply mimetic where you sort of, you copy other people.

Eric Weinstein: So, so just for the folks at home mimetic as in mime rather than memetic is in meme.

Peter Thiel: Yes. Well they're probably closely related. But you imitate people but that's how you learn to speak as a child. You copy your parents language, that's how, but then you also imitate desire and then there are sort of all sorts of aspects of mimesis that can lead to sort of mass violence mass insanity. So it has, it's both what enables human culture to function, but it also is quite, quite dangerous.

Peter Thiel: And you know, when I came across this sort of constellation of ideas as an undergraduate at Stanford, you know, my biases were sort of libertarian, classic liberal, only individuals exist. Individuals are radically autonomous, can think for themselves. And so this was, it was sort of a powerful corrective to that intellectually. But then it also worked on an existential level where you sort of realize, wow, there are all these ways that I've been hyper mimetic, I've been hyper tracked, why am I at Stanford, why does this matter so much? Why, you know, why am I doing all the things I'm doing?

Peter Thiel: And that's, it's a prism through which one when looks at a lot of things that I found to be quite helpful over recent decades. I think the preference falsification you can think of in mimetic terms where, you know, everybody goes along with what everybody else ...
The Power to Make Scientific Breakthroughs
by Liron — 2019-09-08T04:14:14.402Z

This is Part IV of the Specificity Sequence

So far, everything we've accomplished with our specificity powers has been admittedly kind of destructive: demolishing bad arguments, judging startup ideas as being either definitely bad or only maybe bad... it would be nice to accomplish something more constructive.

Let's try activating our specificity powers for science. Because at least in science, if we destroy a currently-accepted theory, then it's a good sign that we're probably on our way to inventing a better theory!

Choosing Where to Zoom In

In The Power to Demolish Bad Arguments, we zoomed into generic claims by inspecting specific examples of those claims. Actually, "zooming in" is an incomplete description of what we did. There's a one-to-many relationship between generic claims and specific examples. How do you choose the examples? How do you choose where to zoom into a claim?

It depends if you're trying to check basic coherence or pinpoint a contradiction.

Checking Basic Coherence

If you're confused about what the claim even means, you can just try zooming in anywhere to look for any coherent meaning of the claim.

This was our strategy for zooming into Steve's claim that "Uber exploits its drivers by paying them too little". We asked Steve to give us any specific example of what he was talking about, and there were none to be found, so the discussion came to an abrupt stop.

Some readers accused me of hunting for the weakest part of Steve's claim, so I explained in a comment why it's really not like that:

I'm not saying to hunt for a counterexample that demolishes a claim. I'm saying to ask the person making the claim for a single specific example that's consistent with the general claim.

Imagine that a general claim has 900 examples and 100 counterexamples. Then I'm just asking to see one of the 900 examples :)

Pinpointing a Contradiction

When a general claim has already shown you plenty of specific examples, but it also has some questionable implications that may be contradictory or nonsensical, that's when you want to go on the hunt for a counterexample.

This is how we chose where to zoom into Steve's claim that "Oregon's coastline is too straight. I wish all coastlines were less straight so that they could all have a bay!" We purposely chose to zoom in on the most questionable part of Oregon's coastline, the least straight part, the part that would best con...
Do you have algorithms for passing time productively with only your own mind?
by Rudi C — 2019-09-07T20:48:42.084Z

Things that require nothing, just you. At least in the short term. (e.g., it’s ok to think about what question would be worthwhile to ask on Lesswrong.)
I myself sometimes have specific questions from my studies and/or practical matters, which are excellent for this purpose. But the supply of these questions usually is much less than the time I have for pondering them.
Age gaps and Birth order: Reanalysis
by Bucky — 2019-09-07T19:33:16.174Z

This post follows on from my previous post detailing some areas where I was unable to reproduce Scott’s analysis of how the age gap between siblings modifies the SSC Birth order effect. I suggest you read that post first but here’s the summary:

I attempted to reproduce Scott’s analysis of Birth order effect vs Age gap. I found that:

There appeared to be an error in graphs 2 & 3 where people with one sibling were counted when they shouldn’t have been (graph 2) or were counted twice (graph 3)

Comparing oldest children to youngest children causes a bias in the results which can be prevented by comparing oldest children to 2nd oldest children

I was unable to reproduce Scott’s result on people reporting 0 year age gap – I get a non-significant 58% older siblings compared to Scott’s 70%. I was unable to discover the cause of the difference.

Summary

I reanalysed how sibling age gap modifies the SSC birth order effect. I found that:

The birth order effect is relatively steady for the first 4-8 years of age gap at about 70% respondents being the firstborn vs secondborn. For larger age gaps the effect reduces. There is insufficient evidence to conclude how long this reduction takes or whether the effect is completely removed at very large age gaps.

2 other trends were noted in the data but evidence for them was not strong:

The reduction may not be the same (or might disappear) for larger families
Birth order effect may be lower at 1 year age gap vs 2-7 year age gap

Considering competing theories on the cause of the Birth order effect, two theories fit the data well:

Intra-family dynamics
Decreased parental investment

And three theories fit the data poorly:

Changed parental strategies
Maternal antibodies
Maternal vitamin deficiencies

Introduction

The original reason for me looking at this data was to analyse whether the data support a sudden drop between years 7 and 8 or whether there is an alternative explanation which fits the data.

I will note here that I’m not a trained statistician and am using this as practice of Bayesian model comparison, inspired by johnswentworth’s recent model comparison sequence. I’d say I’m 80% confident in my broad conclusions, less so in the specifics - I'd be fairly confident there are a couple of errors lurking in here somewhere.

Analysis: All family sizes combined

Is there a sudden drop after 7 years?

Getting back to the data, here’s the result th...
Age gaps and Birth order: Failed reproduction of results
by Bucky — 2019-09-07T19:22:55.068Z

Summary

I attempted to reproduce Scott’s analysis of Birth order effect vs Age gap. I found that:

1. There appeared to be an error in graphs 2 & 3 where people with one sibling were counted when they shouldn’t have been (graph 2) or were counted twice (graph 3)

2. Comparing oldest children to youngest children causes a systematic bias in the results. This can be prevented by comparing oldest children to 2nd oldest children

3. I was unable to reproduce Scott’s result on people reporting 0 year age gap – I get a non-significant 58% older siblings compared to Scott’s 70%. I was unable to discover the cause of the difference.

I have reanalysed the data based on points 1 & 2 in a separate post.

Previously in Birth order effect

In the 2018 Slate Star Codex survey Scott asked some questions about what order in the family respondents were born. He found that eldest children were massively overrepresented.

Following on, historical mathematicians and Nobel winning physicists were found to exhibit the same property.

In the 2019 SSC survey Scott included questions about age gaps between respondents and their adjacent siblings. He analysed the results, finding that:

This study found an ambiguous and gradual decline [in Birth order effect] from one to seven years [Age gap between siblings], but also a much bigger cliff from seven to eight years.

Failed reproduction of Scott’s graphs

I had originally intended to analyse the data to see if I could draw any further conclusions. However, when running the analysis I found that I was unable to reproduce Scott’s results.

Scott includes 3 graphs.

The first – comparing % of sample oldest child vs age gap for people with 1 sibling – I was able reproduce almost exactly (Scott also has access to respondents’ data who asked not to be included in the public data so we aren’t exactly the same. There may be other differences too but these are small).

The second – comparing how many oldest vs youngest children there are in the sample for people with more than 1 sibling – I was unable to reproduce. Actually, I was able to reproduce the graph but only if I also included people with 1 sibling.

This is actually what the third graph was supposed to show, but it looks like the third graph double counts the people with 1 sibling.

The graphs are below, with Scott’s on the left and my reproductions on the right. Note the similarity between Scott’s gra...
Are minimal circuits deceptive?
by evhub — 2019-09-07T18:11:30.058Z

This post is part of research I did at OpenAI with mentoring and guidance from Paul Christiano.

One possible inner alignment technique is to structure your learning algorithm’s inductive biases such that it is far more likely for it to produce an aligned model than a misaligned model. Two basic ways in which you might get traction now on how a technique like that will scale to AGI might be:

Do experiments with current regularization techniques where you try to determine to what extent models end up being aligned with the loss function they were trained under depending on the types of inductive biases present during their training. I’m actually fairly excited about this approach, and I am working on a post right now which will hopefully provide some ways in which you might start going about doing that. One concern with this sort of approach, however, is that you might not be able to get any traction on phenomenon that only appear once your capabilities get really close to AGI.

Analyze from a theoretical perspective what happens in the limit of perfect capabilities. The idea here is to try to patch the previous problem where when you do experiments you don’t get to see phenomenon that only happen when your capabilities get really advanced by imagining that your capabilities were perfect. In this case, however, you end up with the opposite problem of not seeing any phenomenon that only appear when your capabilities are weaker.

The nice thing about combining these two approaches is that they tackle the problem from both sides, giving you an idea both of what the weak capability landscape looks like as well as the strong capability landscape. For this post, however, the focus will be on the second approach.

Background

From a theoretical perspective, the two most natural types of inductive biases are simplicity bias and time complexity bias. Paul Christiano has previously argued that simplicity bias can be deceptive in the limit, but left the question of whether the same is true of time complexity bias as an open problem. The goal of this post is to attempt to resolve that open problem. In particular, though one might have hoped that deceptive alignment could be avoided by heavily penalizing time complexity so that the model doesn’t have time to spend reasoning about deception, I will argue that this is not the case due to the possibility of meta-learning.

First, ...
Are we living at the most influential time in history?
by jp — 2019-09-07T18:09:00.411Z
Link: https://forum.effectivealtruism.org/posts/XXLf6FmWujkxna3E6/are-we-living-at-the-most-influential-time-in-history-1

Will MacAskill has been posting several articles on the EA Forum. This one seems especially interesting for a LessWrong audience. In it Will approaches the title question by attempting to create an outside view prior, and then updates based on our inside view evidence. Great discussion follows in the comments.
Integrating the Lindy Effect
by lsusr — 2019-09-07T17:38:27.348Z

Suppose the following:

1. Your intelligence is directly proportional to how many useful things you know.

2. Your intelligence increases when your learn things and decreases as the world changes and the things you know go out-of-date.

How quickly the things you know become irrelevant is directly proportional to how many relevant things you know and therefore proportional to your intelligence I and inversely proportional to the typical lifetime of things you know L. Let's use R to denote your rate of learning. Put this together and we get a equation.

IL+dIdt∝R

If we measure intelligence in units of "facts you know" then the proportionality becomes an equality.

dIdt=R−IL

The solution to this first order differential equation is an exponential function.

I(t)=ce−t/L+RL

We must solve for c. For convenience let's declare that your intelligence is 0 at time t=0. Then c must equal −RL. That gives us a tidy solution.

I(t)=RL(1−e−t/L)

Our solution makes sense intuitively because your intelligence is directly proportional to R and L. But wait a minute. L isn't just a coefficient. It's in the exponential too.

Time t and Lifetime L

Most human beings reading this article will be between 10 years and 100 years old. In other words, t is measured in decades. In other other words, t is on the order of 10 years.

L values, on the other hand, are distributed exponentially across many orders of magnitude.

Order of days. (0.003 years)

daily newspaper

Order of weeks (0.2 years)

Sunday newspaper
political story
sports game outcome

Order of decades (10 years)

programming languages

Order of centuries (100 years)

classic literature
most spoken languages

Order of millennia (1,000 years)

cooking
history

Order of 10,000 years

human psychology

Order of gigaannum (billion years)

biology
physics

Forever

math

The details of whether exactly each of these things fit on the scale is not important. What is important is that most things you can know have a useful lifetime at least one order of magnitude away from the human timescale of decades. In other words, we can assume that either L is much greater than t or much less than t.

Suppose L is much less than t. Then the exponential vanishes and we're left with I=RL. In other words, if L≪t then how long you have been learning for t is irrelevant. I is constant with respect to time. Years and years of s...
Russian x-risks newsletter, summer 2019
by avturchin — 2019-09-07T09:50:51.397Z

This is the first Russian x-risks newsletter, which will present news about Russia and global catastrophic risks from the last 3 months.

Given the combination of high technological capabilities, poor management, high risk tolerance and attempts to catch up with West and China in the military sphere, Russia is prone to technological catastrophes. It has a 10 times higher level of aviation catastrophes and car accidents than developed countries.

Thus it seems possible that a future global catastrophe may be somehow connected with Russia. However, most of the work in global catastrophic and existential risk (x-risks) prevention and policy efforts are happening in the West, especially in US, UK and Sweden. Even the best policies adopted by the governments of these countries may not help if a catastrophe occurs in another country or countries.

This newsletter will help to inform the Western public of activity relative to x-risks in Russia. All information presented here is based on open sources. However, these sources require some systematisation and debiasing. For example, some Russian news sources tend to present situation as worse than they are just to attract attention. Another source of biases is the ongoing new cold war.

This is the first test issue of the newsletter, which I will issue every 3 months. I will publish it on LessWrong, the existential risk subreddit and existential risk Facebook page. I do not currently have a subscription option available. Below are news for summer 2019:

1. “Planetary catastrophe” was prevented after a fire aboard the deepwater nuclear submarine Losharick “by the dead heroes” on its crew, as was said during the funeral for the victims. Most likely, this was just rhetoric. However, a more ominous but less probable interpretation is that the catastrophe was somehow connected with the Poseidon nuclear torpedo, which is said to be capable of carrying a 100 megaton cobalt-salted bomb (still not a planetary catastrophe, even if it explodes in a harbour, but surely a regional one). This is, however, unlikely, as Losharik is a small submarine, incapable of carrying such a large system.

2. Russia’s biggest state-owned bank, Sberbank, said that AI could be the last invention of humanity. Sberbank chairman German Gref is known for his technoprogressivist views; as well as one of the main investors in AI development; the bank invited Bos...
AI Safety "Success Stories"
by Wei Dai — 2019-09-07T02:54:15.003Z

AI safety researchers often describe their long term goals as building "safe and efficient AIs", but don't always mean the same thing by this or other seemingly similar phrases. Asking about their "success stories" (i.e., scenarios in which their line of research helps contribute to a positive outcome) can help make clear what their actual research aims are. Knowing such scenarios also makes it easier to compare the ambition, difficulty, and other attributes of different lines of AI safety research. I hope this contributes to improved communication and coordination between different groups of people working on AI risk.

In the rest of the post, I describe some common AI safety success stories that I've heard over the years and then compare them along a number of dimensions. They are listed in roughly the order in which they first came to my attention. (Suggestions welcome for better names for any of these scenarios, as well as additional success stories and additional dimensions along which they can be compared.)

The Success Stories

Sovereign Singleton

AKA Friendly AI, an autonomous, superhumanly intelligent AGI that takes over the world and optimizes it according to some (perhaps indirect) specification of human values.

Pivotal Tool

An oracle or task AGI, which can be used to perform a pivotal but limited act, and then stops to wait for further instructions.

Corrigible Contender

A semi-autonomous AGI that does not have long-term preferences of its own but acts according to (its understanding of) the short-term preferences of some human or group of humans, it competes effectively with comparable AGIs corrigible to other users as well as unaligned AGIs (if any exist), for resources and ultimately for influence on the future of the universe.

Interim Quality-of-Life Improver

AI risk can be minimized if world powers coordinate to limit AI capabilities development or deployment, in order to give AI safety researchers more time to figure out how to build a very safe and highly capable AGI. While that is proceeding, it may be a good idea (e.g., politically advisable and/or morally correct) to deploy relatively safe, limited AIs that can improve people's quality of life but are not necessarily state of the art in terms of capability or efficiency. Such improvements can for example include curing diseases and solving pressing scientific and technological problems.

(I wa...