trentmkelly/LessWrong-43k huggingface dataset:
Intuitions on Universal Behavior of Information at a Distance
by johnswentworth — 2020-04-20T21:44:42.260Z

What’s special about

information at a distance

? Some examples of what I mean:

What information about a little patch of pixels in a photo is relevant to another little patch of pixels far away in the same photo?
Relationship between exact voltages in two transistors which are far apart on the same CPU
When planning a road trip from San Francisco to LA, which details of the route within SF are relevant to the details of the route within LA?

The unifying feature in all these examples is that we imagine high-level observables which are “far apart” within some low-level model, in the sense that interactions between any two observables are mediated by many layers of hidden variables. In an undirected causal model, we could draw the picture like this:

There’s a handful of observed variables (black nodes), whose interactions are mediated by many layers (dotted groups) of unobserved variables (white nodes). Somewhere in the middle, everything connects up. We’re interested in the limit where the number of layers of hidden variables between any two observables is large.

In the limit of many hidden variables intermediating interactions, I suspect that there’s some kind of universal behavior going on - i.e. all models (within some large class) converge to behavior with some common features.

This post will give some hand-wavy, semi-mathematical arguments about what that behavior looks like. The main goal is to show some intuitive pictures; later posts can more carefully build out the details and boundaries of the phenomena sketched here.

Outline:

Background intuition on information distributed across variables, which is not contained in any one variable. Intuitively, it seems like this information should be quickly wiped out in large, noisy systems.
How to quantify distributed information.
Characterization of systems with zero distributed information.
Quick test: in a large system of normal variables, do we actually see the expected behavior? We do, and it looks a lot like a prototypical phase change in complex systems theory.

Intuition on Distributed Information

Fun fact: with three random variables X, Y, Z, it’s possible for X to contain zero information about Z, and Y to contain zero information about Z, but for the pair (X,Y) together to contain nonzero information about Z.

Classic example: X and Y are u...
What are the externalities of predictions on wars?
by Mati_Roy — 2020-04-20T17:37:09.510Z

To which extent, if at all, does (probabilistically) predicting peace increases or decreases peace in expectation?

To which extent, if at all, does (probabilistically) predicting war increases or decreases in expectation?

Do those predictions have any other significant impact on the world?

Answers might very well depend on the type of wars.
Training Regime Day 16: Hamming Questions
by Mark Xu — 2020-04-20T14:51:31.310Z

Note: I have acquired enough slack to resume writing, but cannot guarantee that it will be consistent.

Introduction

Sometimes, it makes sense to view your life from a narrative lens - to pretend that you're the main character in a book of your favorite genre. From there, you can try and make guesses at what the plot might be. For some of you, there will actually be a plot - an ambition/mission/purpose that shines through you. For others, such a plot will be less obvious, and that's ok; remember, "not all those who wander are lost."

However, given that your life has a plot, you should take a moment to remember that you don't actually live in a book; there's no real reason why the plot can't just end at chapter two. This is a different problem - one that is tackled by other techniques.

The goal of Hamming questions is to figure out what the heart of your plot is - to find the biggest problems in your life, so you know which problems you need to solve.

Warning

By ancient literary tradition, all great magics require an equally great sacrifice. The power of hamming questions is to rapidly accelerate the course of your life, identifying key milestones and obstacles for you to confront. The cost is unique to each individual, but might include the pain of an accurate self-assessment, uncomfortable and difficult conversations, and the possibility of catastrophic failure.

This isn't a light warning. There are people who will have net-worse lives if they try to seriously answer certain hamming questions. If you find it extremely difficult to think of an answer, that's a sign you should stop looking - most of the time, people can't think certain thoughts for very good reasons.

The rule is that you should be able to do things that you're slightly uncomfortable with, but to not force yourself to do things you don't want to do. Be kind to your parts. Don't avoid all pain, but there is some threshold of pain that you should definitely avoid.

There's some meta-hazard where people might think that not being able/ready to answer hamming questions is a sign of weakness and then try to force themselves to answer. I'm not sure pointing out the meta-hazard resolves it, but I don't think it makes it worse.

Hamming questions

Step one

Sometimes, all of the problems in someone's life revolve around a single nexus. Sometimes, this is extremely obvious. Hence the first question: what's the b

...
AI Services as a Research Paradigm
by VojtaKovarik — 2020-04-20T13:00:40.276Z
Link: https://docs.google.com/document/d/1Jk2GgJnF9pLIQqg9hgX0Tvdukq6MR2qmXW3FyvqiiUg/edit?usp=sharing

Introduction

I would hereby like to advertise a document Systems of Services
as a Paradigm for AI Alignment
. I hope that it can serve as a starting point for investigating AI alignment through the lens of systems of AI services. An alternative framing for the text is that it is a collection of pieces I have found particularly helpful for having productive conversations about the topic. In this post, I briefly recap the motivation behind the document and outline its contents. I also argue for why service-systems are a useful paradigm and mention my best guesses for promising future work.

Motivating ideas

As part of a recent collaboration, we wanted to look over technical problems in Comprehensive AI Services (CAIS), then pick one of them and study it in more detail. However, it soon turned out that a lot more conceptual work was needed before we can formalize any of our thoughts in a way that would be useful. We did study Drexler's Reframing Superintelligence prior to embarking on this project, and the text indeed provides a lot of useful insights into the nature of advanced systems of AI services. However, it puts less emphasis on how to use the service framework for technical research. So from our starting position, we weren't sure how to define the basic concepts, how to model problems in CAIS, and what these problems even are.

We also decided to focus on a slightly different framing, mostly in two ways. First, Reframing Superintelligence primarily discusses comprehensive AI services, which are expected to be on par with a artificial general intelligence. Since we can reasonably expect AI X-risk to be associated with AI that is radically transformative, we wanted to deal with a broader class of systems. Second, it is likely that even very advanced AI services will be deeply entangled with services that are "implemented on humans". (Especially since we can view human organizations and institutions as clusters of services.) From this reason, the document studies hybrid systems of services that consist of both AI- and human services. Even if our interest is primarily in AI services, studying the whole system (including its non-AI parts) should make it easier to fully understand their impact on the world.

Contents of the document

While we did eventually make some progress on more specific problems, I thought it might be useful to write down a separate "introdu

...
What confidence interval should one report?
by Jsevillamol — 2020-04-20T10:31:54.107Z

TL;DR: How to choose the width of the confidence intervals one reports?

I am working on a paper on statistical technological forecasting.

Our model produces confidence intervals for the date when a particular milestone will be achieved. I am now thinking about how to report the results.

One line of reasoning is that you want the results to be as informative as you can, so we would report 90% confidence intervals.

Another one is that you dont want to overstate your results, specially on a domain such as the one I am working on where model error could be as high as 1/3 of the models. So your confidence interval should be commesurate with the expected base rate of model error (eg 70% confidence intervals in this case).

I think I am not framing this problem correctly, so any thoughts would be appreciated.
covid-19 notes, 4/19/20
by nostalgebraist — 2020-04-20T05:30:01.873Z

Brain-dumping some miscellaneous Covid-19 thoughts.

“Mind viruses”

[if you’re skimming, this is the least interesting part of the post IMO]

In late March I wrote this big, long dramatic proclamation about information cascades and stuff.

Back then, it felt like the situation in the US was at an inflection point – at various kinds of inflection point – and I was feeling this particular combination of anxiety and passion about it. A do-or-die emotion: something was happening quickly, we had a limited window in which to think and act, and I wanted to do whatever I could to help. (”Whatever I could do” might be little or nothing, but no harm in trying, right?)

I felt like the intellectual resources around me were being under-applied – the quality of the discussion simply felt worse than the quality of many discussions I’d seen in the past, on less important and time-sensitive topics. I did my best to write a post urging “us” to do better, and I’m not sure I did very well.

In any event, those issues feel less pressing to me now. I don’t think I was wrong to worry about epistemically suspect consensus-forming, but right now the false appearance of a consensus no longer feels like such a salient obstacle to good decision-making. We’ve seen a lot of decisions made in the past month, and some of them have been bad, but the bad ones don’t reflect too much trust in a shaky “consensus,” they reflect some other failure mode.

Bergstrom

Carl Bergstrom’s twitter continues to be my best source for Covid-19 news and analysis.

Bergstrom follows the academic work on Covid-19 pretty closely, generally discussing it before the press gets to it, and with a much higher level of intellectual sophistication while still being accessible to non-specialists.

He’s statistically and epistemically careful to an extent I’ve found uncommon even among scientists: he’s comfortable saying “I’m confused” when he’s confused, happily acknowledges his own past errors while leaving the evidence up for posterity, eloquently critiques flawed methodologies without acting like these critiques prove that his own preferred conclusions are 100% correct, etc.

I wish he’d start wri...
What Happens During a Recession
by Elizabeth — 2020-04-20T03:03:56.586Z

3 weeks ago I asked

“What happens during a recession anyway?”

I’ve since researched and given answers on 5 different aspects of recessions, plus there’s a secret six aspect I haven’t published because it requires many graphs, and images are a pain in comments. None of these was worth a full post on their own, but I believe in aggregate they add up to more than one post’s worth of content. Answers currently have low visibility, which the LessWrong team is working on changing, but in the meantime it seemed worth aggregating all of these comments plus the new content in a top level post.

All data is for the United States unless otherwise specified.

Unemployment

My prediction:

Unemployment increases in a recession.This creates a long lasting negative effect on people who enter the labor force during a recession (unemployment scarring).
Women’s employment is more stable than men’s

By Sector

According to

Have employment patterns in recessions changed?

(which was published in 1981), recessions universally (for n=4) concentrate employment in the service sector, by 1-3 percentage points

Looked at in more detail, from the

Bureau of Labor Statistics

By Gender

From 1953-1980, women have a higher unemployment rate than men, during both expansions and recessions. From 1980 on, men and women have nearly identical unemployment rates in good times, but men’s unemployment has higher peaks during recessions.

Unemployment over time, by gender

At first I thought this was because men are more likely to work in manufacturing, which is more procyclic (see next section), but the pattern holds even within sectors

But unemployment typically means ‘is looking for work’. Perhaps women who lose their jobs are more likely to call themselves Stay-At-Home-Parents and stop looking for work. What happens to the labor force participation rate?

So that’s not it either.

By Race

Black and Latino people typically have a higher unemployment rate than white people (I did not find equivalent data for Asians over a long enough time period), and are hit harder during recessions

By Age

Unfortunately I could only find comparative data going back to 1990, but it looks like youth unemployment is continually higher than older adults, by a similar amount over time.

This isn’t the whole story though, because unemployment can have long ...
Bass Whistle in the Browser
by jefftk — 2020-04-20T02:10:02.864Z

Last summer I wrote

some code that would
let you control a bass synthesizer by whistling (

details). I wrote it in a

low-level
language so I could minimize latency, but the downside is that
it's a bit of a pain to install and try out. After playing around
with the

Web
Audio API

yesterday, however, I
realized I could make a demo in JavaScript. Here it is:

bass-whistle-js.

Currently this only works in Chrome, because Firefox and Safari haven't implemented
AudioWorklet yet. It has enough latency to be painful, even
though the AudioContext is running with minimum
latency settings. I suspect this is Chrome layering several
buffers, and comes from optimizing for something other than what I
want (cross-platform, simplicity, minimizing dropouts, power
consumption). Still, it's neat that this works at all!

Debugging my AudioWorklet port was painful, as browser debugging goes.
Devtools doesn't understand the worklet very well, and so several
syntax errors in the worklet were reported as syntax errors in the UI
thread code. The devtools "disable cache" setting also doesn't seem
to apply to worklet modules, and they don't even show up in the
Network tab, so I put in a cachebuster during development.

Comment via: facebook
Database of existential risk estimates
by MichaelA — 2020-04-20T01:08:39.496Z

This post was written for Convergence Analysis, though the opinions expressed are my own. Cross-posted from the Effective Altruism Forum.

This post:

Provides a spreadsheet you can use for making your own estimates of existential risks (or of similarly “extreme” outcomes)

Announces a database of estimates of existential risk (or similarly extreme outcomes), which I hope can be collaboratively expanded and updated

Discusses why I think this database may be valuable

Discusses some pros and cons of using or making such estimates

Key links relevant to existential risk estimates

Here’s a spreadsheet listing some key existential-risk-related things people have estimated, without estimates in it.

The link makes a copy of the spreadsheet so that you can add your own estimates to it.

I mention this first so that you have the option of providing somewhat independent estimates, before looking at (more?) estimates from others.

Some discussion of good techniques for forecasting, which may or may not apply to such long-range and extreme-outcome forecasts, can be found here, here, here, here, and here.

Here’s a database of all estimates of existential risks, or similarly extreme outcomes (e.g., reduction in the expected value of the long-term future), which I’m aware of. I intend to add to it over time, and hope readers suggest additions as well.

The appendix of this article by Beard et al. is where I got many of the estimates from, and it provides more detail on the context and methodologies of those estimates than I do in the database.

Beard et al. also critically discuss the various methodologies by which existential-risk-relevant estimates have been or could be derived.

In this post, I discuss some pros and cons of using or stating explicit probabilities in general.

Why this database may be valuable

I’d bet that the majority of people reading this sentence have, at some point, seen one or more estimates of extinction risk by the year 2100, from one particular source.[1] These estimates may in fact have played a role in major decisions of yours; I believe they played a role in my own career transition. That source is an informal survey of global catastrophic risk researchers, from 2008.

As Millett and Snyder-Beattie note:

The disadvantage [of that survey] is that the estimates were likely highly subjective and unreliable, especially as the survey did not a

...
Every system seems random from the inside
by  — 2020-04-19T23:02:55.151Z

[Cross-posted from Grand, Unified, Crazy. This builds on a lot of stuff I wrote from before I was cross-posting to Less Wrong, but should be mostly intelligible on a general intuition of what a "computational system" is.]

I’ve been working on a post on predictions which has rather gotten away from me in scope. This is the first of a couple of building-block posts which I expect to spin out so I have things to reference when I finally make it to the main point. This post fits neatly into my old (2014!) sequence on systems theory and should be considered a belated addition to that.

Systems can be deterministic or random. A system that is random is, of course… random. I’m glad the difficult half of this essay is out of the way! Kidding aside, the interesting part is that from the inside, a system that is deterministic also appears random. This claim is technically a bit stronger than I can really argue, but it guides the intuition better than the more formal version.

Because no proper subsystem can perfectly simulate its parent, every inside-the-system simulation must ultimately exclude information, either via the use of lossy abstractions or by choosing to simulate only a proper, open subsystem of the parent. In either case, the excluded information effectively appears in the simulation as randomness: fundamentally unpredictable additional input.

This has some interesting implications if reality is a system and we’re inside it, as I believe to be the case. First it means that we cannot ever conclusively prove whether the universe is deterministic (a la Laplace’s Demon) or random. We can still make some strong probabilistic arguments, but a full proof becomes impossible.

Second, it means that we can safely assume the existence of “atomic randomness” in all of our models. If the system is random, then atomic randomness is in some sense “real” and we’re done. But if the system is deterministic, then we can pretend atomic randomness is real, because the information necessary to dispel that apparent randomness is provably unavailable to us. In some sense the distinction doesn’t even matter anymore; whether the information is provably unavailable or just doesn’t exist, our models look the same.
Explaining the Rationalist Movement to the Uninitiated
by Mellivora — 2020-04-19T20:53:59.154Z

Edited for LessWrong from my Original Post.

On a personal level, I have struggled greatly to adequately explain to other people not already familiar with the community and its values, the core of what Rationality is, and why they should care. The Sequences are obviously a great start, and once someone has "read the sequences", they are highly likely to have a good idea of what the Rationalist Movement is all about, but someone unsure of whether they are interested or not is unlikely to commit to reading such a lengthy set of articles.

In my quest to find a way to short-cut this process, I have come across many other posts trying to do something similar, but all are still quite... long winded. (And if you are already thinking that this post looks too long, just skip to the end, where I get to the point!)

What Do We Mean By "Rationality"? (1671 words)

Twelve Virtues of Rationality (2205 words)

Biases: An Introduction (1787 words)

Whilst all of these are perfectly reasonable, if either technical, mystical or both, they are difficult to boil down to something short and pithy. It is not that one couldn't necessarily convince someone to read these, but they can't exactly be rolled off the tongue in mid conversation. Anything that I have previously read that is a short and pithy description of Rationality can either easily be woefully misinterpreted, resulting in “so Rationalists want us to all be emotionless robots then? Count me out!”, or is really a description of a small subset of the community, making it sound far more specific and exclusive than it actually is. Here are another two articles that come remarkably close:

The Secret Society for Suppressing Stupidity (3815 words)

Why I Am Not Rene Descartes (3480 words)

These articles focus heavily on defending Rationality against its detractors, which is again very reasonable, however unfortunately these still don’t easily boil down into a short, hard to misinterpret definition. Therefore, I will make an attempt to describe what I consider Rationality and the Rationalist Movement to be, and not to be, in a way that can be boiled down without losing all subtlety.

I. What it is not

Firstly, I shall deal with what Rationality is not. Unfortunately, there are several schools of philosophy that Rationality is often conflated with, so I will go through each of them, and br...