trentmkelly/LessWrong-43k huggingface dataset:
Bayesian Inference with Partially Specified Priors
by Christopher Clark — 2020-03-01T05:02:38.075Z

In this post, I am going to write about the practical difficulty of specifying priors that reflect internal beliefs, and about the possibility of performing probabilistic inference with partially specified or imprecise priors. For the most part, I will try to give the general intuition and then link to some relevant scholarship. I will quickly review Bayes rule and probabilistic inference to start, but this post is primarily intended for readers that are already familiar with these concepts (you can read introductions

in

several

places

).

Bayes rule specifies how to compute the probability of a hypothesis, H, given some data, D:

p(H|D)=P(H)P(D|H)P(D)∝P(H)P(D|H)

Where p(H|D) is called the posterior, P(H) is the prior, and P(D|H) is the likelihood. If we assume rational people should update their beliefs in a way that is consistent with the laws of probability, then rational people should update their beliefs using Bayes rule. It follows from this view that Bayes rule is not just a mathematical observation, it is prescriptive in the sense that it describes how people (or algorithms, AIs, or other agents that aspire to be rational) should update their beliefs. In that case, the prior can be understood as stating how probable you think the hypothesis is given all the data that is available to you, except for D, and the posterior describes how probable you ought to think the hypothesis is after accounting for D.

A basic understanding of what all this means is that you should account for your prior belief when considering new evidence. For example, say you get tested for the presence of a particular genetic sequence in your DNA, and that the test is 99% accurate (meaning it has a 99% true positive rate and true negative rate) and returns positive. If your prior belief that you have the genetic sequence is very low (maybe you happen to know the base rate of the sequence in the general population is less than 0.001%), then you should conclude the test was most likely a false positive.

This is all well and good in theory, but if we try to explicitly use Bayes rule in practice we might find specifying our priors to be very difficult. Even worse, it can feel arbitrary or a matter of guesswork. Consider the previous example, only now imagine that you do not have convenient knowledge about the genetic sequence's base rate. In theory, a Bayesian would say, there ... ,
An Analytic Perspective on AI Alignment
by DanielFilan — 2020-03-01T04:10:02.546Z

This is a perspective I have on how to do useful AI alignment research. Most perspectives I’m aware of are constructive: they have some blueprint for how to build an aligned AI system, and propose making it more concrete, making the concretisations more capable, and showing that it does in fact produce an aligned AI system. I do not have a constructive perspective - I’m not sure how to build an aligned AI system, and don’t really have a favourite approach. Instead, I have an analytic perspective. I would like to understand AI systems that are built. I also want other people to understand them. I think that this understanding will hopefully act as a ‘filter’ that means that dangerous AI systems are not deployed. The following dot points lay out the perspective.

Since the remainder of this post is written as nested dot points, some readers may prefer to read it in workflowy.

Background beliefs

I am imagining a future world in which powerful AGI systems are made of components roughly like neural networks (either feedforward or recurrent) that have a large number of parameters.

Furthermore, I’m imagining that the training process of these ML systems does not provide enough guarantees about deployment performance.

In particular, I’m supposing that systems are being trained based on their ability to deal with simulated situations, and that that’s insufficient because deployment situations are hard to model and therefore simulate.

One reason that they are hard to model is the complexities of the real world.

The real world might be intrinsically difficult to model for the relevant system. For instance, it’s difficult to simulate all the situations in which the CEO of Amazon might find themselves.

Another reason that real world situations may be hard to model is that they are dependent on the final trained system.

The trained system may be able to affect what situations it ends up in, meaning that situations during earlier training are unrepresentative.

Parts of the world may be changing their behaviour in response to the trained system…

in order to exploit the system.

by learning from the system’s predictions.

...
Cortés, Pizarro, and Afonso as Precedents for Takeover
by Daniel Kokotajlo — 2020-03-01T03:49:44.573Z

Crossposted from AI Impacts.

Epistemic status: I am not a historian, nor have I investigated these case studies in detail. I admit I am still uncertain about how the conquistadors were able to colonize so much of the world so quickly. I think my ignorance is excusable because this is just a blog post; I welcome corrections from people who know more. If it generates sufficient interest I might do a deeper investigation. Even if I’m right, this is just one set of historical case-studies; it doesn’t prove anything about AI, even if it is suggestive. Finally, in describing these conquistadors as “successful,” I simply mean that they achieved their goals, not that what they achieved was good.

Summary

In the span of a few years, some minor European explorers (later known as the conquistadors) encountered, conquered, and enslaved several huge regions of the world. That they were able to do this is surprising; their technological advantage was not huge. (This was before the scientific and industrial revolutions.) From these cases, I think we learn that it is occasionally possible for a small force to quickly conquer large parts of the world, despite:

Having only a minuscule fraction of the world's resources and power
Having technology + diplomatic and strategic cunning that is better but not that much better
Having very little data about the world when the conquest begins
Being disunited

Which all suggests that it isn’t as implausible that a small AI takes over the world in mildly favorable circumstances as is sometimes thought.

EDIT: In light of good pushback from people (e.g. Lucy.ea8 and e.g. Matthew Barnett) about the importance of disease, I think one should probably add a caveat to the above: "In times of chaos & disruption, at least."

NEW EDIT: After reading three giant history books on the subject, I take back my previous edit. My original claims were correct.

Three shocking true stories

I highly recommend you read the wiki pages yourself; otherwise, here are my summaries:

Cortés: [wiki] [wiki]

April 1519: Hernán Cortés lands in Yucatan with ~500 men, 13 horses, and a few cannons. He destroys his ships so his men won't be able to retreat. His goal is to conquer the Aztec empire of several million people.
He makes his way towards the imperial capital, Tenochtitlán. Along the way he encounters various local groups, fighting some and allying with some. He is constantly

...
What Noise Does a Tambourine Make?
by jefftk — 2020-03-01T02:10:02.735Z

I recorded myself playing around with a tambourine, and I was somewhat
surprised to see what it looked like:

(mp3)

I'm thinking of it as four notes, and I hear four notes, but there are
really more like eight. As I start to move my wrist to play a note
there's a small jingle, and then a larger jingle when I complete the
movement. The larger jingle is on the beat, but without the early one
it doesn't sound like a tambourine. This two part sound is probably
why our onomatopoetic term is "jingle".

If I want to recreate this with sampling, one important question is
whether the spacing depends on tempo. If the jingle is longer at
lower tempos and shorter at faster tempos, then I need to treat it as
two sounds, while if the jingle is always the same length but played
more or less frequently, then I can take the whole jingle as a sample.

It looks to me like it's the latter:

The distance between peaks of the jingle looks like ~100ms with both
fast and slow playing. In the slow playing (top) the peaks of a
jingle are closer to each other than to the peaks of adjacent jingles,
while in faster playing (bottom) the peaks are more evenly spaced.

It looks like I only need two samples. First, a free jingle (1, 2,
4):

(wav)

And then a, louder, stopped jingle (3):

(wav)

One thing that makes this tricky is that this is very "slow
to sound". You need to trigger the sample ~100ms before you want the
beat to fall. In my stage setup I tap the downbeats and it
predicts the upbeats, so I handle this by, for the tambourine,
additionally predicting the next downbeat so it can anticipate it by
100ms.
What should be my triggers for initiating self quarantine re: Corona virus
by Eli Tyre — 2020-02-29T20:09:49.634Z

[Some answer here, where I asked this question in comments.]

Is there some reliable source on the triggers for quarantining one's self? Both immediate triggers (when X happens, I will not leave the house again until y happens), and warning signs (when Z happens, I'm going to leave the city, in preparation for quarantine, etc.).
UConn Effective Altruism
by jcz208 — 2020-02-29T19:46:26.776Z

Hey y'all! I'm not sure how many people from UConn browse LW, but I figured it'd be worth a shot to let y'all know that I'm starting up the UConn Effective Altruism club on the Storrs campus! Check out our

Facebook page

, or shoot me an email at jeffrey.duan@uconn.edu if this sort of thing piques your interest. We just got our active status through UConn, so we'll be starting meetings . I hope to hear from some of you guys!
Training Regime Day 15: CoZE
by Mark Xu — 2020-02-29T17:13:42.685Z

Note: this is going to be short because I spent some time doing COVID-19 preparations, which I encourage you do to also. I think that it is better for my commitment to post at least one thing a day, even if it's bad. I anticipate heavily revising this later, when I have more time.

Introduction

CoZE stands for Comfort Zone Expansion. The model is that people have a lot of beliefs about themselves and due to things like positive bias, some non-trivial chunk of these beliefs haven't ever been tested.

The goal of CoZE is to take beliefs about yourself and test them experimentally, both making your identity smaller and discovering new things about yourself. Remember to try things.

Finding your comfort zone

The first step is to find your comfort zone. You should take 10 minutes and write down all of the beliefs that you have about yourself. You should include "obvious" things to avoid stopping idea generation.

My list included:

I don't like dancing

I'm slightly more attractive than average, but not that attractive

I'm only slightly socially awkward

I'm bad at singing

I'm OK at writing

Find tests for your beliefs

An idea test for one of your beliefs should make you say "ugh fine." A test is too weak if you don't react emotionally to it. A test is too strong if it makes you go "ahhhhhhh".

For example, if I wanted to test "I'm not good at dancing", I might take a single beginner's dance class. I wouldn't want to go to a party and start dancing in front of everyone I knew. I also wouldn't want to take an advanced dance class.

It is important that you're fair to your belief. If I wanted to test "I'm bad at singing", then I should pick an easy song that I enjoy to try singing, I shouldn't pick a really hard song to sing. The test should actually behave differently depending on whether or not the belief is true. I might sing a hard song badly even if I'm good at singing.

Exercise

Find one of the beliefs that you found above, devise a test for it, and do the test.
Usable implementation of IDA available
by rmoehn — 2020-02-29T08:54:09.155Z

Paul Christiano published his and Buck Shlegeris' implementation at https://github.com/paulfchristiano/amplification. It's the code behind the article Supervising strong learners by amplifying weak experts.

With William Saunders' permission, I published a version modified by him and later me: https://github.com/rmoehn/amplification This one has changes and more documentation that allow you to run it almost out of the box.
COVID-19 - a good or bad time for extended travel?
by wolverdude — 2020-02-29T08:18:20.992Z

A recent loosening of my employment and relational commitments has afforded me the opportunity to do some unencumbered international travel. My intention is to spend a month or more in a single locale so I can get to know the culture and learn from their perspectives. This is something I've wanted to do for a while, and this would be the perfect time, but... COVID-19.

How COVID-19 changes things

New failure modes:

I get turned away or quarantined upon arrival in my destination due to outbreaks in the US, particularly in the Bay Area (which has the most cases in the US so far, not to mention being a densely-populated metropolitan area with a lot of ties to China).

There's an outbreak in the country I'm visiting and it becomes difficult, expensive, or impossible to return to the US due to travel restrictions.

I carry COVID-19 to a new location and become responsible for a new outbreak.

It becomes more difficult to socialize at my destination due to fears around COVID-19.

I contract COVID-19 during travel due to the cramped/crowded conditions.

I develop pneumonia from COVID-19 in a country where:

I don't know anyone who can take care of / watch out for me.

I don't know how to navigate the medical system and/or don't have medical insurance coverage.

The hospitals are insufficiently resourced to be able to handle my case.

New benefits:

Cheaper flights due to other people's travel cancellations.

Escaping the Bay Area during the possible pending outbreak.

My questions

Should I go?

Where should I go?

Is there poor thinking or relevant information I haven't considered?

Should I go?

Concerns:

Any of the bad scenarios listed above. I realize some of these can be mitigated, but not all.

I have traveled abroad before, but always with a group that had established connections and a clear itinerary, so this is a self-stretching exercise to begin with.

I am a software engineer, so I can probably get a job that permits full-time remote work, enabling me to do this at a later time. But I will still have the obligation of work, and I may also become committed to relationships that make this much harder. I may not get another chance this good.

Timing: It seems the best policy is to get out of here ASAP while the likelihood that I am infected remains minuscule and travel restrictions/quarantines can be avoided. But the longer I wait, the more information will come out a

...
Approaches for collecting and analyzing data about yourself?
by DivineMango — 2020-02-29T00:09:09.167Z

As step one in actually improving my life, I am embarking on a massive expedition for data, both about myself (e.g. triggers for moods, when I feel the most focused) and about the effects of certain interventions (e.g. meditation, taking short naps). However, I'm not quite sure what the best way is to go about this.

I've been using the iPhone apps iMoodJournal and Track Your Happiness for about a week for hourly check-ins, and they've worked fine, but they don't quite have the flexibility and breadth of functionality I'm looking for.

Ideally, I'd have something that would let me add check-in entries about my mood, focus/productivity, state of physical discomfort, or anything else I'd want to measure on a numbered scale. I'd also like to be able to enter in these check-ins things that I'd done in the previous hour and then see an analysis of how these activities/interventions affect me (e.g. how my state is different when I do X vs not-X not just at that moment, but in the hours after as well).

People who have done something like this: what was your approach? Is this much precision necessary? Does something like I describe exist?

My general question is this: what are some ideas for ways to systematically notice and track your internal state to find correlations between events/actions and your state?
Trace: Goals and Principles
by johnswentworth — 2020-02-28T23:50:12.900Z

In terms of research, I decided to devote the month of February mainly to foundations and tools. One project was to come up with a notation/language/framework which matches the way I’ve been thinking about computation - i.e. DAGs with symmetry and “clouds” representing DAG-structure-as-data. The tool I’ve been building - a Python library tentatively called Trace - isn’t stable enough that I want to show it off yet, but I do think I’ve nailed down the core goals and principles, so it’s time to write them up.

Goals

The main thing I need for my research is a data structure suitable for automated causal abstraction algorithms on arbitrary computations. Some subgoals:

Universality: data structure should be able to represent any computation performed by a program

Data structure needs to be finite, which means leveraging symmetry to represent infinite computational DAGs

Computations must be straightforward both for a human to specify directly and for an algorithm to manipulate

Want to be able to do causal-DAG-like things, like query for parents/children of a node or perform interventions/counterfactuals along the lines of Pearl’s do()

Need to handle DAGs with dynamic structure

Eventually I’ll want to do reflective things with this data structure (i.e. self-modeling agents), so simplicity matters in the specification and core algorithms.

I’ll give a bit more detail...

The first two subgoals basically amount to “I need a data structure representing the computation performed by an arbitrary program” - i.e. the trace of an arbitrary program. I do need to actually use the data structure, so it needs to be finite. (We could use a lazy infinite structure, but then I want to know what finite data structure is used to actually represent the infinite data structure.) The computational DAGs of programs are usually infinite but symmetric, so the solution probably needs to leverage symmetry in the representation.

I (a human) need to be able to specify computations - so I either need an interface which would basically be a programming language, or I need to transpile from an existing programming language. Ideally the human-facing representation would directly reflect the computer-facing data structure, which weighs against transpilation from an existing language. Also, existing languages have way more bells and whistles than I really want to deal with for this project, even sett

...
What to make of Aubrey de Grey's prediction?
by Rafael Harth — 2020-02-28T19:25:18.027Z

Aubrey de Grey (head of SENS) just had an appearance on Joe Rogan's podcast. At some point during the interview, he made a fairly specific and extremely bold prediction about the near future.

See 56:40 - 58:45+. The short version is: sometime in the near future ("could easily happen in the next 3 to 5 years"), there will be an extremely sudden shift in the public perception on aging through which the alleged fact that it can be reversed becomes common knowledge. At that point, there will be enormous pressure towards funding the field, and you won't get elected without promising to throw money at the problem.

A relevant reference class here could be something like "expert in a field predicting a massive near-term shift of awareness about something related to their field." Predictions in this class probably come true about 0% of the time. On the other hand, there are some reasons to think Aubrey is unusually credible.

My questions are

How seriously do you take this claim? Aubrey didn't specify a testable criterion in this conversation, but a reasonable one could be something like "a candidate in the 2024 presidential general election lists fighting aging as a campaign issue on their official website."
Suppose Aubrey is correct, and there is a massive shift within the next five years. Is there a way for a normal person to benefit financially out of knowing this now?
Seeing the Smoke
by Jacob Falkovich — 2020-02-28T18:26:58.839Z

Cross-posted from Putanumonit.

COVID-19 could be pretty bad for you. It could affect your travel plans as countries impose quarantines and close off borders. It could affect you materially as supply chains are disrupted and stock markets are falling. Even worse: you could get sick and suffer acute respiratory symptoms. Worse than that: someone you care about may die, likely an elderly relative.

But the worst thing that could happen is that you’re seen doing something about the coronavirus before you’re given permission to.

I’ll defend this statement in a minute, but first of all: I am now giving you permission to do something about COVID-19. You have permission to read up on the symptoms of the disease and how it spreads. Educate yourself on the best ways to avoid it. Stock up on obvious essentials such as food, water, soap, and medicine, as well as less obvious things like oxygen saturation monitors so you know if you need emergency care once you’re sick. You should decide ahead of time what your triggers are for changing your routines or turtling up at home.

In fact, you should go do all those things before reading the rest of the post. I am not going to provide any more factual justifications for preparing. If you’ve been following the news and doing the research, you can decide for yourself. And if instead of factual justifications you’ve been following the cues of people around you to decide when it’s socially acceptable to prep for a pandemic, then all you need to know is that I’ve already put my reputation on the line as a coronaprepper.

Instead this post is about the strange fact that most people need social approval to prepare for a widely-reported pandemic.

Smoke Signals

As Eliezer reminded us, most people sitting alone in a room will quickly get out if it starts filling up with smoke. But if two other people in the room seem unperturbed, almost everyone will stay put. That is the result of a famous experiment from the 1960s and its replications — people will sit and nervously look around at their peers for 20 minutes even as thick smoke starts obscuring their vision.

The coronavirus was identified on January 7th and spread outside China by the 13th. American media ran some stories about how you should worry more about the seasonal flu. The markets didn’t budge. Rationalist Twitter star...