trentmkelly/LessWrong-43k huggingface dataset:
Motivating Abstraction-First Decision Theory
by johnswentworth — 2020-04-29T17:47:31.896Z

Let’s start with a prototypical embedded decision theory problem: we run two instances of the same “agent” from the same source code and in the same environment, and the outcome depends on the choices of both agents; both agents have full knowledge of the setup.

Roughly speaking,

Functional decision theory

and its predecessors argue that each instance of the agent should act as if it’s choosing for both.

I suspect there’s an entirely different path to a similar conclusion - a path which is motivated not prescriptively, but descriptively.

When we say that some executing code is an “agent”, that’s an abstraction. We’re abstracting away the low-level model structure (conditionals, loops, function calls, arithmetic, data structures, etc) into a high-level model. The high-level model says “this variable is set to a value which maximizes <blah>”, without saying how that value is calculated. It’s an agent abstraction: an abstraction in which the high-level model is some kind of maximizer.

Key question: when is this abstraction valid?

Abstraction = Information at a Distance

talks about what it means for an abstraction to be valid. At the lowest level, validity means that the low-level and high-level models return the same answers to some class of queries. But the linked post reduces this to a simpler definition, more directly applicable to the sort of abstractions we use in real life: variables in a high-level model should contain all the information in corresponding low-level variables which is relevant “far away”. We abstract far-away stars to point masses, because the exact distribution of the mass and momentum within the star is (usually) not relevant from far away.

Or, to put it differently: an abstraction is “valid” when components of the low-level model are independent given the values of high-level variables. The roiling of plasmas within far-apart stars is (approximately) independent given the total mass, momentum, and center-of-mass position of each star. As long as this condition holds, we can use the abstract model to correctly answer questions about things far away/far apart.

Back to embedded decision theory.

We want to draw a box around some executing code, and abstract all those low-level operations into a high-level “agent” model - some sort of abstract optimizer. The only information retained by the high-level model is the functional form of the abstr...
[AN #97]: Are there historical examples of large, robust discontinuities?
by Rohin Shah — 2020-04-29T17:30:02.043Z

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

HIGHLIGHTS

Discontinuous progress in history: an update (Katja Grace) (summarized by Nicholas): One of the big questions in AI alignment is whether there will be a discontinuous AI takeoff (see here (AN #62) for some reasons why the question is decision-relevant). To get a better outside view, AI Impacts has been looking for large discontinuities in historical technological trends. A discontinuity is measured by how many years ahead of time that value is reached, relative to what would have been expected by extrapolating the trend.

They found ten 100-year discontinuous events, for example in ship size (The SS Great Eastern), the average speed of military payload across the Atlantic Ocean (the first ICBM), and the warmest temperature of superconduction (yttrium barium copper oxide).

There are also some interesting negative examples of discontinuities. Particularly relevant to AI are AlexNet not being a discontinuity on the ImageNet benchmark and chess performance not having any discontinuities in Elo rating.

Nicholas' opinion: Ignoring the George Washington Bridge (which confuses both me and the authors), I’d roughly categorize the causes of these discontinuities as

- 3 of them were due to a concerted but apparently misplaced effort towards something others weren’t trying to do. These are Pyramid of Djoser, SS Great Eastern, and the Paris Gun.

- 2 of them were due to the Atlantic Ocean causing a threshold effect (as they explain in the post). These are the ICBM and the first nonstop transatlantic flight.

- 4 of them were due to a new technological breakthrough followed by increased investment and a faster rate of progress. These are the two telegraph cables, nuclear weapons, and superconductors.

Of these, the final category seems the most relevant to AGI timelines, and I could imagine AGI development following a similar trajectory, where a major breakthrough causes a large amount of investment and then we have much faster progress on AI going forward.

I was quite surprised that AlexNet did not represent a discontinuity on ImageNet performance. I...
Facebook AI: A state-of-the-art open source chatbot
by FactorialCode — 2020-04-29T17:21:25.050Z
Link: https://ai.facebook.com/blog/state-of-the-art-open-source-chatbot

Abstract:

Building open-domain chatbots is a challenging area for machine learning research. While prior work has shown that scaling neural models in the number of parameters and the size of the data they are trained on gives improved results, we show that other ingredients are important for a high-performing chatbot. Good conversation requires a number of skills that an expert conversationalist blends in a seamless way: providing engaging talking points and listening to their partners, both asking and answering questions, and displaying knowledge, empathy and personality appropriately, depending on the situation. We show that large scale models can learn these skills when given appropriate training data and choice of generation strategy. We build variants of these recipes with 90M, 2.7B and 9.4B parameter neural models, and make our models and code publicly available under the collective name Blender. Human evaluations show our best models are superior to existing approaches in multi-turn dialogue in terms of engagingness and humanness measurements. We then discuss the limitations of this work by analyzing failure cases of our models.

Blog post: https://ai.facebook.com/blog/state-of-the-art-open-source-chatbot

Paper: https://arxiv.org/abs/2004.13637

Project page: https://parl.ai/projects/blender/
Peter's COVID Consolidated Brief - 29 Apr
by Peter Wildeford — 2020-04-29T15:27:35.638Z

It’s been almost a month since my last COVID Consolidated Brief and I hope you are all doing ok. I’ve personally been settling into the new normal. On the other hand, I’ve witnessed first hand some of the risks that might be coming with COVID. I had to take shelter in a tornado warning for the first time in my life. While the tornado and the destruction were luckily quite minimal, there was a power failure for about a day and social distancing made it a lot harder to wait out the power failure in a nearby library or Starbucks. Overall, I’m lucky my life is so safe that this is the biggest problem, but I am worried about people who might be a lot less likely and face strong hurricanes or wildfires while also having to maintain social distancing. More on this in a bit.

If you’re just joining us, I follow COVID-19 a lot and this is my third semi-regular installment of a public consolidated brief that tries to consolidate everything I read into one short, actionable list so other people can stay up to date without reading a ton on their own. For this issue, I spent over 25 additional hours trying to get to the bottom of everything so that you don’t have to. This way I can save time and

fight research debt

and save you time from having to read all of this yourself. That being said, do keep in mind that I am not an expert and I have not been able to cover everything going on - I had to be fairly selective to make this brief actually somewhat brief.

I’m not sure how often I will do these, but I still intend to do them as I am able. Maybe it will be a monthly newsletter. Maybe I’ll be able to do it every other week. We’ll see!

Previously:

2 April Brief

29 March Brief

My research questions

(27 March)

See also:

LessWrong links database

EA Coronavirus Facebook Group

Doing Your Part! How You Can Stay Safe and Help the Fight!

Masks

Masks are a good idea - tell your friends! The opinion on masks has changed a lot since I last reported about a month ago.

WHO is now onboard

: “The World Health Organisation says it supports government initiatives that require or encourage the public wearing of masks, marking a major shift from previous advice amid the Covid-19 pandemic. [...] The WHO added that surgical masks should be reserved for medical professionals, while the public should use mainly cloth or home-made face coveri...
I do not like math
by lsusr — 2020-04-29T11:27:17.214Z

I do not like math very much.

I do not find math completely boring. If I were stuck in a prison cell I'd happily do math until I expired from a ruptured spline. Insofar as I have escaped prison, I have attempted to escape math too.

Avoiding math was particularly difficult while I earned my bachelor's degree with majors in physics and (a-hem) pure mathematics. Undaunted, I found many ways to avoid doing real math while ostensibly doing math.

The lowest-hanging fruit to go unpicked was proofs. Many math classes are nothing but a professor proving a single theorem. We were never tested on anything but the conclusion. Students majoring in math are expected to pay intrinsic interest to these lectures. I daydreamed through them all. A mathematician writing a proof of a theorem is strong Bayesian evidence the theorem is true. Lots of other students watched the professors for errors. Transparency is the best disinfectant. This is good enough for me. With so much transparency, reading the proofs felt redundant, like irradiating my hands and then washing them. I wash my hands of, well, washing my hands.

My favorite method of avoiding math is proof by contradiction. Proof by contradiction assumes a theorem is false and then contradicts itself with a counterexample. It's obscenely dissatisfying.

Alas, some things are not provable by contradiction. This called for last resorts. Unable to do real math, I performed a physics substitution. I translated the math problem into physics, solved the problem and then translated it back into pure mathematics. In this way, calculus became kinematics, real analysis became electrodynamics, complex analysis became quantum mechanics, group theory became gauge theory and linear algebra became machine learning.

I didn't start gauge theory and machine learning until after I had nearly failed my algebra classes. Oops. Aside from this hiccup, my degree in pure mathematics became a cozy math supplement to my physics degree.

My professors taught me lots of math despite my best efforts to the contrary. For this I am grateful. Math was the second-hardest subject I've ever studied. Some day I hope to be good enough at math that I can work out how to not come across as "weird" in an informal social setting.
Negative Feedback and Simulacra
by Elizabeth — 2020-04-29T02:00:01.734Z

Part 1: Examples

There’s a thing I want to talk about but it’s pretty nebulous so I’m going to start with examples. Feel free to skip ahead to part 2 if you prefer.

Example 1: Hot sauce

In this r/AmITheAsshole post, a person tries some food their their girlfriend cooked, likes it, but tries another bite with hot sauce. Girlfriend says this “…insults her cooking and insinuates that she doesn’t know how to cook”.

As objective people not in this fight, we can notice that her cooking is exactly as good as it is whether or not he adds hot sauce. Adding hot sauce reveals information (maybe about him, maybe about the food), but cannot change the facts on the ground. Yet she is treating him like he retroactively made her cooking worse in a way that somehow reflects on her, or made a deliberate attempt to hurt her.

Example 2: Giving a CD back to the library

Back when I would get books on CD I would sometimes forget the last one in my drive or car. Since I didn’t use CDs that often, I would find the last CD sometimes months later. To solve this, I would drop the CD in the library book return slot, which, uh, no longer looks like a good solution to me, in part because of the time I did this in front of a friend and she questioned it. Not rudely or anything, just “are you sure that’s safe? Couldn’t the CD snap if something lands wrong?.” I got pretty angry about this, but couldn’t actually deny she had a point, so settled for thinking that if she had violated a friend code by not pretending my action was harmless. I was not dumb enough to say this out loud, but I radiated the vibe and she dropped it.

Example 3: Elizabeth fails to fit in at martial arts

A long time ago I went to a martial arts studio. The general classes (as opposed to specialized classes like grappling) were preceded by an optional 45 minute warm up class. Missing the warm up was fine, even if you took a class before and after. Showing up 10 minutes before the general class and doing your own warm ups on the adjacent mats was fine too. What was not fine was doing the specialized class, doing your own warm ups on adjacent maps for the full 45 minutes while the instructor led regular warm ups, and then rejoining for the general class. That was “very insulting to the instructor”.

This was a problem for me because the regular warm ups hurt, in ways that clearly meant they were bad for me (and this is at ...
Estimates of COVID risk rankings for different activities when together with people in the same household?
by nonzerosum — 2020-04-29T01:13:33.911Z

My questions in this thread may be relevant to people with roommates, or people who are visiting a small number of friends or family in person, but who want to still minimize risk.

I've seen and liked: http://www.overcomingbias.com/2020/04/covid19-pizza-risk-estimator.html

Here are some questions I have based on situations I'm running into most frequently:

what's the risk level of hugging an extended family member that I'm not living with?
what's the risk level of being together inside, but 6 ft apart? how much better is 6ft than 3ft or 9ft?
what's the risk level of shaking hands and then washing hands before touching anything else?
if I'm at an extended family member's house generally 4-6 ft apart, and eating off their silverware, using their serving utensils, etc, but I'm also washing my hands before eating, using tissues to open doors, etc, am I better off or are my protective measures moot if I'm spending a few hours with them?

And of course all of these would be estimates.

My biggest overall questions are:

how much benefit do protective measures do against people who could be (non-obviously i.e. no fever) sick in the same household, where no one is isolating? is it worth keeping 6 ft from extended family or if you're in the same house does it not do anything?
if protective measures are actually helpful even against people you're visiting for a few hours, what are the things that are particularly critical to avoid, and what are the things that are less important?

These questions are meaningful because right now I'm in the spot where our shared desire for human interaction has us going to one family member's house for a couple hours a couple times a week, but where my fear of getting sick (from one of them that's either asymptomatic or early before they realize) is high enough that I'm being a germaphobe while there (though I'm not willing to ask them to wear masks, basically I'm just doing the stuff where the burden is on me not them), while at the same time wishing I knew if it was worth being a germaphobe or whether all my efforts are moot anyway.

If you're in the same household as someone or visiting a family member's house for multiple hours e.g. for dinner, should you assume you'll get what they have and vice versa, or is spread far from guaranteed? And what...
Chris Masterjohn on Coronavirus, Part 2
by philh — 2020-04-28T21:50:01.430Z

Update (April 29th 2020): At the original source, I've made a signifiant update to the vitamin D section.

Background on this in part one.

General info: Interferon

Interferon is an antiviral that our immune system naturally releases when it detects invasion. But SARS-CoV and MERS-CoV (the virus that causes MERS) manage to avoid this response. The trigger is the presence of double-stranded RNA, which viruses need but our own cells don't. (We make single-stranded RNA from DNA, and then proteins from single-stranded RNA. We never need to copy our RNA. But viruses do need to copy their RNA, and apparently that means viral RNA needs to be double-stranded.) SARS-CoV and MERS-CoV hide their double-stranded RNA inside "double-membrane vesicles" to avoid detection.

They also have a bunch of other ways to limit production of interferon, and on top of that they limit the response to the interferon that does get produced.

This all sounds like "so we should ramp up interferon". But that's probably a bad idea. During early stages of the virus, interferon is suppressed, so the virus can replicate quickly. But when the infection is established, macrophages generate a lot of interferon, leading to a cytokine storm. In a mouse trial, deleting the genes for the main interferon receptor made the subjects very resistant to SARS-CoV, but very vulnerable to mouse hepatitis virus and influeza A, compared to normal mice. (0% died to a dose of SARS-CoV1 that killed 85% of normal mice; but 100% died to a dose of the other viruses that killed 10-20% of normal mice.)

(Question: "During the replication of the virus, macrophages are recruited to the lung". These are what release the interferon. What is a macrophage and what is recruiting them and why?)

We don't yet know that any of this applies to SARS-CoV-2 as well, but it seems likely. So high levels of interferon might be slightly helpful in early stages, but make later stages much worse. Thus, Chris recommends avoiding them.

(Question: LW user CellBioGuy is bullish on inhaled interferon pretreatment, which I take it means "inhaling interferon before the disease gets bad". Does this square with Chris' recommendations? It wouldn't surprise me if inhaling interferon increases your levels in the short term but not the long term, which is exactly what we want. On the other hand, he links a paper whose abstract says SARS-CoV-2 "is much more sens...
Training Regime Day 24: Resolve Cycles 2
by Mark Xu — 2020-04-28T19:00:09.060Z

Introduction

Two anecdotes that I claim have something in common:

Scene: me at office hours having worked on an algebra problem for 3+ hours.

TA: The solution involves matrices

Me: <realizes answer>

Scene: friend asking me for help solving a reasonably difficult CS problem that I don't know the answer to.

Friend: I tried X, Y and Z and nothing is working.

Me: Well, can't you just do A? Seems like it should work.

Friend: ...yeah that works.

In the example case, I already knew the solution involved matrices, but telling me "there exists a solution" let me solve the problem almost immediately. In the second example, someone was like "can you help me solve this?" and I figured out how to do it faster than I would have normally

The important part of the first example is that telling someone "there exists a solution" is implicitly telling them "there exists a solution and I expect you to be able to find it." The important thing about the second part is that asking someone to help you with something suggests that you expect them to be able to help.

It's all about expectation, you see.

Expectation Cycles

When someone expects me to be able to solve a problem, then I start thinking about the problem in a different way. I go from trying-to-solve to just solving. I stop thinking about how I might approach the problem, and just start approaching it. I stop wondering about the tools I might need and just do the obvious thing.

To turn this into a technique, you need to figure out a way to reliably generate expectation for yourself. People's brains work differently - there is no reliable and consistent way that works for everyone. Sometimes, the standard resolve cycle can accomplish this for you. If it does not, there are other tricks to try:

1. Tell your friend that you're going to solve your problem in N minutes

2. Pretend your friend is the one with the problem and they're asking you for help

3. Pretend someone you trust has told you that your problem has a solution they expect you to be able to find

A caveat with this technique is that if you can't solve the problem after you've expected yourself to be able to solve it, then it hurts. In order to leverage your full power to solve a problem, you have to expect to solve it. The corresponding implication is that if you don't solve it, then you will fe...
A Book Review
by Zack_M_Davis — 2020-04-28T17:43:07.729Z
Link: http://unremediatedgender.space/2020/Apr/book-review-human-diversity/

Someone wrote a blog post reviewing a book by some sociologist named Murray. Never heard of him. Anyway, I couldn't get through the whole thing because the reviewer has this really obnoxious writing style that uses way too many italics and exclamation points (as well as occasional weirdly out-of-place cuss words?!), but I did notice that he (?) links to Less Wrong a few times, which is something I don't see "in the wild" very often these days, so I "thought it couldn't hurt" to share the link here in case one of you happens to find it interesting?? (April 2020, ~11,200 words)
UAP and Global Catastrophic Risks
by avturchin — 2020-04-28T13:07:21.698Z

DoD has confirmed yesterday the authenticity of UAP videos and Yudkowsky made a bet: “So apparently there’s some kind of thing right now about supposed aliens. I haven’t looked into it at all, but I am happy to blindly bet against anything to do with visible intelligent aliens at 100:1 odds.”

So where are the other 99 odds – and how they affect our future and the probability of global risks? I created the biggest possible list of explanations of UAP and estimations of their effects on x-risks. There are several more extreme explanations than “aliens”, which challenge our model of the world: travellers between branches of Everettian multiverse, glitches in the Matrix, absurdity in Boltzmann Brains chains, etc.

Abstract: After 2017 NY Times publication, the stigma of the scientific discussion of the problem of so-called UAP (Unidentified Ariel Phenomena) was lifted. Now the question arises: how UAP will affect the future of humanity, and especially, the probability of the global catastrophic risks? To answer this question, we assume that the Nimitz case in 2004 was real and we will suggest a classification of the possible explanations of the phenomena. The first level consists of mundane explanations: hardware glitches, malfunction, hoaxes and lies. The second level involves explanations which are not changing completely our model of the world, but only updating it: new military tech, large psyop operation or new physical effect like ball lightning. The third level of explanations requires a complete overhaul of our world model and includes a large set of fantastic hypothesis: alien starships, interdimensional beings, glitches in the matrix, projections from the collective unconsciousness, Boltzmann brain’s experiences etc. The higher is the level of the hypothesis, the less probable it is, but the bigger is its consequences for possible global catastrophic risks. Thus, “integrating” over the field of all possible explanations, we find that UAP increases catastrophic risks. Technological progress increases our chances of direct confrontation with the phenomena. If the phenomenon has intelligence, the change of its behavior could be unexpected after we pass some unknown threshold. But we could lower the risks if we assume some reasonable policy of escaping confrontations.

The whole a...