trentmkelly/LessWrong-43k huggingface dataset:
How important are MDPs for AGI (Safety)?
by michaelcohen — 2020-03-26T20:32:58.576Z

I don't think finite-state MDPs are a particularly powerful conceptual tool for designing strong RL algorithms. I'll consider the case of no function approximation first.

It is certainly easier to do RL in a finite-state MDP. The benefit of modeling an environment as a finite-state MDP, and then using an MDP-inspired RL algorithm, is that when the agent searches for plans to follow, it doesn't evaluate the same plans twice.

Instead, it caches the (approximate) "value" for each possible "state", and then if a plan would take it to a state that it has already evaluated, it doesn't have to re-evaluate what the plan would be from that point on. It already knows, more or less, how much utility it could get thereafter. Compare that to naïve approach of using a world-model to do full expectimax search at each timestep.

The model-the-environment-as-finite-state-MDP-then-do-dynamic-programming approach, or just "the MDP approach" for short, is, I think, all about not searching the same region of the planning search space twice. This is clearly a good thing, but I don't think the MDP approach in RL contains much more conceptual progress toward AGI than that. If I were to try to do a pre-natum of a fairly advanced RL agent, that is, if I tried to anticipate a response to "things went well; why did that happen?", my guess would be that a big part of the answer would be:

It avoids searching much of the planning search space even once, certainly not twice.

The MDP approach with function approximation is more powerful, depending on how good the function approximation is. There's no upper bound on how good the MDP approach with function approximation could be, because buried inside the function approximation (whether that's approximation of the value, or the optimal policy, or both) could be some clever RL algorithm that does most of the work on its own. A good function approximator that is able to generate accurate predictions of the value and/or the optimal policy might appear to us to "generalize" well across "similar states". But it's not clear to me to what extent it is a useful abstraction to say that the function approximator thinks in terms of the agent bouncing around a set of states that it classifies as more or less similar to each other.

I don't mean to say th...
What is the safe in-person distance for COVID-19?
by Raemon — 2020-03-26T20:29:52.732Z

I'm not sure where the 6' number comes from, and I'm skeptical it really holds up as something I'd be comfortable maintaining for an extended period of time (If someone with c19 coughed at me from 6' away I would not feel very safe). I'm guessing the 6' is more like a quick rule for people who are only interacting briefly.

How much does it matter whether you're up/downwind? I've heard conflicting things about how airborne it might be.

I'm interested in this largely for "Okay, assuming we need to be careful about this for months at a time, what sort of practices could we use to maintain in-person social ties, indefinitely, without risk?" (i.e. going on long walks, visiting each other's house where 1-2 people hang out in the street or sidewalk and house denizens hang out on the porch, etc)

I'm guessing this has separate answers for "outdoor" and "indoor."

Current Answers:

Indoors: Basically it's not safe.
Outdoors: 10 meters (30 feet) seems safe, if nobody is upwind/downwind of each other. But I'm unclear what the falloff range is.
March 24th: Daily Coronavirus Link Updates
by habryka — 2020-03-26T02:22:35.214Z

As part of the LessWrong Coronavirus Link Database, Ben, Elizabeth and I are publishing update posts with all the new links we are adding each day that we ranked a 3 or above in our importance rankings. Here are all the top links that we added yesterday (March 24th), by topic.

You can find the full database here:

https://www.lesswrong.com/coronavirus-link-database

Dashboards

Dashboard with estimates and predictions of true prevalence

A dashboard that gives you estimates for current prevalence by country, as well as predictions for the future based on varying amounts of mitigation.

Economics

What will the economic effects of quarantine be?

LW attempts to predict what the effects of a short or long quarantine will be

Medical System

Flexport CEO explains why scaling PPE is hard

Outlines the difficulties in scaling, including QA and legal issues

Other

Stories of C19 layoffs

Reddit thread of people who lost their jobs due to coronavirus or quarantine

Spread & Prevention

Review of mask efficacy

They're useful but the gains may be overwhelmed by any risk compensation, and they need to be saved for medics

(EV) He left out a swath of studies on mask use in mass gathering

LW Advice Summary

A summary of the best suggestions from the justified practical advice thread

Work & Donate

LessWrong C19 Agenda

A list of questions we want answered to inform future decisions, and assembly of answers as they're created

Full Database Link
Price Gouging and Speculative Costs
by jefftk — 2020-03-26T01:50:02.432Z

Let's say you see a potential pandemic coming, and you produce a
product that could be critical. Maybe you make respirator masks,
maybe you make ventilators, maybe you make PCR test reagents. You can
see that if you and your competitors don't ramp up production and the
pandemic happens, there will be a shortage. What do you do?

One option is to do nothing: keep producing at your regular rate. If
the pandemic fears were overblown then you're fine. If the pandemic
happens you quickly sell out, and start scrambling to ramp up
production.

Another option is to ramp up production now, speculatively. Start
paying workers extra to work longer shifts and run your assembly lines
around the clock. Train extra workers. Find what you're bottlenecked
on and figure out how to get that ramped up too. If the pandemic
fears were overblown you lose a lot of money, but if the pandemic
happens people need what you have so much that you can charge high
prices. How much to ramp up production in advance depends on how
likely you think the pandemic is, and how much you'd be able to
increase prices if it does happen.

Except we have laws and customs against price gouging: if the
pandemic does happen, you are going to have a lot of trouble raising
your prices. The laws generally do allow passing
along increased costs, but the problem here is that your costs
were speculative. Let's work an example.

Imagine your ventilators normally cost $35k to make, and the market
price is $40k each. If you push really hard to ramp up production you
can make a lot more, but your cost goes up to $90k/each. In New
Jersey, the state I looked at last time, you're
allowed to pass along costs but can't increase your profit on
"merchandise which is consumed or used as a direct result of an
emergency or which is consumed or used to preserve, protect, or
sustain the life, health, safety or comfort of persons or their
property" by more than 10% (56:8-107,
108,
109).
Your normal profit is 14%, so you would be allowed to sell them for $104k. Your possibilities are:

No pandemic: you spent $90k each, but the market
price is $40k. You lose $50k each.

Yes pandemic: you spent $90k each, and the market price is way
above that, but you can legally only charge $104k. You make $14k
each.

If you think the pandemic is >78% likely to happen then you'll
expect to make money by ...
AGI in a vulnerable world
by AI Impacts — 2020-03-26T00:10:01.942Z

I’ve been thinking about a class of AI-takeoff scenarios where a very large number of people can build dangerous, unsafe AGI before anyone can build safe AGI. This seems particularly likely if:

It is considerably more difficult to build safe AGI than it is to build unsafe AGI.
AI progress is software-constrained rather than compute-constrained.
Compute available to individuals grows quickly and unsafe AGI turns out to be more of a straightforward extension of existing techniques than safe AGI is.
Organizations are bad at keeping software secret for a long time, i.e. it’s hard to get a considerable lead in developing anything.

This may be because information security is bad, or because actors are willing to go to extreme measures (e.g. extortion) to get information out of researchers.

Another related scenario is one where safe AGI is built first, but isn’t defensively advantaged enough to protect against harms by unsafe AGI created soon afterward.

The intuition behind this class of scenarios comes from an extrapolation of what machine learning progress looks like now. It seems like large organizations make the majority of progress on the frontier, but smaller teams are close behind and able to reproduce impressive results with dramatically fewer resources. I don’t think the large organizations making AI progress are (currently) well-equipped to keep software secret if motivated and well-resourced actors put effort into acquiring it. There are strong openness norms in the ML community as a whole, which means knowledge spreads quickly. I worry that there are strong incentives for progress to continue to be very open, since decreased openness can hamper an organization’s ability to recruit talent. If compute available to individuals increases a lot, and building unsafe AGI is much easier than building safe AGI, we could suddenly find ourselves in a vulnerable world.

I’m not sure if this is a meaningfully distinct or underemphasized class of scenarios within the AI risk space. My intuition is that there is more attention on incentives failures within a small number of actors, e.g. via arms races. I’m curious for feedback about whether many-people-can-build-AGI is a class of scenarios we should take seriously and if so, what things society could do to make them less likely, e.g. invest in high-effort info-security and secrecy work. AGI development seems much more ...
ODE to Joy: Insights from 'A First Course in Ordinary Differential Equations'
by TurnTrout — 2020-03-25T20:03:39.590Z

Foreword

Sometimes, it's easier to say how things change than to say how things are.

From 3Blue1Brown: Differential Equations

When you write down a differential equation, you're specifying constraints and information about e.g. how to model something in the world. This gives you a family of solutions, from which you can pick out any function you like, depending on details of the problem at hand.

Today, I finished the bulk of Logan's A First Course in Ordinary Differential Equations, which is easily the best ODE book I came across.

A First Course in Ordinary Differential Equations

As usual, I'll just talk about random cool things from the book.

Bee Movie

In the summer of 2018 at a MIRI-CHAI intern workshop, I witnessed a fascinating debate: what mathematical function represents the movie time elapsed in videos like The Entire Bee Movie but every time it says bee it speeds up by 15%? That is, what mapping t↦x(t) converts the viewer timestamp to the movie timestamp for this video?

I don't remember their conclusion, but it's simple enough to answer. Suppose f(t) counts how many times a character has said the word "bee" by timestamp t in the movie. Since the viewing speed itself increases exponentially with f, we have x′(t)=1.15f(x(t)). Furthermore, since the video starts at the beginning of the movie, we have the initial condition x(0)=0.

This problem cannot be cleanly solved analytically (because f is discontinuous and obviously lacking a clean closed form), but is expressed by a beautiful and simple differential equation.

Gears-level models?

Differential equations help us explain and model phenomena, often giving us insight into causal factors: for a trivial example, a population might grow more quickly because that population is larger.

Equilibria and stability theory

This material gave me a great conceptual framework for thinking about stability. Here are some good handles:

Let's think about rocks and hills. Unstable equilibria have the rock rolling away forever lost, no matter how lightly the rock is nudged, while locally stable equilibria have some level of tolerance within which they'll settle back down. For a globally stable equilibrium, no matter how hard the perturbation, the rock comes rolling back down the parabola.

Resonance

A familiar example is a playground swing, which acts as a pendulum. Pushing a person in a swing in time with the natural interv

...
Tales From the Borderlands
by Jacob Falkovich — 2020-03-25T19:11:48.373Z

Cross-posted as always from Putanumonit.com

Dear United States Customs and Border Protection, I was honored to receive your invitation to apply to the Border Patrol.

Not at first: at first I was bemused, and joked about this being the most mistargeted targeted ad in the history of the internet. After all, I represent a dramatic failure of your agency: an immigrant from the Middle East who infiltrated your nation, took your jobs, seduced your women, and

undermined your democracy

. But perhaps that is your strategy — if I could beat you, let me join you.

But then I remembered: half a lifetime ago I did wear a uniform that gave me a purpose. I spent a good deal in that uniform patrolling a border. Perhaps I went beyond. You must be familiar with that time since your ad found me, but allow me to share some details anyway. A young man sees and learns many things while on guard duty.

My country is small, shaped like a long chef’s knife, and surrounded by hostile neighbors. It has a very high ratio of miles-of-border-that-need-to-be-watched to teenagers. So on any given day many of its teenagers are employed by watching its borders. Demand creates its own supply, as they say.

I just turned 18 when they gave me a rifle and two sets of uniforms. One crisp and clean that I would almost never wear, one grimy and rough and suitable for the work of infantry boot camp: running, push-ups, shooting, washing toilets, catching naps, guard duty.

Guard duty is no one’s main job in the army, but almost everyone has to do a lot of it throughout their service. It has a lot in common with Vipassana meditation — a quiet, solitary affair whose ultimate goal is “seeing”. One’s practice starts in boot camp doing 30 minute stints, then one graduates to practicing for 4 or 8 hours in a row and going on retreats.

I wish I knew about Vipassana when I enlisted, I surely would have been enlightened twice over for all the hours I spent standing guard. Instead, I mostly entertained myself by singing Russian pop songs in my head. I learned the lyrics to hundreds of those, along with the entire Marshall Mathers LP and an eclectic selection of French hip hop and German metal. It has always been my gift and curse: I have a perfect memory for song lyrics in any language, but I can’t carry a tune to save my life. I sing often to myself, but at karaoke I’m resigned to cringing as my vocally talented friends ...
[AN #92]: Learning good representations with contrastive predictive coding
by Rohin Shah — 2020-03-25T17:20:02.043Z

Newsletter #92

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

HIGHLIGHTS

Representation Learning with Contrastive Predictive Coding (Aaron van den Oord et al) (summarized by Rohin): This paper from 2018 proposed Contrastive Predictive Coding (CPC): a method of unsupervised learning that has been quite successful. At its core it is quite simple: it simply combines the ideas of predictive coding and contrastive losses, both of which have been significantly studied in the past.

The simplest form of unsupervised learning would be data compression via generative models (as in e.g. VAEs), in which, to model the data p(x), you attempt to encode x into a latent (hidden) state z in such a way that you can then recover the original data point x from z. Intuitively, we want z to have high mutual information with x.

For sequential data in a partially observed setting, you need to deal with the full sequence. Consider natural language: in this setting, each x would be a single word. Consider the sentence "I sat on the chair". If the z corresponding to the word "the" only has to reconstruct the word "the", it's not going to "remember" that the past context involved sitting, and so that z would be terrible at predicting that the next word will be chair. To fix this, we can use predictive coding, where we instead require that we can predict future words using z. This now incentivizes z_t to have high mutual information with x_{t+k}.

There is still a problem: reconstructing the entire input x would require a lot of irrelevant information, such as e.g. the background color of the environment in RL, even if that never changes. How can we get rid of these irrelevant features? Contrastive losses allow us to do this: intuitively, since the irrelevant features are the ones that are common across all the xs (and so are fully captured by p(x) ), if we train the neural net to distinguish between various xs, we can incentivize only the relevant features. In particular, given a latent state z_t, we take the true x_{t+k}, and throw in a bunch of other xs sampled from p(x) (known as negative ...
True Habits
by MartinB — 2020-03-25T14:00:01.669Z

Photo by My Name on Unsplash

Habits have been extensively written about in pop-literature for the last couple of years. Many claims have been made!

Want to get an awesome body? Habitify it!

Want to become great at your studies? Place gummy-bears on each page to train your pesky mind.

Want to get rid of stress? Sit in the same spot and breathe through your nose for 5 minutes, and to ensure you keep doing it, track it in a spreadsheet.

But how well does this fit with the habit literature? And why don’t more people train themselves with gummy-bears? Let’s find out!

A note on “why now”

Many of us are in unfamiliar contexts, being forced to work from home to reduce contagion.

This presents to us both a challenge and an opportunity.

We’re forced to work in unfamiliar contexts and, if you’re anything like me, this means that your work time is now being intruded upon by a bunch of thoughts relating to habits that are great when you relax – but gaming isn’t going to get your thesis done.

Luckily, habits are more easily changed when contexts change. If you want to get new, appropriate habits at home, or get rid of something you’ve been doing, the time is now!

(A short side-note: This will not be a review of all habit-literature, but rather small dives into results I’ve encountered that were interesting and/or ran counter to what I expected).

What defines a true habit?

If you’re like me, you’re slightly provoked by the title. “True habits” you say? What are “false” habits?

I used to think that doing the same task repetitively would be sufficient to form habits, or was in fact, the entire definition of a habit.

To question that assumption, we must first ask ourselves, “why do we prefer habits to to-do lists?”. If we can just do what we want, when we want to, without getting tired, why bother with this habit business at all?

The answer lies in automaticity. We want to be able to perform our morning routine without thinking too much about it, while tired, stressed and under time-pressure.

It turns out that some actions that change behavior are detrimental to forming automaticity. This is (obviously) really important, and runs counter to much of what’s written in the pop-psych litterature.

But first, what is automaticity?

In the habit literature, it’s operationalised as a high score on the Self-Report Behavioural Automaticity Index (SRBAI) (Gardner 2012). Once again, sc...
3 Interview-like Algorithm Questions for Programmers
by lsusr — 2020-03-25T09:05:10.875Z

These are my favorite interview-style algorithm questions for programmers. I like these questions because they test how well you understand information density instead of your ability to regurgitate facts.

Comments to help clarify puzzle details such as how the Enigma machine works are allowed. Otherwise, comments with hints, answers and spoilers will be deleted, even if they use special formatting to hide spoilers. PM me if you are unsure. Edit: I can't actually do this with less than 2000 karma. Please self-moderate responsibly.

1. Sorting

What is the fastest you can sort a list of ints in Java?

Answer: O(n) time where n is the length of the list

2. Search

You have 1000 saliva samples. Exactly 1 of them is from someone infected by COVID-19. You have unlimited testing strips. Each day the CDC has resources to process up to 10 testing strips for you. You must send all 10 testing strips to the CDC at the same time. The next day, the CDC will tell you which of the 10 strips has been exposed to infected saliva. What is the fewest number of days required to figure out which sample is infected?

Answer: 1 day

3. Cryptography

It is the year 1932. You work for the Polish government. They have captured a German Enigma machine. The Enigma machine has three rotating rubber rotors (disks). Each rotor has 26 copper wires feeding from one side to the other. At the back of the machine is a reflector, reflecting 13 of the contact points on the back of rotor 3 to the 13 other contact points on the back of rotor 3. In this way, the current gets scrambled forward through rotors 1, 2, and 3 and then scrambled backwards through rotors 3, 2 and 1. Each time someone presses a key, the first rotor rotates forward one letter. Every full rotation of the first rotor, the second rotor rotates one letter. Every full rotation of the second rotor, the third rotor rotates one letter.

Each letter of the alphabet is thus mapped to another letter of the alphabet in a symmetric cipher. This cipher changes each keystroke. You can configure the Enigma machine by setting the dials with a 3-letter code. The Germans change this code once per day. Today the daily code might be SEC. Tomorrow it could be UNK. The Germans also create a unique 3-letter code separately for each message. The daily code is used to transmit the message code twice.

For example, if the daily code is SEC and the message code

...