trentmkelly/LessWrong-43k huggingface dataset:
Pasteur's quadrant
by jasoncrawford — 2020-06-05T17:47:58.569Z
Link: https://rootsofprogress.org/pasteurs-quadrant

In my recent post on the case study of the transistor, we saw that the research that led to its invention did not fall neatly into the categories of “basic” vs. “applied”, but in fact cycled rapidly between them.

An entire book—Pasteur’s Quadrant, by Donald Stokes—is dedicated to the thesis that “basic” vs. “applied” is a false dichotomy that is harming science funding.

The core idea of Pasteur’s Quadrant is that basic and applied research are not opposed, but orthogonal. Instead of a one-dimensional spectrum, with motion towards “basic” taking you further away from “applied”, and vice versa, he proposes a two-dimensional classification, with one axis being “inspired by the quest for fundamental understanding” and the other being “inspired by considerations of use”:

I find these phrases somewhat cumbersome, given their centrality to the thesis, so let’s call them “discovery” and “invention” for short.

In Stokes’s classification, research aiming at discovery, with no thought of invention—“pure basic research”—is characterized by Bohr’s search for a model to explain the atom, and hence labeled “Bohr’s quadrant”. The opposite, “pure applied research”, or invention with no attempt at discovery, is characterized by Edison and his lab, who tinkered endlessly to engineer products such as the light bulb, with little heed to theory: “Edison’s quadrant”. But, crucially for Stokes, there is research that combines discovery and invention, motivated by both at the same time. “Pasteur’s quadrant” is characterized by the career of the great scientist who gave us both the germ theory of disease and the first engineered vaccines.

Stokes traces the history of the basic-applied dichotomy back to ancient Greece, where they had different words for different types of knowledge: episteme for scientific knowledge, and techne for the practical arts. Long before the Baconian program, episteme was not seen as the basis of techne—nor was it, at the time. Mathematics, astronomy, and natural philosophy were ends in themselves, pursued for the pleasure of knowing, not with practical goals in mind. ...
Emergence and Control: An examination of our ability to govern the behavior of intelligent systems
by meanderingmoose — 2020-06-05T17:10:08.295Z

As researchers and philosophers discuss the path towards human-equivalent / superhuman general artificial intelligence, they frequently examine the concept of control. Control of our world has always been of central importance to our species, so it’s no surprise that we’d look to extend our control in the future. However, most of the currently discussed control methods miss a crucial point about intelligence – specifically the fact that it is a fluid, emergent property, which does not lend itself to control in the ways we’re used to. These methods treat the problem as an algorithmic one, directly addressed with specific instructions that can be encoded to ensure proper behavior (harkening back to Asimov’s Three Laws of Robotics) – but the AI of tomorrow will not behave (or be controlled) like the computers of today. Emergent intelligence can’t be boxed up directly at an algorithmic level because this level is fluid, with constant shifts and changes that give the instructions no base to latch onto; the best we can hope for is to control it through a system of indirect rewards and punishments (much as we control human behavior today).

Before diving into the concepts of AI and control, it may be helpful to take a step back and align on the concept of emergence. An emergent property of an arrangement of objects is a property which could not be predicted based on the attributes of the objects themselves. This definition is a bit dense – so as an example, think of a river with a rock in the middle of it. For a common observer, with standard concepts of “river” and “rock”, the expectation would be that the water flows cleanly around the rock. However, upon further examination, they would find that the river + rock system results in whirlpools forming behind the rock as the water flows around it. In this example, the whirlpool is an emergent property of the river + rock system. This example highlights another key facet of emergence – it is dependent on the observer. In the situation just reviewed, the whirlpools being an emergent property depends on the observers limited concepts of “river” and “rock”; an experienced kayaker would expect the system to generate whirlpools (although they would lack predictive power of the type / size of whirlpool forme...
Reply to Paul Christiano on Inaccessible Information
by Alex Flint — 2020-06-05T09:10:07.997Z

In Inaccessible Information, Paul Christiano lays out a fundamental challenge in training machine learning systems to give us insight into parts of the world that we cannot directly verify. The core problem he lays out is as follows.

Suppose we lived in a world that had invented machine learning but not Newtonian mechanics. And suppose we trained some machine learning model to predict the motion of the planets across the sky -- we could do this by observing the position of the planets over, say, a few hundred days, and using this as training data for, say, a recurrent neural network. And suppose further that this worked and our training process yielded a model that output highly accurate predictions many days into the future. If all we wanted was to know the position of the planets in the sky then -- good news -- we’re done. But we might hope to use our model to gain some direct insight into the nature of the motion of the planets (i.e. the laws of gravity, although we wouldn’t know that this is what we were looking for).

Presumably the machine learning model has in some sense discovered Newtonian mechanics using the training data we fed it, since this is surely the most compact way to predict the position of the planets far into the future. But we certainly can’t just read off the laws of Newtonian mechanics by looking at the millions or billions or trillions of weights in the trained model. How might we extract insight into the nature of the motion of the planets from this model?

Well we might train a model to output both predictions about the position of the planets in the sky and a natural language description of what’s really going on behind the scenes (i.e. the laws of gravity). We’re assuming that we have enough training data that the training process was already able to derive these laws, so it’s not unreasonable to train a model that also outputs such legible descriptions. But in order to train a model that outputs such legible descriptions we need to generate a reward signal that incentivizes the right kind of legible descriptions. And herein lies the core of the problem: in this hypothesized world we do not know the true laws of Newtonian mechanics, so we cannot generate a reward signal by comparing the output of our model to ground truth during training. We might instead generate a reward signal that (1) measures how accurate the predictions of the ...
Three characteristics: impermanence
by Kaj_Sotala — 2020-06-05T07:48:02.098Z

This is the sixth post of the "a non-mystical explanation of the three characteristics of existence" series.

Impermanence

Like no-self and unsatisfactoriness, impermanence seems like a label for a broad cluster of related phenomena. A one-sentence description of it, phrased in experiential terms, would be that “

All experienced phenomena, whether physical or mental, inner or outer, are impermanent

”.

As an intellectual claim, this does not sound too surprising: few people would seriously think that either physical things or mental experiences last forever. However, there are ways in which impermanence does contradict our intuitive assumptions.

A conventional example of this is

change blindness

. In a typical change blindness experiment, people report having good awareness of the details of a picture shown to them: but when details are changed during an eye saccade, subjects fail to notice any difference. Maybe a person’s hat looks red, and people who have been looking right at the hat fail to notice that it looked green just a second ago: the consciousness of the green-ness has vanished, replaced entirely with red.

People are typically surprised by this, thinking that “if it was red a second ago, surely I would remember that” - a thought that implicitly assumes that sense percepts leave permanent memories behind. But as long something does not explicitly store a piece of conscious information, it is gone as soon as it has been experienced.

This is a natural consequence of the Global Neuronal Workspace (GNW) model of consciousness from neuroscience. As I

have previously discussed

, studies suggest that the content of consciousness corresponds to information held in a particular network of neurons called the "global workspace". This workspace can only hold a single piece of conscious content at a time, and new information is constantly trying to enter it, replacing the old information.

Now if the content of your consciousness happens to be something like this:

0 milliseconds: Seeing a red hat
53 milliseconds: Thinking about cookies
200 milliseconds: Seeing a green hat

Then at the 200 millisecond mark, unless some memory system happened to explicitly store the fact of seeing a red hat before, no trace of it remains in consciousness for the person to compare with. One can train particular subsystems to monitor the cont...
Does taking extreme measures to avoid the coronavirus make sense when you factor in the possibility of a really long life?
by Adam Zerner — 2020-06-05T00:58:49.775Z

Some back of the napkin math. Suppose we:

Value a QUALY at $50k.
Use an expected lifespan of 10k years. Perhaps you expect a 1% chance of living 1M years due to the possibility of a friendly superintelligence or something.

That would mean:

The value of your life would be $500M.
A 1% chance of death would cost $5M.
A 100x smaller chance of death of 0.01% would cost $50k.
Decreasing your chance of death 100x would be worth ~$5M.

There seem to be various ways to decrease your chance of death from the coronavirus by 100x or more by going from "normal careful" to extremely careful.
Thinking About Super-Human AI: An Examination of Likely Paths and Ultimate Constitution
by meanderingmoose — 2020-06-04T23:22:34.430Z

In this post, we'll examine how we might get to artificial general intelligence, and will put some thought into what this type of “superintelligence” might look like. We’ll start out by taking a step back, and looking at the most likely potential pathways by which to achieve this goal – as the behaviors exhibited by any “superintelligent” agent will have significant pathway dependence. We’ll then explore the necessary attributes for any intelligent system – and the unnecessary elements that we’re used to seeing in humans (and other species) but that may not appear in an artificial intelligence. Once we’ve laid this groundwork for thinking about artificially intelligent systems / agents, we’ll take a look at how to think about the goals these systems would have, and will consider how they might compare to our own human goals. As we’re currently still quite far from constructing any sort of generally intelligent machine, intuitions formed from the observation of existing AI systems may serve as poor guides (intuitions from observing human agents will be far more applicable) – this post looks to help guide you to new intuitions, but that process will be aided by the temporary setting aside of existing views based on the significant limitations of artificial systems as they exist today.

Crafting an artificial intelligence of any sort will be extremely difficult – seemingly magnitudes more difficult than anything accomplished in the course of human history. As covered in another post on my blog, Defining Intelligence, there’s significant difficulty associated with just understanding what intelligence is – let alone building it ourselves. Our species is designed to be good at top-down architecture, with limited dependencies between the different parts of the system (think about building a house – each brick has an easily defined role to play, and plays only that role); intelligence, on the other hand, is a bottom-up phenomena, arising from interactions between all parts of the system (the stock market is another good example of a bottom-up type phenomena). Our difficulties with bottom-up phenomena are exemplified by the enormous amount of work it has taken to understand how the C. Elegans nervous system drives behaviors – only 302 neuro...
How to validate research ideas?
by kwiat.dev — 2020-06-04T21:37:44.150Z

Let's say you have an idea that you think might be interesting to investigate, possibly a new aspect in AI safety, maybe some new algorithm.

If you're an experienced researcher, you probably have plenty of intuition to think through it, consider the possible outcomes and decide whether it's worth investigating.

If you have a decent academic network, you can probably bring it up even in casual conversations with people who are as good or better than you in the field to get a sense of their intuitions.

What if you have none of those things? Is there an online forum for such discussions, just like I'd go to reddit when looking for e.g. game recommendations? Maybe *this* is the forum?

And are there any good recommendations how to try and work it out by yourself, without referring to the wisdom of online crowds?
FHI paper on COVID-19 government countermeasures
by Bird Concept — 2020-06-04T21:06:51.287Z
Link:  https://doi.org/10.1101/2020.05.28.20116129

In brief: this is the largest data-driven study trying to disentangle the effect of individual countermeasures, and one of the most thoroughly validated ones. The authors ran multiple predictions with heldout data and various sensitivity analyses. The results seem robust. It's still unclear how generalisable they are, since it's essentially an observational study. (I managed the project who wrote the paper, but I wasn't involved as a co-author.)

The effectiveness and perceived burden of nonpharmaceutical interventions against COVID-19 transmission: a modelling study with 41 countries

Background: Existing analyses of nonpharmaceutical interventions (NPIs) against COVID19 transmission have concentrated on the joint effectiveness of large-scale NPIs. With increasing data, we can move beyond estimating joint effects towards disentangling individual effects. In addition to effectiveness, policy decisions ought to account for the burden placed by different NPIs on the population.

Methods: To our knowledge, this is the largest data-driven study of NPI effectiveness to date. We collected chronological data on 9 NPIs in 41 countries between January and April 2020, using extensive fact-checking to ensure high data quality. We infer NPI effectiveness with a novel semi-mechanistic Bayesian hierarchical model, modelling both confirmed cases and deaths to increase the signal from which NPI effects can be inferred. Finally, we study how much perceived burden different NPIs impose on the population with an online survey of preferences using the MaxDiff method.

Results: Eight NPIs have a >95% posterior probability of being effective: closing schools (mean reduction in R: 50%; 95% credible interval: 39%-59%), closing nonessential businesses (34%; 16%-49%), closing high-risk businesses (26%; 8%-42%), and limiting gatherings to 10 people or less (28%; 8%-45%), to 100 people or less (17%; -3%-35%), to 1000 people or less (16%; -2%-31%), issuing stay-at-home orders (14%; -2%-29%), and testing patients with respiratory symptoms (13%; -1%-26%). As validation is crucial for NPI models, we performed 15 sensitivity analyses and evaluated predictions on unseen data, finding strong support for our results. We combine the effectiveness and preference results to estimate effectiveness-to-burden ratios.

Conclusions: Our results suggest a surprisingly large role for schools

...
Covid-19 6/4: The Eye of the Storm
by Zvi — 2020-06-04T20:50:01.199Z

Still standing by this: Covid-19: My Current Model

Previous update posts: Covid-19 5/29: Dumb Reopening, Covid 5/14: Limbo Under

Remember last week when I opened with this?

I remember when people on Twitter had constant reminders that today was, indeed, only Wednesday, or whatever day it happened to be. Time moved that slowly.

Time has sped up again.

Well, yeah. Not so much anymore.

In March and April I found myself constantly checking Twitter and the financial markets for news, frantically hunting for ways to get a handle on what was happening in the world, worried everything would fall apart. Would our supply chains hold? Would we be able to maintain civil order? Would millions die? How can I keep my family and friends safe?

My beloved New York City was no longer a place one could live a reasonable life. So we fled. Even after that, great worry.

Then things started to calm down. Most of May, I increasingly managed to relax. We learned how to grill properly. I played a bunch of Assassin’s Creed Odyssey. I stopped checking Twitter or the stock market.

My biggest worries were dealing with getting the Emergents Alpha ready and Magic: The Gathering melting down under the weight of the companion mechanic in particular and its new design philosophy in general.

End of the Beginning

Then, on May 25, 2020, in Minneapolis, four policemen murdered George Floyd.

The resulting cycle of protests against police brutality causing more police brutality, which in turn amplifies the protests, has been ongoing each day since then. Huge crowds flood the streets. The crowds come disproportionately from communities with more Covid-19 cases than average. The crowds often yell. The police use tear gas on them, which causes coughing. The police arrest them and lock them in tight quarters, which is going to spread the virus even if the outdoor activities don’t, hopefully in small enough numbers this isn’t a huge deal.

To the extent other things are also happening, that only makes the infection situation worse.

It would be very surprising if all this did not lead directly to a surge of new Covid-19 infections.

It would also be very surprising if all this did not lead indirectly to a much bigger surge of new Covid-19 infections.

Effectively, the lock down is over.

Nuance is hard in the best of circumstances. The community I know who got most ahead of the game on Covid-19, the Rationalist community,...
Visual Babble and Prune
by DirectedEvolution — 2020-06-04T18:49:30.044Z

How do human beings produce knowledge? When we describe rational thought processes, we tend to think of them as essentially deterministic, deliberate, and algorithmic. After some self-examination, however, I've come to think that my process is closer to babbling many random strings and later filtering by a heuristic.

-alkjash

Introduction

I want to describe my first and so far only experience inventing a math problem. It's probably not an original or important problem. But I think there is value in describing the origins of ideas, especially when those ideas are actionable.

Here is the problem:

Draw a square bounding box of side length N. One at a time, drop a square of side length L, oriented identically to the bounding box, at a random location such that it is entirely enclosed within the bounding box, until two of the dropped squares overlap. What is the relationship between the average total area of the dropped squares (excluding the last one dropped), and L^2/N^2?

I did some analysis on this problem. As L increases, each square fills more area (increasing L^2/N^2), but the chance of an overlap increases as well (decreasing L^2/N^2). Until L^2/N^/2 = 0.25, the relationship closely follows a power law. After that point, the relationship is linear, as it becomes impossible to fit more than one square in the bounding box, meaning that the only remaining factor is the increasing size of the single dropped square.

It would be interesting to work out why this relationship holds, but my primary goal here is to describe how I formed this problem.

Circumstances

I am studying math, CS, and chemistry at the undergraduate level (as a post-bacc student). I was not passionate about mathematics until a couple years ago, and do not consider myself an unusually talented math student compared to my classmates.

I came up with this problem while smoking weed and listening to my girlfriend play the violin on the porch. I often notice that weed can make me ask more expansive or deep questions, follow a train of thought with more focus, and allow me to explore a vague idea that I might otherwise dismiss.

It also dramatically enhances my ability to visualize. I have a poor sensory imagination. But when I smoke weed, I can see and manipulate mental pictures and audio with much greater vividness and sustained attention.

Developing the Idea

Babble

In this case, I started imagining k...
Pantry Staples for DIY
by jefftk — 2020-06-04T18:10:02.955Z

It's often helpful to be able to build or fix things without needing
to go to the store: perhaps it's urgent, perhaps you don't have a car,
perhaps there's a pandemic on, perhaps you have an idea that you want
to try out before your interest fades. In cooking, the foods that you
keep on hand are often called "pantry staples"; what are the
equivalent consumables worth stocking for DIY projects?

For me, the key "ingredients" I want to have on hand are:

2x4s: 1½" by 3½" by generally 8', very strong
though not infinitely so, lots of wood to sink a fastener into. Can
be doubled up if you need to make a post. It can be worth getting
some 2x3s as well (cheaper; useful when a 2x4 is more than you need)
and 2x6s (stronger; can span a wider distance; might happen to be a
useful thickness). I wouldn't stock 2x8s or bigger since if I need
those I'm probably working on a big enough project that I'm getting
more materials anyway.

¾" plywood: comes in 4x8 sheets, but these are
unwieldy and too big to fit down my basement stairs so I usually have
the sheets cut into three 16" or two 24" strips on the store's panel
saw first. I usually get AB (one nice side, one decent side) softwood
since it's reasonably cheap, easy to work with, and attractive enough
when sanded that I don't mind it indoors. Having some ¼" plywood
can be useful for lighter things and spacing adjustments.

screws: you need some way to attach these together, and
screws are a good default. Traditionally the standard
fastener would be nails, but I find screws are a better fit for small
projects. I like that there's no hammer blow to knock things out of
alignment or need to be braced against. Historically people have
preferred nails because of higher shear strength, but screws are
actually very
good here. Star-drive (torx) screws are a lot more pleasant than
Phillips (plus-shaped) because the bit stays on the screw more
easily. You want at least 3" (attaching two 2x4s), 2¼" (attaching
¾" plywood to the flat side of a 2x4), and 1½" (attaching ¾"
plywood to more ¾" plywood) though I've also found ½", ¾", 1
¼", and 3½" sizes useful enough to keep on hand.

glue: wood glue, or you can just use white glue. Before
screwing two pieces of wood together consider putting in a bit of
glue. You'll get a much stronger joint, and the screws provide
clamping with no extra effort. Since the screws

...
Why isn’t assassination/sabotage more common? 
by Rudi C — 2020-06-04T18:04:40.509Z

People can short stocks of companies and then sabotage or assassinate important people to make the stocks drop.
Political goals seem ripe for assassination. For example, the US recently killed Iran’s general Soleimani. Couldn’t they do this without taking responsibility (i.e. with plausible deniability)? If so, aren’t further assassinations beneficial?
Political goals also seem ripe for sabotage. Currently, sometimes nations do this with cyberattacks. But I haven’t heard of physical sabotage attempts. For example, poisoning the water supply of enemy cities, burning their government buildings, cutting their electric cables, etc.