trentmkelly/LessWrong-43k huggingface dataset:
What is the appropriate way to communicate that we are experiencing a pandemic?
by Filipe Marchesini — 2020-03-04T00:02:01.879Z

After spending 3 days reading all of Lesswrong's posts and comments (and sources cited) related to coronavirus, I started taking action (summarizing the information collected and planning the least harmful ways to alert my closest peers to start preparing - we only have two confirmed cases still in Brazil). In my entire network of friends, I have not yet seen a single person who even takes the matter seriously. When I show a small amount of data (2~5 minutes of reading) and its extrapolations for the next weeks/months, the natural reaction is "you are forgetting a lot, someone will invent a cure/vaccine, there is nothing to worry about". They don't care about the smoke.
However, by showing a reasonable amount of data (1 hour reading) about coronavirus around the planet to my girlfriend and its long-term consequences (up to a year), she was convinced that it was a pandemic, that the consequences would be very bad, and her reaction was to cry. I showed an amount of data equivalent to 30 minute reading for my closest friend, his reaction was "tell me what you gonna do and when you gonna do, I'll do the same"
As there are a reasonable amount of people who don't think the subject is relevant or even know anything about it, I'm creating a video for YouTube in Portuguese. But then I thought the psychological consequences for a not inconsiderable amount of people would be disastrous, as it seems to my girlfriend. However, I see a lot of valuable information that I can only find on Lesswrong and no other source in Portuguese and it seems it would be a mistake not to share. At the same time, I don't want to be a information hazard source.

On one hand, I can simply create the video, translate it to English and share with Lesswrong community to see if the community approves.

On the other hand, you could say what you think it would be an adequate sequence to explain the subject, like: [what is coronavirus > why you should pay more than 1 minute attention to it > current number of deaths vs expected number of deaths > who is probably going to die > you have permission to be worried and take precautions > what are the precautions > what data should trigger behavior changing for more aggressive precautions > etc]
Robustness to fundamental uncertainty in AGI alignment
by Gordon Seidoh Worley — 2020-03-03T23:35:30.283Z
Link: https://arxiv.org/abs/1807.09836

Publication cycles are long, so I've talked about some of the ideas in this paper before, but since it's freshly out in print, I thought I'd take the chance to share it and summarize its contents for those who don't want to slog through an entire academic paper. I'll aim for a more friendly, informal, and relatively brief approach in this post as an alternative to the academic, formal, and more detailed approach you'll find in the paper. Take your pick of which style you prefer, or enjoy both.

Summary

The distribution of risks from AGI suggest we can reduce the risk of catastrophe in attempts to build aligned AGI by trading off false positives for false negatives.

In more words, since we operate under uncertainty about how to build aligned AGI, assuming we are already along the Pareto frontier of interventions that are most likely to work, we are safer if we avoid trying some things that counterfactually would have worked had we tried them in exchange for trying fewer things that would fail to produce aligned AGI since those failed attempts put us at risk of unleashing unaligned or otherwise unsafe AGI and losing more that we value than we would have lost from wasted opportunity to develop aligned AGI sooner.

Stated in less formal terms, my thesis is that it's safer to be "conservative" in choices of assumptions when designing alignment schemes where "conservative" is specifically conservation of false positives.

To demonstrate this, I work through two examples of making choices about necessary assumptions to designing alignment schemes, and I know that these are necessary assumptions that lie along the frontier because they are epistemologically fundamental choices. I show what reasoning about tradeoffs between false positives and false negatives in alignment looks like, and present two recommended tradeoffs, albeit without considering how those tradeoffs might be operationalized when designing alignment mechanisms.

Reducing false positive risk

The risks of AGI appear to be so large such that even if you believe a catastrophic outcome from AGI is very unlikely, that catastrophe would be so bad that its expected value is still massively negative, and such negative outcomes represent a far larger loss of value than failures-of-opportunity to obtain the best positive outcomes. If you disagree with this premise th...
What are sensible ways to screen event participants to reduce COVID-19 risk?
by ChristianKl — 2020-03-03T20:43:39.280Z

This Sunday I'm co-hosting a SlateStarCodex meetup in Berlin. There currently seem to be a handful of cases in Berlin, so I'm wanting to take reasonable precautions.

I don't think canceling the event is reasonable at this point in time. At the same time I do want to have a policy of keeping potentially ill people away.

I have ordered a contactless thermometer. Most were sold out at Amazon.de, so if you want to hold any events in the coming months, now might be the last time to order. I also got normal mouth thermometers.

But I don't know about what values would be good cutoffs for fever testing.
What does Solomonoff induction say about brain duplication/consciousness?
by riceissa — 2020-03-02T23:07:28.604Z

Back in 2012, in a thread on LW, Carl Shulman wrote a couple of comments connecting Solomonoff induction to brain duplication, epiphenomenalism, functionalism, David Chalmers's "psychophysical laws", and other ideas in consciousness.

The first comment says:

It seems that you get similar questions as a natural outgrowth of simple computational models of thought. E.g. if one performs Solomonoff induction on the stream of camera inputs to a robot, what kind of short programs will dominate the probability distribution over the next input? Not just programs that simulate the physics of our universe: one would also need additional code to "read off" the part of the simulated universe that corresponded to the camera inputs. That additional code looks like epiphenomenal mind-stuff. Using this framework you can pose questions like "if the camera is expected to be rebuilt using different but functionally equivalent materials, will his change the inputs Solomonoff induction predicts?" or "if the camera is about to be duplicated, which copy's inputs will be predicted by Solomonoff induction?"

If we go beyond Solomonoff induction to allow actions, then you get questions that map pretty well to debates about "free will."

The second comment says:

The code simulating a physical universe doesn't need to make any reference to which brain or camera in the simulation is being "read off" to provide the sensory input stream. The additional code takes the simulation, which is a complete picture of the world according to the laws of physics as they are seen by the creatures in the simulation, and outputs a sensory stream. This function is directly analogous to what dualist/epiphenomenalist philosopher of mind David Chalmers calls "psychophysical laws."

Carl's comments pose the questions you can ask/highlight the connection, but they don't answer those questions. I would be interested in references to other places discussing this idea, or answers to these questions.

Here are some of my own confused thoughts (I'm still trying to learn algorithmic information theory, so I would appreciate hearing any corrections):

If the camera is duplicated, it seems to take a longer program to track/read off two cameras in the physical world instead of just one, so Solomonoff induction would seem to prefer a bit sequence where only one camera's inputs are visible. However, there seems to be a symm

...
How should I be thinking about the risk of air travel (re: Coronavirus)?
by Eli Tyre — 2020-03-02T20:10:40.617Z

Is there any information about how much risk of exposure increases by air-travel?

Like Ŀady Jade Beacham, I am considering flying out to my parents house in Arizona. If I need to quarantine for multiple weeks to months, that would be a much more comfortable place to be, for me.

The crux is how much risk going on an airplane entails. Does anyone have an estimate of how much the probability of infection goes up, given (~ 3 hours of) air travel?

What things can you do to make it safer? (Things like wearing a mask, wiping down the seat with disinfectant wipes, buying the seats on either side of you so that you're more isolated.) How much do those interventions help?
Epistemic standards for “Why did it take so long to invent X?”
by jasoncrawford — 2020-03-02T18:58:42.783Z
Link: https://rootsofprogress.org/epistemic-standards-for-why-it-took-so-long

In seeking to understand the history of progress, I keep running across intriguing cases of “ideas behind their time”—inventions that seem to have come along much later than they could have, such as the cotton gin or the bicycle. I’ve started collecting a list here, and will update that page with new analyses as I find them.

Debates on these questions sometimes oddly devolve into arguments, with people fruitlessly talking past each other (although if people on the Internet can argue over how many days are in a week, I guess we can argue about anything). So I want to comment on how we think about such cases and what the standards for evidence are.

Aeolipile, from Hero's Pneumatica
Wikimedia

To start, there is a need for precision. For example, take the steam engine. There was a thing in antiquity called an engine, that used steam: “Hero’s engine”, also known as the aeolipile. Some people see this and conclude that “steam engines existed in the 1st century” or that there’s no reason ancient Rome couldn’t have used this widely in industrial applications.

This is a mistake. The aeolipile is nothing like the steam engines of the 18th century and later: it’s a turbine, which means it is rotary, rather than using the reciprocating (back-and-forth) motion of a piston, as in Newcomen’s engine. Why does this matter? Because the aeolipile doesn’t generate enough torque for practical applications—one analysis says that Watt’s engine generated a quarter of a million times more torque.

Newcomen's steam engine

Even before finding that analysis, I had an hunch it would be the case—indeed, that’s how I knew to search for “aeolipile torque”, which led me to that link on the first page of Google results. My intuition was based on a few things. First, if a simple, primitive turbine like the aeolipile could be put to practical use, why didn’t anyone reinvent it in the 18th or even 17th century? Why did Newcomen, Watt, and others focus on much more complicated piston engines? They were smart people and were obviously working hard on the problem, it seems impossible that such a simple solution would have escaped all of them. Second, the aeolipile is small—Hero’s sketch above shows it sitting on a table—but Newcomen’s engine was very large, to the point where a separate shed would be built to house one. Why did the engine have to be that large if a tiny one would do?

Again, a precise

...
Chapter 10: What does it Mean?
by ld97 — 2020-03-02T18:21:09.463Z

Let's take a look at the picture we've made.

How does our mind process information?

You receive signals from your sensory mechanisms, and they activate neuronal networks in our brains. Most strongly coupled parts of the system enable as a whole, and that gives you a sense of the object. We can describe that part as a graph of objects that we named "Knowledge Graph."
Information about that activations goes by another neuronal path to our Attention Filter. We don't copy the whole object. We receive a reference.
Our attention looks at the references it received and check their combinations with other references we already have in working memory, using recall mechanisms.
Our working memory is what we can "see" by some part of our model that we name "I."
By choosing the most strong signals, we travel our Knowledge Graph and try to bind together two parts of it that currently disconnected. That's how we solve problems.
When we have a problem, we feel curious. If we can't solve it for a long time - we losing patience, and that's how we abort the task. Curiosity feeling - is what helps us to move. Motivation points. Patience is what helps us to stop solving tasks that seem useless to our brains.
To notify us about task solving results, we have regulation mechanisms. If we solved a problem - we feel inspired. If our solution gives us a result that was predicted - we'll feel high. But if our Answer was wrong, if we had expectations that were not supported sensory - we will feel disappointed.
All that regulation mechanisms enable L-LTP that allows us to remember results for a long period.
If you try to use something that will solve the problem of bad feelings without finding it cause - you can run into addiction circle. Remember, in most cases, feeling yourself bad means that you have some problems. Sometimes it becomes a separate problem that will disturb you from finding a cause. In that case, you can use something to release your pain, but irresponsible usage of painkillers will lead to another problem. That's because your brain will try to bind painkiller as a solution and will remember it.
To find a way out from the circle that became your problem or to prevent its creation, you can use your Curiosity. It will switch your attention from disappointment and will lead you to the solution.

One small picture ...
Coronavirus is Here
by Zvi — 2020-03-02T14:10:00.443Z

Even in the best case scenarios, things are going to get a lot worse from here before they get better.

Please, please, please, do not rely on me here to tell you what is going on or what to do. Even more than that, please, please, please, do not take this as saying that you shouldn’t do a lot more than the things I’m saying here.

This is me doing what various stupid reasons prevented me from doing earlier, deciding that saying something is a lot better than saying nothing, and hoping it will do some good.

This is not a model of what is happening, or an attempt to justify what you should do, because attempting to do that would cause me to continue to say nothing, and that seems worse.

If you already are taking the situation seriously and making serious preparations, this probably won’t tell you anything new. I am totally fine with that.

This is me seeing something and saying something.

If you need some sort of permission to yourself to acknowledge that this is happening, and that you need to take action now to prepare, this is one more instance of that. You have it.

It will be important that you retain the ability, when things get bad, to keep your head on straight. Prepare now, including mentally, as best you can, for people you know and care about getting sick and dying, because this might well happen to you regardless of what actions everyone involved takes.

Here is an article describing the symptoms, if you are not yet familiar with that. It’s primarily dry cough and pneumonia.

From the statistics I have seen, with super wide error bars, overall risk of death if you do get infected could be up to about 2%. But that is only an average. It varies wildly based on age and prior health, and presumably access to health care. I’ve also seen reasonable claims that initial degree of exposure matters here.

Risk of death chart by age:

Age Death Rate

80+ years old 14.8%

70-79 years old 8.0%

60-69 years old 3.6%

50-59 years old 1.3%

40-49 years old 0.4%

30-39 years old 0.2%

20-29 years old 0.2%

10-19 years old 0.2%

0-9 years old no fatalities

Co-morbidity is very high for conditions such as heart attacks, cancer and diabetes. If you are elderly and/or have conditions such as cancer, a prior heart attack or diabetes, you are at much greater risk, and should take all precautions more aggressively.

If you have not yet prepared for quarantine in place for at least several weeks, ...
Anthropics over-simplified: it's about priors, not updates
by Stuart_Armstrong — 2020-03-02T13:45:11.710Z

I've argued that anthropic reasoning isn't magic, applied anthropic reasoning to the Fermi question, claimed that different anthropic probabilities answer different questions, and concluded that anthropics is pretty normal.

But all those posts were long and somewhat technical, and needed some familiarity with anthropic reasoning in order to be applied. So here I'll list what people unfamiliar with anthropic reasoning can do to add it simply[1] and easily to their papers/blog posts/discussions:

Anthropics is about priors, not updates; updates function the same way for all anthropic probabilities.

If two theories predict the same population, there is no anthropic effect between them.

Updating on safety

Suppose you go into hiding in a bunker in 1956. You're not sure if the cold war is intrinsically stable or unstable. Stable predicts a 25% chance of nuclear war; unstable predicts a 75% chance.

You emerge much older in 2020, and notice there has not been a nuclear war. Then, whatever anthropic probability theory you use, you update the ratio

P(stable):P(unstable) by 75:25=3:1.

Population balancing

Suppose you have two theories to explain the Fermi paradox:

Theory 1 is that life can only evolve in very rare conditions, so Earth has the only life in the reachable universe.

Theory 2 is that there is some disaster that regularly obliterates pre-life conditions, so Earth has the only life in the reachable universe.

Since the total population predicted by these two theories is the same, there is no anthropic update between them[2].

These points are a bit over-simplified, but are suitable for most likely scenarios. ↩︎

If you use a reference class that doesn't include certain entities - maybe you don't include pre-mammals or beings without central nervous systems - then you only need to compare the population that is in your reference class. ↩︎
If I were a well-intentioned AI...
IV: Mesa-optimising
by Stuart_Armstrong — 2020-03-02T12:16:15.609Z

Here I apply my "If I were a well-intentioned AI" filter to mesa-optimising

Now, I know that a mesa-optimiser need not be a subagent (see 1.1 here), but I'm obviously going to imagine myself as a mesa-optimising subagent.

An immediate human analogy springs to mind: I'm the director of a subdivision of some corporation or agency, and the "root optimiser" is the management of that entity.

There is a lot of literature on what happens if I'm selfish in this position; but if I'm well-intentioned, what should I be doing?

One thing that thinking this way made me realise: there is a big difference between "aligned with management" and "controlled by management".

We'll consider each one in turn, but to summarise: aligned mesa-optimisers are generally better than controlled mesa-optimisers, but it is hard to tell the difference between an aligned and a dangerous unaligned mesa-optimiser.

Control vs alignment

First let's flesh out the corporate/management example a bit. Me-AI is in charge of making widgets, that are used by the company for some purpose. That purpose is given by - the base utility for the corporation.

My role is to make as many widgets as possible within my budget; this is , the mesa-objective I have been given by management.

My true utility function is Ume. Management don't fully know what Ume is - or at least don't fully understand it, or all of its implications. This is needed, of course, because if management fully understood the implications of Ume, there would be no uncertainty at all on their part, and they could make me do exactly what they wanted - or they would turn me off.

Because of this uncertainty, management have added some extra levels of control over me. Let's assume one typical precaution: if I underspend the budget, the budget is cut next year. If I overspend, I'm reprimanded (and fired if I do it too often), but the budget is not cut.

There are three possible situations I could confront:

S1: I've made as many widgets as I can this year, and spent 90% of my budget. I predict that, next year, I will only need 90% of this year's budget.

S2: I've made as many widgets as I can this year, and spent 90% of my budget. I predict that, next year, I will need 100% of this year's budget.

S3: I've spent 100% of my budget on widgets. I predict that widgets are particularly valuable to the company this year, much moreso than next year.

Aligned mesa

...
Change Your Mind
by Slimepriestess — 2020-03-02T10:41:44.879Z
Link: https://hivewired.wordpress.com/2020/03/02/change-your-mind/

This is a post of updates and places where I've changed my mind about things I'd previously written about. I will continue to update it over time as my endorsements of the things I've written change. I think it's really important to have this sort of log of the evolution of one's beliefs and would strongly encourage others to do things similar to this instead of just deleting old content they don't agree with anymore. Recontextualize old content and describe how you've updated on it instead of just hiding the uncomfortable beliefs of the past; that seems like the more honest and mature thing to do.
Chapter 9: Why can it Select?
by ld97 — 2020-03-02T10:11:37.047Z

Our final problem. Let's take a look at our task:

We know that memories are coupled subnetworks in the Knowledge Graph.
We know that what we can recall them using our Working memory.
We understood the regulation mechanism that motivates us to do it. We also realized it's binding with Late Long-Term Potentiation, which allows us to store data for long periods.
We know that we can receive signals about activations of memory parts. And their strengths.

How to select what signal or Reference will pass the spam-filter of attention? What criteria for filtering will we use? How should it work in terms of activations in neurons?

Let's take a simple example. Please fill blank spaces, and try to observe how are you doing that:

The whale shark is ... than elephant
The ... is smaller than Jupiter.
Do you ...?

I am quite sure that it was like that:

Bigger or smaller? I don't know. The average whale is bigger. That's obvious. But whale shark?
Guy, are you serious? There are a lot of things smaller than Jupiter. All the planets in the solar system smaller than it!
Do I what? Oh, I understood, you've gone completely mad.

What you were doing is creating hypotheses. Bigger or smaller? What planet did he mean? Had he gone mad? You've been trying to explain what is happening with the information you've had. You've been trying to guess.

And you always do it! While watching serials, while talking with someone, while trying to choose between chocolate and strawberry ice-cream, reading this sequence of articles.

But how to describe the prediction you making and how to choose between them?

Prediction is an easy one thing - you have information, you activate object in the knowledge graph, you receive the results of your mind-search. The activations had some power; we choose the strongest of them and... And here, we have a problem because we want different meanings for the same things in different contexts.

Context is what we currently think about, so let's simplify it to object-references in our Working Memory.

How should work the filter that has one reference that is context, and a bunch of other references that could be related to this object? How to pass only meaningful, only bound references?

Bound. BOUND!

What if filter coactivates all the references by pairs, looks on results, and chooses the strongest signals? That will allow it to filter c...