trentmkelly/LessWrong-43k huggingface dataset:
Forecasting Newsletter: Looking back at 2021
by NunoSempere — 2022-01-27T20:08:16.038Z
Link: https://forecasting.substack.com/p/looking-back-at-2021

Table of contents

The American Empire Has Alzheimer's.
Prediction Markets: VC money, searching for DraftKings, predatory pricing, and the race to be last.
For skilled forecasters, crypto prediction markets are much more profitable than forecasting platforms.
Best forecasting pieces from 2021

You can sign up for or view this newsletter on substack, where there are already a few thoughtful comments.

The American Empire has Alzheimer's

It is 1964. Sherman Kent is a senior intelligence analyst. While doing some rudimentary experiments, he realizes that analysts themselves disagree about the degree of confidence that words convey. He suggests that analysts clearly state how certain they are of their conclusions, by using words that correspond to probabilities. This will allow keeping track of how analysts do, while being simple enough for less detail-focused politicians to understand. His proposal encounters deep resistance, and doesn't get implemented.

It is the 30th of April of 1975. With the fall of Saigon, the US finally pulls out of a bloody war with Vietnam. There are embarrassing images of people flying out of the US embassy at the last moment. Biden is a newly-minted senator from Delaware.

It is 2001. The US intelligence agencies are very embarrassed by not having been able to predict the September 11 attacks. The position of the Director of National Intelligence, and an associated Office of the Director of National Intelligence, is established to coordinate all intelligence agencies to do better in the future.

At the same time, Robin Hanson pushes for a "Policy Analysis Market", which would have covered topics of geopolitical interest. This proposal becomes too controversial, and gets dropped.

It is 2008. A bunch of Nobel Prize winners and other luminaries publish a letter urging the Commodity Futures Trading Commission (CFTC) to make prediction markets more legal.

It is 2010. IARPA, an intelligence agency modeled after DARPA, which incubates high-risk, high payoff projects, creates a tournament to find out which forecasting setups do best. Philip Tetlock had done some experiments which found that pre-selecting participants does pretty well. He repeatedly wins the IARPA tournament, and creates Good Judgment Inc to provide the services of his preselected high-performance forecasters. The US doesn't buy their services, and Good Judgment Inc survives by selling v...
Covid 1/27/22: Let My People Go
by Zvi — 2022-01-27T17:00:01.543Z

The moment’s here. My people are all the people. It is time to let my people go.

While case counts in many places remain high, we are on the way back down the mountain. The hospitals will hold. People can choose, based on their preferences and situation and the local conditions, whether they want to go now or wait a few more weeks before going. That is their call. It needs to be their call.

One could argue, as Tyler Cowen did in this excellent talk this week at Yale, that the moment is not quite here yet, on the theory that in a month cases will be an order of magnitude lower and thus it will be politically and socially easier to make the transition. There would be less opposition then, so better to wait, the price for doing so is small.

Would I take that deal? Absolutely I would take that deal, if we agreed on an end date or on explicit end conditions. A few more weeks is a small price in the grand scheme, and getting these things to happen takes time, so ‘a few weeks from now’ is the second best time to end pandemic restrictions.

But there’s no need for that. The best time is right now.

Remember that the case counts are seven-day averages and there is a several-day delay between infection and positive test, so we are living continuously living, for better and for worse, ‘in the future.’

Today I go back to the excellent Da Umberto, to celebrate (barring another variant, and ignoring the writing of posts and the pro forma wearing of masks and showing vaccination cards) the end of my pandemic.

Executive Summary

Cases now declining most places.
Restrictions mostly remain in place.
Let my people go.

Let’s run the numbers.

The Numbers

Week-Over-Week Predictions

Prediction from last week: 4.4mm cases (-10%) and 14,500 deaths (+15%).

Results: 4.05mm cases (-17%) and 15,964 deaths (+26%).

Prediction for next week: 2.85mm cases (-30%) and 20,000 deaths (+25%).

Overall it seems we peaked faster and more in unison than I expected, while other places did the opposite, and had softer declines than I expected. Based on those smaller declines elsewhere, I still expect the decline to accelerate as the West turns the corner, but I don’t expect a crash as quick as the rise up. A much faster decline is possible, but it would make our observations elsewhere quite weird.

The most confusing and surprising place of all right now is India. Cases are somehow flat there, and have been for a week, wit...
Newcomb's Lottery Problem
by Heighn — 2022-01-27T16:28:11.609Z

Inspired by and variant of The Ultimate Newcomb's Problem.

In front of you are two boxes, box A and box B. You can either take only box B (one-boxing) or both (two-boxing). Box A visibly contains $1,000. Box B contains a visible number X. X is guaranteed to be equal to or larger than 1 and smaller than or equal to 1000. Also, if X is composite, box B contains $1,000,000. If X is prime, B contains $0. You observe X = 226. Omega the superintelligence has predicted your move in this game. If it predicted you will one-box, it chose X to be composite; otherwise, it made X prime. Omega is known to be correct in her predictions 99% of the time, and completely honest.

The Having Fun With Decision Theory Lottery has randomly picked a number Y, which is guaranteed to fall in the same range as X. Y is displayed on a screen visible to you. The HFWDT Lottery is organized by Omega - but again, Y is picked at random and therefore completely separately from X. If both X and Y are prime, the HFWDT Lottery gives you $4,000,000. Otherwise it gives you $0. You observe Y = 536.

Do you one-box or two-box?

Newcomb's Lottery Problem 2: Everything is the same as before, except the HFWDT Lottery price is now $8,000,000. Do you one-box or two-box?
Causality, Transformative AI and alignment - part I
by Marius Hobbhahn — 2022-01-27T16:18:57.942Z

TL;DR: transformative AI(TAI) plausibly requires causal models of the world. Thus, a component of AI safety is ensuring secure paths to generating these causal models. We think the lens of causal models might be undervalued within the current alignment research landscape and suggest possible research directions.

This post was written by Marius Hobbhahn and David Seiler. MH would like Richard Ngo for encouragement and feedback.

If you think these are interesting questions and want to work on them, write us. We will probably start to play around with GPT-3 soonish. If you want to join the project, just reach out. There is certainly stuff we missed. Feel free to send us references if you think they are relevant.

There are already a small number of people working on causality within the EA community. They include

Victor Veitch

,

Zhijing Jin

and

PabloAMC

. Check them out for further insights. There are also other alignment researchers working on causal influence diagrams (authors: Tom Everitt, Ryan Carey, Lewis Hammond, James Fox, Eric Langlois, and Shane Legg) whose work is very much related.

Causality - a working definition:

Just to get this out of the way: we follow a broad definition of causality, i.e. we assume it can be learned from (some) data and doesn’t have to be put into the model by humans. Furthermore, we don’t think the representation has to be explicit, e.g. in a probabilistic model, but could be represented in other ways, e.g. in the weights of neural networks.

But what is it? In a loose sense, you already know: things make other things happen. When you touch a light switch and a light comes on, that’s causality. There is a more technical sense in which no one understands causality, not even Judea Pearl (where does causal information ultimately come from if you have to make causal assumptions to get it? For that matter,

how do we get variables out of undifferentiated sense data

?). But it's possible to get useful results without understanding causality precisely, and for our purposes, it's enough to approach the question at the level of

causal models

.

Concretely: you can draw circles around phenomena in the world (like "a switch" and "a lightbulb") to make them into nodes in a graph, and draw arrows between those nodes to represent their causal relationships (from the switch to the lightbulb if you think the switch causes the l...
Arguments about Highly Reliable Agent Designs as a Useful Path to Artificial Intelligence Safety
by riceissa — 2022-01-27T13:13:11.011Z
Link: https://arxiv.org/abs/2201.02950

This paper is a revised and expanded version of my blog post Plausible cases for HRAD work, and locating the crux in the "realism about rationality" debate, now with David Manheim as co-author.

Abstract:

Several different approaches exist for ensuring the safety of future Transformative Artificial Intelligence (TAI) or Artificial Superintelligence (ASI) systems, and proponents of different approaches have made different and debated claims about the importance or usefulness of their work in the near term, and for future systems. Highly Reliable Agent Designs (HRAD) is one of the most controversial and ambitious approaches, championed by the Machine Intelligence Research Institute, among others, and various arguments have been made about whether and how it reduces risks from future AI systems. In order to reduce confusion in the debate about AI safety, here we build on a previous discussion by Rice which collects and presents four central arguments which are used to justify HRAD as a path towards safety of AI systems.

We have titled the arguments (1) incidental utility,(2) deconfusion, (3) precise specification, and (4) prediction. Each of these makes different, partly conflicting claims about how future AI systems can be risky. We have explained the assumptions and claims based on a review of published and informal literature, along with consultation with experts who have stated positions on the topic. Finally, we have briefly outlined arguments against each approach and against the agenda overall.

See also this Twitter thread where David summarizes the paper.
Coping with Undecidability
by Morpheus — 2022-01-27T10:31:00.520Z

Mostly just reflections on things I've been learning about theory of computer science and trying to figure my philosophical stance towards them, in order to not be a trollable mathematician

In the last few months, my mind has constantly been trying to argue against something like the "naive Church Turing Thesis" in an attempt to better understand what it means for "the real world" that some things we can ask ourselves with a Turing machine can't be answered with "yes" or "no" by any Turing machine (formally called an undecidable problem). (See the halting problem, for an example).

My confusion stems from this weird dichotomy[1] that basically everything we can compute in the real world can be computed by a finite state automaton, because our world does not contain infinite memory tapes (as far as I know from the probabilistic evidence I was able to gather). But (for convenience?), mathematicians/computer scientists use Turing machines/Lambda Calculus/(your favorite programming language) for most purposes instead to model things, even though they have these really peculiar undecidability properties.

What I've learned so far is that applying your formal model of computation to say things about problems in the real world is a super philosophically slippery domain (though I feel the reason for this has more to do with the space of all algorithms just being a really resistant hydra and less with peculiarities of the human mind[2]). The replies I got from people on Lesswrong along with Scott Aaronsons writing have been very helpful here[3].

Maybe I am just a pedant, but after tripping myself for a while, I see other people commit philosophical blunders in this area. As an example, here is this motte-and-bailey [4]which I fell for:

Motte-and-Bailey computability

Motte:

"intuitively computable" => a (Turing Machine)/(Lambda Calculus)/(Your favorite equivalent model C) can do it.
The converse does not necessarily hold, because no one has infinite memory. When doing theoretical computer science, we want to abstract away the details, so it is convenient to use C to be able to extrapolate Algorithm X to infinity in order to force us to characterize how X is behaving asymptotically (which is itself of course only a convenient way to roughly characterize the runtime of an algorithm).

Bayley:

"Let me show you a problem that even computers can't solve":
"shows the halting ...
Ranking Cultures by Rationality
by Gadersd — 2022-01-27T02:25:10.690Z

I have been interested in different cultures for a while, probably because I find so many deficiencies in my own culture. Which countries/cultures do you find to be generally the most rationally based? Japan and Sweden especially seem to have quite a secular populace mostly immune to silly ideologies and superstition. Hong Kong also seems to be a place of practically minded people who do not put much stock into religion. I cannot recall any specific data on these places other than a general lack of religion. What other measures would be useful for ranking countries by rationality?
Vavilov Day Discussion Post
by Elizabeth — 2022-01-27T00:26:27.649Z

When I first wrote up my plan to fast in honor of Nikolai Vavilov and his institute staff, I conceived of it as basically a thing for myself, that might interest a few other people. It both got much more response than I expected, and more response (positive and negative) treating it as a bid for community structure rather than one of many things that someone, somewhere was doing. That wasn’t my goal and to be honest I feel trepidatious about the concept: I was going for a very specific thing and I’m simultaneously really happy it resonated with so many people and concerned about dilution

But the Vavilov Day concept clearly has enough energy behind it that I don't feel right blocking discussion. I continue to not want to host that discussion on a personal post about my own choices, so have created this post instead.

Some things I think might be good to share here:

Your specific observation of Vavilov Day and how it felt (I was surprised a lot of people defaulted to a 36 hour fast; to the extent other people joined in I expected them to pick their own thing)
Feelings about the core concept and ontology

Ideally keeping discussion about the larger patron saints/holiday anarchy discussion over on

that post

instead.

What you’d like to be different or the same for you next year.

In my ideal world, this conversation is a bunch of individuals discussing actions and ideas to figure out what works best for them. But some of the pent-up discussion is about that attitude (vs. something more community built), and I don't get to declare my view correct by fiat. However I will ask that you keep that frame in mind and, if someone's responses don't make sense or seem oppressive, consider that this might be why.
Event: Moral Foundations of Progress Studies, March 4–6 at UT Austin
by jasoncrawford — 2022-01-26T21:57:16.932Z
Link: https://rootsofprogress.org/moral-foundations-of-progress-studies-workshop

I’m excited to announce a workshop on the Moral Foundations of Progress Studies.

The progress studies community has had a lot of discussion about technology, economics, history, and politics. However, there is no consensus on the moral basis for valuing or pursuing “progress,” and there are key open questions about how progress is to be judged and measured, who should benefit from it, and what type of progress we should pursue.

The goal of this workshop is to reach a consensus on what major moral/ethical questions are at the foundations of a study of progress, and what broad answers to these questions have been proposed. A few designated attendees will take notes and draft a short article afterwards summarizing the discussion. (We’re currently looking for the appropriate place to publish this; it may be in a journal or on a blog.)

Apply to attend here. Space is limited; we’ll be prioritizing people in or with a connection to academia, and public intellectuals who write about progress or adjacent topics.

When: March 4–6, 2022

Where: University of Texas at Austin

Agenda (subject to change):

Friday:

Survey of writers on progress (including Cowen, Deutsch, Pinker)
Theories of well-being
Panel: Steven Pinker, David Deutsch (via video)
Metrics and standards of value
Challenges to the claim that the last two centuries represent progress

Saturday:

Interrogating the idea of moral progress
Progress & safety (including the Precautionary Principle and existential risk)
Challenges in assessing possible futures

Sunday morning:

Wrap-up

Except for the Friday panel, each session will be a 90-minute discussion led by one or a small panel of participants who give a brief intro.

Co-hosts:

Jason Crawford, founder of The Roots of Progress
Gregory Salmieri, director of the Program for Objectivity in Thought, Action, and Enterprise at the Salem Center at UT Austin

Attendees are being invited from the progress studies and Effective Altruism communities, plus moral philosophers familiar with the virtue-ethics and well-being literatures.

There is no cost to attend. We have some funding to pay for travel and lodging for a limited number of participants; you can request funding when you apply.
Freudianism among the 17th century Wendat
by David Gross — 2022-01-26T18:15:32.042Z

I’m close to finishing David Graeber & David Wengrow’s The Dawn of Everything: A New History of Humanity. The most interesting thing I’ve found in it so far is a quote from The Jesuit Relations, a series of field reports from missionaries in North America published in the 17th century. This one is from 1649:

In addition to the desires that we generally have that are free—or, at least, voluntary in us—which arise from a previous knowledge of some goodness that we imagine to exist in the thing desired, the Hurons believe that our souls have other desires, which are, as it were, inborn and concealed.…

Now they believe that our soul makes these natural desires known by means of dreams, which are its language. Accordingly, when these desires are accomplished, it is satisfied; but, on the contrary, if it be not granted what it desires, it becomes angry, and not only does not give its body the good and the happiness that it wished to procure for it, but often it also revolts against the body, causing various diseases, and even death.

…And they call this Ondinnonk—a secret desire of the soul manifested by a dream.

Nevertheless—just as, although we did not always declare our thoughts and our inclinations by means of speech, those who by means of supernatural vision could see into the depths of our hearts would not fail to have knowledge of them—in the same manner, the Hurons believe that there are certain persons, more enlightened than the common, whose sight penetrates, as it were, into the depths of the soul. These see the natural and hidden desires that it has, though the soul has declared nothing by dreams, or though he who may have had the dreams has completely forgotten them. It is thus that their Medicine-men—or, rather, their Jugglers—whom they call Saokata, acquire credit, and make the most of their art by saying that a child in the cradle, who has neither discernment nor knowledge, will have an Ondinnonk—that is to say, a natural and hidden desire for such or such a thing; and that a sick person will have similar desires for various things of which he has never had any knowledge, or anything approaching it. For, as we shall explain further on, the Hurons believe that one of the most efficacious remedies for rapidly restoring health is to grant the soul of the sick person these natural desires.

(the quote above is more extensive than the excerpt in The Dawn o...
Disambiguating the ‘Observable Universe’
by Sam Enright — 2022-01-26T16:50:17.435Z

I’ve seen a lot of confusion over what precisely the term ‘observable universe’ refers to. This post is an attempt to remedy that. Crossposted from my personal website.

In 1929,

Edwin Hubble

discovered that the universe is expanding. He observed that light emitted from distant celestial objects was redder than expected, due to the downward shift in frequency as their light receded from Earth. And the further away the objects were, the proportionately greater their redshift. Since objects were getting further apart from each other, figures like

Lemaître

and

Friedmann

reasoned that there must have been some point in the past at which the whole universe was compressed into a single point.

Embarrassingly, linear extrapolation implied an age of the universe of 1-2 billion years, shorter than the known age of the oldest rocks on Earth. One of the difficulties is that expansion flipped from slowing down to speeding up after several billion years, and this requires complex observations of supernovae to account for. In any case, astronomers eventually measured the age of the universe at about 13.8 billion years. This implies three tempting definitions for the observable universe — that part of the whole universe which we can theoretically see — only one of which is correct.

Universe age in light-years: You would be forgiven for thinking that the observable universe is a sphere with a radius of 13.8 billion light-years centred on the Earth, since that’s how long light has had to reach us. However, this assumes the universe isn’t expanding or contracting.

The Hubble radius: Hubble found that the distance to a given galaxy was proportional to its recessional velocity, with a constant of proportionality now called the Hubble constant H. This implies there is a sphere past which everything is travelling away from us faster than the speed of light. This is known as the Hubble sphere, and it has a radius of around 14.4 billion light-years.

You may worry that this faster-than-light travel violates Einstein’s theory of relativity, which says that nothing can travel faster than light. However, what relativity says is that matter within space can’t travel faster than light, but nothing about how fast space itself can travel.

The Hubble sphere isn’t the observable universe either. The radius of the Hubble sphere will be given by&...