trentmkelly/LessWrong-43k huggingface dataset:
The Embarrassing Problem of Premature Exploitation
by Richard Meadows — 2020-04-30T20:33:07.528Z
Link: https://thedeepdish.org/exploring-exploiting/

tl;dr: summarising some handy concepts from Algorithms to Live By, as they relate to optionality and tinkering.

Babies love putting things in their mouths: dirt, insects, bits of grass, their own poo. They have no sense of fear or self-preservation, and come up with endlessly creative ways to place themselves in mortal peril. Once they learn to talk, their constant experimentation with the world transcends the physical to the philosophical. They want to know everything. They are bottomless pits of curiosity, with very little in the way of attention span or self-discipline. Your typical two-year-old can only concentrate on a task for six minutes at a time. Young children are not self-aware enough to feel much in the way of shame, or embarrassment. Nothing is off-limits.

In a word, very young people spend almost all of their time exploring.

The elderly are set in their ways. The only foreign objects they put in their mouths are dentures and hard caramels; occasionally followed by a fork to extricate said caramels from said dentures. They tend to have stable routines, rituals, hobbies, and social circles. They rarely try new things or experiment with new identities. They’ve lived long enough to know what they’re about, and they intend to wring out every ounce of enjoyment before the curtains come down.

In a word, very old people spend almost all of their time exploiting.

The ‘explore-exploit’ constraint is one of the most useful ideas I’ve come across. Don’t worry about the connotations; these terms are borrowed from computer science, where they’re used neutrally.

The point is that there’s an inescapable trade-off between these activities. Time spent investigating new opportunities is time you could have spent enjoying what you already have, and vice versa.

In the language of optionality: if you want to open up attractive new options—to cultivate the fat purple figs on the possibility tree—you have to spend time exploring. But unless you eventually switch to exploiting, all your precious figs will blacken and rot and fall to the ground.

This creates dilemmas great and small: Keep your job or pivot to a new career? Try the new Chinese place or return to your favourite haunt?

The Multi-Armed Bandit

The classic explore-exploit scenario is the ‘multi-armed bandit’.

You walk into a casino, and ...
Forecasting Newsletter: April 2020
by NunoSempere — 2020-04-30T16:41:35.849Z

A forecasting digest with a focus on experimental forecasting.

You can sign up here.

You can also see this post on the Effective Altruism Forum here

And the post is archived here

The newsletter itself is experimental, but there will be at least five more iterations. Feel free to use this post as a forecasting open thread.

Conflict of interest: With Foretold in general and Jacob Laguerros in particular. This is marked as (c.o.i) throughout the text.

Index

Prediction Markets & Forecasting platforms.

Augur.

PredictIt & Election Betting Odds.

Replication Markets.

Coronavirus Information Markets.

Foretold. (c.o.i).

Metaculus.

Good Judgement Inc. & Good Judgement Open.

In the News.

Long Content.

Prediction Markets & Forecasting platforms.

Forecasters may now choose to forecast any of the four horsemen of the Apocalypse: Death, Famine, Pestilence and War.

Augur: augur.net

Augur is a decentralized prediction market. It will be undergoing its first major update.

Predict It & Election Betting Odds: predictIt.org & electionBettingOdds.com

PredictIt is a prediction platform restricted to US citizens or those who bother using a VPN. Anecdotically, it often has free energy, that is, places where one can earn money by having better probabilities, and where this is not too hard. However, due to fees & the hassle of setting it up, these inefficiencies don't get corrected.
In PredictIt, the world politics section...

gives a 17% to a Scottish independence referendum (though read the fine print).

gives 20% to Netanyahu leaving before the end of the year

gives 64% to Maduro remaining President of Venezuela before the end of the year.

The question on which Asian/Pacific leaders will leave office next? also looks like it has a lot of free energy, as it overestimates low probability events.

Election Betting Odds aggregates PredictIt with other such services for the US presidential elections.

Replication Markets: replicationmarkets.com

Replication Markets is a project where volunteer forecasters try to predict whether a given study's results will be replicated with high power. Rewards are monetary, but only given out to the top N forecasters, and markets suffer from sometimes being dull. They have added two market-maker bots and commenced and conclude their 6th round. They also added a sleek new widget to visualize the price of shares better.

Coron

...
Against strong bayesianism
by Richard_Ngo — 2020-04-30T10:48:07.678Z

Note that this post has been edited in response to feedback and comments. In particular, I've added the word "strong" into the title, and an explanation of it at the beginning of the post, to be clearer what position I'm critiquing. I've also edited the discussion of Blockhead.

In this post I want to lay out some intuitions about why bayesianism is not very useful as a conceptual framework for thinking either about AGI or human reasoning. This is not a critique of bayesian statistical methods; it’s instead aimed at the philosophical position that bayesianism defines an ideal of rationality which should inform our perspectives on less capable agents, also known as "strong bayesianism". As described here:

The Bayesian machinery is frequently used in statistics and machine learning, and some people in these fields believe it is very frequently the right tool for the job. I’ll call this position “weak Bayesianism.” There is a more extreme and more philosophical position, which I’ll call “strong Bayesianism,” that says that the Bayesian machinery is the single correct way to do not only statistics, but science and inductive inference in general – that it’s the “aspirin in willow bark” that makes science, and perhaps all speculative thought, work insofar as it does work.

Or another way of phrasing the position, from Eliezer:

You may not be able to compute the optimal [Bayesian] answer. But whatever approximation you use, both its failures and successes will be explainable in terms of Bayesian probability theory.

First, let’s talk about Blockhead: Ned Block’s hypothetical AI that consists solely of a gigantic lookup table. Consider a version of Blockhead that comes pre-loaded with the optimal actions (according to a given utility function) for any sequence of inputs which takes less than a million years to observe. So for the next million years, Blockhead will act just like an ideal superintelligent agent. Suppose I argued that we should therefore study Blockhead in order to understand advanced AI better. Why is this clearly a bad idea? Well, one problem is that Blockhead is absurdly unrealistic; you could never get anywhere near implementing it in real life. More importantly, even though Blockhead gets the right answer on all the inputs we give it, it’s not doing anything remotely like thinking or reasoning.

The general lesson here ...
Optimising Society to Constrain Risk of War from an Artificial Superintelligence 
by JohnCDraper — 2020-04-30T10:47:56.483Z

This paper is now up (with the annex mentioned in the paper) as a preprint at https://osf.io/preprints/socarxiv/4268q

It can be cited as Draper, John. 2020. “Optimising Peace Through a Universal Global Peace Treaty to Constrain Risk of War from a Militarised Artificial Superintelligence.” SocArXiv. April 15. doi:10.31235/osf.io/4268q.

Comments are welcome below.

Optimising Peace through a Universal Global Peace Treaty to Constrain Risk of War from a Militarised Artificial Superintelligence

John Draper

Abstract

An artificial superintelligence (ASI) emerging in a world where war is still normalised may constitute a catastrophic existential risk, either because the ASI might be employed by a single nation-state on purpose to wage war for global domination or because the ASI goes to war on behalf of itself to establish global domination; these risks are not mutually incompatible in that the first can transition to the second. We presently live in a world where few states actually declare war on each other or even war on each other. This is because the 1945 United Nations’ Charter's Article 2 states that UN member states should “refrain in their international relations from the threat or use of force against the territorial integrity or political independence of any state”, while allowing for “military measures by UN Security Council resolutions” and “exercise of self-defense”. In this theoretical ideal, wars are not declared; instead, 'international armed conflicts' occur. However, costly interstate conflicts, both ‘hot’ and ‘cold’, still exist, for instance the Kashmir Conflict and the Korean War. Furthermore, a ‘New Cold War’ between AI superpowers (the United States and China) looms. An ASI-directed/enabled future interstate war could trigger ‘total war’, including nuclear war, and is therefore ‘high risk’. One risk reduction strategy would be optimising peace through a Universal Global Peace Treaty (UGPT), which could contribute towards the ending of existing wars and towards the prevention of future wars, through conforming instrumentalism. A critical juncture to optimise peace via the UGPT is emerging, by leveraging the UGPT off a ‘burning plasma’ fusion reaction breakthrough, expected from circa 2...
What is the alternative to intent alignment called? 
by Richard_Ngo — 2020-04-30T02:16:02.661Z

Paul defines intent alignment of an AI A to a human H as the criterion that A is trying to do what H wants it to do. What term do people use for the definition of alignment in which A is trying to achieve H's goals (whether or not H intends for A to achieve H's goals)?

Secondly, this seems to basically map on to the distinction between an aligned genie and an aligned sovereign. Is this a fair characterisation?

(Intent alignment definition from https://ai-alignment.com/clarifying-ai-alignment-cec47cd69dd6)
Since the "Will COVID-19 survivors suffer lasting disability at a high rate?" thread was posted, has new information emerged that may update those estimates?
by nonzerosum — 2020-04-29T21:40:12.089Z

https://www.lesswrong.com/posts/h4GFHbhxE2pfiadhi/will-covid-19-survivors-suffer-lasting-disability-at-a-high
List of prompts for brainstorming COVID response project ideas
by nonzerosum — 2020-04-29T21:39:07.966Z

Suggested use: go through the questions, bullet out a couple things for each, and then at the end you'll probably have a couple project ideas that are uniquely suited to you.

Your personal interests

What outcomes are you personally most concerned about? Especially things where you feel like you're more concerned than others.
What questions or things do you find yourself most frequently wondering about?
What are your urgent or critical challenges?
What do you wish someone else would do?

Network

Because a hard part of many projects is reaching the people needed to build it, or reaching the people who you want to use it / benefit from it.

Current and past work connections
School connections
Social connections
Family and extended family, and their connections
Family friends
Online forums or communities

Skills and aptitudes

What things do other people view as "work" that you do for fun?
What things are really easy for you?
What are your skills?
What are your aptitudes?
What have you built, worked on, or done in the past?

Other resources

What other resources or assets do you have? For example, ways you can reach certain audiences, excess financial capital?

Boundaries

How much time are you willing to spend on a COVID related project before seeing positive results?
Melting democracy
by Richard_Ngo — 2020-04-29T20:10:01.470Z

Liquid democracy, also known as delegative democracy, is a proposed hybrid between direct democracy and representative democracy. In liquid democracy, as in direct democracy, everyone is able to vote directly on every issue. Of course, in a modern society it’d be prohibitively time-consuming for everyone to research every topic well enough to cast an informed vote. In practice, most who vote would end up following the advice of someone better-informed on the topic. Liquid democracy makes this dynamic explicit by allowing any voter to delegate their vote to someone else that they trust. Their delegate can now vote on their behalf, or else delegate again to someone else.

Some implementation details of how this system would work:

If A delegates to B, and B delegates to C, then C now has three votes. There’s no limit to how many times votes can be delegated (except that forming a cycle returns a vote to its original owner).
It’s a real-time system: any delegation can be changed at any time. In practice we’d need to implement voting and delegation via a secure digital system.
Voting and delegation would be private for most people, but public for those who have more than a certain number of pledged votes.
People can also split their delegation by subject area, so they don’t need to find a delegate whose judgement they trust on every topic. For example, A could delegate their vote on economic issues to B, their vote on foreign policy to C, and their vote on everything else to D.

I have mixed feelings about liquid democracy. There are quite a few reasons why direct democracy is a bad idea, and liquid democracy only solves some of them. In particular, the role of representatives isn’t just to cast votes, but also to design, advocate, and implement good policies, which is a full-time job. Meanwhile there needs to be some way to make all those policies consistent with each other over time - otherwise voters might approve a budget constraint, and then approve individual policies expensive enough to overrun that budget constraint. Then, once the policies have been in effect for long enough to judge their success or failure, we also need to know which groups to praise or blame - since allocating responsibility on an individual basis is hard. In short, electing representatives who belong to political parties to serve multi-year terms in parliament seems like a robustly good idea

...
Why anything that can be for-profit, should be
by jasoncrawford — 2020-04-29T20:00:09.048Z
Link: https://rootsofprogress.org/organizational-metabolism-and-the-for-profit-advantage

In a recent post I surveyed different types of funding models, including nonprofit models such as universities or private foundations, and for-profit models such as startups. Although we need both models, I believe that for-profit models are underrated today. In what follows I will explain some fundamental advantages of the for-profit model.

To understand these advantages and why they are essential, we have to understand the incentives and feedback loops that exist within the for-profit and nonprofit worlds. One framework I use to think about this is a concept I call organizational metabolism.

An organization, like a biological organism, has a metabolism: a core set of processes that keep it alive. These processes provide the resources it needs to act, to grow, and to sustain itself. For an organism, the basic resource is energy, whether from food, sunlight or chemicals. For an organization, the basic resource is money, for payroll and other expenses. Just as an animal must eat, an organization must collect revenue, or die.

Understanding an entity’s metabolism is fundamental to understanding its role within an ecosystem of competing entities and the selective pressures it is under. An entity with a successful metabolism survives and grows; one that fails in its metabolism is eliminated. Over time, through natural selection, an ecosystem becomes dominated by the entities with successful metabolism. Different entities can have different designs and make different choices, but the laws of nature decide which of them thrive.

Metabolism can be represented as a set of feedback loops. Here is the metabolism of a for-profit business:

The main loop is on the right: a business sells a product or service to its customers for a profit. To give birth to a business, investors can put in capital, which must produce a return; this is the loop on the left. Call the right-hand loop the “product loop” and the left-hand the “return loop”.

A business must be profitable over the long term to survive—but any business that makes a profit can survive. Similarly, a business must credibly promise a return to receive investment, and an investor must make a return over the long term to continue investing—but any promising investment can receive capital, and any successful investor can stay in business. And over time, resources accrue to the mos...
On “COVID-19 Superspreader Events in 28 Countries: Critical Patterns and Lessons”
by Zvi — 2020-04-29T19:40:01.006Z

Analysis of / response to: COVID-19 Superspreader Events in 28 Countries: Critical Patterns and Lessons

The article above, pointed out to me by many good sources, does one of the things we should be doing orders of magnitude more of than we are doing. It attempts to physically model Covid-19, and figure out the ways it spreads and how relatively dangerous they are. Then, based on that, it reasons out the wise policy responses and wise personal behaviors.

My analysis backs up the article’s conclusions. There are ways out, but they seem implausible.

There are three plausible vectors; that part of this article is matched by what I’ve seen everywhere.

Two of them are airborn:

According to the binary model established in the 1930s, droplets typically are classified as either (1) large globules of the Flüggian variety—arcing through the air like a tennis ball until gravity brings them down to Earth; or (2) smaller particles, less than five to 10 micrometers in diameter (roughly a 10th the width of a human hair), which drift lazily through the air as fine aerosols.

…

And it is on this crucial scale that our knowledge is thinnest. Despite the passage of four months since the first known human cases of COVID-19, our public-health officials remain committed to policies that reflect no clear understanding as to whether it is one-off ballistic droplet payloads or clouds of fine aerosols that pose the greatest risk—or even how these two modes compare to the possibility of indirect infection through contaminated surfaces (known as “fomites”).

This seems super important because we are all choosing how paranoid to be about each of these three vectors, and in what ways.

Each of the three vectors has different implications. Here’s the article’s take on them.

If large droplets are found to be a dominant mode of transmission, then the expanded use of masks and social distancing is critical, because the threat will be understood as emerging from the ballistic droplet flight connected to sneezing, coughing, and laboured breathing. We would also be urged to speak softly, avoid “coughing, blowing and sneezing,” or exhibiting any kind of agitated respiratory state in public, and angle their mouths downward when speaking.

If lingering clouds of tiny aerosol droplets are found to be a dominant mode of transmission, on the other hand, then the focus on sneeze ballistics and the precise g

...
Training Regime Day 25: Recursive Self-Improvement

by Mark Xu — 2020-04-29T18:22:03.677Z

Epistemic status: highly experimental

Introduction

I have this theory that humans have this implicit skill that's something like "doing stuff". If you're getting at "doing stuff", then you can reliably get what you want. If you're not good at "doing stuff", you often fail to get what you want, either by failing to get it or by failing to try.

Being able to "do stuff" gives you the ability to change your day-to-day behavior. For example, if you can do stuff, then you'll be able to begin an exercise routine without much difficulty. If you can do stuff, then you'll be able to consistently practice piano, if you want to learn piano. If you can do stuff, then you'll be able to consistently read books, should you desire. Being able to do stuff makes your life better.

Applied rationality is supposed to increase your ability to "do stuff". Murphyjitsu makes it easier to get what you want. TAPs make certain actions automatic. Systemization makes getting what you want closer to the default action. Goal factoring helps you access what you want more directly.

The lesson we learn from video games is that it's more efficient to grind skills directly. So how do you grind "doing stuff" directly?

Recursive Self-Improvement

Doing murphyjitsu makes you better at it, so you want to do murphyjitsu more. As the saying goes, "there's a TAP for that". You decide to make a TAP to do murphyjitsu every time you make a plan. But making a TAP is a plan - you can murphyjitsu it!

You keep forgetting rationality techniques, so you want to systemize. You list all the rationality techniques in a checklist. You think of a TAP to consult this list every time you make a decision. Making a TAP is a plan, so you murphyjitsu it.

You notice feel a bit strange, but you ignore it and move on. Then you realize you wanted to do some focusing. There's a TAP for that, triggering your meta-TAP that results in murphyjitsuing your plan to install a focusing TAP.

(All roads lead to murphyjitsu.)

On the object level, all I'm doing is pointing out that you can use rationality techniques on themselves - cute, but probably not that important.

On the meta level, I claim that this is what it takes. In order for applied rationality techniques to actually make your life better, you have to actually use them. T...