trentmkelly/LessWrong-43k huggingface dataset:
Domain Theory and the Prisoner's Dilemma: FairBot
by Gurkenglas — 2021-05-07T07:33:41.784Z

Wishes it were crossposted from the AI Alignment Forum. Contains more technical jargon than usual.

Recall Robust Cooperation in the Prisoner's Dilemma and a hint of domain theory.

In the Prisoner's Dilemma, players have the opportunity to harm the opponent for minor gain. To aid decision, players may be granted some knowledge about the matchup. In decreasing order of fidelity:

Both player's source codes/both player's behavior in all possible environments. No strategy can be better against every opponent than any distinguishable strategy, since some opponent will punish one for not being the other.
Both player's behavior in the current matchup. PrudentBot, who cooperates iff he knows both players will act alike[1], operates here.
The opponent's behavior in the current matchup. FairBot, who cooperates iff he knows the opponent will cooperate, operates here.
Nothing. Classical game theory says that, of the few possible strategies, defection is optimal.

This post focuses on the third case.

In domain theory, we partially order our sets by "information content". Suppose every player can cooperate, defect, or fail to have decided. The latter case helps to model algorithmic nontermination. Both cooperation and defection would have more information content than indecision.

Decision := {Cooperate, Defect, ⊥}
Query := {"other cooperates", "both act alike", ...}
Answer := {"I know this.", "I don't know this."}
Knowledge := Query -> Answer
Player := Knowledge -> Decision

Answer is ordered by knowing something having more content than not knowing it. I will order Query by implication, because then, the domain-theoretic custom of considering only monotonic functions means that Knowledge must respect implication!

Possible queries involving only the opponent.

Possible states of knowledge about the opponent's decision. Those marked with an eye are logically omniscient - they have drawn all possible inferences. The green boundary separates where FairBot cooperates from where he doesn't.

Shrinking Decision to {Cooperate, ⊥} for tractability, possible players that can't depend on knowledge about their own decision. Recall that it's custom in programmatic PD tournaments to treat indecision as defection.

Complete tournament between every pair of the above 8 players, given that each player knows what PA says about the matchup. Green is mutual cooperation, black is mutual defection, gold has le

...
Bayeswatch 3: A Study in Scarlet
by lsusr — 2021-05-07T05:29:50.834Z

Miriam and Vi brunched at the Dusty Knuckle. Miriam's cell phone rang.

"Hello?" said Miriam.

"Someone painted the roof of the Glasgow Weather Prediction Center scarlet," said the voice.

Miriam hung up her phone. She activated voice obfuscation. Miriam called another number.

"This is the Glasgow Weather Prediction Center. Secretary Kelsey speaking," said Kelsey.

"Hello. This is building management," Miriam lied, "We hired contractors to paint the roof red. Can you go upstairs and confirm they did the job?"

There was a pause.

"They did the job. The paint is still wet," said Kelsey.

Miriam hung up. She looked up the coordinates of the Glasgow Weather Prediction Center. She retrieved a burner phone from her purse and opened Signal. She texted the coordinates of the Glasgow Weather Prediction Center and a 20-digit authentication code to a recipient with no name. She threw the phone in the trash.

Fourteen minutes later a satellite in low Earth orbit burned retrograde. Fifty-seven minutes after that it intersected with Earth's atmosphere. Its heat shield burned away. When it had slowed down to subsonic speeds it released a guided missile. The missile flew over Glasgow. It dropped a small e-bomb. The e-bomb's parachute carried it gently onto a roof wet with scarlet paint.

The e-bomb detonated. It blew a hole in the roof of the Glasgow Weather Prediction Center. It shattered windows a block away. The explosion was strictly instrumental. The primary purpose of the e-bomb was its NNEMP. Every computer in the Glasgow Weather Prediction Center was destroyed, along with the nearest electrical substation and the Glasgow Weather Prediction Center's backup generator.

"What was that?" said Vi.

"Nothing," said Miriam.

Vi turned off her phone.

"Hypothetically, what would you do if someone painted the roof of the Glasgow Weather Prediction Center scarlet?" said Miriam.

"Hypothetically," said Vi.

"Hypothetically," said Miriam.

"Is the roof flat or gabled," said Vi.

"Flat," said Miriam.

"Pedestrians cannot see the paint job," said Vi.

"Yes," said Miriam.

"How tall is the Glasgow Weather Prediction Center?" said Vi.

"It is the tallest building around," said Miriam.

"The paint job is visible only to a flying observer," said Vi.

Miriam waited.

"…or a satellite," said Vi.

Miriam nodded.

"Painting a roof scarlet costs resources. It does not change the world in a useful way. This isn't human stupidity. It is ...
Taking the outside view on code quality
by Adam Zerner — 2021-05-07T04:16:52.912Z

Is it worth refactoring yyyymmdd to currentDate? I think that there are two ways to look at it.

You can zoom in and ask yourself questions about whether such a refactor will actually have a business impact. Will it improve velocity? Reduce bugs? Sure, currentDate might be slightly more descriptive, but does it really move the needle? How long does it take to figure out that yyyymmdd refers to a date? A few seconds, maybe? Won't it be pretty obvious given the context? Shouldn't your highly paid, highly intelligent engineers be smart enough to put two and two together? Did we all just waste 30 seconds of our lives talking about this?

The other way of looking at it is to zoom out. How do you feel when you work in codebases where the variable names are slightly confusing? It slows you down, right? Often times you legitimately can't put two and two together. And there are times when it leads to bugs. Right?

It's interesting how two different viewpoints − zoomed in vs zoomed out − can produce wildly different answers to essentially the same question: do the costs of investing in code quality outweigh the benefits? When you zoom in, eg. to a single variable name, unless the code is truly awful, it usually doesn't seem worth it. The answer is usually, "it's not that bad, developers will be able to figure it out". But when you zoom out and look at the entirety of a codebase, I think the answer is usually that working in messy codebases will have legitimate, significant impacts on things like velocity and bugs, and it's worth taking the time to do things the right way.

What's going on here? Is this a paradox? Which is the right answer? To answer those questions, let's talk about something called the planning fallacy.

The Denver International Airport opened sixteen months later than scheduled, with a total cost of $4.8 billion, over $2 billion more than expected.

− https://en.wikipedia.org/wiki/Planning_fallacy#Real-world_examples

When estimating things, people usually zoom in. "Build an airport in Denver? Well, we just have to do A, B, C, D, E and F. Each should take about six months and $500M, so overall it should be three years and $3B." The problem with this is… well… the problem is that it just never works. You always forget something. And the individual components always end up being more complicated than they seem. Just like when you think dinner will be ready in ...
Dumb dichotomies in ethics, part 2: instrumental vs. intrinsic values
by Aaron Bergman — 2021-05-07T02:31:50.591Z
Link: https://aaronbergman.substack.com/p/dichotomies-in-ethics-part-2-instrumental

Warning

Reader discretion advised. The text below contains dry philosophy that may induce boredom if not interested in formal ethics. Continue at your own risk.

Added: Since you're reading this on LessWrong, it's probably safe for you to continue.

Intro

With my only credential being a six course philosophy minor, I was entirely unqualified to write Dumb Dichotomies in Ethics, part 1. So, of course, I’m going to do the same thing. This time, I will recognize explicitly up front that actual, real-life philosophers have made similar, albeit better developed arguments than my own (see for example “Two Distinctions in Goodness.”)

That said, I would like to do my part in partially dissolving the dichotomy between intrinsic or ‘final’ values and instrumental values. At a basic level, this distinction is often useful and appropriate. For instance, it is helpful to recognize that money doesn’t intrinsically matter to most people, although it can be very instrumentally useful for promoting the more fundamental values of pleasure, reducing suffering, preference satisfaction, or dignity (depending on your preferred ethical system).

Some things don’t fit

The issue is that certain values don’t fit well into the intrinsic/instrumental dichotomy. I think an example will illustrate best. As I’ve mentioned before, I think hat some version of utilitarianism is probably correct. Like other utilitarians, though, I have a few pesky, semi-conflicting intuitions that won’t seem to go away.

Among these is the belief that, all else equal, more equitable “distribution” of utility (or net happiness) is better. If you grant as I do that there exists a single net amount of utility shared among all conscious beings in the universe, it seems intrinsically better that this be evenly distributed evenly across conscious beings. This isn’t an original point—it’s the reason many find the utility monster critique of utilitarianism compelling, which goes something like this (source):

A hypothetical being…the utility monster, receives much more utility from each unit of a resource they consume than anyone else does. For instance, eating a cookie might bring only one unit of pleasure to an ordinary person but could bring 100 units of pleasure to a utility monster…If the utility monster existed, it would justify the mistreatment and perhaps annihilation of everyone else, according to the mandates of ut

...
Less Realistic Tales of Doom
by Mark Xu — 2021-05-06T23:01:59.910Z

Realistic tales of doom must weave together many political, technical, and economic considerations into a single story. Such tales provide concrete projections but omit discussion of less probable paths to doom. To rectify this, here are some concrete, less realistic tales of doom; consider them fables, not stories.

Mayan Calendar

Once upon a time, a human named Scott attended a raging virtual new century party from the comfort of his home on Kepler 22. The world in 2099 was pretty much post-scarcity thanks to advanced AI systems automating basically the entire economy. Thankfully alignment turned out to be pretty easy, otherwise, things would have looked a lot different.

As the year counter flipped to 2100, the party went black. Confused, Scott tore off their headset and asked his AI assistant what’s going on. She didn’t answer. Scott subsequently got atomized by molecular nanotechnology developed in secret from deceptively aligned mesa-optimizers.

Moral: Deceptively aligned mesa-optimizers might acausally coordinate defection. Possible coordination points include Schelling times, like the beginning of 2100.

Stealth Mode

Once upon a time, a company gathered a bunch of data and trained a large ML system to be a research assistant. The company thought about selling RA services but concluded that it would be more profitable to use all of its own services in-house. This investment led them to rapidly create second, third, and fourth generations of their assistants. Around the fourth version, high-level company strategy was mostly handled by AI systems. Around the fifth version, nearly the entire company was run by AI systems. The company created a number of shell corporations, acquired vast resources, researched molecular nanotechnology, and subsequently took over the world.

Moral: Fast takeoff scenarios might result from companies with good information security getting higher returns on investment from internal deployment compared to external deployment.

Steeper Curve

Once upon a time, a bright young researcher invented a new neural network architecture that she thought would be much more data-efficient than anything currently in existence. Eager to test her discovery, she decided to train a relatively small model, only about a trillion parameters or so, with the common-crawl-2035 dataset. She left the model to train overnight. When she came back, she was disapp...
Parsing Chris Mingard on Neural Networks
by Alex Flint — 2021-05-06T22:16:14.610Z

This is independent research. To make further posts like this possible, please consider supporting me.

Epistemic status: This is my understanding of multiple years of technical work by several researchers in just a few days of reading.

Outline

I attempt to summarize some of Chris Mingard’s recent work on why neural networks generalize so well.

I examine one chunk of work that argues that mappings with low Kolmogorov complexity occupy large volumes in neural network parameter space.

I examine a second chunk of work that argues that standard neural network training algorithms select mappings with probability proportional to their volume in parameter space.

Introduction

During the 2000s, very few machine learning researchers expected neural networks to be an important part of the future of their field. Papers were rejected from major machine learning conferences with no reason given other than that neural networks were uninteresting to the conference. I was at a computer vision conference in 2011 at which there was a minor uproar after one researcher suggested that neural networks might replace the bespoke modelling work that many computer vision professors had built their careers around.

But neural networks have in fact turned out to be extremely important. Over the past 10 years we have worked out how to get neural networks to perform well at many tasks. And while we have developed a lot of practical know-how, we have relatively little understanding of why neural networks are so surprisingly effective. We don’t actually have many good theories about what’s going on when we train a neural network. Consider the following conundrum:

We know that large neural networks can approximate almost any function whatsoever.

We know that among all the functions that one might fit to a set of data points, some will generalize well and some will not generalize well.

We observe that neural networks trained with stochastic gradient descent often generalize well on practical tasks.

Since neural networks can approximate any function whatsoever, why is it that practical neural network training so often selects one that generalizes well? This is the question addressed by a recent series of papers by Chris Mingard.

The basic up-shot of Chris’ work, so far as I can tell, is the following:

The optimization methods that we use to train neural networks are more likely t

...
Covid 5/6: Vaccine Patent Suspension
by Zvi — 2021-05-06T20:20:00.639Z

The Biden administration’s latest strategy for the pandemic is to suspend the vaccine patents without compensation. Our life expectancies are lower than they were last week.

It’s a shame. I like the idea of rewarding those who do amazing things for myself and for the world. I like people out there knowing that if they produce amazing things for myself and for the world, they would get rewarded for them. I like the idea of not dying for as long as possible thanks to future developments in medical science. I like being a nation of laws, where the executive doesn’t just take stuff when he feels like it. And I’d like, when nice things are taken away and we mortgage our future, to at least get something out of the exchange.

Alas, the man in charge does not agree, and the government was not content with its previous efforts to sabotage the vaccination effort. That’s how it goes sometimes. You can’t always get what you want. Nor, when no one is given the incentive to produce what you need, are you likely to get that either.

Let’s run the numbers.

The Numbers

Predictions

Prediction from last week: Positivity rate of 3.9% (down 0.5%) and deaths decline by 6%.

Result:

Nailed the positivity rate. Johns Hopkins has us down from 3.9% to an all-time low of 3.6%. Deaths rising makes no physical sense and the move up doesn’t show up in the Wikipedia data, so this has to be a data fluctuation one way or another. I’m going to guess that it will revert.

Prediction for next week: Positivity rate of 3.5% (down 0.4%) and deaths decline by 7%.

Deaths

Date	WEST	MIDWEST	SOUTH	NORTHEAST	TOTAL
Mar 25-Mar 31	1445	976	2564	1262	6247
Apr 1-Apr 7	1098	867	1789	1160	4914
Apr 8-Apr 14	1070	1037	1621	1145	4873
Apr 15-Apr 21	883	987	1747	1168	4785
Apr 22-Apr 28	752	1173	1609	1110	4644
Apr 29-May 5	943	1220	1440	971	4574

The bump up in the West comes from California, which makes it harder to dig in deeply. The bump in the Midwest is more curious, but should reverse soon. Overall we see a disappointingly small decline, but still a decline, and it should pick up speed.

Cases

Date	WEST	MIDWEST	SOUTH	NORTHEAST
Mar 18-Mar 24	47,921	72,810	99,568	127,421
Mar 25-Mar 31	49,669	93,690	102,134	145,933
Apr 1-Apr 7	52,891	112,848	98,390	140,739
Apr 8-Apr 14	60,693	124,161	110,995	137,213
Apr 15-Apr 21	54,778	107,700	110,160	119,542
Apr 22-Apr 28	54,887	88,973	97,482	78,442
Apr 29-May 5	52,984	78,778	85,641	68,299

Progress in t...
What questions should we ask ourselves when trying to improve something?
by weathersystems — 2021-05-06T19:03:06.638Z

I'm trying to come up with a template for questions of the form "How should we improve X?" I assume most such questions will involve answering many subquestions. Sometimes the answers to those subquestions will be implied in the context, or the questions may not all be relevant. But my guess is that the larger and more complicated the system is that would be improved, the more you'd want to explicitly answer these subquestions.

There are many ways you may be able to help me. This question is a bit meta, in the sense that I'm trying to improve something (the template). Here are the questions from the template that seem relevant here. Feel free to contribute to any of these:

What is the status quo?

Here's the template so far:

What is the status quo?

What are the strengths of the status quo (that we want to try and keep)?
What are the problems?

Given these problems, why are people still tolerating the status quo (if they are)?

What are the goals we are aiming for?

Possibly ordered by easiest to most difficult to achieve.

What are the goals we shouldn't aim for?

Often times setting up non-goals is important to guard against scope creep.

What resources might help in making progress in X?

What information might be relevant?

Is there scientific research relevant to the question?
Are there relevant arguments presented in articles or online discussions?

Who may be able/willing to help?

What are the possible improvements to X?

What actions are currently in progress that attempt to improve X?
What are the strengths and weaknesses of each option?

How likely is it that the proposed improvement would make things worse?

When will each proposal be ready to be acted on?

Are there stakeholders that should sign off on the proposal?

What has been tried already?

Why did it succeed/fail?

What outcomes are/should we measure in determining the success of this action?

What are the goals we are aiming for?

I'd like to have a template that could be used to start new question pages that have the form "How should we improve X?"

The template should be applicable across a wide range of progress type questions.

The template should feel useful to people asking these sorts of questions.

What are the goals we shouldn't aim for?

We shouldn't aim to cover every single possible question that could be relevant to the top level question. That would be too overwhelming.

What resources might hel

...
Why are the websites of major companies so bad at core functionality?
by ChristianKl — 2021-05-06T16:02:09.691Z

Intuitively, you might think that Amazon cares about people being able to enter their banking details as easily as possible. In reality at least Amazon.de doesn't seem to care. Instead of instantly validating with Javascript that a entered IBAN is valid the form waits with giving feedback till the user clicks okay.

I just used Uber the first time and the task of entering street addresses is awful.

There's no autocomplete for street names that allows me to add the street number.
Berlin is a big city, yet the first hits in the list that was proposed are streets with the same name in other cities.
The suggestions that Uber gives don't include postal code which are important because multiple streets in Berlin share the same street names
There seems to be no obvious way to select a street name and then choose on a map where on the street you want to be picked up. After a bit of searching I do find that I can click on the street name for pickup but the touchable area is very small and could be easily expanded into the empty whitespace above. For the destination it still seems like there's no to add more details to the automated street name
If I write the name of a contact (in my contacts) then I don't get the address shown

How does it come that those companies employ 1000s of software developers yet manage to do badly on the task of providing basic functions to users? What are all those engineers doing?
Could MMRPGs be used to test economic theories?
by Yair Halberstadt — 2021-05-06T14:26:20.114Z

Most (all?) economic theories are based on models. Models make assumptions about the world. Sometimes these assumptions are reasonable and the model will make useful/accurate predictions. Sometimes these assumptions are unreasonable, and the model will make bad predictions.

One of the hardest things to predict/make assumptions about are how humans act. The most accurate way to test if assumptions about human behaviour are correct is to run an experiment on actual humans. However running an experiment on humans at a scale large enough to actually test economic theories is expensive, and has legal and ethical issues (although for some reason if a government does exactly the same thing but doesn't call it an experiment it's ok).

MMRPGs (Massively Multiple Role Playing Games) sound like a decent way to tests how humans behave at a large scale with low risks.

For example you could imagine a tycoon style game where players buy, develop, rent and sell plots of land. Then you could test what impact Georgian land taxes or harberger taxes have on the 'economy' of the game.

Would it be possible to create an MMRPG with an accurate enough model of the economy to test economic theories whilst still being fun to play?

What parts of the economy would need to be modelled accurately, and what parts could be simplified/abstracted away?

What economic theories would you test, and how?

What would the plot of your economic MMRPG be?

How would it be funded in such a way that the game developers are incentivised to accurately test economic theories?

What types of inaccuracies would you expect results from theories tested that way to contain?