trentmkelly/LessWrong-43k huggingface dataset:
Paper review: A Cryptographic Solution to a Game Theoretic Problem
by victorsintnicolaas — 2021-04-24T11:54:20.015Z

Crossposted from: Equilibria Club

It's time for a puzzle involving two fields: game theory and cryptography! Both fields model self-interested agents in a competitive setting. In game theory, we list actions and payoffs, after which we can determine which actions will benefit agents the most. In cryptography, we model actions to ensure agents only get a limited amount of information.

Comparison of concepts between cryptography and game theory

The problem: game of chicken

To illustrate how cryptography can be used to achieve better outcomes in a game, we're going to look at the "Game of Chicken". Imagine two players driving towards each other in a car, in principle wanting to avoid a collision, but also not wanting to chicken out. The standard two-player game is characterized by the following payoffs:

Player 1 cooperates	Player 1 defects
Player 2 cooperates	(4, 4)	(1, 5)
Player 2 defects	(5, 1)	(0, 0)

"While the “wisest” pair of actions is (C, C), this is not a Nash equilibrium, since both players are willing to deviate (believing that the other player will stay at C)."

We can display the three Nash equilibria (the optimal actions for non-coordinating self-interested players) on a graph. The players will either (cooperate, defect), (defect, cooperate), or have a 50% chance of (cooperate, defect) and (defect, cooperate).

Three Nash equilibria in the extended game

In the previous post about game theory, I referred to the Folk Theorem of game theory, which says that many equilibria can be enforced by the players in iterated games. However, those enforceable equilibria all fall in the so-called convex hull of the Nash equilibria - which is any situation contained by the Nash equilibria. In the Game of Chicken, situations in the convex hull are all situations below the purple diagonal, leading to a maximum total payoff of 6. Is there any way we can do better? Well yes! We can try to achieve a correlated equilibrium ( 1/3(C, D) + 1/3(D, C) + 1/3(C, C)), giving the two players a total expected payoff of 2*(5*1/3 + 1*1/3 + 4*1/3)=6.33.

"In a Nash equilibrium, each player chooses its move independently of the other player. (Hence, the induced distribution over the pairs of moves is a product distribution.) Yet, Aumann [2] showed that in many games, the players can achieve much higher expected payoffs, while preserving the “self-enforcement” property, if their strategies are corr

...
Election results in Central Europe match some pre-WW1 borders
by ForensicOceanography — 2021-04-24T06:24:16.207Z

I have seen this image of Quora:

Polish presidential election, 2010

Election results are arguably one of the most apparent and easy to measure effect of complicated cultural diffences (provided that the country is sufficiently diverse to make the cultural differences relevant). After reading Albion's seed, I think that I understand much better the origin of some of the cultural differences in the United States. The author of Albion's seed devotes his final chapter to an analysis of the US election results until the XX century. Cultures are different through countless axes, but election results may be seen as a sort of spontaneous principal component analysis: the population spontaneously sorts itself in some political coalitions (which tend to become only two in countries with first-pass-the-post systems, and a few more in countries with proportional systems).

This looks a Prussia - Russian Empire split, rather than a city countryside split (though big cities like Warsaw vote with Prussia). Looking at the population density, the orange part does not seem much more urbanized than the blue part.

First of all, maybe this particular year is cherry-picked? Kind of: the map of other presidential elections does not correspond so neatly to the former Prussian-Russian border. But the general pattern remains.

These two regions of Poland are sometimes called Poland A and Poland B, and the developement gap between the two is widely discussed (much like the gap between northern and southern Italy). This might also be seen as a split across the Hajnal line. But the population that lives Poland A is not the same population that lived in the orange zone when it belonged to Prussia. Following the massive resettlements of the postwar period, the majority of the modern day Polish-speaking inhabitants of the Recovered Territories are not descendents of the (mostly German-speaking) inhabitants of the former Eastern Prussia.

As culture is mostly inherited, it is often hard to distinguish between inherited traits and cultural traits. To settle the issue, a possible experiment may be replacing all the inhabitants of a geographic area with a new populations, and see what happens. History has provided us with several approximations of such an experiment, and the Polish recovered territories are one. This looks to me as evidence that the economic gap between Poland ...
How to test tiny skills?
by Nicholas / Heather Kross — 2021-04-24T05:02:45.347Z

It's hard to tell how good you are at something, until you try it.

Most people don't try most things, so advice like "follow your passion" or "unlock your inner child" are drawing on a terrifyingly small pool of things you've actually tried. And our best skills may not have obvious correlations, and a person's distribution of skills could be lumpy and all-over-the-place. (The exception being IQ, which kinda helps a lot of things. But you can't "specialize" just in "tasks that are easier at this IQ level".).

Example: "gaming" seems like a specific, not-super-useful skill. But video games can be wildly different from each other. I have bad aim and awareness in FPS games, but I'm unusually good at The Wiki Game. I think this is a combination of some tiny subskills I excel at, like "bus-ticket obsession with some trivia" and "making random torturous connections between topics".

Being good at The Wiki Game isn't that valuable, but it matches some subskills. And some things that are (or seem) valuable, don't match your subskills.

So here's what I'm looking for:

Large lists of tiny, oddly-specific skills. Less "Programming", more "pattern recognition (logic)" and "problem precisification" and "interpreting what someone wants" and "efficient googling".

Free/cheap quick ways to test skills. What's the quickest thing I can do, to see if my brain easily turns sideways in that particular dimension?

Large or systematic attempts to match tiny skills with jobs, hobbies, fields, and things-you-can-do-for-others. "You're really fast at that shape task, have you considered going into surveying?"

EDIT: see also: discussion on reddit.
Apps / tools for combatting sleep procrastination?
by emily.fan — 2021-04-24T01:40:44.010Z

I've recently written about combatting sleep procrastination and was wondering if anyone has found any apps / tools similar to a spreadsheet system where we iteratively:

enumerate failure modes contributing to sleep procrastination

brainstorming strategies on how to combat sleep procrastination

References:

Sleep procrastination article: https://www.lesswrong.com/posts/KuuafiWtxG6mMtmPa/on-sleep-procrastination-going-to-bed-at-a-reasonable-hour

Spreadsheet template: https://docs.google.com/spreadsheets/d/1WqwnDj0eOdDjBnocN_1gPaeKlO--DtCzNlWhg5IyRwc/edit#gid=1670365040
Vaccination and House Rules
by jefftk — 2021-04-24T01:00:03.030Z

One of the effects of

living with
others during a pandemic is that risks you take don't just affect
yourself. While this is also true even if you live alone, you can't
really negotiate with society at large over acceptable levels of risk.
When our house decided to

lock
down a year ago,

open up a bit in
summer, and

lock down
again this winter, each needed quite a bit of discussion.

As of today, all the adults in our house have had at least one shot,
so we're starting to think about how this should change how we'll
handle risk.

We've already updated our approach a little bit: after discussion
in February we decided that
each week each of us can choose one fully-vaccinated person and spend
unrestricted indoor time with them.

Once all the adults are fully vaccinated, though, what someone does
outside of the house likely doesn't have much of an effect on risk to
the rest of us. The vaccines are very good, decreasing both your
chances of getting coronavirus and passing it on if you get it. The
risk isn't zero, but our housemates aren't the kind to go to an
anti-vax song convention, so I'm not too worried. At that point, what
you do outside of the house can go back to being your own decision.

Shared spaces are a bit different. While we haven't decided for sure
yet, I think it's probably going to end up being that you can invite
anyone in who is fully vaccinated.

There's also the question of how to handle the intermediate stage,
when some of us or our friends have had some time since an initial
shot but are not yet fully vaccinated. After a couple weeks this is
probably pretty safe, but it is hard to tell how much. Since this is
only about a month and we've been isolating for over a year we'll
probably just wait with some amount of case-by-case (one
housemate is going to see their partner when both are 2wk after their
first doses), but I'm open to arguments otherwise.

With the kids, risk is very low overall, to the point where if
everyone was affected the way children are we wouldn't have even tried
to prevent the spread. The baby will be at elevated risk, and I think
their main risk will probably be via the older kids, especially when
they're in in-person school? Part of the risk to the baby wouldn't be
from covid directly, but instead that infant fevers generally require
aggressive treatment and covid makes a fever more li...
Let's Rename Ourselves The "Metacognitive Movement"
by DirectedEvolution — 2021-04-23T21:06:58.422Z

Associations, Definitions, Prosody

The name "rationality movement," or "rationalism," has several problems. It sounds self-congratulatory. It's the name of a 17th-century anti-empirical movement. It's defined as "the quality of being based on or in accordance with reason or logic," which seems to oppose intuition, tradition, and emotion, whereas we're open to those sources of information.

Sometimes, people try to use names like "aspiring rationalist," "rationalish," or "LessWrong community." None are great. "LessWrong" community doesn't explain much to outsiders. "Aspiring rationalist" comes with the baggage of the word "rationalist." And "rationalish" fails to speak in the positive.

"Metacognition" is defined as "thinking about thinking." That's exactly what we do. It's untainted by history or politics, and intuitively seems more attractive and accessible to outsiders. It has a more humble vibe. And it's as, if not more popular than the word "rationality" on Google Trends.

One review article defines metacognition as a combination of knowledge and regulation. Knowledge is about oneself as a learner, factors affecting performance, strategies for meta-cognition, and use cases for those strategies. Regulation is about self-monitoring one's own thinking, including planning, self-awareness, and evaluation.

Mapping "rationality" words to "metacognition" words shows that metacognition is mostly either the same length, or shorter.

"Rationality" word	"Metacognition" word	Effect of switch on prosody
Rationalist	Meta-thinker	Same length.
Less aggressive, more academic.
Could be good or bad.
Rationalism, Rationality	Metacognition,
Meta-Thinking,
Meta-Thought	Same length or shorter.
Uses a common word ("think").
Rationalist Movement	Metacognitive Movement,
Meta-Thinking Movement	Longer or same length.
Creates alliteration.

One further factor is that "rationalists" can be shortened to "rats," which is nifty, though occasionally used/confused as an insult. I don't have an equivalently cute shortening for "meta-thinkers." Maybe "cogs?"

Building Links To Scientific Literature

Metacognition has an evidence base, including findings that metacognition can be trained, causes performance improvements, and is independent of IQ. One review article (below) also links metacognition to "cumulative culture," which seems to be a term for "cultural transmission of knowledge," i.e. The Secret of Our Success.

I've li...
How to Play a Support Role in Research Conversations
by johnswentworth — 2021-04-23T20:57:50.075Z

There’s a programming technique where you keep a rubber duck on your desk, and when you run into a confusing bug, you try to explain it to the duck. This turns out to be surprisingly useful for resolving bugs.

In research conversations, it is often very useful to have someone serve as an upgraded rubber duck. This is a support role: the rubber duck’s job is not to solve the problem, but to help their partner sort out their own thoughts. Playing this support role well is a skill, and if you’re good at it, you can add value to other researchers in a very wide range of contexts; you can be a general-purpose force multiplier. This post contains a handful of tips for how to add value this way, beyond what a literal rubber duck has to offer.

Figure Out The Picture

My partner is drawing a picture in their head. I want to accurately copy that picture into my head. This is the core of the technique: in order to build an accurate copy in my head, my partner will need to flesh out the picture in their own head.

For sake of discussion, let’s say the conversation is part of a

conjecture workshop

. The goal is to take some fuzzy intuition and turn it into a mathematical conjecture. When playing the support role, I’m trying to build a mathematical conjecture in my head which matches the picture in my partner’s head. By asking questions to fill out my picture of the conjecture, I push my partner to translate their fuzzy intuitions into math.

(We’ll use conjecture workshop examples throughout this post, but the ideas generalize beyond just math.)

Some useful lines:

Can you give an example?
Can you give 2-3 examples which are as different as possible?
Can you give an anti-example, i.e. a case where this would not apply?
Here’s the picture in my head so far. <explain> Does that match what’s in your head? Am I missing anything important?
Let me repeat back what I understood from what you just said. <explain> Did I miss anything?
We’ve been using the analogy of X. How does Y translate into that analogy?
It seems like you’re trying to point to X. Is that right?
Here’s a few different things which I think you might be trying to point to. <explain> Is one of these what you’re saying? Do none of them match what you’re trying to say?
Am I understanding your framing correctly?

Side note: since this all relies on building out a picture in my head, it might be useful to write down some...
The Fall of Rome, II: Energy Problems?
by LukeOnline — 2021-04-23T17:58:23.877Z

What is the core reason for our current dramatic increase in prosperity (seen as the Industrial Revolution, since 1750/1800)? The usage of new energy sources like coal and oil, to replace the physical labor of humans and animals with machines that dramatically boost our productive capacity.

Could the same have happened to our Roman ancestors? Are their technologies linked to energy usage? Let's look at some distinct Roman practices that largely disappeared after the fall of Rome and see if they're linked to the usage of large amounts of energy.

Baths

The Romans were renowned for their elobarate bathhouses. They contained large, heated water basins. The heating process required significant amounts of fuel. They seem to have fell out of use in the Medieval Period. Large spas reappeared in Europe during the end of the 18th century - at the start of the Industrial Revolution.

Concrete

It was known before Rome, but Romans improved the recipe and used it extensively. Descriptions mostly focus on the ingredients and not the fuel usage. The Romans seem to have used a mixture of quicklime, pozzolana and pumice. It has been suggested that this knowledge was lost, explaining why their Medieval descendants barely used concrete. But let's go back to Wikipedia:

After the Roman Empire, the use of burned lime and pozzolana was greatly reduced. Low kiln temperatures in the burning of lime, lack of pozzolana, and poor mixing all contributed to a decline in the quality of concrete and mortar.

Apparently, the Romans used high kiln temperatures and their descendants struggled to reach these temperatures. After many centuries where only small amounts of low-quality concrete were used, concrete reappeared as a major construction material during the... Industrial Revolution! What a coincedence. Portland cement was patented in 1824, and reinforced concrete was invented in 1849. And once again, Wikipedia:

Portland cement clinker is made by heating, in a cement kiln, a mixture of raw materials to a calcining temperature of above 600 °C (1,112 °F) and then a fusion temperature, which is about 1,450 °C (2,640 °F)

Producing concrete seems to require very high temperatures!

Armor

On the left: peak-Roman legionaries. On the right: late-Roman comitatenses.

Rome was famous for its military strength. On the left, we see the equipment of the famous Roman legiona...
The Fall of Rome: Why It's Relevant, And Why We're Mistaken
by LukeOnline — 2021-04-23T12:15:44.902Z

The standard view of the fall of the Roman Empire is horrifying and deeply troubling. A long time ago, there was a prosperous empire. They created advanced technology and complex philosophy. After existing for many centuries, violent barbarians invaded, ending the prosperity and ushering in The Dark Ages. Knowledge and technology was lost and civilization disappeared. After a thousand years of depressing mud hovels and Viking raids, the Renaissance happened - the rebirth of the classical world and the start of our modern era.

Standard view of the Fall of Rome

It's frightening, because it demonstrates that progress isn't inevitable. We are not guaranteed to continously get more wealthy, more peaceful and more civilized. Setbacks do happen, and they doom many millions of human beings to live horrible lives that could have been avoided.

As someone who did have a strong believe in 'Progress', this unsettled me. I had all kinds of questions. How do you lose knowledge and technology? How do you "uninvent" things? How did a bunch of unorganized, primitive barbarians defeat a civilization that was way more advanced than them? For many years, I studied the subject, spoke with leading experts and got access to the most recent data. Things merely became more confusing.

Lots of historians and archaeologists have made graphs presenting their findings from the Roman era. The same curve returns over and over again. Here it is:

It's extraordinary. Things start out at very low levels, nearly zero, and in the centuries before 1AD, they suddenly rise dramatically. They peak somewhere in or around the first century AD, and then decline again with the same pace. In 476, when the last Western Roman Emperor is deposed by a Germanic chieftain, little had been left of the former Roman economic activity. Europe becomes quiet once again.

It does make sense. Although classical Rome lasted from 753BC to 476AD, all the things it's famous for concentrate in the area around the peak in the graph. The Colosseum was completed in 80AD; the Pantheon was built between 113 and 125AD; Julius Ceasar died in 44BC; Pliny the Elder died in 79AD; the Pont du Gard was constructed between 40 and 60AD.

In my opinion, the standard narrative surrounding the rise and fall of ancient Rome has not properly integrated these new datasets. It barely explains the wealth of Rome, seeing it a...
Probability theory and logical induction as lenses
by Alex Flint — 2021-04-23T02:41:25.414Z

This is independent research. To make it possible for me to continue writing posts like this, please consider supporting me.

Abstract

I frame probability theory as a lens for understanding the real-world phenomenon of machines that quantify their uncertainty in their beliefs. I argue in favor of using multiple lenses to look at any phenomena, even when one of those lenses appears to be the most powerful one, in order to keep track of the boundary between the phenomena we are examining and the lens we are using to examine it. I argue that logical induction is another lens for looking at the real-world phenomena of machines that quantify their uncertainty in their beliefs, addressing the specific case of machines that are Turing-computable. I argue that tracking uncertainty in purely logical claims is merely a necessary consequence of addressing this general issue. I conclude with remarks on the importance of general theories of intelligent systems in the engineering of safety-critical AI systems.

Introduction

Last year I began studying the logical induction paper published by MIRI in 2016. I did so partially because I was unsure how to contribute within AI safety. I had been reading articles and writing posts but was at a bit of a loss as to what to do next, and I had been curious about, although intimidated by logical induction for some time, and so I felt inclined to study it. I felt a little guilty about spending time looking into something that I was merely curious about, at the expense perhaps of other projects that might be more directly helpful.

But my curiosity turned out to be well-founded. As I read through the logical induction paper, and particularly as I came back to it many days in a row, I started to see that it was actually significant in a different way than what I had expected. I had previously understood that logical induction was about having uncertainty about purely logical claims, such as putting 10% probability on the claim that the one billionth digit of pi is a 3. That is certainly in there, but as I read through the paper, I realized that logical induction paints a bigger picture than this, and that in fact the maintaining of uncertainty in logical claims is a mere consequence of this bigger picture.

The bigger picture is this: logical induction provides an account of what it means for machines to quantify their uncertainty in their b...
Is there anything that can stop AGI development in the near term?
by Wulky Wilkinsen — 2021-04-22T20:37:41.163Z

Assume that short timeline arguments are correct. (For previous discussion, see

"What if AGI is near?"

.)

Some possible ideas:

A multilateral, international governmental/legal agreement to halt GPU production or ban AI research

Surveillance systems that detect when someone is about to launch an AI system and report them to the authorities. Obviously this would be just an implementation detail of the above idea.

An agreement among prominent AI researchers (but not necessarily governments) in multiple countries that further progress is dangerous and should be halted until alignment is better understood
A global nuclear war or some other disaster that would halt economic progress and damage supply chains

Nuclear EMPs that would damage many electrical systems, possibly including computing hardware, while limiting casualties

Even in these scenarios it seems like further progress could be possible as long as at least one research group with access to sufficient hardware is able to scale up existing methods. So I'm just curious, would it even be possible to stop AI research in the near term? (This is different from asking whether it would be good to do it -- there are obviously reasons why the ideas above could be quite terrible.)

Also, should we expect that

due to anthropic/observer selection effects

, we will find ourselves living in a world where something like the dystopian scenarios discussed above happens, regardless of how unlikely it is a priori?