trentmkelly/LessWrong-43k huggingface dataset:
The Telephone Theorem: Information At A Distance Is Mediated By Deterministic Constraints
by johnswentworth — 2021-08-31T16:50:13.483Z

You know the game of Telephone? That one where a bunch of people line up, and the first person whispers some message in the second person’s ear, then the second person whispers it to the third, and so on down the line, and then at the end some ridiculous message comes out from all the errors which have compounded along the way.

This post is about modelling the whole world as a game of Telephone.

Information Not Perfectly Conserved Is Completely Lost

Information is not binary; it’s not something we either know or don’t. Information is uncertain, and that uncertainty lies on a continuum - 10% confidence is very different from 50% confidence which is very different from 99.9% confidence.

In Telephone, somebody whispers a word, and I’m not sure if it’s “dish” or “fish”. I’m uncertain, but not maximally uncertain; I’m still pretty sure it’s one of those two words, and I might even think it’s more likely one than the other. Information is lost, but it’s not completely lost.

But if you’ve played Telephone, you probably noticed that by the time the message reaches the end, information is more-or-less completely lost, unless basically-everyone managed to pass the message basically-perfectly. You might still be able to guess the length of the original message (since the length is passed on reasonably reliably), but the contents tend to be completely scrambled.

Main takeaway: when information is passed through many layers, one after another, any information not nearly-perfectly conserved through nearly-all the “messages” is lost. Unlike the single-message case, this sort of “long-range” information passing is roughly binary.

In fact, we can prove this mathematically. Here’s the rough argument:

At each step, information about the original message (as measured by

mutual information

) can only decrease (by the

Data Processing Inequality

).
But mutual information can never go below zero, so…
The mutual information is decreasing and bounded below, which means it eventually approaches some limit (assuming it was finite to start).
Once the mutual information is very close to that limit, it must decrease by very little at each step - i.e. information is almost-perfectly conserved.

This does allow a little bit of wiggle room - non-binary uncertainty can enter the picture early on. But in the long run, any information not nearly-perfectly conserved by the later steps will be lost.

Of course, t...
What could small scale disasters from AI look like?
by CharlesD — 2021-08-31T15:52:50.968Z

If an AI system or systems goes wrong in the near term and causes harm to humans in a way which is consistent with or supportive of alignment being a big deal, what might it look like?

I'm asking because I'm curious about potential fire-alarm scenarios (including things which just help to make AI risks salient to the wider public), and also looking to operationalise a forecasting question which is currently drafted as

By 2032, will we see an event precipitated by AI that causes at least 100 deaths and/or at least $1B 2021 USD in economic damage?

to allow a clear and sensible resolution.
Grokking the Intentional Stance
by jbkjr — 2021-08-31T15:49:36.699Z

Considering how much I’ve been using “the intentional stance" in my thinking about the nature of agency and goals and discussions of the matter recently, I figured it would be a good idea to, y’know, actually read what Dan Dennett originally wrote about it. While doing so, I realized that he was already considering some nuances in the subject that the Wikipedia summary of the intentional stance leaves out but that are nonetheless relevant to the issues we face when attempting to e.g. formalize the approach, or think more clearly about the nature of agency in the context of alignment. I don’t expect many LessWrongers will read the original book in full, but I do expect that some additional clarity on what exactly Dennett was claiming about the nature of agency and goals will be helpful in having less confused intuitions and discussions about the subject.

In what follows, I provide an in-depth summary of Dennett’s exposition of the intentional stance, from Chapter 2 of The Intentional Stance (“True Believers: The Intentional Strategy and Why It Works”), which Dennett considers “the flagship expression” of his position. Then, I discuss a few takeaways for thinking about agency in the context of AI safety. In brief, I think 1) we should stop talking about whether the systems we build will or won’t “be agents,” and instead debate how much it will make sense to consider a given system as “an agent,” from the information available to us, and 2) we should recognize that even our internally-experienced beliefs and desires are the result of parts of our minds “applying the intentional stance” to other parts of the mind or the mind as a whole.

This work was completed as a Summer Research Fellow at the Center on Long-Term Risk under the mentorship of Richard Ngo. Thanks to Richard, Adam Shimi, Kaj Sotala, Alex Fabbri, and Jack Auen for feedback on drafts of this post.

Summarizing Dennett's position

TLDR: There is no observer-independent “fact of the matter” of whether a system is or is not an “agent”. However, there is an objective fact of the matter about how well-modeled a particular system’s behavior is modeled by the intentional stance, from the point of view of a given observer. There are, objectively, patterns in the observable behavior of an intentional system that correspond to what we call “beliefs” and “desires”, and these patterns explain or predict the behavio...
An Apprentice Experiment in Python Programming, Part 4
by gilch — 2021-08-31T07:56:04.711Z

[Note to readers: The Jupyter notebook version of this post is here]

Previously: https://www.lesswrong.com/posts/fKTqwbGAwPNm6fyEH/an-apprentice-experiment-in-python-programming-part-3

Python Objects in Memory (from comments)

In the previous post, purge commented:

Due to Python's style of reference passing, most of these print statements will show matching id values even if you use any kind of object, not just True/False. Try to predict the output here, then run it to check:

def compare(x, y):
print(x == y, id(x) == id(y), x is y)
a = {"0": "1"}
b = {"0": "1"}
print(a == b, id(a) == id(b), a is b)
compare(a, b)
c = a
d = a
print(c == d, id(c) == id(d), c is d)
compare(c, d)

When I was coming up with an answer to this question, I got stuck on what the operator is did. I only had a vague sense of how to use it—I knew comparison with None was done via is but didn't know why—so I had to look up what is actually did.

Identity comparisons

The operators "is" and "is not" test for an object’s identity: "x is y" is true if and only if x and y are the same object. An Object’s identity is determined using the "id()" function. "x is not y" yields the inverse truth value.

Here's the doc for id():

id(obj, /)
Return the identity of an object.
This is guaranteed to be unique among simultaneously existing objects.
(CPython uses the object's memory address.)

Then I understood that is would literally check if two objects are the same object. So in the above example we'd get True False False from print(a == b, id(a) == id(b), a is b) and True True True from print(c == d, id(c) == id(d), c is d).

Object Storage in Memory

Speaking of checking if two objects being the same object stored in the same location in memory, gilch made more comments about object storage models (paraphrased):

Compared to C/C++, Python has a more consistent object storage model: everything is an object, only references to objects are stored on the stack, pointing to the actual objects stored in the heap. This means that Python objects are scattered all over the place. One important aspect of CPU optimization is caching contiguous blocks of memory in CPU caches, but Python's model cause cache-miss to be high since two objects adjacent to each other in memory are likely unrelated. This performance degradation is the price for Python's simple memory model.

For computing tasks that have high requir...
World War Zero
by lsusr — 2021-08-31T07:22:39.706Z

In the 1750s, British settlers in America wanted to trade in Ohio Country. The French didn't want them to. In 1753 the French began construction of a series of forts in the Ohio Country. Nobody stopped them because Ohio Country was de facto French territory. The French had been trading there for years and had boots on the ground.

The fact that Ohio Country was French territory didn't stop other powers from claiming it. Indians claimed the territory on the grounds they were there first. British colonies claimed it on the grounds they outnumbered the French and the Indians.

In December 1753, the royal governor of Virginia dispatched Lieutenant Colonel George Washington on a 1000-mile round-trip to order the French to abandon their posts and forts. It was an adventure. They literally went to Murdering Town.

The Day following, just after we had passed a Place called the Murdering Town, where we intended to quit the Path, and steer across the Country for Shannopins Town, we fell in with a Party of French Indians, who had lain in wait for us; one of them fired at Mr. Gist or me, not 15 Steps, but fortunately missed. We took this Fellow into Custody, and kept him till about 9 o'Clock at Night, and then let him go, and walked all the remaining Part of the Night without making any Stop, that we might get the Start so far, as to be out of the Reach of their Pursuit the next Day, as we were well assured they would follow our Track as soon as it was light: The next Day we continued travelling till quite dark, and go to the River about 2 Miles above Shannapins; we expected to have found the River frozen, but it was not, only about 50 Yards from each Shore; the Ice I suppose had broke up above, for it was driving in vast Quantities.

There was no Way for getting over but on a Raft, which we set about, with but one poor Hatcher, and got finished just after Sun setting, after a whole Day's Work; we got it launched, and on board of it, and set off; but before we were half Way over, we were jammed in the Ice in such a Manner that we expected every Moment our Raft to sink, and ourselves to perish: I put out my setting Pole to try to stop the Raft, that the Ice might pass by, when the Rapidity of the Stream threw it with so much Violence against the Pole, that it jirked me out into 10 Feet Water, but I fortunately saved myself by catching hold of one of the Raft Logs; notwithstand

...
Exercise Trade Offs
by Elizabeth — 2021-08-31T02:40:05.381Z

Update 9/2: A friend pointed out that I was ignoring the time costs of exercise, which ended up being pretty significant. See new numbers here. I then double checked the math on the microlife numbers and the news is not good.

Tl;dr: under my current conditions, outdoor exercise is slightly safer than indoor for me, but the risks of both are dwarfed by the benefits of exercise.

Recently I’ve been weighing trade offs around exercise. At the gym I’m risking covid exposure. I can reduce that by wearing a mask, at the cost of making the exercise less effective or enjoyable. I could use my friend’s outdoor gym, but it’s fire season here in California so there are prolonged periods where I don’t want to be sucking in all that unfiltered air. This is also addressable with a mask, but at the same cost. I could exercise indoors in my own home, but I do not have that much space and it gets miserable really fast. I could not exercise until conditions improve, but that has its own health costs. So I did some math.

Wikipedia says 10 minutes of exercise = 1 micromort lost (as in, you live longer). That’s obviously going to depend a lot on the type of exercise but we’ll use it.

This calculator translates time * AQI into cigarette equivalents. At 50 AQI, it takes 12 minutes to generate .01 cigarettes. I’m going to treat that as 10 minutes because exercising is slightly worse than merely existing out doors and it makes the math much easier.

Wikipedia lists an equivalent of 1.4 cigarettes = 1 micromort.

N95 masks block 95% of PM2.5 particles (which is what the AQI is based on). I couldn’t immediately find a translation of that to micromorts so let’s assume it’s linear discounting. EDIT: On Twitter Divia Eden points out that 95% assumes a perfect seal, which you probably don’t have. This isn’t material at my current air quality; I did this whole thing without including masks at all and then added them in afterwords, but when you do your own math you should include that.

That means that 10 minutes unmasked outdoor cardio, at 50 AQI = .01/1.4 = .007 micromorts, which is clearly dwarfed by the 1 micromort lost from exercise (even if you assume it’s 10x worse for me due to the existing chest congestion, and don’t give the exercise a corresponding impact bump). If I wear a mask the risk is probably below the significant figures I’m allowed. It’s so negligible compared to the benef...
How much COVID protection is gained from protecting your eyes?
by Davis_Kingsley — 2021-08-31T01:18:03.952Z

So, as I understand it there are three main spots on the body where COVID infection typically takes place -- the nose, the mouth, and the eyes. It is relatively easy to obtain a very protective mask for the nose and mouth (n95/p100), but a large majority of people seem to be using masks that do not provide eye protection. How much extra protection can be gained by protecting your eyes?
(Obviously, the degree of protection one might use can vary -- on the low end we might have "wear glasses", while then we might progress further with things like "disposable face shield over mask", "wear a full face respirator rather than nose/mouth only", and then even "full head positive pressure hood" on the very high end.)
Quote Quiz
by Gunnar_Zarncke — 2021-08-30T23:30:52.067Z

This is a fun question/quiz. Who said the following?

You are in doubt in a situation in which you should be uncertain. Do not accept anything because it is rumoured to be, because it is the traditional belief, because the majority hold to it, because it is found in textbooks, because it is the product of metaphysical argument and speculation, or after a superficial investigation of facts, or because it conforms with your inclinations, because it is authoritative or because of the prestige value of your teacher.
If anyone were to speak ill of me, my codex, or my followers do not bear any ill towards them, be upset or agitated, for if you were to be so it will only cause you harm. If on the other hand anyone were to speak well of me, my codex, or my followers, do not be overjoyed, thrilled or elated, for if so it will only be in your way of forming a correct judgement as to whether the qualities praised are real and actually found.

The original quote is somewhat altered to make it harder to google and more interesting to guess.
We Live in an Era of Unprecedented World Peace
by lsusr — 2021-08-30T22:25:33.118Z

[Edit: This data seems to underestimate the violent death tolls in state societies. I think the general thesis is still valid but the data I used in this article should be taken very skeptically.]

Beside not dying of infectious disease my favorite thing about living in the 21st century is the fact I live in an era of unprecedented world peace. For most of human history, people didn't live in civilizations. For 90% of Homo sapiens existence, everyone lived as hunter-gatherers, pastoralists, or just small-scale farmers. Primitive societies are violent.

A few of primitive societies stand out as having relatively less violence, particularly the Eskimos and the !Kung. The Eskimos live in the Arctic tundra. The !Kung live in the Kalahari desert. They have so little violence because there is so little to fight over.

Most people live in places with high population densities. Places with lots of food can support larger populations. I suspect that, prior to civilization, most people lived in the most plentiful regions like modern New Guinea. In The World Until Yesterday, Jared Diamond writes about how in New Guinea, entire tribes would commit regular genocide against each other.

The !Kung and the Eskimos were paragons of nonviolence by the standards of primitive societies. Yet the 20th century (which included the Russo-Japanese War, World War I, World War II, the Armenian Genocide, the Chinese Civil War, the Korean War, the Vietnam War, the Soviet-Afghan War and many other conflicts) had a fraction of the per capita violent deaths of the !Kung and the Eskimos.

I was surprised to discover that conflict death rate went up by an order of magnitude between the Colombian Exchange and the modern era. (The chart uses a log scale.) But even at its height, civilization's per capita violent death rate was a fraction of the most peaceful primitive societies we have numbers for.

If we lived with the violent death rate of the early 20th century then we would be fortunate. Global violence today is even lower than it was in the early 20th century.

World War Two was the biggest conflict in human history. It ended in 1945. Let's pessimistically cherry-pick our data by examining only 1947 and later. The violent death rate still goes down over time.
Alignment Research = Conceptual Alignment Research + Applied Alignment Research
by adamShimi — 2021-08-30T21:13:57.816Z

Instead of talking about alignment research, we should differentiate conceptual alignment research and applied alignment research.

I expect these two categories to be quite obvious to most people around here: conceptual alignment research includes deconfusion, for example work on deception, HCH, universality, abstraction, power-seeking, as well as work searching for approaches and solutions for the problems raised by this deconfusion; whereas applied alignment research focuses on experimentally testing these ideas and adapting already existing fields like RL and DL to be more relevant to alignment questions. Some work will definitely fall in the middle, but that’s not a problem, because the point isn’t to separate “good” from “bad” alignment research or “real” from “fake” alignment research, just to be able to frame these clusters and think about them meaningfully.

Isn’t that decomposition trivial though? It’s indeed obvious, just like the result of almost any deconfusion. And yet, committing to that framing clears the air about so many internal issues of the alignment research community and guards discussions of field-building against frustrating everyone involved.

Thanks to Connor Leahy, Logan Smith and Rob Miles for feedback on a draft.

(Note that the

AI tag

has a distinction that sounds similar but with in my opinion far worse names (and weird classification). It might make sense to change the names following this post if the distinction makes sense to enough people.)

It’s all a Common-Payoff Game

Another obvious yet fundamental point is that neither conceptual alignment research nor applied alignment research is enough by itself -- we need both. Only succeeding in conceptual alignment research would result in a very good abstract understanding of what we should do, but no means of concretely translating it in time; only succeeding in applied alignment research would result in very good ability to build and understand models that are susceptible to alignment yet no principled understanding of what to be careful of and how to steer them.

So it doesn’t make any sense to say that one is more important than the other, or that only one is “real alignment research”. Yet frustration can so easily push one to make this mistake. I personally said in multiple conversations that I was talking about “real alignment research” when talking about conceptual research, only to ...
Antidotes to Number Numbness
by Collisteru — 2021-08-30T20:17:39.077Z

Humans can’t grasp large numbers. True, when we hear “one hundred”, we might imagine ten rows of ten or a few written paragraphs. Some of the more number-savvy might hear one thousand and see half the stars in the sky. But when we reach for higher powers of ten—ten thousand, one hundred thousand, and the formidable -illions—we come up short. These numbers are beyond the reach of our intuition. Our innumeracy leads us to undervalue big issues because we can’t visualize just how big they are (scope insensitivity), making utilitarian calculations difficult.

However, humans make up for a lack of inborn numeracy with a talent for creating and manipulating mental images. This opens a shortcut where we use our natural visual capabilities to improve intuition with numbers.

Large Numbers

One remedy for number numbness is chunking. Chunking happens when we mentally represent a quantity by comparing it to a reference that’s easy to visualize, as in, "one megaton of TNT could destroy Paris, so twenty well-placed megatons will destroy a country." Writers litter comparisons like this in news and popular science articles because they help readers grasp the wide-ranging quantities of science and mathematics.

These comparisons are helpful when writers supply them, but they're no solution we can broadly apply. A single repository of these comparisons that we can memorize would help in more situations.

This is an attempt at such a repository.

Instead of trawling the internet for poorly sourced examples, we can multiply tiny units of volume by powers of ten and visualize the resulting volume. I tried two different units for this: raindrops and grains of sand. These units are useful because they never change, unlike population, and because we can easily visualize sand grains and raindrops, unlike seconds, meters, dollars, years, or breaths.

For each power of ten between one hundred and one trillion, the chart below gives the volume filled by that many grains of sand and that many raindrops. I’ve included links to images when possible to help visualization. For smaller numbers, I’ve also thrown in a few miscellaneous examples, though these become hard to find as the numbers grow.

Power of Ten	Short Scale Name	Volume Filled by That Many Grains of Sand	Volume Filled by That Many Raindrops	Miscellaneous
102	One Hundred	A tiny wisp	Two-thirds of a US teaspoon (3.2 mL)

US senators
Years between the fir

...
Causal without correlation, how?
by MoritzG — 2021-08-30T16:43:30.862Z

There is correlation without causation, but
there is also causation without correlation.
Why, when is the latter? Is there one reason or more and if so how can they be structured and by what?
If one of the observables does not change, because there is a controlling observer (prediction+feedback), there is no way to establish correlation.
I am displeased by bayesian probability combined with graphs (DAG), it so obviously lacks the nonlinear activation function.
If two random binary streams feed into a XOR gate, the output is uncorrelated with anyone of the streams even though there is plenty of change to observe and perfect causality.