trentmkelly/LessWrong-43k huggingface dataset:
Range and Forecasting Accuracy
by niplav — 2022-05-27T18:47:44.315Z

cross-posted from niplav.site

This text looks at the accuracy of forecasts in
relation to the time between forecast and resolution, and
asks three questions: First; is the accuracy higher between
forecasts; Second; is the accuracy higher
between questions; Third; is the accuracy
higher within questions? These questions are
analyzed using data from PredictionBook
and Metaculus, the answers turn
out to be yes, unclear and yes for Metaculus data; and no, no and yes
for PredictionBook data. Possible reasons are discussed. I also try to
find out how far humans can look into the future, leading to various
different results.

Range and Forecasting Accuracy

Above all, don’t ask what to believe—ask what to anticipate. Every
question of belief should flow from a question of anticipation, and that
question of anticipation should be the center of the inquiry. Every guess
of belief should begin by flowing to a specific guess of anticipation,
and should continue to pay rent in future anticipations. If a belief
turns deadbeat, evict it.

— Eliezer Yudkowsky, “Making Beliefs Pay Rent (in Anticipated Experiences)“, 2007

Judgmental
forecasting
in which humans aggregate both qualitative and quantitative
methods
to make predictions, and become better at doing so, is a comparatively
simple idea. Basically, one needs to have only very few tools at one's
disposal to being ready to start forecasting:

View of belief as probabilistic (perhaps with some bayesian epistemology)

Track records (grading results of forecasts using for example brier scores or log scores)

Probability theory (a concept of probabilities, and maybe some simple probability distributions)

Since the 1980s, forecasting has slowly but surely matured from "X is
going to happen because divine revelation told me so" to "my probability
distribution on the outcome of this random variable is an X distribution
with the following parameters", or alternatively "I assign a probability
of X% to this event".

However, since this kind of forecasting is relatively recent, information
about the accuracy of long-range forecasting is basically non-existent:

Long-range forecasts are often stated
too imprecisely to be judged for accuracy.
More

Even if a forecast is stated precisely, it might be difficult to
find the information needed to check the forecast for accuracy.
More

Degrees of confidence for long-range
forecasts are rarely qua

...
Croesus, Cerberus, and the magpies: a gentle introduction to Eliciting Latent Knowledge
by Alexandre Variengien — 2022-05-27T17:58:54.666Z

The article is presented as a fiction stripped out of technical jargon, yet I tried to make it conceptually faithful to

ARC’s first technical report

. I present what is latent knowledge, why eliciting it is a problem, and the type of approaches taken to tackle this open problem.

This is my submission to the

AIMS Distillation Contest

.

Croesus is rich, this goes without saying. The king accumulated so much gold that he cannot hope to get more. The only thing he wants now is to keep what he already amassed. So, he decided to put all his wealth on a remote island that only he knows about. Thieves are not a threat here, but to his disappointment, the island is populated by magpies. Even after moving the pile of gold into a deep cave, these nasty birds regularly find a hole to steal some pieces of the shiny metal.

Croesus, his treasure, and one of his coins stolen by a magpie.

It seems hopeless to perfectly seal the cave. To keep constant control of his wealth, he hired a group of mercenaries to secure the cave. However, he’s also aware of the temptation that will haunt these poor men facing a mountain of gold. To be sure that nobody will collect more than their salary, he also hired a reporter that will have to blend in with the group and report to Croesus what is happening in the group of mercenaries.

Croesus is proud of his idea, but then he thinks a bit more. How can he also be sure that the reporter will tell the truth? He could also be part of a plot to steal gold and would have no interest in reporting this. Since the reporter knows what Croesus wants to hear, the king can only expect to listen to pleasing stories. True or false, he will never have the chance to know.

He could interrogate every mercenary, every day. He could increase their reward, but it will never match what they could steal. He will never be sure about what they report. Nonetheless, the fact that there exists, or not, a plot to withdraw nuggets of gold from the pile, has to be present in the head of at least one of them.

How to train the mercenaries, and how to choose their salary scheme such that the soldiers will behave honestly without relying on their good faith?

Here, Croesus wants to discover thoughts hidden in mercenaries' heads. In the next parts we'll focus on a more general, and weirder case: elicit...
Deconfusing Landauer's Principle
by EuanMcLean — 2022-05-27T17:58:23.353Z

I’m going to assume knowledge of the basics of information theory. I won’t assume any physics knowledge (but you’ll probably find it easier if you know physics).

Thanks to Francis Priestland, Anson Ho, Nico Macé, Dan Hatton and Kenneth Freeman for their comments.

This was the result of me being nerd-sniped by a friend who wanted to talk to me about the connection between cognition and thermodynamics. I thought “yea, I know thermodynamics, it should take me a nice afternoon or two of straightening things out in my head to come up with something interesting to say”. How naive of me. In the process, I fell down a number of holes of confusion. I realised that properly understanding Landauer’s principle would be a good waypoint on my quest. This is the result of my clearing up these confusions.

Landauer’s principle

is often summarised as something like:

1 bit of memory in a register is erased, destroying 1 bit of information. Because information is entropy(?!), this means the memory register loses 1 bit of entropy. Total entropy always increases over time, so the environment must increase its entropy by at least 1 bit to compensate, resulting in a corresponding minimum amount of heat dissipation, along with a minimum energy requirement for deleting the bit.

But whoa hey hold your horses there; what does entropy really mean here? Is the

Shannon entropy

of a logic gate really connected to its

thermodynamic entropy

, and if so, in what way? I always found Landauer’s principle a bit mysterious because it seems, at first glance, to connect something we tend to consider as a subjective construct (a probability distribution) with a measurable physical outcome (the release of heat). And I’m clearly not alone, there have been a

bunch

of

people

coming

out

and saying that the principle is widely misunderstood or even that the principle itself is wrong (but don’t panic, it’s been

pretty well

experimentally

verified

). So fleshing out what’s really going on here was helpful for me to understand more intuitively what computation (and by extension cognition maybe?) means from a physics point of view. The central insight is that, as Landauer put it; “information is physical”.

The relationship between Shannon entropy and thermodynamic entropy

Thermodynamic entropy is a pretty objective property of some physical system. It’s related t...
On The Spectrum, On The Guest List: (ii) LAVO
by party girl — 2022-05-27T16:45:17.205Z
Link: https://onthespectrumontheguestlist.substack.com/p/lavo?s=w

(See also: Part One and the Prologue)

"I found, however, that it takes considerable coordinated effort to mobilize people into what looks like a spontaneous waste of money..."

Hey do you wanna join us at the lavo nightclub tonight? We have an 11:30pm walk-in let me know brightens my lockscreen while I’m at work the following Thursday. I let it sit for hours, then come back with a lame tiebreaker: is bird poop really good luck?

The widespread belief of bird droppings as good luck is based on how uncommon it is to be pooped on by one. With countless birds in the sky and numerous people, getting pooped on by one is extremely unlikely—in fact, it is said to be even more unlikely than winning the lottery ticket, but the bird chose you.

The bird chose me. hey yeah sure i can make it! I scroll up, past the previous Thursday’s coordinations, past several times he asked and I was busy, until I rubber-band at the top:

I swipe out and left and peck my Uber app, scheduling in advance sometimes saves me a few dollars, but I'm in "Uber jail," I heard someone call it that, so I take the goddamn selfie to verify I'm masked up for my ride eight hours from now, cursing whichever of R’s former teammates is doing well at Uber. My officemates probably think I’m some closet hashtag-grindset influencer now.

Which reminds me. Later at home, I scoop out my banish pile from the back corner of my closet and sift through it, until I find an itty-bitty black shirtdress, about two-hundred dollars' worth of polyester and markup:

Models, especially, were considered so beautiful that they could wear a basic dress out and still be considered top quality.

During summer the dress was my go-to for first dates, until I almost had to tear myself out of it after spending ten claustrophobic minutes tangled up in this sort of Mobius loop-fold it does at the waist. But it’s short, so short they don’t even bother with the -dress. And I'm tall enough people will argue with me about what sports I played in high school, so the combo should turn heads.

Or rather, not V’s. I grab a fuzzy, floral Jacquard coat/shaw/robe thing I just bought, I can wrap it around me for instant modesty. And some heels, and a purse with a little puffball charm I hope is J.Crew enough to not attract any thieving hands. As I throw things together I nurse a shot of whiskey with my name on it—my real name, if it does exist Partygirl Whiske...
Can growth continue?
by jasoncrawford — 2022-05-27T16:19:33.374Z
Link: https://rootsofprogress.org/can-growth-continue-ignite-talk

I gave a five-minute talk talk for Ignite Long Now 2022. It’s a simplified and very condensed treatment of a complex topic. The video is online; below is a transcript with selected visuals and added links.

Humanity has had a pretty good run so far. In the last two hundred years, world GDP per capita has increased by almost fourteen times:

Our World in Data

But “past performance may not be indicative of future results.” Can growth continue?

One argument against long-term growth is that we will run out of resources. Malthus worried about running out of farm land, Jevons warned that Britain would run out of coal, and Hubbert called Peak Oil.

Fears of shortages lead to fears of “overpopulation”. If resources are static, then they have to be divided into smaller and smaller shares for more and more people. In the 1960s, this led to dire predictions of famine and depredation:

But predictions of catastrophic shortages virtually never come true. Agricultural productivity has grown faster than Malthus realized was possible. And oil production, after a temporary decline, recently hit at an all-time high:

Wikimedia / Plazak

One reason is that predictions of shortages are based on conservative estimates from only proven reserves. Another is that when a resource really is running out, we transition off of it—as, in the 1800s, we switched our lighting from whale oil to kerosene.

But the deeper reason is that there’s really no such thing as a natural resource. All resources are artificial. They are a product of technology. And economic growth is ultimately driven, not by material resources, but by ideas.

In the 20th century, the crucial role of ideas was confirmed by formal economic models. The economist Robert Solow was studying how output per worker increases as we accumulate capital, such as machines and factories. There are diminishing returns to capital accumulation alone. A single worker who is given two machines can’t be twice as productive. So if technology is static, output per worker soon stops growing:

Marginal Revolution University

But technology acts as a multiplier on productivity. This makes each worker more productive, and creates more headroom for capital accumulation:

Marginal Revolution University

So we can have economic growth if, and only if, we have technological progress.

How does this happen? Another economist, Paul Romer, pointed out the key feature of ideas: In ...
An academic journal is just a Twitter feed
by David Hugh-Jones — 2022-05-27T11:14:13.555Z
Link: https://wyclif.substack.com/p/a-journal-is-just-a-twitter-feed?s=w

This was inspired by a tweet exchange with Michel Nivard:

There’s a lot of thinking going on about how to rework academic journals and the peer review process. It’s easy to succumb to featuritis. Can we automate publication using a pre-analysis plan? We need to be on the blockchain! Maybe we should build a special web platform for double-blind peer review?

Look at things differently. Here is a proposal for the minimum viable format for an academic journal.

I’ll talk in terms of Twitter. If you prefer, substitute your favourite open/decentralized alternative, like Mastodon. Twitter is just a concrete example.

A journal is just a Twitter feed

Journal article publications are tweets tagged #article. They contain at most four things:

Hashtags, including the #article hashtag.
The article title and authors, as text, in the form ‘A journal is just a Twitter feed’, by David Hugh-Jones (@hughjonesd). Arbitrary contact details go in brackets after author names. If you run out of characters, it’s fine to write “et al.”
A link to the article itself.
Optionally, a hash of the article’s file content, e.g. SHA256:8c53730… This lets people check that the linked article hasn’t changed.

Subsequent tweets may discuss the article, add an abstract etc. But it’s a published article if and only if it has the #article tag. This leaves journal editors free to tweet about other stuff, advertise forthcoming special issues, forward cat pictures, et cetera.

It’s up to the journal where to store the linked articles. Most journals will want to provide a relatively durable URL. There are many options, such as Github. Or you can provide a URL containing a DOI. But you really can start a journal out of a Dropbox folder if you want.

Comments, updates and peer review

Post-publication comments in the journal are replies to the article tweet, also in the article format. Comments in a different journal may also be replies to the tweet, again in the article format. Of course, anyone may comment informally.

Updated articles are replies to the article tweet, and have the same title and authors as the original, along with #article and #updated hashtags. Withdrawn articles are replies to the article tweet with the #withdrawn hashtag and an optional explanation.

Journals can also publish drafts. These take the same format as articles, but with an #article-draft tag instead of #article.

Journals can organize peer revie...
Grabby Animals: Observation-selection effects favor the hypothesis that UAP are animals which consist of the “field-matter”: 
by avturchin — 2022-05-27T09:27:36.370Z

TL;DR: Life based on exotic matter could start space travel before reaching general intelligence and it will colonise a large part of the universe without creating visible transformative changes.

Epistemic status: I don’t actually believe in this, but think that this line of thought has its right to exist and is at least as reasonable as other theories about UAP like mass hoaxes, military experiments and the “ET hypothesis.”

By “UAP = animals” I mean that: a) the observed objects are “bodies”, not technological instruments, b) their intelligence is limited and doesn’t have a human level of intelligence or superintelligence, though they could be above humans in some tasks, like fly control and mimicry; + they don’t create complex texts or instruments, с) they are not evolving and not accumulating cultural knowledge, but act based on some drives and according to the current situation. The closest thing on Earth is dolphins.

Assumptions

1. Stable objects consisting entirely of fields are possible. For the start, we assume that some combination of electromagnetic fields is relatively stable (not dissipating) in the absence of atomic matter, at least if it has an external energy source. Maybe such field combinations are knots in the magnetic field – but it is not the central point; other exotic fields will suffice (gravitational, dark matter or dark energy – you name it, but electromagnetic nature seems the closest fit). It has to satisfy the instant acceleration condition discussed below. I will call this state of matter – “field-matter”.

2. “Field-matter” could form complex objects and self-replicating life, “field-animals”. We also assume that such a stable combination of fields can interact with each other and thus it can perform computations and form structures of any complexity, which may look similar to ordinary matter, if condensed, but could be made less condensed and move through ordinary matter (this explains the ability of UAP to move through the air without creating a sonic boom as well as to move underwater.)

3. Instant acceleration property. We also suggest that the filed matter has a special property: an instant non-destructive acceleration is possible. This is because the field acts on the whole “mass” of the object, which is very small, and thus almost without inertia. This property enables quick space col...
Iterated Distillation-Amplification, Gato, and Proto-AGI [Re-Explained]
by Gabe M — 2022-05-27T05:42:41.735Z

Note: This is a joint distillation of both

Iterated Distillation and Amplification

by Ajeya Cotra (summarizing Paul Christiano) and

A Generalist Agent

by DeepMind.

Audience: New to alignment. Mostly non-technical, but a basic ML understanding helps.

Amplify, Distill, Rinse, Repeat

Suppose you want to use a friend to help you write really great novels. Ideally, you could give them some simple prompt like “write an epic fantasy book where

ingesting metals

gives you magical powers,” and they would write you a book better than any professional author. But your friend isn’t a professional author. They aren’t even a decent author, and when you ask them to write you this book, they don’t give you much better than “A girl ate some gold, then she could fly, but the gold made her sick and she died. The end.” Not very epic.

But your publisher just sent you an email with an all-caps subject line. You need to get a book out soon, and you know you can’t do a good job on your own. And while your friend isn’t very clever, you happen to know they’re fast, they don’t mind you asking them questions, and they can give you a bunch of (meh-quality) answers. Maybe you could still use them to accelerate your book writing process…

You decide that although you can’t trust the whole book-writing process to your friend, you can split up the work and use them to help out with little bits. You ask them things like “give me a visual description for this character who plays the role of the mentor” or “finish the end of this scene,” and they give you answers. Usually, you have to ask them the same question multiple different times and choose the best version of their answer, and sometimes none of their answers are good and you end up writing your own. But they generate some ideas you wouldn’t think of, and it feels like you’re actually getting a slightly better book written than you would have alone. This takes longer than you thought, but you eventually submit the finished novel the day of the deadline, and it gets accepted!

Another email from your publisher. Your book didn’t do as well as you thought, and now you have even less time to write a better book! You think using your friend for help was a good idea, but it took a lot of time to go through all those answers, and only a few of them were good. Desperate, you wonder if there’s a way to make your friend ...
Where Utopias Go Wrong, or: The Four Little Planets
by ExCeph — 2022-05-27T01:24:18.729Z
Link: https://ginnungagapfoundation.wordpress.com/2022/05/22/where-utopias-go-wrong-or-the-four-little-planets/

Cross-posted from The Ginnungagap Foundation. This exploration of human attempts to improve human society began as a nonfiction essay, but I decided making it fiction would be more engaging.

Some concepts like ethics are elaborated upon in my previous articles, which is why I did not go into more detail about them in this story. I may cross-post those here as well if there is interest.

Content warning: AI safety skepticism (voiced by, ironically enough, a friendly AI).

Thoughts? Questions? Criticisms?

***

Once upon a time, there were four planets, in all different corners of the galaxy. They existed unbeknownst to each other, but each was filled with turmoil and misery. Their people fought and stole from and hurt and killed each other. One by one, each planet decided enough was enough. Independently, each one decided to create a utopia, where everyone was happy and nobody fought.

The First Planet

The first planet built its utopia out of rules. The people attempted to design a perfect set of laws to keep order. They made laws against murder and laws against theft. They made laws against violence and vandalism. They made laws about what people could buy and sell. Laws would prevent conflict and make happiness possible, they agreed.

However, they soon found that it was not possible to write enough laws to cover every conflict. There was always a way around a law for a clever and determined mind. People came up with ways to hurt and steal from each other that were not technically illegal.

Wealthy people had no need to engage in theft when they could perform hostile acquisition, and destroy any evidence the former owner had owned anything in the first place. Instead of threatening people, they could introduce incentives–it’s amazing how much safer a place becomes when people pay their “insurance” premiums. Instead of murdering their rivals, they staged “accidents”. Instead of lying, they deceived, getting victims to agree to ambiguous but binding contracts based on misleading language and false assumptions.

The people of the first planet were outraged that those of ill intent had figured out how to get away with these new approaches to conflict. In response, they decided to appoint officials to keep their system of laws worki...