trentmkelly/LessWrong-43k huggingface dataset:
Explaining the Twitter Postrat Scene
by Jacob Falkovich — 2022-04-05T22:23:27.125Z

You are reading this post on LessWrong. If you're like most of my rationalist friends, you believe that LessWrong's culture of decoupled and straightforward communication and focus on timeless issues of rationality and world-optimization has created a community of like-minded people that you enjoy interacting with and learn a lot from. Once in a while, I tell my LessWrong friends that they should get on Twitter, that the culture of memes and half-joking deeply contextualized and obscure banter on the latest breaking news and culture war topics has also created a wonderful community of like-minded people you can learn a lot from. My rationalist friends are horrified at the suggestion.

They're almost as horrified as people who've tweeted for years about sex and astrology and pineal glands are to discover that half their mutuals are actually

LessWrongers

.

So, what the hell is going on? Is rationalist twitter a thing and if so, why? Are postrats just rats who post? And am I writing this off the cuff with little editing because that faithfully communicates

my Twitter style

or because it's Goodheart week? Read on to find out!

Twitter Counterculture

The problems with Twitter are obvious on the surface:

It mostly consists of partisans of competing culture war tribes fighting over polarizing object-level topics with no desire or ability to go meta.
No matter how clearly you express yourself,

people will misunderstand it

in the least charitable way.
There's no way to enforce barriers to entry, any rando can comment on anything.
But well-known journalists and other bluechecks are even worse, using Twitter either to parrot the same coordinated narrative or to viciously settle personal scores.
The nuggets of vital breaking info and true deep insight are buried under a torrent of bullshit.
Anonymity + audience + zero personal stake in relationships =

people being total fuckwads

.

But also Twitter is the place where

Elon Musk challenges Vladimir Putin

to hand to hand combat for the future of Ukraine. People don't want to miss out on that.

To get the best parts of Twitter without the worst, you'd want to join as part of a community with norms that defuse and ameliorate Twitter's worst tendencies. A community where:

Object-level pontificating is looked-down upon, and partisanship is outright scorned.
Illegibility to outsiders is prized, so that only those who

...
Prioritise Tasks by Rating not Sorting
by Neel Nanda — 2022-04-05T22:20:19.394Z
Link: neelnanda.io/blog/48-rating

I often get paralysed by indecision when I have a bunch of tasks to do - it’s hard to figure out where to start! My standard solution is sorting the tasks into priority order - usually implicitly with a comparison-based sorting method where I go through the list one-by-one and put each new item in the right position. But I’ve recently gotten a surprisingly large improvement by rating - giving each item a score out of 10 for priority, and going in order by score. I have no idea how much this generalises to other people, but I’m sharing this quick post making the case for why this method is better in case anyone else vibes! (note - this post contains major amounts of overly opinionated overthinking of simple ideas! Read on at your own risk)

Reason 1: I avoid thinking about the hardest comparisons

Unlike in standard computer-implemented sorting algorithms, comparing any pair of tasks does not take the same amount of time. Most comparisons are obvious, while comparing close tasks feels much harder and more ambiguous. In practice, putting a task in exactly the right place among similar tasks is what absorbs the most time and attention when priority ordering, and is where I most easily get discouraged.

Rating, in comparison, feels easy for any task - I can just convert my gut feel into a number in a few seconds. If it feels ambiguous, eg I can’t decide between a 7 or 8, I just put a 7.5! Ratings can be continuous. And then, if I end up with several tasks with the same rating, I can just arbitrarily tie break (eg, by the order that they were entered into the list) - if they have the same rating, then prioritising further can’t be that useful! In hindsight, the hardest comparisons are in fact the least useful - if I do the fourth most important task before the third most important, that’s no big deal, but doing the tenth before the first is a screw-up!

Reason 2: Rating means going through the list once, sorting goes back-and-forth.

I find that this makes things much faster, and cognitively easier - to sort each task into the list I need to read back through for each item to find the right position, which is distracting and flow-breaking. While for rating, I just read through once and consider each task in isolation

Technical aside: I’m comparing insertion sort, which is quadratic time, with bucket sort, which is linear time with a worse constant, but I think time complexi...
Supervise Process, not Outcomes
by stuhlmueller — 2022-04-05T22:18:20.068Z

We can think about machine learning systems on a spectrum from process-based to outcome-based:

Process-based systems are built on human-understandable task decompositions, with direct supervision of reasoning steps.
Outcome-based systems are built on end-to-end optimization, with supervision of final results.

This post explains why

Ought

is devoted to process-based systems. The argument is:

In the short term, process-based ML systems have better differential capabilities: They help us apply ML to tasks where we don’t have access to outcomes. These tasks include long-range forecasting, policy decisions, and theoretical research.
In the long term, process-based ML systems help avoid catastrophic outcomes from systems gaming outcome measures and are thus more aligned.
Both process- and outcome-based evaluation are attractors to varying degrees: Once an architecture is entrenched, it’s hard to move away from it. This lock-in applies much more to outcome-based systems.
Whether the most powerful ML systems will primarily be process-based or outcome-based is up in the air.
So it’s crucial to push toward process-based training now.

There are almost no new ideas here. We’re reframing the well-known outer alignment difficulties for traditional deep learning architectures and contrasting them with compositional approaches. To the extent that there are new ideas, credit primarily goes to Paul Christiano and Jon Uesato.

We only describe our background worldview here. In a follow-up post, we explain why we’re building

Elicit

, the AI research assistant.

The spectrum

Supervising outcomes

Supervision of outcomes is what most people think about when they think about machine learning. Local components are optimized based on an overall feedback signal:

SGD optimizes weights in a neural net to reduce its training loss
Neural architecture search optimizes architectures and hyperparameters to have low validation loss
Policy gradient optimizes policy neural nets to choose actions that lead to high expected rewards

In each case, the system is optimized based on how well it’s doing empirically.

MuZero

is an example of a non-trivial outcome-based architecture. MuZero is a reinforcement learning algorithm that reaches expert-level performance at Go, Chess, and Shogi without human data, domain knowledge, or hard-coded rules. The architecture has three parts:

A representation network, mapping obs

...
My agenda for research into transformer capabilities - Introduction
by p.b. — 2022-04-05T21:23:11.303Z

I recently applied for funding for my research agenda. I probably did a pretty bad job. In fact just now before writing this second sentence, it occured to me to check my emails and, lo and behold, this specific application has just been denied.

This is not particularly surprising or depressing. However, the process of writing a grant application and also the aftermath of thinking about what I should have written led me to a clearer understanding of where I would like to go with my research.

I want to pour that new-found clarity into a series of posts, where I describe the questions I would like to investigate and how I plan on doing this with relatively minimal resources. Given that I have absolutely zero slack right now (writing this post is already cutting into sleep) and funding seems a remote possibility, this is also a way to put these ideas and approaches out there for others to be inspired.

My motivation

As a chess fanatic and a Deep Learning enthusiast, I have long dreamt about and occasionally worked on combining these two passions. Originally the idea was simply to model human concepts with Deep Learning based classifiers and integrate these classifiers into a system that could give human understandable feedback on games and moves. [Example: Label positions by whether the player is going to succumb to a mating attack later in the game. Train a classfier to predict this. Call it "king safety" and give feedback like "This move weakens your king's position."]

However, as I got deeper into Deep Learning my motivation changed towards "I want to try out all these cool techniques and ideas on my own little usecase". Now, however, as Transformers are seemingly scaled in the general direction of AGI, my motivation has changed again towards understanding what transformers are capable of in the limit.

In late 2014 I had gotten interested in Deep Learning and came across a paper that used a CNN to quite accurately predict human moves in the game of Go. Having some interest in and a little knowledge about chess and Go engines, I realized that this was the missing piece in current Go engines. I briefly considered trying to implement this myself, but then decided that some industry giant would probably create a human competitive Go engine within a relatively short time.

Maybe the certainty I felt was misplaced. But obviously (in hindsight) this turned...
Forecasting Newsletter: March 2022
by NunoSempere — 2022-04-05T20:23:04.373Z

Highlights

Comparing top forecasters and domain experts

finds that past studies mainly were not comparing apples to apples and that the assertion that superforecasters were 30% better than intelligence analysts was unjustified.

Samotsvety's Nuclear Forecasts

got picked up in the

Spanish press

and criticized by a

pessimistic nuclear expert

.

Forecasting Wiki launched

Polymarket is inflating its volume by incentivizing wash trading. (edit: apparently not the case, will issue a correction in the next issue)

Index

The state of forecasting
Notable news
Platform by platform
Relevant research

You can sign up for this newsletter on

substack

, or browse past newsletters

here

.

The state of forecasting

On account of getting a plug on one of Spain's most-read newspapers, this newsletter has reached 1,000 subscribers:

You can find a market on when it will reach 2000 here.

So I thought I would summarize the state of forecasting as I see it, striving to be informative to new readers. If you're already familiar with the key points, you might want to skip to the next section.

The main problem is bullshit or lack of epistemic virtue and ability. The US misled itself into thinking that Iraq still had weapons of mass destruction or that

everything would be okay in Afghanistan

(

a

). People were not expecting covid to last so long. And everyone keeps expecting a better brand of politician to show up.

What is the alternative? The alternative is to develop better models of the world and then use those better models to make better decisions.

But how do we know which models of the world are good? How do we differentiate real understanding from fake understanding? It's tricky, but to a first approximation, we make our hypotheses about the world output predictions, and we

reduce our confidence in the hypotheses that make worse predictions

(

a

). The book Superforecasting is a neat introduction to the practices involved. E.T. Jaynes' Probability Theory: The Logic of Science is a hardcore introduction to the math behind it. Both books are probably available for free in the

z library

(

a

).

A graphical representation of Bayes' rule, from Arbital.

You could keep track of your probabilities in a spreadsheet. But it would also be convenient to collaborate and compete with others. And here come various forecasting platforms, like

Metaculus

(

a

),

Manifold Markets

(

a

), Good Judgment Open, or INFER. These forecasting plat...
What Would A Fight Between Humanity And AGI Look Like?
by johnswentworth — 2022-04-05T20:03:30.232Z

When I imagine what a fight between humanity and a moderately-superhuman AGI looks like… well, mostly I imagine there is no fight, humanity just gets wiped out overnight. But if humanity turns out to be more resilient than expected, and the AGI doesn’t immediately foom for some reason, then I start to think about what it’s like to fight an opponent smarter than us.

Story 1: OODA Loops

I once played laser tag against someone who was really good at laser tag. They’d shoot from one spot, and by the time you turn to look, they’d already be moving somewhere else. Then they’d loop around, and hit you from the back while you were looking where they were previously. Then you’d try to retreat, but they’d already be headed that way and hit you again. So you’d try to loop around and get the drop on them, but they’d see where you’re going, and they’d once again pop out from someplace you didn’t expect.

This sort of thing is straight out of standard US military doctrine: it’s all about “getting inside the opponent’s OODA loop”. You want to observe what the opponent is doing and react to it faster than they can orient to your own actions. When someone is “inside your OODA loop” like this, it feels confusing and disorienting, you’re constantly being hit from surprising directions and you have no idea where the opponent is or what they’re doing.

This sort of fight is very cognition-heavy. Not just in the generic sense of “being smart”, but also in the sense of noticing things, tracking what’s going on in your environment, predicting your opponent, making good decisions quickly, acting efficiently, etc. It’s the sort of thing you’d expect even relatively weak AGI to be very good at.

Story 2: Technology Indistinguishable From Magic

Imagine a medieval lord in a war against someone with slightly more advanced technological knowledge. We're not talking modern weaponry here, just gunpowder.

To the lord, it doesn’t look like the technologist is doing anything especially dangerous; mostly the technologist looks like an alchemist or a witch doctor. The technologist digs a hole, stretches a cloth over, dumps a pile of shit on top, then runs water through the shit-pile for a while. Eventually they remove the cloth and shit, throw some coal and brimstone in the hole, and mix it all together.

From the lord’s perspective, this definitely looks weird and mysterious, and they may be somewhat ...
Ukraine Post #9: Again
by Zvi — 2022-04-05T19:40:00.555Z

This is once again a ‘sources and information on everything happening’ post. In the last few days after Russia’s retreat from the Kyiv area, focus has shifted from military and other issues to the atrocities discovered in Bucha. This covers both.

Military Progress and Conditions

The Battle of Kyiv is over, Russia has withdrawn, not entirely smoothly. Here is a claim that sabotage of railway connections with Belarus forced Russia to withdraw. This is in contrast to previous reports of Russia digging in around Kyiv and more in line with earlier reports from late March they were destroying bridges.

The entire Kyiv front is now back in Ukrainian hands, including Chernobyl. The Russians may or may not have been playing chicken with nuclear power plants, but they do not seem inclined to use them as hostages as some originally feared.

It appears they are indeed instead going into some sort of Phase Two, with more limited war goals involving claiming territory in the east. The primary effort looks indeed to be in Donbass. The goal appears to be to encircle Ukraine’s forces. If Russia’s efforts to do this fail, there does not appear to be another path to success even in the east beyond grinding down over a long period.

That does not mean this was ever the plan. It wasn’t, although the encirclement aspect was. Bret Devereaux points out in this thread that Russia’s attacks are entirely consistent with trying to encircle the Ukrainian army and capture Kyiv with a quick knockout punch to achieve a fait accompli, and complete massive overkill for the goal of taking the rest of Donbass.

Also there are any number of other things Russia could attempt to do instead, including something outside of Ukraine.

Here is the new map as of 3 April.

And here’s April 4 from a different source, for contrast

Here is a third source from April 5. Yellow is reclaimed territories.

Russia will attempt to use what remains of the withdrawn invasion forces in the Donbas, where they will try to claim a victory that much more urgently. Ukraine will be able to free up a lot of its own resources to do the same. At one point there were 80,000 defenders in Kyiv, many of which are now unnecessary.

According to some reports, Russia then made preparations to attack Kharkiv (4 April). They had a ‘covert mobilization’ starting on 3 April but the covertness level leaves something to ...
Prompt Your Brain
by En Kepeig — 2022-04-05T18:42:29.575Z

Summary: You can prompt your own brain, just as you would GPT-3. Sometimes this trick works surprisingly well for finding inspiration or solving confusing problems.

Language Prediction

GPT-3 is a language prediction model that takes linguistic inputs and predicts what comes next based on what it learned from training data. Your brain also responds to prompts, and often does so in a way that (on the surface) resembles GPT-3. Consider the following sequences of words:

A stitch in time saves _____.
The land of the free and the home of the _____.
Harry Potter and the Methods of _____.

If you read these one at a time, you’ll likely find that the last word automatically appears in your mind without any voluntary effort. Your language prediction process operates unconsciously and sends the most likely prediction to your conscious awareness.

But unlike GPT-3, your brain takes many different types of input, and makes many different types of predictions. When we listen to music, watch movies, drive cars, buy stocks, or publish blog posts, we have an intuitive prediction of what will likely come next. If we didn’t, we would be constantly surprised.

You can take advantage of this knowledge by prompting your brain and causing it to activate the relevant mental processes for whatever you're trying to do.

Writer’s Block

When you stare at a blank page and struggle to find inspiration, the easy explanation is that you have no prompt. The brain has nothing to predict. That’s why one of the most common solutions to writer’s block is to put something, anything down on the page. Now the brain has a prompt!

If you’re writing fiction, you can start with a template like “my protagonist lives in _____ and wants to _____”. If you’re writing nonfiction, you can use an information-based template like “more people should know about _____” or “I wish I knew about _____ sooner”.

The need for creative ideas might seem obvious to your conscious mind, but often the rest of your brain just doesn’t know what you want it to do. The more context you provide, the more likely your brain is to give you a good answer.

Babble

The babble method (generate lots of output with low standards) works in part because it gives the brain a prompt for generating ideas.

A babble prompt might look like this:

50 ways to earn more money next year:

Switch jobs
Ask for a promotion
Start a new business
Etc.

Before writing down the prompt,...
Optimizing crop planting with mixed integer linear programming in Stardew Valley
by hapanin — 2022-04-05T18:42:01.887Z

Abstract

TL;DR: plant your crops like this and you will make the most money possible

Motivation

If you haven’t yet tried, consider escaping 2020’s cyberpunk hellscape into the 2D world of Stardew Valley, an open-world farming RPG where almost all actions lead to positive outcomes. The game is as non-punishing as they come, making it a superb unwinding after a long day or recurring date with your long-distance quarantined non-gaming significant other. Whether you spend your time watering blueberries, reeling in trout, or romancing the local townsfolk, most would say there is no wrong way to play as long as you’re having fun.

I disagree. A farm is a business and businesses should make as much money as possible by any means necessary[1]. Since farming provides the majority of a Stardew farmer’s income, it is therefore necessary to determine the rotation of crops which maximizes profit within the constraints of energy and available capital. If your fun is not optimized, you’re wasting valuable time.

Background

You arrive on your late grandfather’s farm with little more than a hoe[2] and a watering can. The mayor welcomes you with 15 parsnip seeds and you inherit 500 gold pieces. To plant a seed, you must first till the ground with your hoe, expending 2 of your 270 daily energy. For another 2 energy you then water the seeds that and every subsequent t days until they are harvested. Harvesting takes no energy, so you should quickly pick your produce, sell it over at Pierre’s General Store and immediately buy more seeds, rinse, and repeat. Each season consists of 28 days, after which all crops die and cannot be harvested.

Additionally, you can forage materials at no monetary cost by cutting down trees and selling the wood and byproducts to Robin, the carpenter. Trees on average contain about 50g worth of materials and take 30 energy to harvest. Unlike crops there is no growing period and we will assume an infinite number of available trees[3].

In order to maximize profit, in general you should plant a lot of seeds, but not so many seeds that you cannot water them all in a single day, and also spend your excess energy foraging. One such algorithm could be:

On each of day d where d%t=0 plant as many crops as energy or budget will allow

Spend all excess energy foraging

Assuming we only plant parsnips, our farming and foraging strategy should look like this:

Benchmark f...