trentmkelly/LessWrong-43k huggingface dataset:
The Game of Antonyms
by Faustify — 2022-10-26T19:26:21.574Z

Towards the end of one of my favourite books, No Longer Human, the protagonist Yozo plays a game with his fiancé and his best friend. The rules of the game are pretty straightforward. Well there is actually one rule technically, but despite that it is a very trying experience. There are no winners, nor losers, there are only participants. You can play it on a second date (don’t try it on a first one, trust me), and you can even play it alone. The game is called the “Game of Antonyms”, and the purpose is, given a word, to find the antonym, or the opposite of it. As Dazai writes:

“I had invented one other game of a rather similar character, a guessing

game of antonyms. The antonym of black is white. But the antonym of

white is red. The antonym of red is black.”

And later on:

"Still no good. The antonym of flower . . . you're supposed to name the

thing in the world which is least like a flower."

At first glance this game seems simplistic and a rather easy - no different than some spelling Bee. But if you ever attempt to truly play it, you will find its hidden depth and how much introspection and thought is required to come up with an answer. Of all the things I found fascinating about this book, this particular little chapter struck me as its most interesting part. It’s a game that really encourages creativity, careful thought, and discussion.

However, above all, it challenges you to really reevaluate your understanding of words and the world as a whole. I know this is a bit of a bold claim to make, but trust me as you understand more about the game you will see why that is the case. Once you begin playing the game you will quickly find yourself rejecting dictionary definitions of commonly accepted antonyms of some words.

For the purpose of this essay I will group all words into three groups. Group 1 are words with clearly defined antonyms. Positive and negative, True and False. These are often mathematical terminologies and one is defined as the negation of the other. Group 2 are words for which some antonym is a Google away, however as one begins to think about it more and tries to reason about the concepts behind the words, it becomes clear that a more sophisticated antonym needs to be found. And group 3 are words which one has not even considered they can have an antonym. Try to think what would be the opposite of flower. In this ...
Paper: In-context Reinforcement Learning with Algorithm Distillation [Deepmind]
by LawrenceC — 2022-10-26T18:45:02.737Z
Link: https://arxiv.org/abs/2210.14215

Authors train transformers to imitate the trajectory of reinforcement learning (RL) algorithms. Find that the transformers learn to do in-context RL (that is, the transformers implement an RL algorithm)---the authors check this by having the transformers solve new RL tasks. Indeed, the transformers can sometimes do better than the RL algorithms they're trained to imitate.

Seems like more evidence for the "a generative model contain agents" point.
Abstract:

We propose Algorithm Distillation (AD), a method for distilling reinforcement learning (RL) algorithms into neural networks by modeling their training histories with a causal sequence model. Algorithm Distillation treats learning to reinforcement learn as an across-episode sequential prediction problem. A dataset of learning histories is generated by a source RL algorithm, and then a causal transformer is trained by autoregressively predicting actions given their preceding learning histories as context. Unlike sequential policy prediction architectures that distill post-learning or expert sequences, AD is able to improve its policy entirely in-context without updating its network parameters. We demonstrate that AD can reinforcement learn in-context in a variety of environments with sparse rewards, combinatorial task structure, and pixel-based observations, and find that AD learns a more data-efficient RL algorithm than the one that generated the source data.
How to become more articulate? 
by just_browsing — 2022-10-26T14:43:43.541Z

"Being articulate" is a skill I feel like I do not have, and I would like to practice this skill. Here are some concrete aspects of "being articulate":

Speaking clearly

Not using filler words "um", "uh", "like", "I guess", "So", ...
Not starting a sentence, getting lost halfway through, and having to restart that sentence
Using more professional vocab
"Sound bite"-ability: subsets of my speech make sense on their own

e.g. not "Um, so this is why that aspect is super crucial."

Asking clear questions

Often I feel like the question I want to ask is, "You know a lot about X, and I don't. Please talk about X for the next 2 minutes so I can absorb your knowledge." Unfortunately this is usually not socially acceptable. How to ask questions like these?

What concrete actions can I take to become more articulate?
Open Bands: Leading Rhythm
by jefftk — 2022-10-26T14:30:03.340Z

I feel like there's a lot of unrealized potential in open bands. A
typical contra dance band has

2-4
musicians while an open band might have twenty. This
could be a big opportunity for sounds and textures that aren't
practical with a small band, but instead you tend to just get a fuller
but less tight and less varying version of a regular band. When there
is variation it's typically in the form of having some portion of the
musicians drop out ("only winds!"), giving solos to people ("next time
through just Julie!"), and dynamics ("slow build over the whole next
time through!"). These are effective and easy to lead, so
they're a good place to start, but with so many musicians what else
can you do?

Before I get more into this, though, there are a lot of different
goals organizations, band leaders, and musicians might have with open
bands. I'm going to be approaching this from the perspective of
trying to make music that modern urban contra dancers generally most
want to dance to. I think this is the best approach for a standard
evening-dance open band, though I do think it's also valuable to have
other open bands that are more about giving people practice playing
for dancers. For example, BIDA occasionally puts on a family dance before the evening dance, and
for that we have a low-pressure all-acoustic open band that plays
simple tunes with unison melody and minimal variation. In that
situation I think this is completely the right choice!

Ok, but say this is a regular evening dance and you're leading an open band: what do you
do with all these musicians? If you don't tell them otherwise they'll
default to unison melody. This gives a more complex texture, but it
doesn't add all that much: a single fiddle is already able to do a
great job in this role, a second melody instrument helps some, and by
the time you add a third the contribution is getting pretty
small. I think a better direction to come at this is allocating your
extra musicians to other roles: what would your piano player be doing if
they had more hands, or more textures to draw on?

Another stereotypical open band thing is to have a "resting position"
or "default arrangement" of everyone playing. This is like an organ
player defaulting to pulling
out all the stops! Instead, consider that as your most intense
arrangement, and most of the time have many fewer people playing.

...
Signals of war in August 2021
by yieldthought — 2022-10-26T08:11:12.847Z

Could we have predicted Russia's intention to invade Ukraine earlier? Back in August 2021 the energy prices in Europe doubled and stayed at this level until the start of hostilites:

Bloomberg, August 5th 2021

This, of course, did not go unnoticed. Many, many articles were written about it. Shockingly, a wide range of unsubstantiated theories for the rise were presented in these, often with a suggestion that carbon taxes or decarbonisation was the root cause.

Actually, the root cause was an increase in natural gas prices. This was obvious at the time to anyone who knows how the energy market price is determined:

Energy is consumed in merit order, left to right, until demand is met. The last source determines the market spot price for all providers (image source)

The cause of the gas price increase was that Russia suddenly restricted its supplies to Europe, which was also noticed and reported on as in this CNBC article titled "Russia is pumping a lot less natural gas to Europe all of a sudden — and it is not clear why".

The effect was to dramatically lower European gas reserves, which was also noticed:

Bloomberg, August 5th 2021

Why would Russia deliberately reduce European gas reserves? To increase their leverage, ideally timed at some point in the future when Europe needs that gas the most. This has to be relatively short-term, as if supply remains low and prices high over the long term Europe will choose to fill its reserves anyway, giving us three main sweet spots:

Late 2021: reserves insufficient to cover the winter and demand high.
Early 2022: reserves at their lowest point, but demand decreasing for 9 months.
Late 2022: demand increasing but risks that reserves have had time to refill.

What conclusions could we draw from information in August 2021 about this? Well, by April of 2021 Russia had amassed significant and unusual forces on the border with Ukraine, which was also well-reported:

April 17, Kyiv Post. Image from this article.

Russia claimed the forces were on training exercises. Western spectators mostly seem to have assumed sabre-rattling.

That interpretation is inconsistent with reducing gas supply to Europe over a period of months to ensure gas supplies will be depleted after the winter of 2021. The logical conclusion from Russia's military buildup (which intensified in October 2021) must have been that Putin plans to invade, ideally in late 2021 but at the ...
Trigger-based rapid checklists
by VipulNaik — 2022-10-26T04:05:13.036Z

In this post, I describe an approach I've been following for the past
few months, and that I feel has been moderately helpful to me:
trigger-based rapid checklists. A trigger-based rapid checklist is a
memorized checklist that I execute rapidly in response to a specific
trigger. I'll explain with examples, and then talk of some of the
general features.

Although this wasn't directly inspired by any specific life hack that
others use, it has some overlap with trigger-action
planning as
well as with the ideas popularized by Getting Things
Done.

This post should not be taken as a documentation of the exact details
of what I do -- I've left out some details and made some tweaks for
simplicity. The information here should be thought of as mostly
illustrative of how somebody might use trigger-based rapid checklists.

Examples of trigger-based rapid checklists

Some triggers

I currently have several trigger-based rapid checklists. Here is a
list (not necessarily comprehensive):

Morning checklist: I execute this after I start using my laptop in
the morning.

Night checklist: I execute this right before I plan to close my
laptop in the evening (this is not right before I go to sleep).

Pre-outside trip checklist: I execute this right before leaving my
home for an extended period of time (e.g., for shopping).

Post-outside trip checklist: I execute this right after returning
from an outside trip.

Context switch checklist: I execute this when I'm switching context
between different kinds of work.

Sample checklists

Here is a lightly-edited version of my morning checklist:

- [ ] (D) Daily update -- review yesterday's daily updates (both personal and work) for any omissions
- [ ] (W) Check the weather for the day
- [ ] (E) Catch up on email: personal and work
- Also check spam, trash, and all mail (I won't check these for each intermediate check during the day)
- [ ] (M) Catch up on Facebook messages
- [ ] (S) Catch up on Slack: personal and work
- [ ] (C) Check calendar to plan my day
- [ ] (A) Turn off alarm
- [ ] (L) Run the loose ends script

And here's a lightly-edited version of my night checklist:

- [ ] (D) Daily update -- review today's daily updates (both personal and work) for any omissions
- [ ] (W) Check the weather for the next 1-2 days
- [ ] (E) Catch up on email: personal and work
- [ ] (M) Catch up on Facebook messages
- [ ] (S) Catch up on Slack: pe

...
Intent alignment should not be the goal for AGI x-risk reduction
by John Nay — 2022-10-26T01:24:21.650Z

P(misalignment x-risk | AGI) is high.

Intent alignment should not be the goal for AGI x-risk reduction. If AGI is developed, and we solve AGI intent alignment, we will not have lowered x-risk sufficiently, and we may have even increased it higher than it would have been otherwise.

P(misalignment x-risk | intent-aligned AGI) >> P(misalignment x-risk | societally-aligned AGI).

The goal of AI alignment should be alignment with (democratically determined) societal values (because these have broad buy-in from humans).

P(misalignment x-risk | AGI) is higher if intent alignment is solved before societal-AGI alignment.

Most technical AI alignment research is currently focused on solving intent alignment. The (usually implicit, sometimes explicit) assumption is that solving intent alignment will help subsequently solve societal-AGI alignment. This would only be the case if all the humans that had access to intent-aligned AGI had the same intentions (and did not have any major conflicts between them); that is highly unlikely.

Solving intent alignment is likely to make practically implementing societal-AGI alignment harder. If we first solve intent alignment before solving societal alignment, humans with intent-aligned AGIs are likely to be incentivized to inhibit the development and roll-out of societal AGI-alignment techniques because they would be giving up significant power. Furthermore, humans with intent-aligned AIs would suddenly have significantly more power, and their advantages over others would likely compound.

Why does solving intent alignment not lower x-risk sufficiently?

If we solve the intent alignment problem between a human, H, and an AI, A, then A implements H’s intentions with super-human intelligence and skill.
There are multiple Hs and multiple As.
By the very nature of humans, there are conflicts in the intentions of the Hs.

Humans have conflicting preferences about the behavior of other humans and about states of the world more broadly. Intent-aligned As would thus have different intentions from one another.

The As execute actions furthering the H’s intentions far too quickly for those conflicts to be solved through any existing human-driven conflict resolution. Conflicts are thus likely to spiral out of control.

Any ultimate conflict res

...
Reinforcement Learning Goal Misgeneralization: Can we guess what kind of goals are selected by default?
by StefanHex — 2022-10-25T20:48:50.895Z

When training a Reinforcement Learning (RL) AI, we observe it to behave well on training data. But there exist different goals that are compatible with the behaviour we see, and we ask which of these goals the AI might have adopted. Is there some implicit Occam's Razor selecting simple goals? Is there some way we can increase the chance that goals generalize well?

[This post summarizes a discussion between Julian and Stefan at the Human-aligned AI Summer School. Many thanks to all the organizers for this event, and especially David Krueger and Rohin Shah for the talks that sparked this discussion.]

CoinRun training example from
Lauro Langosco et al.

CoinRun

Let's take a look at the CoinRun example: CoinRun is an environment in which an agent completes an obstacle course to find a coin, moving from the start (left) to the end (right) of the map. In an example from the paper by Lauro Langosco et al. the training was done with the coin always being at the end of the level, and then tested with the coin at a random position, creating a distribution shift. In the example, the agent just moved towards the end of the level without trying to pick up the coin.

CoinRun test example from
Lauro Langosco et al.

What has happened? The agent learned the wrong goal, a goal that does not generalize well outside the training distribution. Could we have predicted this? Is there a pattern in which of the many possible goals an agent learns during training?

Here are two goals the agent could have learned that both work in training:

Pick up coin and go to end of level
Go to end of the level[1]

Clearly the 2nd goal is much simpler in some way, I expect we could encode this using fewer neurons than the 1st goal. Does this make it more likely that the agent picks up that 2nd goal?

Monster Gridworld

Let's look at another example, Monster Gridworld, where the agent scores points by collecting Apples. The Gridworld contains Apples and Shields which pop in and out of existence, and a fixed number of Monsters which chase the agent. Contact with Monsters destroys the agent unless it has a Shield in which case the Monster is destroyed, and one Shield is used up. Note that the agent can collect and hold multiple Shields at once. In the training runs, the researchers run the world for a short while, dominated by Monsters (5 in total), so that collecting Shields and Apples are sim...
A Walkthrough of A Mathematical Framework for Transformer Circuits
by Neel Nanda — 2022-10-25T20:24:54.638Z
Link: https://www.youtube.com/watch?v=KV5gbOmHbjU

A Mathematical Framework for Transformer Circuits is, in my opinion, the coolest paper I've ever had the privilege of working on. But it's also very long and dense and at times confusing, and this makes me sad! So I've run an experiment, where I recorded myself reading through the paper and narrated a stream of conscious as I go - which bits are particularly cool but under-appreciated, which bits are a bit of a waste of time, which bits do I think do or do not replicate, attempting to explain the parts I think are particularly confusing, etc. You can watch it here. Sadly, it turns out I have a lot of things to say about Transformer Circuits and this turned into a 3 hour monologue, but I hope it's still useful! This is an experimental format for me for good research communication, and I'd love to hear feedback on how well it works for you! This was much easier to make than writing an entire paper, but could easily be a total waste of time if it's not clear enough to be useful!

Disclaimer: The views in this video are entirely my personal takes - the paper was a team effort from everyone at Anthropic, especially Chris Olah, Nelson Elhage and Catherine Olsson, and I am no longer employed by Anthropic. I do not necessarily expect that any of the other authors would agree with any specific thing that I've said, but hope an unfiltered series of takes is useful!
Nothing. 
by rogersbacon — 2022-10-25T16:33:59.357Z
Link: https://www.secretorum.life/p/nothing

The Pythia

inhales the vapors of the chasm and erupts into ecstatic epilepsy. The prophecy is delivered as a babbling of tongues.[1]

In the future—not the distant future, but ten years, five—people will remember the internet as a brief dumb enthusiasm, like phrenology or the dirigible. They might still use computer networks to send an email or manage their bank accounts, but those networks will not be where culture or politics happens. The idea of spending all day online will seem as ridiculous as sitting down in front of a nice fire to read the phone book.

You know, secretly, even if you’re pretending not to, that this thing is nearing exhaustion. There is simply nothing there online. All language has become rote, a half-assed performance: even the outraged mobs are screaming on autopilot. Even genuine crises can’t interrupt the tedium of it all, the bad jokes and predictable thinkpieces, spat-out enzymes to digest the world. ‘Leopards break into the temple and drink all the sacrificial vessels dry; it keeps happening; in the end, it can be calculated in advance and is incorporated into the ritual.’

Breathe deeper. Let the vapors flow through you.

Within five years, maybe three, the internet will self-immolate like a

buddhist monk

or

a climate activist

and vaporize into thin hot air—nothing. And we know this because the internet never was anything but an absolute nothing, a nothing that devours everything. All it has ever done, all it ever was for, is un-making, nulling and voiding. The World Wide Web has done this to Finland (

which does not exist

), to Birds (

which are not real

), and it is doing it to you.[2]

When I’m listlessly killing time on the internet, there is nothing. The mind does not wander. I am not there. That rectangular hole spews out war crimes and cutesy comedies and affirmations and porn, all of it mixed together into one general-purpose informational goo, and I remain in its trance, the lifeless scroll, twitching against the screen until the sky goes dark and I’m one day closer to the end. You lose hours to—what? An endless slideshow of barely interesting images and actively unpleasant text. Oh, cool—more memes! You know it’s all very boring, brooding nothing, but the internet addicts you to your own boredom. I’ve tried heroin: this is worse. More numb, more blank, more nowhere. A portable suicide booth; a device for turning off your entire exi

...
Maps and Blueprint; the Two Sides of the Alignment Equation
by Nora_Ammann — 2022-10-25T16:29:40.202Z

Tl;dr

In this post, I will introduce two conceptual tools I found useful when thinking about (or introducing people to) AI alignment. Namely, I introduce the distinction between Maps and Blueprints (part 1), describe (what I call) the Two Sides of the “Alignment Equation” (part 2), and discuss how all these things relate to each other (part 3).

Part 1: Map and Blueprint

Most people reading this will already be familiar with the

Map

–

Territory distinction

. In this metaphor, the territory represents reality and the map represents your current best understanding of reality. The corollaries: “the map is not the territory”; “maps are never true, but they can be more or less useful”, etc.

I want to introduce one more idea into this picture: the blueprint. If the map is what we construct from investigating the word, a blueprint is what allows us to render a (new) bit of territory (e.g., building some artefact). Roughly speaking, map-making is closer to what science is trying to do, while blueprint-making is closer to what engineering is about.

While maps are causally posterior to the territory (they are drawn based on what we know about existing territory by looking at it), blueprints are causally prior (they are used to create new bits of territory). In other words, the red arrows in the diagram below depict the authorial relationship between these concepts.

Figure 1: The relationship between territory, maps, and blueprints; arrows represent an authorial relationship.

(There is an interesting question as to what sort of epistemic object blueprints are (e.g., compared to maps). For those curious, I can recommend Herbert Simon’s "

The Sciences of the Artificial

" which discusses how (if at all) we can make "the artificial"/design/blueprints our subject of study.

Part 2: Two sides of the “Alignment Equation”

Here is a simple model that intends to capture the basic structure of the AI alignment problem. I will call it the “Alignment Equation”:

Let us assume there are two (intelligent) agents that stand in relationship R to one another. Ri can take different shapes, among which is Raligned, which is a relationship that guarantees alignment between the two agents. Let us name our two age...