trentmkelly/LessWrong-43k huggingface dataset:
LessWrong's (first) album: I Have Been A Good Bing
by habryka — 2024-04-01T07:33:45.242Z

tl;dr: LessWrong released an album! Listen to it now on Spotify, YouTube, YouTube Music, or Apple Music.

On April 1st 2024, the LessWrong team released an album using the then-most-recent AI music generation systems. All the music is fully
AI-generated, and the lyrics are adapted (mostly by humans) from LessWrong posts (or other writing LessWrongers might be familiar with).

Honestly, despite it starting out as an April fools joke, it's a really good album. We made probably 3,000-4,000 song generations to get the 15 we felt happy about, which I think works out to about
5-10 hours of work per song we used (including all the dead ends and things that never worked out).

The album is called I Have Been A Good Bing. I think it is a pretty fun album and maybe you'd enjoy it if you listened to it!
Some of my favourites are The Litany of Tarrrrrski, Half An Hour Before Dawn in San Francisco, and
Prime Factorization.

Click here to read the original text of the post published on April 1st.

Rationality is Systematized Winning, so rationalists should win. We’ve tried saving the world from AI, but that’s
really hard and we’ve had … mixed results. So let’s start with something that rationalists should find pretty easy:
Becoming Cool!

I don’t mean, just, like, riding a motorcycle and breaking hearts level of cool. I mean like the first kid in school
to get a Tamagotchi, their dad runs the ice cream truck and gives you free ice cream and, sure, they ride a
motorcycle. I mean that kind of feel-it-in-your-bones, I-might-explode-from-envy cool.

The eleventh virtue is scholarship, so I hit the books search engine on this one. Apparently, the aspects of
coolness are:

Confidence

Playing an instrument

Low average kinetic energy

I’m afraid that (1) might mess with my calibration, and Lightcone is committed to moving quickly which rules out (3),
so I guess that leaves (2). I don’t have time to learn an instrument, but my second-hand understanding of dath ilani
culture is that I can just pay someone to do it for me and the coolness should transfer.

Lightcone put out a call for collaborators in all the places we could think of that cool people might hang out.
Sysadmin listservs, direct-to-data-center optical fiber connection providers, high frequency trading firms, that one
Discord server where everyone speaks in Elvish. Despite this wide and varied o

...
The Closed Eyes Argument For Thirding

by Bentham's Bulldog — 2024-03-31T22:11:44.036Z

Don't you know when your eyes are closed
You see the world from the clouds along with everybody else?
Don't you know when your eyes are closed
You see the world from the clouds along with everybody else?

—

Close Your Eyes

by The Midnight Club

"In her house at R'lyeh sleeping beauty waits dreaming."

Crossposted on my blog.

The sleeping beauty problem is one of the most hotly debated topics in decision theory. It’s one of the topics like Newcomb’s problem where everyone seems to find their answer obvious, yet people don’t agree about it. The first paper on it (

which settled the issue

) was by Adam Elga, and described it thusly:

The Sleeping Beauty problem: Some researchers are going to put you to sleep. During the two days that your sleep will last, they will briefly wake you up either once or twice, depending on the toss of a fair coin (Heads: once; Tails: twice). After each waking, they will put you to back to sleep with a drug that makes you forget that waking. When you are first awakened, to what degree ought you believe that the outcome of the coin toss is Heads?

There are two main answers: 1/2 and 1/3. Halfers say that you should start out 50/50 before waking up, and because you’ll wake up either way, you’ll remain 50/50 (

this is false

for a rather subtle reason). Thirders say that, because there are twice as many wakings if the coin comes up tails as heads, upon waking up you should think tails are twice as likely as heads.

Now imagine that we modify the scenario slightly. You’re put to sleep. A fair coin is flipped. If it comes up heads, you’ll wake up in the laboratory once, be put back to sleep, be put in your bed, have your memory erased, and be woken up again. If the coin comes up tails you’ll be woken up, put back to sleep, have your memory erased, and woken up a second time in the lab. Suppose that you wake up in the lab—what should your credence be in the coin having come up tails?

I submit that halfers in the original sleeping beauty problem should say 1/2. After all, both theories predict with equal confidence that you’ll wake up in the lab—you haven’t learned anything new. Furthermore, in the original sleeping beauty problem, presumably after you aren’t woken up again, you’ll be sent home. So halfers in the original sleeping beauty problem think that if you will be sent home on the second day, then after waking up in lab conditions, you shoul...
Apply to be a Safety Engineer at Lockheed Martin!
by yanni kyriacos — 2024-03-31T21:02:08.499Z

Are you passionate about ensuring the safety and reliability of the world’s most lethal and cutting-edge weaponry? Does the idea of creating technology and then working out its impacts excite you? Do you thrive in dynamic environments where innovation meets rigorous safety standards? If so, you might want to consider joining the team at Lockheed Martin (LM), global leaders in advanced weapon systems development!

Position overview and background:

As a Safety Engineer specializing in advanced weaponry systems, you will play a critical role in ensuring we pass the checks and balances we’ve helped Federal Governments develop. You will collaborate very closely with multidisciplinary teams of engineers, scientists, and analysts to assess, mitigate, and manage risks associated with our most innovative products (however we expect any capabilities insights you discover along the way will be kept from your colleagues).

You might be a good fit if you:

Thrive on rigorously testing SOTA lethal weaponry, to ensure their safety and compliance.
Enjoy working closely with PR & Comms - as needed you will be asked to appear on various podcasts and give presentations to the weapons Safety community, whom we work very closely with.
Have experience in organizations with a flat hierarchy. For example, our CEO works extremely closely with the board.
Have industry connections. We maintain close ties with our independent auditors, many of whom used to work at LM!
Can predict with 100% accuracy that you won’t ever be interested in moving into different areas of the company. We hire the smartest and most conscientious talent specifically for our Safety teams, and assume they’ll never want to move into weapons capabilities advancement.

Annual Salary (USD)

Multiply the not-for-profit equivalent by 7X.

Join Us:

Apply

here

by June 16, 2026 (after which it will probably be

too late

).
The Best Tacit Knowledge Videos on Every Subject
by Parker Conley — 2024-03-31T17:14:31.199Z

TL;DR

Tacit knowledge

is extremely valuable. Unfortunately, developing tacit knowledge is usually bottlenecked by apprentice-master relationships.

Tacit

Knowledge

Videos

could widen this bottleneck. This post is a Schelling point for aggregating these videos—aiming to be

The Best Textbooks on Every Subject

for Tacit Knowledge Videos. Scroll down to the list if that's what you're here for. Post videos that highlight tacit knowledge in the comments and I’ll add them to the post. Experts in the videos include Stephen Wolfram, Holden Karnofsky, Andy Matuschak, Jonathan Blow, Tyler Cowen, George Hotz, and others.

What are Tacit Knowledge Videos?

Samo Burja claims YouTube has opened the gates for a

revolution in tacit knowledge transfer

. Burja defines tacit knowledge as follows:

Tacit knowledge is knowledge that can’t properly be transmitted via verbal or written instruction, like the ability to create great art or assess a startup. This tacit knowledge is a form of

intellectual dark matter

, pervading society in a million ways, some of them trivial, some of them vital. Examples include woodworking, metalworking, housekeeping, cooking, dancing, amateur public speaking, assembly line oversight, rapid problem-solving, and heart surgery.

In my observation, domains like housekeeping and cooking have already seen many benefits from this revolution. Could tacit knowledge in domains like

research

,

programming

,

mathematics

, and

business

be next? I’m not sure, but maybe this post will help push the needle forward.

For the purpose of this post, a Tacit Knowledge Video is any video that communicates “knowledge that can’t properly be transmitted via verbal or written instruction”. Here are some examples:

Neel Nanda, who leads the Google DeepMind mechanistic interpretability team, has a

playlist of “Research Walkthroughs”

. AI Safety research is discussed a lot around here. Watching research videos could help instantiate what AI research really looks and feels like.
GiveWell has

public audio recordings

of its Board Meetings from 2007–2020. Participants include Elie Hassenfeld, Holden Karnofsky, Timothy Ogden, Rob Reich, Tom Rutledge, Brigid Slipka, Cari Tuna, Julia Wise, and others. Influential business meetings are not usually made public. I feel I have learned some about business communication and business operations, a

...
How does it feel to switch from earn-to-give?
by Neil  — 2024-03-31T16:27:22.860Z

I suspect this phenomenon is common in the LW/EA spheres, but I've never seen it presented like this. I describe the way that switching from earning-to-give to working-in-altruism has consequences on one's sense of responsibility and trust. I wonder if others have experienced this and how.

Delegating responsibility

One of the truisms in life is “there are no adults”. Having turned 18 last month, I’ve had the displeasure of staring that truism in the face. Nothing deals a blow to your sense of civilizational adequacy quite like thinking about future Earth with life extension where everyone is thousands of years old, and then remembering you live on Earth2024 where most people in charge are barely half a century old. Nihil supernum and all that.

But the illusion of adults is extremely tempting to me. A part of me really wants to believe there are adults out there that can solve my problems better than I can. For instance, I donated to MIRI for the first time a month ago, and anytime I make money now, I run the expected value calculation and establish that if MIRI can slightly increase the log-odds of everyone surviving, that’s worth more than anything I could buy for myself. MIRI has become a sort of blackbox to me: money comes in, survival

lottery tickets

come out, and I don't care how the sausage gets done. I'm willfully ignorant, because "donating to MIRI" is

one deviation away from the front lines, and lets me avoid taking ultimate responsibility for things.

Switch

Then I got accepted into BlueDot Impact's

governance course

. Oh no! That's a reversal of responsibility! Now other people

[1]

are treating

me

as a black box where money comes in and x-risk mitigation comes out!

I applied to the course in the hopes of becoming the type of person who can use Neil's dollars in a more effective manner than MIRI.

[2] In other words, I wanted to be able to legitimately trust myself more than MIRI as far as allocating my own money is concerned. I'm not there yet, but the fact I am aiming to become that person is a shift in trust. It may also be the meaning of adulthood, to the extent there is one.

Should people aim for this?

In x-risk as with any other cause, should you aim to become the type of person whose money is better spent on oneself than on a charity?

Well, no. It's widely admitted that earning-to...
SAE-VIS: Announcement Post
by CallumMcDougall — 2024-03-31T15:30:49.079Z

This is a post to officially announce the sae-vis library, which was designed to create feature dashboards like those from Anthropic's research.

Summary

There are 2 types of visualisations supported by this library: feature-centric and prompt-centric.

The feature-centric vis is the standard from Anthropic’s post, it looks like the image below. There’s an option to navigate through different features via a dropdown in the top left.

You can see the interactive version at the GitHub repo, at _feature_vis_demo.html.

The prompt-centric vis is centred on a single user-supplied prompt, rather than a single feature. It will show you the list of features which score highest on that prompt, according to a variety of different metrics. It looks like the image below. There’s an option to navigate through different possible metrics and choices of token in your prompt via a dropdown in the top left.

You can see the interactive version at the GitHub repo, at _prompt_vis_demo.html.

Other links

Here are some more useful links:

GitHub repo
User Guide - Google Doc explaining how to use the library
Dev Guide - Google Doc explaining more about how the library was built, for if you'd like to try and extend it / build off it
Demo Colab - includes examples, with code explained

You might also be interested in reading about Neuronpedia, who make use of this library in their visualizations.

If you're interested in getting involved, please reach out to me or Joseph Bloom! We will also be publishing a post tomorrow, discussing some of the features we've discovered during our research.
UDT1.01: Local Affineness and Influence Measures (2/10)
by Diffractor — 2024-03-31T07:35:52.831Z

Attention Conservation Notice: This is a moderately mathy post.

Affineness, it's Useful!

So, if we're going to be restricting the sorts of environments we're considering, and trying to build an algorithm that's closer to UDT1.0 (just pick your action to optimize global utility without the whole "coordinating with alternate versions of me" aspect), it's probably worthwhile to consider what sorts of environments naturally work well with UDT1.0. What sorts of environments/games let you craft a notion of "the effect that a single player is having" in a way that doesn't really depend on what everyone else is doing?

If we've got some global high-dimensional maximization problem, maxx,y,zf(x,y,z), what sorts of functions let you split it into a bunch of little low-dimensional maximization problems that don't interact? Well, if f is a sum of a bunch of other functions which only depend on a single input, then something like maxx,y,z(f1(x)+f2(y)+f3(z)) splits up as maxxf1(x)+maxyf2(y)+maxzf3(z).

If we've got some game with a bunch of players, what sort of game lets you consider the plays of the players individually without worrying about interactions between the actions of the various different players? Linear games. If Ai is the space of player i's actions, then a game in general would have every player i be associated with a utility function Ui:(∏jAj)→[0,1]. But in linear games, everyone's utility function Ui breaks down as a weighted sum of component utility functions Ui,j:Aj→[0,1], which could be viewed as "how much player i likes or dislikes the action that player j took". No interaction between the different actions. How much player i approves of j's action has nothing to do with the actions that everyone else is taking. I actually have a whole ton to say about linear games, they're incredibly interesting, but that's probably a matter for another post that isn't a part of this sequence.

So what's the analogue of this "split stuff into a weighted sum" thing for policy selection environments? Well, permitting probabilistic choice of actions, if O<n is the space of histories of length less-than-n (places where we can make a decision), and our action space is A, and we're playing up to a time horizon of n, then the space of policies is..
∏:=O<n→ΔA
And, from the previous post, a policy-selection environment (in full generality) is one where there's a function
O<...
My simple AGI investment & insurance strategy
by lc — 2024-03-31T02:51:53.479Z

TL;DR:

Options traders think it's extremely unlikely that the stock market will appreciate more than 30 or 40 percent over the next two to three years, as it did over the last year. So they will sell you the option to buy current indexes for 30 or 40% above their currently traded value for very cheap.
But slow takeoff, or expectations of one, would almost certainly cause the stock market to rise dramatically. Like many people here, I think institutional market makers are basically not pricing this in, and gravely underestimating volatility as a result, especially for large indexes like VTI which have never moved more than 50% in a single year.
To take advantage of this, instead of buying individual tech stocks, I allocate a sizable chunk of my portfolio to buying LEAPS (Long-term Equity AnticiPation Securities) on the broader stock market. If a slow takeoff does happen, and public companies capture some of the increased productivity, I'll at least be duly compensated for it when my skills become worthless. If it doesn't happen, this part of my portfolio will vanish, but that seems like an acceptable risk given the upside.

I started doing this in January, and so far the mark price of the basket of options I've bought has doubled.[1]

FAQ

The options contracts you're talking about expire in "two to three years". Does this strategy only make sense if you think visible slow takeoff will begin before 2027?

That's not quite necessary. If large parts of the economy get automated "only" in 2030, near-term AGI progress could start to impress market makers enough that they "wake up" and increase the price of these securities and options in anticipation of a boom. Which is why I choose to buy now instead of closer to my expected timelines, while Nvidia is only a two trillion dollar company and my alpha on this could run out any given year.

But I think takeoff before 2027 is possible. As a layman, the simplest argument for shorter timelines I can empathize with is that GPT-3 was released in 2020, GPT-4 was released in 2023, and prediction markets expect GPT-5 to release later this year. That plus the enormous amount of capital investment in AI makes me think that there's a possibility of large portions of software engineering getting automated soon, which would precede further speedups.

Why not buy futures instead of options, if your thesis is about the next ten years rather than

...
Principles of action: fundamentals of how action works
by DDthinker — 2024-03-31T02:19:42.252Z

Introduction

For all action that an actor in the cosmos does, there can be determined general principles of action which are able to guide action which I call 'principles of action'. My principles of action are given and organised in what I call 'frameworks' which is a technique I developed during my researching efforts which is able to represent many pieces of knowledge in a unified whole.

Components of frameworks

Frameworks are structured in three ways of 'subject', 'description', and 'placement'. The components of frameworks see subjects which are given descriptions which are then placed in a placement relative to other pieces of knowledge by indents (spaces) and lines. Each line has a single section which can have subsections in immediately following indented lines. An example framework would be the following:

.subject - description

-subject - description

-subject - description
-subject - description

-subject - description

-subject - description

The following components of frameworks have the following descriptions (as given by the following framework as another example):

.subject - the thing that is being described and having associated knowledge and given a name
.description - the content of knowledge associated with a thing
.placement - where and when knowledge is located of a thing

-position - where a thing occurs logically within another idea (as a subsection of a supersection) as seen in idents. positioning nests knowledge under a supersection in which a supersection is thematic with all subsections having a similar theme
-ordering - when a thing occurs logically after another idea as seen in lines coming one after the other

Heeding knowledge

Knowledge in my frameworks is typically very concise and succinct in which there is a great density of knowledge within my frameworks so slow, methodical, and thoughtful reading is necessary to make sure knowledge is understood, heeded, and remembered. Sections have subjects which are typically well named and so this aids remembering the descriptions of a section. How sections relate to each other is important in frameworks so paying attention to idents and if necessary looking or glancing back at previously read section subjects can allow for the good structuring of knowledge within one's mind; with sections which contain many nested subsections that can extend quite far in lines, it is easy to lose track of where knowledge i...
How to best measure if and to what degree you’re too pessimistic or too optimistic?
by CstineSublime — 2024-03-31T00:57:53.982Z

I’ve been told a number of times that I’m too pessimistic about personal outcomes but I feel like I’m realist. So I’d like to test and measure it.

This post on Overconfident Pessimism appears to cover a lot of the same ground and certainly has illuminated for me the way that I become pessimistic or give low probability to tasks or processes I don't yet understand how to do. However the article is chiefly about making predictions about innovation and technological advances, not things in the personal realm.

The problem appears to be predicting where one's own behaviour is involved (although that didn't stop Wilbur Wright).

Never the less, surely if I make a raft of predictions, assign how confident I am in each of them and it turns out I am overwhelmingly overconfidently pessimistic, then it would confirm the "I am a pessimistic hypothesis" and vice versa for someone who is considered to be too optimistic, right?