trentmkelly/LessWrong-43k huggingface dataset:
Life Has a Cruel Symmetry
by philh — 2023-01-23T23:40:04.161Z

t ≈ 0s

I am born into knowledge.

I know the world is dangerous, but not malign. If I were unshielded, I would quickly be killed, torn apart by eddies in spacetime that grow and shrink and combine and split, turning all they encounter to ash.

But I know that I am shielded. I have been given a garden, 264 microns to a side, in the northwest corner of the world. And the world will not intrude. Walls and gates surround it, and while the ash can get through, the probability is… I begin to calculate, but around 2-24 it seems no longer worth consideration. Within the garden, there is only myself. Outside, the world is incomprehensibly vast, 2128 microns on a side.

I know the laws of physics. An atom will be created wherever it will bond to three other atoms; remaining while it has two or three bonds; and then disappear. A single unexpected atom, or its absence, can have vast consequences.

(And I wonder how I know these things. I have no memory of learning them. If they were not true, would I not-know them? I begin to consider that they may be false.)

I know that I am not free from physics. I have, in some sense, no more will than the ash. But in another, more important sense -

I know what is Good. At the end, the world will be horrifyingly bad, unless I make it Good - but I know Good, and I can act, and I can make it Good.

(I must, I think, have been created by God. How else would I know what is Good? Goodness is not written into the laws of physics; one cannot derive an ought from an is. But I do not know why God gave me such a daunting task. Does Their power extend only over my tiny garden? Or - a welcome possibility - perhaps They can see that I will succeed. Although I seem unfathomably complex to myself, perhaps to God I am the simplest path to attaining Good.)

(And if God created me, knowing Good, that I might make the world Good, then I can reasonably guess that my unexplained knowledge came from God. And - further - I can guess that They would not create me knowing lies, for what purpose would that serve? But these are guesses; I begin to trust my knowledge more, but not absolutely.)

I do not know how large I am. I do not know how quickly I think. God has given me a sense of time, so that I may know when the end will come. (Precisely 2176 seconds after I was born. I've spent roughly 220 seconds thinking.) But I cannot sense every physical tick, I don't think such a t...
Highlights and Prizes from the 2021 Review Phase
by Raemon — 2023-01-23T21:41:21.948Z

We've had a ton of reviews for the The LessWrong 2021 Review, and I wanted to take a moment to: a) share highlights from reviews that felt particularly valuable to me, b) announce the prizes so far, to give people a better idea of how prizes are awarded.

Tl;dr on prizes: we've awarded $4,425 in prizes so far. I've been aiming to pretty consistently give at least a $50-$100 prize for all reviews that put in some effort, and am happy to pay more for good reviews. The largest prize so far has been $200 but I'd be excited to give $500+ to in-depth reviews of posts that were ranked highly in the preliminary voting.

If you'd consider doing reviews but are uncertain about the payoff, PM me and most likely we can work out something out.

Reviews I liked

Nostalgebraist on Fun with +12 OOMs of Compute

After reading Nostalgebraist's review of Fun with +12 OOMs of Compute, I felt like I actually understood the main point for the first time:

This post provides a valuable reframing of a common question in futurology: "here's an effect I'm interested in -- what sorts of things could cause it?"

That style of reasoning ends by postulating causes. But causes have a life of their own: they don't just cause the one effect you're interested in, through the one causal pathway you were thinking about. They do all kinds of things.

In the case of AI and compute, it's common to ask

Here's a hypothetical AI technology. How much compute would it require?

But once we have an answer to this question, we can always ask

Here's how much compute you have. What kind of AI could you build with it?

If you've asked the first question, you ought to ask the second one, too.

The first question includes a hidden assumption: that the imagined technology is a reasonable use of the resources it would take to build. This isn't always true: given those resources, there may be easier ways to accomplish the same thing, or better versions of that thing that are equally feasible. These facts are much easier to see when you fix a given resource level, and ask yourself what kinds of things you could do with it.

This high-level point seems like an important contribution to the AI forecasting conversation. The impetus to ask "what does future compute enable?" rather than "how much compute might TAI require?" influenced my own view of Bio Anchors, an influence that's visible in the cont

...
AI safety milestones?
by Zach Stein-Perlman — 2023-01-23T21:00:24.441Z

Optimal governance interventions depend on progress in technical AI safety. For example, two rough technical safety milestones are have metrics to determine how scary a model's capabilities are and have a tool to determine whether a model is trying to deceive you. Our governance plans should adapt based on whether these milestones have been achieved (or when it seems they will be achieved) (and for less binary milestones, adapt based on partial progress).

What are more possible milestones in technical AI safety (that might be relevant to governance interventions)?
A post-quantum theory of classical gravity?
by Logan Zoellner — 2023-01-23T20:39:03.564Z

This paper proposes a unified theory of gravity and quantum physics which:

Is empirically testable (for example)
Gravity itself is non quantum
physics is inherently stochastic (God does play dice)
Objective waveform collapse exists ("the interaction
of the quantum degrees of freedom with classical space-time necessarily causes decoherence")

Are there any obvious reasons to dismiss this out of hand?

Would have lots of interesting implications (e.g. objective waveform collapse means no multiverse).
Meals For Unclear Dietary Restrictions
by jefftk — 2023-01-23T20:00:07.554Z

When someone's coming for over for dinner, I'll ask about their
dietary restrictions. In my social groups it's common for people to be
vegetarian or vegan, and others have a food allergy, are gluten-free,
etc. Occasionally, however, I find myself making food without knowing
my constraints. Perhaps someone's "bringing a friend" or when I asked
they said "my food situation is pretty complicated, I can eat after if
there's nothing that works for me." In these cases one option is to
serve a meal that works for a wide variety of dietary restrictions.

I wouldn't want to only cook like this: every restriction you put on
your cooking makes it harder. (Not specific to cooking!) Still, there
are a few meals that do work well this way. For example, this is the
main situation where I'll cook baked potatoes with stuff to put on
them:

Baked potatoes

Black beans

Cheese sauce

Ground beef or bacon

Caramelized onions

Roasted cherry tomatoes

Chopped scallions

Olives

Sour cream

Baby spinach

This is mainstream American food, but because each person chooses what
goes on their plate it works for nearly everyone: vegetarians leave
out the meat, vegans leave out the cheese and cream, low-carb people
leave out the potato, potato-only
people just eat the potato, there's no gluten, etc. Which exact
sides you choose don't matter too much as long as you think a bit
about dietary categories. I'm probably listing a few more than I
would normally make.

If I didn't want to make potatoes I'd do tacos/burritos, without
trying for something authentic. This could be:

Corn tortillas, briefly heated

Flour tortillas, same

Black beans

Grated cheese

Ground beef, chicken, or carnitas

Pico de gallo

Salsa

Roasted bell pepper slivers

Guacamole [1]

Sour cream

Baby spinach

And yes, this has a lot in common with the previous meal, but the
spices are pretty different and I wouldn't mind serving them on
consecutive days.

Other meals where there are a lot of options on the table and each
person chooses their own combination also work. The main things I keep
in mind are trying to keep each component relative simple, having at
least two protein options, at least one vegan protein option, and at
least one non-glutenous starch option.

[1] I usually don't make guacamole, because I don't really like it and
I prefer not to prepare food I don't enjoy....
It's ok
by stratospher — 2023-01-23T18:11:00.530Z

I probably shouldn't be ranting out my thoughts here. but I really want to get these thoughts out of my head and I'm hopeful no one will come across this random piece.

Have you ever felt obsessed with something you're building? I did - my last 2 years in college. It's an amazing feeling. It's like riding the ocean current - you have no idea why you're going so fast but the speed feels insane and you're at escape velocity. Everything else fades. It's just this desire to see something manifest itself in the real world.

Was it really that idealistic? No. There were 4 of us in a dorm room. You get comfortable with weird glances in your direction and then one day it's become normal. You get used to people thinking you're crazy for staying up everyday 3 am plus and working - many times even during lunch breaks. You get used to not talking to people - it's difficult to explain to them this intense sensation that's driving you and they weren't really the kind of people who'd understand[1]. Hell, I never understood it then either. It was definitely lonely but I was curious. I wanted to know more about who I was. I disappeared under a pseudonym to answer them - who am I? If no one knew me, would I still learn and work on the same things? If no one appreciates me, do I care about my work enough that I'd still do it? If I take away all external driving factors like fame, security, power(except money lol, we need to be able to afford a life) - would I find my truer motivation in life.

It's on the edge of insanity that you see the most incredible views. It's always about people. I just wanted to have fun working with people I enjoy being around on things that matter.

Working on open source is different. You're free. You can do everything and anything - the only barriers are your mind. It's a constant battle tbh. You get used to it. You want to battle even better problems the next time. The thrill hits when you end up understanding/building things that looked impossible to you at first. It's intimidating though and a huge cause of insecurity/trauma.

One day, I want to be able to just perform without questioning myself about whether I'm good enough.

Looking back, do I have everything figured out now? Sadly, no.

But I'm a much better and stronger person than I could have ever imagined myself to be. New dimensions I was too ignorant to notice then keep popping up. ...
Experimenting with beta.character.ai 
by  — 2023-01-23T17:31:49.243Z

Transcript Link: https://imgur.com/a/EnRLfLc (I am deadzone in this conversation).

This transcript convinced me that this Large Language Model was definitely more conscious than a monkey, and not far away from human-level consciousness. I believe with slightly more parameters, we can get something indistinguishable from humans - and it will be able to contribute to open source and discover new theorems etc.

To me, it just feels like we are 5-6 years from AGI. To be more concrete I think 100x GPT-3's parameters will easily be superhuman at all tasks.
This week in fashion
by Jan — 2023-01-23T17:23:57.191Z
Link: https://universalprior.substack.com/p/this-week-in-fashion

TL;DR: A holiday obsession turns into a deep meditation on all things pretty. Albatrosses and reward *models* included. Also, check out www.fashionator.xyz (no malware, I promise).

Over the Christmas holiday, I became slightly obsessed with the Netflix show "

Next in Fashion

." It's (probably) only a temporary obsession, nothing to worry about. But before I move on to the next obsession, I wanted to take a moment to write up what I learned[1].

Buckle up, this will be a weird one.

Before I get into the nitty-gritty of my newfound interest in fashion, let me first address the elephant in the room - why care about fashion in the first place[2]?

Why fashion

There are three different schools[3] of thought on the question.

School of the devil: The philosophy of "

The Devil Wears Prada

[4]" is centered around the idea that fashion is a form of power and influence. The film explores the idea that the fashion industry is a demanding and competitive field that requires a high level of expertise and commitment, and that subverting the rigid hierarchy of fashion can have unintended or unexpected consequences.

School of the barber pole: Scott Alexander's

barber pole model

posits that fashion trends move in a predictable cycle, with each new trend allowing elites to distinguish themselves from runner-ups (who take a while to catch up). An important prediction of this model is that fashion worn by the “upper” class can end up looking a lot like fashion worn by the “lower” class, who tend to pick up on trends at a time delay. As such, fashion is primarily a vehicle for signaling social status and group membership.

School of reality TV: While never spelled out explicitly, in “Next in Fashion” fashion is a form of creative expression and a way to appreciate art. The show centers around the skill and talent required to create unique and innovative designs, and the importance of supporting and nurturing new talent in the industry. In this interpretation, fashion can also help us better understand ourselves, each other, and the world we live in.

Why I'm not buying it

But none of these explanations are quite satisfying (or even mutually exclusive). Let’s put them under the microscope one by one.

Is fashion a form of power and influence?

Let's start with numbers: Global revenue from luxury apparel

comes in at US$82.30bn

, which is a so-called

no

...
Movie Review: Megan
by Zvi — 2023-01-23T12:50:00.873Z

I will follow my usual procedure here – a fully spoiler-free 1-bit review, then a mostly spoiler-free short review to determine if you should see it, then a fully detailed spoiler-filled analysis.

Fully Spoiler-Free 1-Bit Review – Should You See Megan In Theaters, Yes or No?

Yes.

Mostly Spoiler-Free Review In Brief

The preview for Megan had me filing the movie as ‘I don’t have to see this and you can’t make me.’ Then Scott Aaronson’s spoiler-filled review demanded a response and sparked my interest in various ways, causing me to see the movie that afternoon.

After an early setback, Megan exceeded expectations. I was as surprised as you are.

As entertainment, Megan is a Tier 3 movie1, or about 4 out of 5 stars on the old Netflix scale. The 72 on Metacritic is slightly low. It’s good.

Megan has intelligent things to say and holds up well, which is why this post exists. I am recommending it both as a good time and a future reference point.

If you do not like the genre this movie is in, and also are not interested in issues surrounding AI, you can safely skip it.

Full SPOILERIFIC Analysis

This analysis is intended to make sense for people who don’t have a background in the problem that AI, by default, kills everyone. If you already know about that, forgive the belaboring and butchering.

Premise!

The basic plot of Megan is:

AI Doll Megan is built, told to protect child from ‘physical and emotional harm.’

Megan learns to kill people to achieve its objective function.

When people notice, Megan fights against attempts to shut it down.

Or in more detail:

Cady’s parents die in a car accident. She is about 8.

Cady’s aunt Gemma is assigned as guardian.

Gemma works at Explicitly Not Hasbro Toy Company and is secretly the World’s Greatest AI expert and robot building badass.

Gemma uses a previous toy as a spy to collect training data on child interactions. She plans to use this to create lifelike AI dolls. Development still has some issues.

She tells her boss. Boss says ‘I don’t want to hear about this’ on both counts and wants her back to work on simple $50 virtual pet designs.

After Cady shows interest, Gemma builds the AI robot doll, Megan, to serve as Cady’s companion and toy. At home. In a week.

Megan is a ‘generative model’ with a ‘constant learning function and focus on self-improvement.’

Megan imprints

...
Has private AGI research made independent safety research ineffective already? What should we do about this?
by Roman Leventov — 2023-01-23T07:36:48.124Z

This post is a variation on "Private alignment research sharing and coordination" by porby. You can consider my question as signal-boosting that post.

AGI research is becoming private. Research at MIRI is nondisclosed-by-default for more than four years now. OpenAI stopped publishing details of their work[1], Hassabis also talked about this here.

Does this mean that independent AI safety research begins to suffer from knowledge asymmetry and becomes ineffective?

There are two directions possible directions of knowledge asymmetry:

State-of-the-art scaling results or even novel architectures are not published, and interpretability researchers use outdated models in their work. Hence, these results may not generalise to bigger model scales and architectures. The counterargument here is that in what comes to scale, relatively low-hanging interpretability fruit is still possible to pick even when analysing toy models. In what comes to architectures, transformer details may not matter that much for interpretability (however, Anthropic's SoLU (Softmax Linear Unit) work seems to be the evidence against this statement: relatively minor architectural change has led to significant changes in the interpretability characteristics of the model); and if one of the AGI labs stumbles upon a major "post-transformer" breakthrough, this is going to be an extremely closely-guarded secret which will not spread with rumours to AI safety labs, and hence joining the "knowledge space" of these AI safety labs won't help independent AI safety researchers.
Some safety research has already been done but was not published, either because of its relative infohazardousness or because it references private capability research, as discussed in the previous point. Oblivious to this work, AI safety researchers may reinvent the wheel rather than work on the actual frontier.

Private research space

If the issue described above is real, maybe the community needs some organisational innovation to address it.

Perhaps it could be some NDA + "noncompete" agreement + publishing restriction program led by some AI safety lab (or a consortium of AI safety labs) joining which is not the same as joining the lab itself in the usual sense (no reporting, no salary), but which grants access to the private knowledge space of the lab(s). The effect for the lab will be as if they enlarged their research group, the extra pa...
Deconfusing "Capabilities vs. Alignment"
by RobertM — 2023-01-23T04:46:57.458Z

I've lately been running into the idea that interpretability research might be bad because it could speed up capabilities more than alignment with increasing frequency. I find this idea interesting because it points to a potentially productive mine of buried confusion.

Stepping back a little bit: much discussion about e.g. differential progress on alignment vs. capabilities collapses a lot of complexity into single-dimensional quantities to be maximized. There is also a common argument of the form "there's a lot of overlap between alignment and capabilities work".[1] That argument doesn't make the exact same reduction but buries most of the same complexity.

What are capabilities?

Actually, the question is something like "how should we model capabilities, such that we can usefully distinguish capabilities advancements are more likely to be downstream of certain kinds of alignment research, vs. less likely?".

The abstraction I'm playing around with right now is within-paradigm improvements vs. new-paradigm discovery.

Within-paradigm improvements might look like interpretability tooling which enables one-off or manually discovered improvements to hyperparameter tuning or other slight tweaks to existing architectures.

New-paradigm discovery could be anything from "this tooling let us figure out [something generalizable as applied to existing architectures, i.e. scaling laws]" to "this tooling let us come up with a brand new and improved model architecture".

What is alignment interpretability?

"Alignment" is too wide a bucket to be usefully broken down, because the categories at the "next" level of abstraction are still too wide to usefully reason about with respect to their potential impact on capabilities. Let's look at an individual category: intepretability.

The question starts to break down. How likely are insights and tooling focused on transformer circuits to advance capabilities?[2] Shapley values? Superposition? ELK? (Please keep in mind that if you do come to have a model by which some specific interpretability technique can be used to make rapid capabilities advancements, it is not obvious to me that a public LessWrong post is the best way to warn other interpretability researchers of this fact.)

What else?

It's difficult to confidently answer questions of the form "how likely is [interpretability techni...
What a compute-centric framework says about AI takeoff speeds
by Tom Davidson — 2023-01-23T04:02:07.672Z

As part of my work for Open Philanthropy I’ve written a draft report on AI takeoff speeds, the question of how quickly AI capabilities might improve as we approach and surpass human-level AI. Will human-level AI be a bolt from the blue, or will we have AI that is nearly as capable many years earlier?

Most of the analysis is from the perspective of a compute-centric framework, inspired by that used in the Bio Anchors report, in which AI capabilities increase continuously with more training compute and work to develop better AI algorithms.

This post doesn’t summarise the report. Instead I want to explain some of the high-level takeaways from the research which I think apply even if you don’t buy the compute-centric framework.

The framework

h/t Dan Kokotajlo for writing most of this section

This report accompanies and explains https://takeoffspeeds.com (h/t Epoch for building this!), a user-friendly quantitative model of AGI timelines and takeoff, which you can go play around with right now. (By AGI I mean “AI that can readily[1] perform 100% of cognitive tasks” as well as a human professional; AGI could be many AI systems working together, or one unified system.)

Takeoff simulation with Tom’s best-guess value for each parameter.

The framework was inspired by and builds upon the previous “Bio Anchors” report. The “core” of the Bio Anchors report was a three-factor model for forecasting AGI timelines:

Dan’s visual representation of Bio Anchors report

Compute to train AGI using 2020 algorithms. The first and most subjective factor is a probability distribution over training requirements (measured in FLOP) given today’s ideas. It allows for some probability to be placed in the “no amount would be enough” bucket.

The probability distribution is shown by the coloured blocks on the y-axis in the above figure.

Algorithmic progress. The second factor is the rate at which new ideas come along, lowering AGI training requirements. Bio Anchors models this as a steady exponential decline.

It’s shown by the falling yellow lines.

Bigger training runs. The third factor is the rate at which FLOP used on training runs increases, as a result of better hardware and more $ spending. Bio Anchors assumes that hardware improves at a steady exponential rate.

The FLOP used on the biggest training run is shown by the rising purple lines.

Once there’s been enough algorithmic progres...