trentmkelly/LessWrong-43k huggingface dataset:
AI #66: Oh to Be Less Online
by Zvi — 2024-05-30T14:20:03.334Z

Tomorrow I will fly out to San Francisco, to spend Friday through Monday at the LessOnline conference at Lighthaven in Berkeley. If you are there, by all means say hello. If you are in the Bay generally and want to otherwise meet, especially on Monday, let me know that too and I will see if I have time to make that happen.

Even without that hiccup, it continues to be a game of playing catch-up. Progress is being made, but we are definitely not there yet (and everything not AI is being completely ignored for now).

Last week I pointed out seven things I was unable to cover, along with a few miscellaneous papers and reports.

Out of those seven, I managed to ship on three of them: Ongoing issues at OpenAI, The Schumer Report and Anthropic’s interpretability paper.

However, OpenAI developments continue. Thanks largely to Helen Toner’s podcast, some form of that is going back into the queue. Some other developments, including new media deals and their new safety board, are being covered normally.

The post on DeepMind’s new scaling policy should be up tomorrow.

I also wrote a full post on a fourth, Reports of our Death, but have decided to shelve that post and post a short summary here instead.

That means the current ‘not yet covered queue’ is as follows:

DeepMind’s new scaling policy.

Should be out tomorrow before I leave, or worst case next week.

The AI Summit in Seoul.

Further retrospective on OpenAI including Helen Toner’s podcast.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. You heard of them first.

Not Okay, Google. A tiny little problem with the AI Overviews.

OK Google, Don’t Panic. Swing for the fences. Race for your life.

Not Okay, Meta. Your application to opt out of AI data is rejected. What?

Not Okay Taking Our Jobs. The question is, with or without replacement?

They Took Our Jobs Anyway. It’s coming.

A New Leaderboard Appears. Scale.ai offers new capability evaluations.

Copyright Confrontation. Which OpenAI lawsuit was that again?

Deepfaketown and Botpocalypse Soon. Meta fails to make an ordinary effort.

Get Involved. Dwarkesh Patel is hiring.

Introducing. OpenAI makes media deals with The Atlantic and… Vox? Surprise.

In Other AI News. Jan Leike joins Anthropic, Altman signs giving pledge.

GPT-5 Alive. They are training it now. A security commit

...
The 27 papers
by WitheringWeights — 2024-05-30T08:46:39.587Z

List of 27 papers (supposedly) given to John Carmack by Ilya Sutskever: "If you really learn all of these, you’ll know 90% of what matters today."
The list has been floating around for a few weeks on Twitter/LinkedIn. I figure some might have missed it so here you go.
Regardless of the veracity of the tale I am still finding it valuable.
https://punkx.org/jackdoe/30.html

The Annotated Transformer (nlp.seas.harvard.edu)
The First Law of Complexodynamics (scottaaronson.blog)
The Unreasonable Effectiveness of RNNs (karpathy.github.io)
Understanding LSTM Networks (colah.github.io)
Recurrent Neural Network Regularization (arxiv.org)
Keeping Neural Networks Simple by Minimizing the Description Length of the Weights (cs.toronto.edu)
Pointer Networks (arxiv.org)
ImageNet Classification with Deep CNNs (proceedings.neurips.cc)
Order Matters: Sequence to sequence for sets (arxiv.org)
GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism (arxiv.org)
Deep Residual Learning for Image Recognition (arxiv.org)
Multi-Scale Context Aggregation by Dilated Convolutions (arxiv.org)
Neural Quantum Chemistry (arxiv.org)
Attention Is All You Need (arxiv.org)
Neural Machine Translation by Jointly Learning to Align and Translate (arxiv.org)
Identity Mappings in Deep Residual Networks (arxiv.org)
A Simple NN Module for Relational Reasoning (arxiv.org)
Variational Lossy Autoencoder (arxiv.org)
Relational RNNs (arxiv.org)
Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton (arxiv.org)
Neural Turing Machines (arxiv.org)
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin (arxiv.org)
Scaling Laws for Neural LMs (arxiv.org)
A Tutorial Introduction to the Minimum Description Length Principle (arxiv.org)
Machine Super Intelligence Dissertation (vetta.org)
PAGE 434 onwards: Komogrov Complexity (lirmm.fr)
CS231n Convolutional Neural Networks for Visual Recognition (cs231n.github.io)
The Market Singularity: A New Perspective
by azsantosk — 2024-05-30T07:05:11.882Z

"Do you want to be rich, or do you want to be king? — The founder's dilemma.

As we approach the technological singularity, the sometimes applicable trade-off between wealth (being rich) and control (being king) may extend into the realm of AI and governance.

The prevailing discussions around governance often converge on the need for more coordination, effective global standards, and enforcing political regulations. The choice for a more active role of politics is sometimes framed as a choice for human control over the future, not only against the alien wills of future AIs, but also against coordination failures in general. It is a choice for human reasoning against many so-called molochian forces: prisoner's dilemmas, evolutionary races, arms races, races to the bottom, ruthless capitalism.

Centralized control indeed seems to be necessary if we want to ensure that all AI is going to be altruistic, bound by universal ethical standards, or incapable of causing harm.

However, I'll argue that this choice incurs important trade-offs, and may set us up for a much more brutal outcome, one in which we fail to secure human influence in the new institutions that are likely to replace existing ones.

The Choice

The founder's dilemma is the choice some startup founders face between retaining control of a small company, and accepting dilution for a stake in something much bigger. Both choices carry risks. By accepting loss of control, a founder may see their company going in a path they didn't initially approve. By rejecting dilution, a founder risks seeing their company getting outcompeted and eventually disappearing.

Similarly, humans may face a choice between retaining control over institutions, at the risk of making them less efficient, and ceding early control in exchange for a larger stake in whatever comes next. The less efficient institutions can remain in power, but only if they can successfully repress the arrival of new ones.

Repression doesn't need to last forever: some hope our institutions can gradually improve themselves through a long reflection, eventually maximizing something akin to a coherent extrapolated volition. Others think this is a crazy bad idea. But even if you think it is a good idea, it would require the repression to actually be strong enough to work: do it insufficiently and you get a revolution instead. If you choose to rule as a king, you risk yo...
Awakening
by lsusr — 2024-05-30T07:03:00.821Z

This is the story of my personal experience with Buddhism (so far).

First Experiences

My first experience with Buddhism was in my high school's World Religions class. For homework, I had to visit a religious institution. I was getting bad grades, so I asked if I could get extra credit for visiting two and my teacher said yes. I picked an Amida Buddhist church and a Tibetan Buddhist meditation center.

I took off my shoes at the entrance to the Tibetan Buddhist meditation center. It was like nothing I had ever seen before in real life. There were no chairs. Cushions were on the floor instead. The walls were covered in murals. There were no instructions. People just sat down and meditated. After that there was some walking meditation. I didn't know anything about meditation so I instead listened to the birds and the breeze out of an open window. Little did I know that this is similar to the Daoist practices that would later form the foundation of my practice.

The Amida Buddhist church felt like a fantasy novelist from a Protestant Christian background wanted to invent a throwaway religion in the laziest way possible so he just put three giant Buddha statues on the altar and called it a day. The priest told a story about his beautiful stained glass artifact. A young child asked if he could have the pretty thing. The priest, endeavoring to teach non-attachment, said yes. Then the priest asked for it back. The child said no, thereby teaching the priest about non-attachment. Lol.

It would be ten years until I returned to Buddhism.

Initial Search

It is only after you have lost everything that you are free to do anything.

Things were bad. I had dumped six years of my life into a failed startup. I had allowed myself to be gaslit (nothing to do with the startup; my co-founders are great people) for even longer than that. I believed (incorrectly) that I had an STD. I had lost most of my friends. I was living in a basement infested with mice. I slept poorly because my mattress was so broken I could feel the individual metal bedframe bars cut into my back. And that's just the stuff I'm comfortable writing about.

I was looking for truth and salvation. This is about when I discovered LessWrong. LessWrong addressed the truth problem. I still needed salvation.

On top of all this, I had chronic anxiety. I was anxious all the time. I had always been anxious all the time. What was ...
Value Claims (In Particular) Are Usually Bullshit
by johnswentworth — 2024-05-30T06:26:21.151Z

Epistemic status: mental model which I have found picks out bullshit surprisingly well.

Idea 1: Parasitic memes tend to be value-claims, as opposed to belief-claims

By "parasitic memes" I mean memes whose main function is to copy themselves - as opposed to, say, actually provide value to a human in some way (so that the human then passes it on). Scott's old

Toxoplasma of Rage

post is a central example; "share to support X" is another.

Insofar as a meme is centered on a factual claim, the claim gets entangled with lots of other facts about the world; it's the phenomenon of

Entangled Truths, Contagious Lies

. So unless the meme tries to knock out a person's entire epistemic foundation, there's a strong feedback signal pushing against it if it makes a false factual claim. (Of course some meme complexes do try to knock out a person's entire epistemic foundation, but those tend to be "big" memes like religions or ideologies, not the bulk of day-to-day memes.)

But the Entangled Truths phenomenon is epistemic; it does not apply nearly so strongly to values. If a meme claims that, say, it is especially virtuous to eat yellow cherries from Switzerland... well, that claim is not so easily falsified by a web of connected truths.

Furthermore, value claims always come with a natural memetic driver: if X is highly virtuous/valuable/healthy/good/etc, and this fact is not already widely known, then it’s highly virtuous and prosocial of me to tell other people how virtuous/valuable/healthy/good X is, and vice-versa if X is highly dangerous/bad/unhealthy/evil/etc.

Idea 2: Transposons are ~half of human DNA

There are sequences of DNA whose sole function is to copy and reinsert themselves back into the genome. They're called transposons. If you're like me, when you first hear about transposons, you're like "huh that's pretty cool", but you don't expect it to be, like, a particularly common or central phenomenon of biology.

Well, it turns out that something like half of the human genome consists of dead transposons. Kinda makes sense, if you think about it.

Now we suppose we carry that fact over, by analogy, to memes. What does that imply?

Put Those Two Together...

… and the natural guess is that value claims in particular are mostly parasitic memes. They survive not by promoting our terminal values, but by people thinking it’s good and prosocial to tell others...
The Pearly Gates
by lsusr — 2024-05-30T04:01:14.198Z

St. Peter stood at a podium before the Gates of Heaven. The gates were gold, built on a foundation of clouds. A line of people curved and winded across the clouds, beyond what would be a horizon if this plane of existence was positively-curved. Instead, they just trailed away toward Infinity, away from the golden wall securing Heaven.

The worthy would enter eternal paradise. The unforgiven would burn in Hell for just as long. Infinite judgment for finite lives.

"Next please," said St. Peter.

The foremost man stepped forward. He had freckles and brilliant orange hair.

"Tell me about yourself," said St. Peter.

"Me name's Seamus O'Malley, sure, and I was—or still am, begorrah―an Irish Catholic," said Seamus.

"How did you die?" said St. Peter.

"Jaysus, I went and blew meself to bits tryin' to cobble together an auld explosive to give those English occupiers a proper boot, so I did," said Seamus.

"You were a good Catholic," said St. Peter, "You're in."

Seamus entered the Pearly Gates with his head held high.

"Next please," said St. Peter.

A Floridian woman stepped forward.

"My name is Megan Roberts. I worked as a nurse. I couldn't bear to tell people their family members were going to die. I poisoned them so they would die when a less empathetic nurse was on watch," said the nurse.

"That's a grave sin," said St. Peter.

"But it's okay because I'm a Christian. Protestant," said Megan.

"Did you go to church?" said St. Peter.

"Mostly just Christmas and Easter," said Megan, "But moments before I died, I asked Jesus for forgiveness. That means my sins are wiped away, right?"

"You're in," said St. Peter.

"Next please," said St. Peter.

A skinny woman stepped forward.

"My name is Amanda Miller. I'm an Atheist. I've never attended church or prayed to God. I was dead certain there was no God until I found myself in the queue on these clouds. Even right now, I'm skeptical this isn't a hallucination," said Amanda.

"Were you a good person?" asked St. Peter.

"Eh," said Amanda, "I donated a paltry 5% of my income to efficient public health measures, resulting in approximately 1,000 QALYs."

"As punishment for your sins, I condemn you to an eternity of Christians telling you 'I told you so'," said St Peter, "You're in."

"Next please," said St. Peter.

A bald man with a flat face stepped forward.

"My name is Oskar Schindler. I was a Nazi," said Oskar.

"Metaphorical Nazi or Neo-Nazi?" asked St Peter.

"I a...
AXRP Episode 32 - Understanding Agency with Jan Kulveit
by DanielFilan — 2024-05-30T03:50:05.289Z

YouTube link

What’s the difference between a large language model and the human brain? And what’s wrong with our theories of agency? In this episode, I chat about these questions with Jan Kulveit, who leads the Alignment of Complex Systems research group.

Topics we discuss:

What is active inference?

Preferences in active inference

Action vs perception in active inference

Feedback loops

Active inference vs LLMs

Hierarchical agency

The Alignment of Complex Systems group

Daniel Filan:
Hello, everybody. This episode, I’ll be speaking with Jan Kulveit. Jan is the co-founder and principal investigator of the Alignment of Complex Systems Research Group, where he works on mathematically understanding complex systems composed of both humans and AIs. Previously, he was a research fellow at the Future of Humanity Institute focused on macrostrategy, AI alignment, and existential risk. For links to what we’re discussing you can check the description of this episode and you can read the transcript at axrp.net. Okay. Well Jan, welcome to the podcast.

Jan Kulveit:
Yeah, thanks for the invitation.

What is active inference?

Daniel Filan:
I’d like to start off with this paper that you’ve published in December of this last year. It was called “Predictive Minds: Large Language Models as Atypical Active Inference Agents.” Can you tell me roughly what was that paper about? What’s it doing?

Jan Kulveit:
The basic idea is: there’s active inference as a field originating in neuroscience, started by people like Karl Friston, and it’s very ambitious. The active inference folks claim roughly that they have a super general theory of agency in living systems and so on. And there are LLMs, which are not living systems, but they’re pretty smart. So we’re looking into how close the models actually are. Also, it was in part motivated by… If you look at, for example, the ‘simulators’ series or frame by Janus and these people on sites like the Alignment Forum, there’s this idea that LLMs are something like simulators - or there is another frame on this, that LLMs are predictive systems. And I think this terminology… a lot of what’s going on there is basically reinventing stuff which was previously described in active inference or predictive processing, which is another term for minds which are broadly trying to predict their sensory inputs.

And it seems like there is a lot of sim...
US Presidential Election: Tractability, Importance, and Urgency 
by kuhanj — 2024-05-29T23:52:22.420Z

Disclaimer: To avoid harmful polarization of important topics, this post is written in a non-partisan manner, and I’d encourage comments to be written with this in mind.

US presidential elections are surprisingly tractable

US presidential elections are often extremely close.

Biden won the last election by 42,918 combined votes in three swing states. Trump won the election before that by 77,744 votes. 537 votes in Florida decided the 2000 election.

There’s a good chance the 2024 election will be very close too.

Trump leads national polling by around 1% nationally, and polls are tighter than they were the last two elections. If polls were perfectly accurate (which of course, they aren’t), the tipping point state would be Pennsylvania or Michigan, which are currently at +1-2% for Trump.

There is still low-hanging fruit. Estimates for how effectively top RCT-tested interventions to generate net swing-state votes this election range from a few hundred to several thousand dollars per vote. Top non-RCT-able interventions are likely even better. Many potentially useful strategies have not been sufficiently explored. Some examples:

mobilizing US citizens abroad (who vote at a ~10x lower rate than citizens in the country), or swing-state university students (perhaps through a walk-out-of-classes-to-the-polls demonstration).
There is no easily-searchable resource on how to best contribute to the election. (Look up the best ways to contribute to the election online – the answers are not very helpful.)
Anecdotally, people with little political background have been able to generate many ideas that haven’t been tried and were received positively by experts.

Many top organizations in the space are only a few years old, which suggests they have room to grow and that more opportunities haven’t been picked.
Incentives push talent away from political work:

Jobs in political campaigns are cyclical/temporary, very demanding, poorly compensated, and offer uncertain career capital (i.e. low rewards for working on losing campaigns).
How many of your most talented friends work in electoral politics?

The election is more tractable than a lot of other work: Feedback loops are more measurable and concrete, and the theory of change fairly straightforward. Many other efforts that significant resource

...
Thoughts on SB-1047
by ryan_greenblatt — 2024-05-29T23:26:14.392Z

In this post, I'll discuss my current understanding of SB-1047, what I think should change about the bill, and what I think about the bill overall (with and without my suggested changes).

Overall, SB-1047 seems pretty good and reasonable. However, I think my suggested changes could substantially improve the bill and there are some key unknowns about how implementation of the bill will go in practice.

The opinions expressed in this post are my own and do not express the views or opinions of my employer.

[This post is the product of about 4 hours of work of reading the bill, writing this post, and editing it. So, I might be missing some stuff.]

[Thanks to various people for commenting.]

My current understanding

(My understanding is based on a combination of reading the bill, reading various summaries of the bill, and getting pushback from commenters.)

The bill places requirements on "covered models'' while not putting requirements on other (noncovered) models and allowing for limited duty exceptions even if the model is covered. The intention of the bill is to just place requirements on models which have the potential to cause massive harm (in the absence of sufficient safeguards). However, for various reasons, targeting this precisely to just put requirements on models which could cause massive harm is non-trivial. (The bill refers to “models which could cause massive harm” as “models with a hazardous capability".)

[Edit: limited duty exemption have sadly been removed which makes the more costly while not improving safety. I discuss further in this comment.]

In my opinion, I think the bar for causing massive harm defined by the bill is somewhat too low, though it doesn't seem like a terrible choice to me. I'll discuss this more later.

The bill uses two mechanisms to try and improve targeting:

Flop threshold: If a model is trained with <10^26 flop and it is not expected to match >10^26 flop performance as of models in 2024, it is not covered. (>10^26 flop performance as of 2024 is intended to allow the bill to handle algorithmic improvements.)
Limited duty exemption: A developer can claim a limited duty exemption if they determine that a model does not have the capability to cause massive harm. If the developer does this, they must submit paperwork to the Frontier Model Division (a division created by the bill) explaining their reasoning.

From my understandin...
How I designed my own writing system, VJScript
by vkethana — 2024-05-29T23:18:49.170Z
Link: https://www.vkethana.com/vjscript/

I designed my own writing system and wrote about it on my personal website. I discuss why the current system of English spelling is bad, how VJScript fixes these issues, and provide plenty of detailed examples of what the writing system looks like. The name "VJScript" is a portmanteau of my name "Vijay" and the word "script".

As a sneak peak, here's what a typical sentence in VJScript looks like:

If this sounds interesting to you, you can check out the post here: https://www.vkethana.com/vjscript/

This is my first serious blog post, so thanks for reading!