trentmkelly/LessWrong-43k huggingface dataset:
Subjective Questions Require Subjective information
by Ben — 2024-04-23T13:16:16.865Z

This summarizes a (possibly trivial) observation that I found interesting.

Story

An all-powerful god decides to play a game. They stop time, grab a random human, and ask them "What will you see next?". The human answers, then time is switched back on and the god looks at how well they performed. Most of the time the humans get it right, but occasionally they are caught by surprise and get it wrong.

To be more generous the god decides to give them access (for the game) to the entirety of all objective facts. The position and momentum of every elementary particle, every thought and memory anyone has ever had (before the time freeze) etc. However, suddenly performance in the game drops from 99% to 0%. How can this be? They have more information, they know everything!

If you have the memories of every single human up to that point, then you don't know which of them you are. Who is "you" in the "What will you seen next?"? Before the extra information was added you knew which human you were, it was made very obvious by the memories and information at your disposal. But, given the memories of everyone, and all that other information, you suddenly require an additional piece of information to answer the question.

God: "What will you see next?"

Participant: "I know what every human will see next, but I don't know which one I am".

G: "You have been given all the information there is. How can you not know?"

Idea

"What will you see next?" is a subjective question. All objective facts put together is not enough to answer it, because an addition piece of information "Which person am I"? is needed. This final piece of information is subjective, and arguably in some materialistic sense doesn't really exist. But, the question is also subjective, so it should not be surprising that subjective information is needed to answer it.

You can make the situation more extreme in various ways. Instead of providing a snapshot of the universe at a particular time the god could provide the participant the entire history of the universe from beginning to end (a block universe). Then they need to know not only which person they are, but at what time. If you take Many Worlds seriously then a participant given the wavefunction of the entire universe would also need to ask "Which branch am I in?"

In everyday life this doesn't really matter, because we know which human we are, and through cont...
Rejecting Television
by Declan Molony — 2024-04-23T04:59:50.253Z

I didn’t use to be, but now I’m part of the

2%

of

U.S. households

without a television. With its near ubiquity, why reject this technology?

The Beginning of my Disillusionment

Neil Postman’s book Amusing Ourselves to Death radically changed my perspective on television and its place in our culture. Here’s one illuminating passage:

We are no longer fascinated or perplexed by [TV’s] machinery. We do not tell stories of its wonders. We do not confine our TV sets to special rooms. We do not doubt the reality of what we see on TV [and] are largely unaware of the special angle of vision it affords. Even the question of how television affects us has receded into the background. The question itself may strike some of us as strange, as if one were to ask how having ears and eyes affects us. [In the 1960s], the question “Does television shape culture or merely reflect it?” held considerable interest for scholars and social critics. The question has largely disappeared as television has gradually become our culture. This means that we rarely talk about television, only what is on television—that is, about its content.

Postman wrote this in 1985 and unmasked the

gorilla in the room

—a culture that has acquiesced to the institution of television. Having grown up with one in my family home since birth, I took its presence for granted. I didn’t question it anymore than I might have questioned any other utility such as running water or electricity. So who would be crazy enough in the 21st century to forego television?

A Man who was Crazy Enough

One day while exploring YouTube, I came across an obscure 2003

interview

with author David Foster Wallace.

Interviewer: “Do you watch TV?”

Wallace: “I don’t have TV because if I have a TV, I will watch it all the time. So there is my little confession about how strong I am at resisting stuff.”

He elaborates further in the interview

here

:

“One of the reasons I can’t own a TV is…I’ve become convinced there’s something really good on another channel and that I’m missing it. So instead of watching, I’m scanning anxiously back and forth. Now all you have to do is [motions clicking a remote]—you don’t even have to get up now to change [the channel]! That’s when we were screwed.”

Wallace said this twenty years ago. And while

younger generations

aren’t watching cable television...
LW Frontpage Experiments! (aka "Take the wheel, Shoggoth!")
by Ruby — 2024-04-23T03:58:43.443Z

Update: June 20th

After a few rounds of adjustments and careful examination of the data, we've decided to make the Enriched tab be the default for all logged-in users[1]. Anyone is who has not already switched tabs ever will be set to Enriched tab. If you dislike it, you can switch to the Latest tab, the prior default.

We're not completely certain this is the correct choice long-term, but the results seem good enough to progress to a broader roll out for now, though we'll keep monitoring.

We've also enabled a Recommended tab, so the available tabs are now:

Latest: 100% post from the Latest algorithm (using karma and post age to sort)
Enriched (default): 50% posts from the Latest algorithm, 50% posts from the recommendations engine
Recommended: 100% posts from the recommendations engine, choosing posts specifically for you based on your history
Subscribed: a feed of posts and comments from users you have explicitly followed
Bookmarks: this tab appears if you have bookmarked any posts

Update: May 13th

If you're reading this, it's possible you just found yourself switched to the Enriched tab. Congratulations! You were randomly assigned to be fed to the Shoggoth to a group of users automatically switched to the new posts list.

The Enriched posts list:

Is 50% the same algorithm as Latest, 50% ML-algorithm selected posts for you based on your post interaction history.
The sparkle icon next to the post title marks which posts were the result of personalized recommendations.
You can switch back at any time to the regular Latest tab if you don't like the recommendations
We changed the name "Recommended" to "Enriched" to better imply that it contains 50% of the regular Latest posts. (We will probably soon add a Recommended tab that is 100% recommendations.)

You can read further discussion of the experiments in this comment.

Original Post, April 22nd

For the last month, @RobertM and I have been exploring the possible use of recommender systems on LessWrong. Today we launched our first site-wide experiment in that direction.

Behold, a tab with recommendations!

(In the course of our efforts, we also hit upon a frontpage refactor that we reckon is pretty good: tabs instead of a clutter of different sections. For now, only for logged-in users. Logged-out users see the "Latest" tab, which is the same-as-usual list of posts.)

Why algorithmic recommendations?

A core value of LessWrong ...
Thoughts on Zero Points
by depressurize — 2024-04-23T02:22:27.448Z
Link: https://sexandchicago.substack.com/p/a-very-bad-and-uninteresting-post

There’s a cool concept I’ve been thinking about. I first heard of it when reading Jesse Schell’s book “The Art of Game Design”. (Fun fact: Jesse Schell was my professor’s professor, aka my grand-professor.)

Then I heard of it again in the LessWrong post “

Choosing the Zero Point

”. Having been exposed to it twice, I now see it everywhere. I’m not sure how to describe it though, so I’ll just throw a bunch of examples at you:

Jesse Schell: the developers of World of Warcraft wanted players to play just a little each day. In the game, you gain “experience points” as you play. So they decided you would get less experience points once you had already been playing for half an hour. Of course, players hated this. But instead of actually changing the system, the developers just reduced the baseline amount of experience points you got, and then said that you got a bonus during the first half hour.

In the words of Chris Remo from the Idle Thumbs podcast, “It’s EXACTLY the same as it was before, except NOW everyone is like ‘Fuck yeah, Blizzard, this is exactly what I want!’"
Intensionally, it feels very different to get a bonus during the first half hour than it does to get penalized beyond the first half hour.
But extensionally, the two systems are the exact same. You get more experience points during the first half hour of each day than during the rest of the time you play.
P.S. the above is from memory and may not be 100% accurate to how it happened IRL.

LessWrong: some people portray eating meat as evil, and not eating meat as the bare minimum to be a decent person. But it may be more persuasive to portray eating meat as neutral, and to portray not eating meat as an insanely awesome opportunity to do a massive amount of good.

Intensionally, they are very different. Finding out that you’ve actually been perpetuating a great evil and must drastically change your behavior and give up something you enjoy to just get back to the baseline, sounds like bad news.
But finding out about an opportunity to change your behavior for massive moral benefits sounds like good news.
Extensionally, they are the same. From a utilitarian perspective, there is not an obvious reason why we should prefer one interpretation over the other. All that utilitarianism tells us is that not eating meat is much better than eating meat.

The following are examples of the same phenomenon that I noticed:

How most pe

...
Funny Anecdote of Eliezer From His Sister
by Noah Birnbaum — 2024-04-22T22:05:31.886Z

This comes from a podcast called 18Forty, of which the main demographic of Orthodox Jews. Eliezer's sister (Hannah) came on and talked about her Sheva Brachos, which is essentially the marriage ceremony in Orthodox Judaism. People here have likely not seen it, and I thought it was quite funny, so here it is:

https://18forty.org/podcast/channah-cohen-the-crisis-of-experience/

David Bashevkin:

So I want to shift now and I want to talk about something that full disclosure, we recorded this once before and you had major hesitation for obvious reasons. It’s very sensitive what we’re going to talk about right now, but really for something much broader, not just because it’s a sensitive personal subject, but I think your hesitation has to do with what does this have to do with the subject at hand? And I hope that becomes clear, but one of the things that has always absolutely fascinated me about you and really increased my respect for you exponentially, is that you have dedicated much of your life and the focus of your research on relationships and particularly the crisis of experience in how people find and cultivate relationships. And your personal background on this subject to me really provides a lot of contexts of how I see you speaking. I’m mentioning this for two reasons. Your maiden name is?

Channah Cohen:

Yudkowsky.

David Bashevkin:

Yudkowsky. And many of our listeners, though not all of our listeners will recognize your last name. Your older brother is world famous. It’s fair to say, world famous researcher in artificial intelligence. He runs a blog that I don’t know if they’re still posting on it was called LessWrong. He wrote like a massive gazillion page fan fiction of Harry Potter. Your brother is Eliezer Yudkowsky.

Channah Cohen:

Yes.

David Bashevkin:

You shared with me one really beautiful anecdote about Eliezer that I insist on sharing because it’s so sweet. He spoke at your sheva brachos.

Channah Cohen:

Yes.

David Bashevkin:

And I would not think it was not think that Eliezer Yudkowsky would be the best sheva brachos speaker, but it was the most lovely thing that he said. What did Eliezer Yudkowsky say at your sheva brachos?

Channah Cohen:

Yeah, it’s a great story because it was mind-blowingly surprising at the time. And it is, I think the only thing that anyone said at a sheva brachos that I actually remember, he got up at the first sheva brachos and he sai...
How LLMs Work, in the Style of The Economist
by utilistrutil — 2024-04-22T19:06:46.687Z

The Assignment:

(4 hours) Write an Economist-style explainer article on how LLMs work. You’ve just started as an AI reporter at The Economist, and your editor’s realised there’s no good Economist Explains style piece on how LLMs work. They’ve asked you to write one. It should be 500 words, and in the style of other Economist Explains pieces.

Examples: Economist explainer on biological weapons; Economist explainer on diffusion models; FT explainer on transformers.

Thank you to Shakeel Hashim for feedback! Shakeel previously worked at The Economist as an editor.

Since OpenAI released ChatGPT in November 2022, large language models (LLMs) have gained international attention. A language model is a piece of AI software designed for tasks like translation, speech recognition, or—in the case of ChatGPT—conversations with humans. Language models, and even chatbots, are not new. In the 1960s, the first chatbot, ELIZA, was developed at MIT. ELIZA’s programmer had to write down a precise set of instructions for the chatbot to follow, including canned responses like “Tell me more about such feelings.” Modern language models, by contrast, must learn the structure of language from scratch by poring over internet text and compressing this knowledge across billions of numbers, or ‘weights’. In this way, these language models are ‘large’.

When an LLM receives input text from a user, the words are sliced up into ‘tokens’, and these sub-words are ‘embedded’ into numbers. The numbers representing the user’s input are then passed through the weights of the model to produce the first token of the model’s output. By iterating this process, the model generates a complete response. To find a set of weights capable of performing this powerful task, engineers ‘pre-train’ the model on vast quantities of human text. When the model outputs a token that does not match the next token in the training data, the model’s weights are nudged in the direction that would have produced the correct token. In a process known as Reinforcement Learning from Human Feedback, the pre-trained model is then molded into a ‘helpful, harmless, and honest’ assistant by training on feedback from contracted data labelers. The result is a modern chatbot that answers questions, fulfills requests, and occasionally hallucinates false information.

To generate text, an LLM is tasked with predicting what a human w...
Measuring Coherence and Goal-Directedness in RL Policies
by dx26 — 2024-04-22T18:26:37.903Z

This post was produced as part of the Astra Fellowship under the Winter 2024 Cohort, mentored by Richard Ngo.

Epistemic status: relatively confident in the overall direction of this post, but looking for feedback!

TL;DR:

When are ML systems well-modeled as coherent expected utility maximizers? We apply our theoretical model of coherence in our last post to toy policies in RL environments in OpenAI Gym. We develop classifiers that can spot coherence according to our definition and test them on test case policies that intuitively seem coherent or not coherent. We find that we can successfully train classifiers with low loss which also correctly predict out-of-distribution test cases we intuitively believe to have high or low coherence/goal-directedness. We hope that our preliminary results will be validated and extended to existing frontier training methods, e.g. RLHF.

Introduction

It is unclear in the current literature whether future transformative artificial intelligences (TAIs) will be "well-modeled" as maximizing some utility function representing its goals, or whether they will be more broadly "goal-directed", or what those terms even exactly mean. This is relevant because many of the core arguments around why the first superhuman AIs could by default pose an x-risk, and why alignment is fundamentally difficult, were developed with the assumption of future AIs being "goal-directed". For instance, historically one original conception of a misaligned agent was a "paperclip maximizer" or a monomaniacal agent pursuing maximization of some single discrete objective. An agent having an inner misaligned goal that it prioritizes above others in most situations, and thus instrumentally deceiving the training process to prevent that goal from being detected and trained out, is also one of the most studied failure modes in AI alignment. Fundamentally, if our first TAI acts incoherently, or if it has many internal context-dependent goals with none of them being especially dominant or activated in most situations, then we should have lower credence that the classic arguments for misalignment leading to x-risk will hold. Although there are many other possible dangers from TAI that could cause catastrophe like rising dependence on AI systems, misuse/adversarial attacks, and/or multi-polar scenarios, these scenarios seem slower-moving and less extreme in sco...
AI Regulation is Unsafe
by Maxwell Tabarrok — 2024-04-22T16:37:55.431Z
Link: https://www.maximum-progress.com/p/ai-regulation-is-unsafe

Concerns over AI safety and calls for government control over the technology are highly correlated but they should not be.

There are two major forms of AI risk: misuse and misalignment. Misuse risks come from humans using AIs as tools in dangerous ways. Misalignment risks arise if AIs take their own actions at the expense of human interests.

Governments are poor stewards for both types of risk. Misuse regulation is like the regulation of any other technology. There are reasonable rules that the government might set, but

omission bias

and incentives to protect small but well organized groups at the expense of everyone else will lead to lots of costly ones too. Misalignment regulation is not in the Overton window for any government. Governments do not have strong incentives to care about long term, global, costs or benefits and they do have strong incentives to push the development of AI forwards for their own purposes.

Noticing that AI companies put the world at risk is not enough to support greater government involvement in the technology. Government involvement is likely to exacerbate the most dangerous parts of AI while limiting the upside.

Default government incentives

Governments are not social welfare maximizers. Government actions are an amalgam of the actions of thousands of personal welfare maximizers who are loosely aligned and constrained. In general, governments have strong incentives for

myopia

, violent competition with other governments, and negative sum transfers to small, well organized groups. These exacerbate existential risk and limit potential upside.

The vast majority of the costs of existential risk occur outside of the borders of any single government and beyond the election cycle for any current decision maker, so we should expect governments to ignore them.

We see this expectation fulfilled in governments reactions to other long term or global externalities e.g debt and climate change. Governments around the world are happy to impose trillions of dollars in

direct cost and substantial default risk

on future generations because costs and benefits on these future generations hold little sway in the next election. Similarly, governments

spend billions subsidizing fossil fuel

production and ignore potential solutions to global warming, like a carbon tax or geoengineering, because the

long term or extraterritorial costs and benefits of climate ch

...
Priors and Prejudice
by MathiasKB — 2024-04-22T15:00:41.782Z

I

Imagine an alternate version of the Effective Altruism movement, whose early influences came from socialist intellectual communities such as the Fabian Society, as opposed to the rationalist diaspora. Let’s name this hypothetical movement the Effective Samaritans.

Like the EA movement of today, they believe in doing as much good as possible, whatever this means. They began by evaluating existing charities, reading every RCT to find the very best ways of helping.

But many effective samaritans were starting to wonder. Is this randomista approach really the most prudent? After all, Scandinavia didn’t become wealthy and equitable through marginal charity. Societal transformation comes from uprooting oppressive power structures.

The Scandinavian societal model which lifted the working class, brought weekends, universal suffrage, maternity leave, education, and universal healthcare can be traced back all the way to 1870’s where the union and social democratic movements got their start.

In many developing countries wage theft is still common-place. When employees can’t be certain they’ll get paid what was promised in the contract they signed and they can’t trust the legal system to have their back, society settles on much fewer surplus producing work arrangements than is optimal.

Work to improve capacity of the existing legal structure is fraught with risk. One risks strengthening the oppressive arms used by the ruling and capitalist classes to stay in power.

A safer option may be to strengthen labour unions, who can take up these fights on behalf of their members. Being in inherent opposition to capitalist interests, unions are much less likely to be captured and co-opted. Though there is much uncertainty, unions present a promising way to increase contract-enforcement and help bring about the conditions necessary for economic development, a report by Reassess Priorities concludes.

Compelled by the anti-randomista arguments, some Effective Samaritans begin donating to the ‘Developing Unions Project’, which funds unions in developing countries and does political advocacy to increase union influence.

A well-regarded economist writes a scathing criticism of Effective Samaritanism, stating that they are blinded by ideology and that there isn’t sufficient evidence to show that increases in labor power leads to increases in contract enforcement.

The article is widely ...