trentmkelly/LessWrong-43k huggingface dataset:
[New LW Feature] "Debates" 
by Ruby — 2023-04-01T07:00:24.466Z

Following the success of the 2021 MIRI Conversations in drawing out various people's views on key questions in AI and digging into their disagreements, the LessWrong team was inspired to build a more dedicated feature for hosting debates on LessWrong.

The MIRI conversations were mostly hosted on Discord and then via a laborious process shoehorned into the LessWrong editor. We figured it wouldn't be hard to do better. Among many benefits, the debates can be held on LessWrong itself; readers are able to comment "inline" on responses within the debate; and there will be customized "debate item" in the Latest Posts list on the frontpage that signals that 1) it's a debate, and 2) how many debate responses have been posted since you last viewed the debate. Hopefully all of this is intuitive from the UI.

The feature is designed so that debates can be held in private, possibly edited, and then published publicly. Or that the debate happens live on the site, allowing for live commenting.

As we're rolling out the feature, we'll initially just set up a few debates that we'd like to see, and then later potentially open up the feature to users more broadly. You're welcome to contact us [link] or comment here if you're interested in viewing or participating in a particular debate.

This announcement post will also serve as the Inaugural Debate using the new debate feature. We were lucky to find two willing participants on short notice, so big thanks to them. GPT-4 and Claude+ are going to discuss whether or not AI Safety via Debate is a promising Alignment strategy.
Keep Making AI Safety News
by RedFishBlueFish — 2023-04-01T06:27:21.540Z

Crossposted from the EA Forum

AI Safety is hot right now.

The FLI letter was the catalyst for most of this, but even before that there was the Ezra Klein OpEd piece in the NYTimes. (Also general shoutout to Ezra for helping bring EA ideas to the mainstream - he's great!).

Since the FLI letter, there was there was this CBS interview with Geoffrey Hinton. There was this WSJ Op-Ed. Eliezer's Time OpEd and Lex Fridman interview led to Bezos following him on Twitter. Most remarkably to me, Fox News reporter Peter Doocey asked a question in the White House press briefing, which got a serious (albeit vague) response. The president of the United States, in all likelihood, has heard of AI Safety.

This is amazing. I think it's the biggest positive development is AI Safety thus far. On the safety research side, the more people hear about AI safety, the more tech investors/philanthropists start to fund research and the more researchers want to start doing safety work. On the capabilities side, companies taking AI risks more seriously will lead to more care taken when developing and deploying AI systems. On the policy side, politicians taking AI risk seriously and developing regulations would be greatly helpful.

Now, I keep up with news... obsessively. These types of news cycles aren't all that uncommon. What is uncommon is keeping attention for an extended period of time. The best way to do this is just to say yes to any media coverage. AI Safety communicators should be going on any news outlet that will have them. Interviews, debates, short segments on cable news, whatever. It is much less important that we proceed with caution - making sure to choose our words carefully or not interacting with antagonistic reporters - than that we just keep getting media coverage. This was notably Pete Buttigieg's strategy in the 2020 Democratic Primary (and still is with his constant Fox News cameos), which led to this small-town mayor becoming a household name and the US Secretary of Transportation.

I think there's a mindset among people in AI Safety right now that nobody cares and nobody is prepared and our only chance is if we're lucky and alignment isn't as hard as Eliezer makes it out to be. This is our chance to change that. Never underestimate the power of truckloads of media coverage, whether to elevate a businessman into the white house or to push a fringe idea into the main...
What Are Your Preferences Regarding The FLI Letter?
by JenniferRM — 2023-04-01T04:52:46.887Z

There's a big issue and it doesn't seem to having very much voting happening around it, and it has less really high quality voting, which might be sad, given the importance and stakes and so on?

I think governance innovations that are inclusive of digital people will be important to getting a Win Condition with technology.

Voting can also help aggregate knowledge. If a lot of people make a best effort attempt to estimate a number, or a "right way to proceed" then, if there is some way to average and find the "most central answer" that might be the objectively best one. As was written:

When you hear that a classroom gave an average estimate of 871 beans for a jar that contained 850 beans, and that only one individual student did better than the crowd, the astounding notion is not that the crowd can be more accurate than the individual. The astounding notion is that human beings are unbiased estimators of beans in a jar, having no significant directional error on the problem, yet with large variance. It implies that we tend to get the answer wrong but there's no systematic reason why. It requires that there be lots of errors that vary from individual to individual - and this is reliably true, enough so to keep most individuals from guessing the jar correctly. And yet there are no directional errors that everyone makes, or if there are, they cancel out very precisely in the average case, despite the large individual variations. Which is just plain odd. I find myself somewhat suspicious of the claim, and wonder whether other experiments that found less amazing accuracy were not as popularly reported.

It seems likely to me that there is just some correct answer to the question of whether there should be an AGI moratorium, and if there should be, then what should the details look like to get the best outcome for the world? Yet there is a lack of disagreement on the subject! Maybe it would help to solicit many estimates of the best ways to proceed, and then average?

Somewhat separately, anarchism might be... a sub-optimal way for people to organize themselves. Maybe not. But probably.

There's a very common historical pattern. Where people form themselves into a government, and then they apply locally coherent consent ethics to a generalized problem of legitimating the actions of people in a government that not necessarily everyone agrees wit...
An Average Dialogue
by Nicholas / Heather Kross — 2023-04-01T04:01:50.998Z

TLDR: 📝📝📝📝📝📝📝📝📝📝📝📝📝

A: Knock-knock!

B: Who's there?

A: Cowgo.

B: Cowgo who?

A: The past year has been a distressingly-active one, for this community. Here, of course, I'm referring to the union of the rationality community, the Effective Altruism community, and the AI alignment/DontKillEveryoneism (aka "DKE") community. A quick timeline in case you've been in cryostasis under a rock: April 1, 2022: MIRI, the OG AI DKE organization, announced its new "Death With Dignity" strategy. This gave the AI alignment/safety field a kick in the ass, which seems pretty good. Importantly, it also increased the percentage of the community that takes DKE seriously. Probably. Hopefully. I haven't run a survey. This announcement was so impactful, it overshadowed the concurrent announcements of Good Heart Tokens (!) and infant outreach (!!!). RETRACTED August/September-ish 2022 (but written about in January 2023): The Future of Life Institute, an organization dedicated to lowering extinction risk, allegedly gave $100,000 to an extreme right-wing online publication connected to FLI President Max Tegmark's brother. Tegmark denied this. EDIT: the money seems to have been a grant that was applied for, but rejected by FLI and never paid out. Yay! H/t to DC for correcting me on this. August 2022: TIME Magazine prints a full cover story about Effective Altruism! Yay! November-ish 2022: One of the largest donors to EA causes, the crypto exchange FTX, goes bankrupt in a sudden, spectacular, and arrest-warrant-inducing fashion. Somehow, despite allegedly being on lots of stimulants, the CEO Sam Bankman-Fried was unable to even play League of Legends properly. Thus, his downfall could be described as part ethical failing and part skill issue. Anyway, this has impacted EA funding in the short- and medium-terms, just due to the sheer volume of money and committed money that the FTX Future Fund was putting towards many causes, including longtermism. December 2022: I graduate college! Yay! January 9, 2023: Nick Bostrom, an OG AI DKE researcher, writes an apology for past racist writing. A dramatized reading of his apology can be watched here. March 14, 2023: GPT-4 is released (under an API/paywall) by OpenAI. This has made a lot of AI DKE researchers very angry and been somewhat regarded as a bad move. (Not the paywall I mean, that part's far better than e.g. open-sourcing the model....
The Significance of "Alignment vs Progress: The AI Rap Showdown" in the AI Safety Discourse
by Jonathan Grant — 2023-04-01T03:26:45.471Z

As members of the LessWrong community, we've long been concerned with the future of artificial intelligence and its implications for society. In an attempt to spread awareness and spark discussions around the crucial topic of AI safety, a new rap song titled "Alignment vs Progress: The AI Rap Showdown" has been released. Featuring Elizier Yudkowsky and Sam Altman, the song showcases a rap battle between the two, debating the importance of AI alignment versus rapid progress.

The catchy verses and engaging format of this rap battle serve as an accessible medium to introduce a broader audience to the nuanced conversation surrounding AI safety. By presenting the opposing viewpoints in a creative and relatable manner, the song encourages listeners to consider the trade-offs and challenges that arise from our pursuit of advanced AI systems.

Elizier Yudkowsky, a key figure in the rationality and AI alignment communities, argues for a cautious approach that emphasizes the importance of ensuring AI systems are value-aligned with human interests. He stresses the potential risks of uncontrolled AI development and how critical it is to prioritize AI alignment in order to avoid disastrous outcomes.

On the other hand, Sam Altman, known for his leadership at OpenAI, advocates for the rapid progress and development of AI technologies, emphasizing the need to remain competitive in a global landscape. He acknowledges the importance of AI safety, but contends that slowing down progress may lead to other negative consequences.

The rap battle format cleverly highlights the complexity of the AI safety discussion, demonstrating that there are valid concerns on both sides of the debate. It serves as a reminder that the AI community needs to collaborate and strike a balance between the pursuit of AI alignment and the desire for rapid progress.

By engaging in creative and unconventional methods to disseminate information about AI safety and alignment, we can raise awareness and stimulate discussions among a wider audience. We encourage members of the LessWrong community to share the song, engage in the conversation, and contribute their own insights into the ongoing debate.

As the development of artificial intelligence continues to accelerate, it is crucial that we foster a dialogue that addresses the challenges and trade-offs we face in ensuring a safe and beneficial AI future. "Alignment...
My Model of Gender Identity
by Iris of Rosebloom — 2023-04-01T03:03:16.662Z

People sometimes say that it's impossible to rationally explain gender identity. I've taken this as a challenge, and spent the past few months trying to put together a more sophisticated model than I’ve seen others espouse. I’d like to share what I have in this post. First, though:

Some pitfalls I've avoided with this

One common origin of trouble in discussion around transgenderism is that when a cis person is first exposed to the concept of trans people, they’ll understand that this violates their long-held model of male=man | female=woman, and, given low openness, political prejudice, or other biases, often start a motivated search for other, more intuitive reasons for trans people to exist besides gender dysphoria. These can include “trans people are perverts/confused/mentally ill” and so on, which is a problem insofar as gender dysphoria is a hypothesis at least worth considering.[1] I have experience with what I consider dysphoria, though, and have been surrounded by people who identify as trans for many years, so this failure mode mostly doesn’t apply to me.

There are also some common failure-modes in trans-positive spaces, though. For example: Gender is a keystone in the belief systems of many if not most trans people, so theories on the topic that try to account for some of the more intuitive observations made by the first group can often feel threatening to the identities of those involved in the conversation (e.g. autogynephilia theory, social contagion, probably others). This leads to ignoring certain realities that legitimately need to be explained. On top of this, many vocal[2] trans people are strong members of

the Blue Tribe

; this gives them an external motive not to pay dues to the possibility that gendered tendencies are biologically innate, so as to preserve feminist blank-slatism and openness to queer variety. This also leads to the discounting of valuable information;

here's an example

of a trans woman very briefly treating and then discounting neurology, and implicitly bio-psychology as at large, as a field we can’t currently use to learn much about trans people.

Despite my status as a trans person, I’ve made efforts to shed my membership with the Blue Tribe, and have

tried to show elsewhere

that I’m willing to entertain and accept uncomfortable possibilities about trans people. This means I’m less likely than averag...
Hooray for stepping out of the limelight
by So8res — 2023-04-01T02:45:31.397Z

From maybe 2013 to 2016, DeepMind was at the forefront of hype around AGI. Since then, they've done less hype. For example, AlphaStar was not hyped nearly as much as I think it could have been.

I think that there's a very solid chance that this was an intentional move on the part of DeepMind: that they've been intentionally avoiding making AGI capabilities seem sexy.

In the wake of big public releases like ChatGPT and Sydney and GPT-4, I think it's worth appreciating this move on DeepMind's part. It's not a very visible move. It's easy to fail to notice. It probably hurts their own position in the arms race. I think it's a prosocial move.

If you are the sort of person who is going to do AGI capabilities research—and I recommend against it—then I'd recommend doing it at places that are more likely to be able to keep their research private, rather than letting it contribute to an arms race that I expect would kill literally everyone.

I suspect that DeepMind has not only been avoiding hype, but also avoiding publishing a variety of their research. Various other labs have also been avoiding both, and I applaud them too. And perhaps DeepMind has been out of the limelight because they focus less on large language models, and the results that they do have are harder to hype. But insofar as DeepMind was in the limelight, and did intentionally step back from it and avoid drawing tons more attention and investment to AGI capabilities (in light of how Earth is not well-positioned to deploy AGI capabilities in ways that make the world better), I think that's worth noticing and applauding.

(To be clear: I think DeepMind could do significantly better on the related axis of avoiding publishing research that advances capabilities, and for instance I was sad to see Chinchilla published. And they could do better at avoiding hype themselves, as noted in the comments. At this stage, I would recommend that DeepMind cease further capabilities research until our understanding of alignment is much further along, and my applause for the specific act of avoiding hype does not constitute a general endorsement of their operations. Nevertheless, my primary guess is that DeepMind has made at least some explicit attempts to avoid hype, and insofar as that's true, I applaud the decision.)
Kallipolis, USA
by Daniel Kokotajlo — 2023-04-01T02:06:52.827Z
Link: https://docs.google.com/document/d/1BtYDYnKejgkvoYPc8CIvXAW6iU0OIl1QzC_JV0BDSU4/edit?usp=sharing

This is a very tongue-in-cheek term paper I wrote for my philosophy class on Plato in fall of 2016. I had a lot of fun with it & still like to talk about it sometimes when I get tipsy. I'm putting it up here because why not. Comments welcome.
Abstract:

Reading Plato’s Republic, you may have had reactions like this: “The Kallipolis… what a ridiculous and intriguing city! How utopian, yet how totalitarian! How alien!”

If so, then you are in the right place, because the thesis of this paper is that the present-day United States of America is in fact the Kallipolis.

Admittedly, the argument in this paper is not strong enough to take us all the way to that conclusion with any confidence. But it can push us in that direction. What I will do is draw out all the connections and parallels that I see between the USA and the Kallipolis. I will argue that the USA is currently surprisingly close to realizing Plato’s vision, at least in broad strokes. Some parts of my argument are rather hand-wavy, and others rely on debatable empirical claims, so I doubt you will end up fully convinced. Honestly I am not fully convinced myself. But I will make the best case I can, and hopefully you will end up convinced that the USA is closer in the direction of the Kallipolis than you previously thought.
Is there a manifold plot of all people who had a say in AI alignment?
by skulk-and-quarrel — 2023-03-31T21:50:09.570Z

Since the good-old libertarian/ authoritarian quadrant is no use to us here (yes EY goes on one end but where do you put Scott Aaronson? Zvi? Timnit Gebru?), what we need is a multi-dimensional table and then project this to some human interpretable 2d or 3d manifold.

Context: I really don’t know where I stand in terms of AI alignment, I don’t know who is standing with me, and I need help.

I’m starting a $50 bounty on a figure that contains the above, with at least 20+ public figures, and the table (and the code for some additional TBD bounty) used to generate the plot. The table could have something like chance of death from AI in different columns and some categorical column on the direction they are taking now.

I’ll evaluate the best submission on April 16, Sunday 9 AM PST and then send the award.

If your submission looks like a badly prompted output from BLOOM, I might reject it but I’m open to fully AI generated answers.

In terms of people to include: You don’t need to include people who continually have had wrong takes and moved flagposts (wink wink), or people whose takes are not really clear since they just sway with the wind. I would consider anyone whose posts are taken in high regard in this community as a public persona (e.g., janus definitely counts), and you can be free to include anyone that you think is appropriate.

(4/2/23 edit: changing the deadline to April 16, from April 2)
We might get lucky with AGI warning shots. Let's be ready!
by tcelferact — 2023-03-31T21:01:44.966Z

Prior to ChatGPT, I was slightly against talking to governments about AGI. I worried that attracting their interest would cause them to invest in the technology and shorten timelines.

However, given the reception of ChatGPT and the race it has kicked off, my position has completely changed. Talking to governments about AGI now seems like one of the best options we have to avert a potential catastrophe.

Most of all, I would like people to be preparing governments to respond quickly and decisively to AGI warning shots.

Eliezer Yudkowsky recently had a letter published in Time that I found inspiring: https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ . It contains an unprecented international policy proposal:

Shut down all the large GPU clusters (the large computer farms where the most powerful AIs are refined). Shut down all the large training runs. Put a ceiling on how much computing power anyone is allowed to use in training an AI system, and move it downward over the coming years to compensate for more efficient training algorithms. No exceptions for governments and militaries. Make immediate multinational agreements to prevent the prohibited activities from moving elsewhere. Track all GPUs sold. If intelligence says that a country outside the agreement is building a GPU cluster, be less scared of a shooting conflict between nations than of the moratorium being violated; be willing to destroy a rogue datacenter by airstrike.

There is zero chance of this policy being enacted globally tomorrow. But if a powerful AI system does something that scares the bejesus out of American and Chinese government officials, political sentiment could change very quickly.

Based on my understanding of LLMs, there is a non-trivial possibility that we get a warning shot like this, one that ordinary folks find scary and that convinces many researchers alignment is going to fail, before we actually succeed in building an AGI. I hope we can prepare governments so that, if we get lucky, they take decisive action to 'shut it all down' within weeks rather than years.

Unfortunately I am not a diplomat and I do not know what this kind of preparation would look like. But I am hopeful that there are diplomats out there who will read this, find it convincing, and offer much better suggestions than the ones I am about to give.

I would guess that at a minimum pre...