trentmkelly/LessWrong-43k huggingface dataset:
If you're an AI Safety movement builder consider asking your members these questions in an interview
by yanni kyriacos — 2024-05-27T05:46:17.485Z

I've been doing interviews with aspiring members of the Australian & New Zealand AIS community recently (about 35 so far) and I've arrived at the following set of questions (steps 1 - 3) as being pretty effective for understanding their challenges (and then turning into insights for program ideation). Steps 4 and 5 are speculative (I've not actually asked them but they are what I'd ask if I had more time with each interviewee).

You're unlikely to get to Step 3 if you only have 1 hour with the interviewee. So I recommend asking them in order (i.e. start with background, then go into their pain points, then what they perceive as the pain points of other people, then solutions etc).

HMU on yannikyriacos@gmail.com if you have any in depth questions about this.

Step 1: interviewee overview

Give me a quick run down on everything you’ve done in AI Safety so far (education, networking and jobs)
Give me a quick overview of what your plans are moving forward.

What is your plan A?
What is your plan B?
What is your plan C?
What is your plan D?

What have been the single biggest pain points for you personally in attempting to have an impact in AI Safety?

What is the second biggest?
What is the third biggest?

Step 2: what are their pain points (education, networking, jobs)?

What have been the biggest pain points for you personally in attempting to find relevant education in AI Safety?
What have been the biggest pain points for you personally in attempting to create relevant networks in AI Safety?
What have been the biggest pain points for you personally in attempting to find relevant work in AI Safety?

Step 3: what do they see as others’ paint points (education, networking, jobs)?

What do you believe are significant pain points for other aspiring AI people when attempting to find relevant education? (i.e. not you)
What do you believe are significant pain points for other aspiring AI people trying to create relevant networks/do networking? (i.e. not you)
What do you believe are significant pain points for aspiring AI people when it comes to them finding relevant jobs? (i.e. not you)

Step 4: what do they see as the most promising programs (education, networking, jobs)?

What do you believe are promising movement building programs when it comes to helping aspiring AI folk find useful education?
What do you believe are promising movement building programs fo

...
Book review: Everything Is Predictable
by PeterMcCluskey — 2024-05-27T03:33:53.857Z
Link: https://bayesianinvestor.com/blog/index.php/2024/05/26/everything-is-predictable/

Book review: Everything Is Predictable: How Bayesian Statistics Explain
Our World, by Tom Chivers.

Many have attempted to persuade the world to embrace a Bayesian
worldview, but none have succeeded in reaching a broad audience.

E.T. Jaynes'
book
has been a leading example, but its appeal is limited to those who find
calculus enjoyable, making it unsuitable for a wider readership.

Other attempts to engage a broader audience often focus on a narrower
understanding, such as Bayes'
Theorem, rather than the
complete worldview.

Claude's most fitting recommendation was Rationality: From AI to
Zombies, but at 1,813 pages, it's
too long and unstructured for me to comfortably recommend to most
readers. (GPT-4o's suggestions were less helpful, focusing only on
resources for practical problem-solving).

Aubrey Clayton's book, Bernoulli's Fallacy: Statistical Illogic and
the Crisis of Modern
Science,
only came to my attention because Chivers mentioned it, offering mixed
reviews that hint at why it remained unnoticed.

Chivers has done his best to mitigate this gap. While his book won't
reach as many readers as I'd hoped, I'm comfortable recommending it as
the standard introduction to the Bayesian worldview for most readers.

Basics

Chivers guides readers through the fundamentals of Bayes' Theorem,
offering little that's extraordinary in this regard.

A fair portion of the book is dedicated to explaining why probability
should be understood as a function of our ignorance, contrasting with
the frequentist approach that attempts to treat probability as if it
existed independently of our minds.

The book has many explanations of how frequentists are wrong, yet
concedes that the leading frequentists are not stupid. Frequentism's
problems often stem from a misguided effort to achieve more objectivity
in science than seems possible.

The only exception to this mostly fair depiction of frequentists is a
section titled "Are Frequentists Racist?". Chivers repeats Clayton's
diatribe affirming this, treating the diatribe more seriously than it
deserves, before dismissing it. (Frequentists were racist when racism
was popular. I haven't seen any clear evidence of whether Bayesians
behaved differently).

The Replication Crisis

Chivers explains frequentism's role in the replication crisis.

A fundamental drawback of p-values is that they indicate the likelihood
of the data given a hypothesis, which diffe...
Truthseeking is the ground in which other principles grow
by Elizabeth — 2024-05-27T01:09:20.796Z

Introduction

First they came for the epistemology/we don’t know what happened after that.

I’m fairly antagonistic towards the author of that tweet, but it still resonates deep in my soul. Anything I want to do, anything I want to change, rests on having contact with reality. If I don’t have enough, I might as well be pushing buttons at random.

Unfortunately, there are a lot of forces pushing against having enough contact with reality. It’s a lot of work even when reality cooperates, many situations are adversarial, and even when they’re not entropy itself will constantly chip away at your knowledge base.

This is why I think constantly seeking contact with reality is the meta principle without which all (consequentialist) principles are meaningless. If you aren’t actively pursuing truthseeking, you won’t have enough contact with reality to make having goals a reasonable concept, much less achieving them. To me this feels intuitive, like saying air is necessary to live. But I’ve talked to many people who disagree, or who agree in the abstract but prioritize differently in the breach. This was supposed to be a grand post explaining that belief. In practice it’s mostly a bunch of pointers to facets of truthseeking and ideas for how to do better. My hope is that people can work backwards from these to the underlying principle, or flesh out their own relationship with truthseeking.

Target audience

I think these are good principles for almost any situation, but this essay is aimed at people within Effective Altruism. Most of the examples are from within EA and assume a certain amount of context. I definitely don’t give enough information to bring someone unfamiliar up to speed. I also assume at least a little consequentialism.

A note on examples and actions

I’m going to give lots of examples in this post. I think they make it easier to understand my point and to act on what agreement you have. It avoids the failure mode Scott Alexander discusses

here

, of getting everyone to agree with you by putting nothing at stake.

The downside of this is that it puts things at stake. I give at least 20 examples here, usually in less than a paragraph, using only publicly available information. That’s enough to guarantee that every person who reads this will find at least one example where I’m being really unfair or missing crucial information. I welcome corrections ...
If trying to communicate about AI risks, make it vivid
by MNoetel — 2024-05-27T01:00:48.244Z

Crossposted from the EA Forum

If you want to communicate AI risks in a way that increases concern, [1] our new study says you should probably use vivid stories, ideally with identifiable victims.

Tabi Ward led this project as her honours thesis. In the study, we wrote[2] short stories about different AI risks like facial recognition bias, deepfakes, harmful chatbots, and design of chemical weapons. For each risk, we created two versions: one focusing on an individual victim, the other describing the scope of the problem with statistics.

We had 1,794 participants from the US, UK and Australia[3] read one of the stories, measuring their concern about AI risks before and after. Reading any of the stories increased concern. But, the ones with identifiable individual victims increased concern significantly more than the statistical ones.

Why? The individual victim stories were rated as more vivid by participants. A mediation analysis found the effect of identifiable victims on concern was explained by the vividness of the stories:

Stories about AI risk with identifiable victims (Victim type = 0) were seen as LESS vivid compared to stories about groups of people (statistical victims, Victim type = 1)

This finding aligns with prior research on "compassion fade" and the "identifiable victim effect": people tend to have stronger emotional responses and helping intentions towards a single individual in need than towards larger numbers or statistics. Our study extends this to the domain of risk perception.

Communicating about the harms experienced by identifiable victims is a particular challenge for existential risks. These AI risks are defined by their scale and their 'statistical victim' nature: they could affect billions, but have not yet occurred. Nevertheless, those trying to draw attention to concerns should try to make the risks vivid. The most compelling narrative was one with an identifiable victim of an AI-designed 'nerve agent' was the most compelling narrative, but was a hypothetical future story (not a real one from a news report, like the others).

This might influence the way people communicate about AI. For example, when people are trying to increase concern, it might be harder for a reader to imagine how the following request from an AI is dangerous:

Take this strawberry, and make me another strawberry that's identical to this strawber

...
Луна Лавгуд и Комната Тайн, Часть 1
by Kongo Landwalker — 2024-05-26T22:17:17.137Z

Disclaimer: This is Kongo Landwalker's translation of lsusr's fiction Luna Lovegood and the Chamber of Secrets - Part 1 into russian. I recommend reading Harry Potter and the Methods of Rationality first to understand the setting and plot.

Луна Лавгуд прошла через барьер между платформами девять и десять на платформу девять-и-три-четверти. Луна задумалась, что же случилось c платформой девять-с-половиной. Такие числа, как "три четверти" появляются лишь тогда, когда кто-то делит целое число дважды.

Луна огляделась в поисках кого-то, кто может знать ответ, и подметила единорога. Единорог выглядел как человеческая особь женского пола, на ней была одежда, она ходила на двух ногах и имела кудрявые коричневые волосы. Но ничего из этого не обмануло Луну. У единорожки была аура мира, и её ногти были из аликорна.

— Что случилось с платформой девять-с-половиной? — спросила Луна единорожку.

— Не существует никакой платформы девять-с-половиной. — прозвучал ответ.

— С чего ты взяла? — спросила Луна.

— В противном случае это было бы упомянуто в "Истории Хогвартса" — ответила единорожка. — А ещё нет никакого упоминания "Платформы девять-с-половиной" в Современной Волшебной Истории, в Важных Современных Волшебных Открытиях и ни в какой другой книге из библиотеки Хогвартса. Есть лишь платформа девять-и-три-четверти.

— А что насчёт платформы девять-и-семь-восьмых? — спросила Луна.

— Такой тоже не существует. — и единорожка быстро развернулась и ушла до того, как Луна успела бы продолжить цикл новой итерацией: "С чего ты взяла?".

Если платформа девять-и-три-четверти не упомянута в магловских библиотеках, то платформа девять-с-половиной вряд ли будет упомянута в магических библиотеках. Разве что в библиотеках двойных ведьм, но библиотека Хогвартса таковой не является.

— Привет, как дела? — спросила подошедшая первогодка с цветом волос "Уизли".

— Я пытаюсь найти платформу девять-с-половиной. Та единорожка мне сказала, что такой не существует. Если она "не существует", значит она спрятана мощной магией. А ты как? — сказала Луна.

— Какая ещё единорожка?

— Вон там стоит — Луна указала пальцем.

Первогодка взглянула в указанном направлении, и извинилась, придумав какой-то повод, чтоб побыстрее закончить беседу.

Луна не знала, как заводить друзей. У неё была надежда, что для первогодок поездка в Хогвартс Экспресс будет хорошей возможностью подружиться. У Луны было ожерелье, которое она раскрасила вр...
If you are also the worst at politics
by lemonhope — 2024-05-26T20:07:49.201Z

This has been said many times but it must be said many more. Please excuse the fact that this post breaks its own advice.

I am the worst at politics. I always make people dig their heels in on the opposite stance. I could give many examples. Even here on LW where people are about as like-minded to me as I can find online.

Most of the meager success I've had getting people to do things has been among my friends and family. But I still can't convince people to get over their grudges with each other.

The tiny remaining proportion of my 'political success' was due to me accidentally getting someone else to champion my cause instead of me.

You know it's occurred to me many times that when I say "hey everyone, X should happen" it tends to make the opposite more likely. But I usually can't help myself. It's like when you've made 100 bad bets in a row on the stock market but somehow it's impossible to simply start making the opposite trades.

When Robin Hanson says anything, it seems a big opposition magically appears. Full of people who never heard the idea before. Who might've been on the in-favor side had somebody else said it. He has been posting incredibly unique and insightful (IMO) political ideas for 20 years. What has come of it? Besides perhaps some fun prediction websites.

Anyway, if you are like me politics-wise, and if (unlike me) you genuinely have fantastic ideas for ways to make AI go super well (if certain people can be convinced to do certain things), then maybe you should take a long breath before you start posting your ideas around. Maybe don't post your ideas at all. Maybe internet credit for starting an idea is worth actually nothing. Maybe try to make friends with somebody who knows how to say things so that people will listen. Maybe you're hurting your team with all these shots from half-court. Take a moment and really consider this deeply.

And thank you to the silent vast majority of readers who (unlike me) are already avoiding indiscriminately posting political thinkpieces every 5 minutes.
Review: Conor Moreton's "Civilization & Cooperation"
by Duncan Sabien (Inactive) — 2024-05-26T19:32:43.131Z

Author's note: in honor of the upcoming LessOnline event, I'm sharing this one here on LessWrong rather than solely on my substack. If you like it, you should subscribe to my substack, which you can do for free (paid subscribers see stuff a week early). I welcome discussion down below but am not currently committing to any particular level of participation myself.

Dang it, I knew I should have gone with my first instinct, and photocopied the whole book first. But then again, given that it vanished as soon as I got to the end of it, maybe my second instinct was right, and trying to do that would’ve been seen as cheating by whatever magical librarians left it for me in the first place.

It was just sitting there, on my desk, when I woke up six weeks ago. At first I thought it was an incredibly in-depth prank, or maybe like a fun puzzle that Logan had made for me as an early birthday present. But when I touched it, it glowed, and it unfolded in a way that I’m pretty sure we don’t currently have the tech for.

Took me a while to decode the text, which mostly looked like:

…but eventually I got the hang of it, thanks to the runes turning out to be English, somehow, just a weird phonetic transcription of it.

Hilariously mundanely, it turned out to be a textbook (!), for what seemed like the equivalent of seventh graders (!!), for what seemed like the equivalent of social studies (!!!), written by an educator whose name (if I managed the translation correctly) is something like “Conor Moreton”…

…in a place called (if I managed the translation correctly) something like “Agor.”

At first, I thought it was a civics textbook for the government and culture of Agor in particular, but nope—the more I read, the more it seemed like a “how stuff works” for societies in general, with a lot of claims that seemed to apply pretty straightforwardly to what I understand about cultures here on Earth.

(I’ll be honest. By the time I got to the end of it, I was stoked about the idea of living in a country where everybody was taught this stuff in seventh grade.)

I took notes, but not very rigorous ones. I wasn’t counting on the book just disappearing as soon as I finished reading the last page—

(I know, I know, not very savvy of me, I should have seen that coming. 20/20 hindsight.)

—so what follows is a somewhat patchwork review, with a lot of detail in random places and very little detail in o...
The necessity of "Guardian AI" and two conditions for its achievement
by Proica — 2024-05-26T17:39:30.205Z

Summary

Note: Summarybot has created an excellent summary, so I will use it as the summary for this article.

Executive summary: To protect humanity from existential risks posed by advanced technologies, we must develop an aligned superintelligent "Guardian AI" to preemptively eliminate these risks, which requires achieving both technical AI alignment and political AI governance.

Key points:

The "vulnerable world hypothesis" posits that beyond a certain level of technological advancement, existential risks to humanity will dramatically increase unless unprecedented preventive measures are taken.
Eliminating existential risks in advance is likely biologically impossible for humans due to the immense challenges involved, such as making accurate long-term predictions and developing defensive technologies.
Delegating the task of protecting humanity to a superintelligent "Guardian AI" is proposed as the only viable solution, as it could preemptively predict and address existential risks.
Two critical conditions must be met to realize a safe and beneficial Guardian AI: solving the technical challenge of "AI alignment" to ensure the AI follows human values and intentions, and establishing the political frameworks for global "AI governance".
Organizations and decision-makers worldwide should strongly support and prioritize AI alignment research and AI governance initiatives, as they are crucial for safely transitioning to a post-Singularity future.

Preface

If technology continues to develop at its current pace, there is a high likelihood that "technology capable of certainly destroying civilization" will become widely available to the general public within the next few years to decades. As a result, the catastrophic risks (i.e., existential risks) that could lead to the collapse of civilization are expected to increase significantly. This paper examines the "vulnerable world hypothesis" proposed by Nick Bostrom and proposes methods to address the existential risks associated with future technological innovations.

In conclusion, to sufficiently eliminate the existential risks associated with technological innovation, a "guardian AI" with superhuman capabilities will be necessary. This paper argues that by utilizing this "guardian AI," all x-risks can be preemptively and perfectly eliminated. However, two conditions must be met to realize a guardian AI: AI alignment and AI govern...
Notifications Received in 30 Minutes of Class
by tanagrabeast — 2024-05-26T17:02:20.989Z

Introduction

If you are choosing to read this post, you've probably seen the image below depicting all the notifications students received on their phones during one class period. You probably saw it as a retweet of

this tweet

, or in

one of Zvi’s posts

. Did you find this data plausible, or did you roll to disbelieve? Did you know that the image dates back to at least 2019? Does that fact make you more or less worried about the truth on the ground as of 2024?

Last month, I performed an enhanced replication of this experiment in my high school classes. This was partly because we had a use for it, partly to model scientific thinking, and partly because I was just really curious. Before you scroll past the image, I want to give you a chance to mentally register your predictions. Did my average class match the roughly 1,084 notifications I counted on Ms. Garza's viral image? What does the distribution look like? Is there a notable gender difference? Do honors classes get more or fewer notifications than regular classes? Which apps dominate? Let's find out!

Before you rush to compare apples and oranges, keep in mind that I don't know anything about Ms. Garza's class -- not the grade, the size, or the duration of her experiment. That would have made it hard for me to do a true replication, and since I saw some obvious ways to improve on her protocol, I went my own way with it.

Procedure

We opened class with a discussion about what we were trying to measure and how we were going to measure it for the next 30 minutes. Students were instructed to have their phones on their desks and turned on. For extra amusement, they were invited (but not required) to turn on audible indicators. They were asked to tally each notification received and log it by app. They were instructed to not engage with any received notifications, and to keep their phone use passive during the experiment, which I monitored.

While they were not to put their names on their tally sheets, they were asked to provide some metadata that included (if comfortable) their gender. (They knew that gender differences in phone use and depression were a topic of public discussion, and were largely happy to provide this.)

To give us a consistent source of undemanding background "instruction" — and to act as our timer — I played the first 30 minutes of Kurzgesagt's groovy

4.5 Billion Years in 1 Hour

vid...
Show LW: HackerNews but for research papers
by sleno — 2024-05-26T15:14:46.411Z

Hey there,

A few days ago I went semi-viral on HN for posting a side project:
https://news.ycombinator.com/item?id=40455478
Here's a screenshot of the analytics for the last few days:
The site is called papertalk. Basically I made it because I'd love to have a LW/HN type of community centered around the state of the art of every academic field; health, history, cs, economics, etcetera. I've noticed with high-quality communities the discussion can do a really great job of quickly communicating the value or lack thereof in a way a layman like myself can understand.
I didn't expect so much positive feedback from the HN post but in light of that I'm trying to turn this into something really polished and useful. I thought I'd post here as well as this is another community I think it could resonate with.

Any feedback is very much appreciated.
If anyone is an expert in any field and familiar with the landscape of academic papers, I want to have an MVP (most valuable papers) section for each field with the foundational / most significant research.

Thanks,

Stefan