trentmkelly/LessWrong-43k huggingface dataset:
Ackshually, many worlds is wrong
by tailcalled — 2024-04-11T20:23:59.416Z

Thank you to Justis Mills for proofreading and feedback. This post can also be found on my substack.

I mentioned that I disagree with the many worlds interpretation of quantum mechanics in a comment, and I thought I should clarify my position. I title the post "ackshually" because it is a very pedantic objection that I don't think is very important. But I found the philosophy interesting/enlightening when I thought of it.

The TL;DR is that many-worlds theory is a way to embed a quantum system into a non-quantum system, whereas it seems more natural to assume that the world is just Inherently Quantum. To understand what I mean by "embedding a quantum system", it may be clearest to start with an analogy to stochasticity.

The many-worlds theory of stochasticity

To me, the notion of "true randomness" is philosophically coherent. Like yes, it's conceivable that we happen to live in a deterministic universe, but I don't think it's objectionable for the True Generator Of Physics to be a nondeterministic Markov Chain or whatever.

How can we model randomness? One option is sampling. If your modelling tool supports randomness, then you can turn the randomness of the system you're modelling into randomness in the tool itself, for instance in programming you could define an interface like such:

trait Distribution[A] {
def sample(): A
}

val uniform = new Distribution[Double] {
def sample() =
/* somehow produce a stochastic number in [0, 1], using True Randomness */
}

... with the property that each time you call uniform.sample(), you get a new random number.[1]

Sampling is not the way randomness is usually modelled in mathematics, partly because mathematics is deterministic and so you can't model randomness in this way. Instead, it is usually modelled using probability, which in the finite setting we can think of as a function P:A→R. This means that to each possible value a:A, we have a real number P(a) quantifying the "realness" of this A.

Now, what happens if we take P literally? It seems like it is postulating "many worlds" of A, with quantifiable levels of realness. This isn't true if the universe is truly stochastic. It's also not true if the P is modelling uncertainty. One could perhaps say it's sort of true if the P is modelling propensities, but even then it's sort of a stretch. Maybe the place where i...
What does Eliezer Yudkowsky think of the meaning of life now?
by metaqualia — 2024-04-11T18:36:25.153Z

https://docs.google.com/document/d/12xxhvL34i7AcjXtJ9phwelZ7IzHZ_xiz-8lGwpWxucI/edit

In this Google doc, if you go to 2.5, you'll find a section called "What is the meaning of life?"

Yudkowsky concluded at that time that the meaning of life is the singularity. It looks very logical to me, but in the end the singularity is an extreme existential threat to humanity. He's clearly terrified of it now, so I'm curious what his new idea of meaning is.
Structured Transparency: a framework for addressing
use/mis-use trade-offs when sharing information
by habryka — 2024-04-11T18:35:44.824Z
Link: https://arxiv.org/pdf/2012.08347.pdf

Andrew Trask together with Ben Garfinkel, Allan Dafoe and Eric Drexler (and others I know less well) released a paper on transparency and infohazards that seemed interesting on a first skim:

Abstract: Successful collaboration involves sharing information. However, parties may disagree on how the information they need to share should be used. We argue that many of these concerns reduce to ‘the copy problem’: once a bit of information is copied and shared, the sender can no longer control how the recipient uses it. From the perspective of each collaborator, this presents a dilemma that can inhibit collaboration. The copy problem is often amplified by three related problems which we term the bundling, edit, and recursive enforcement problems. We find that while the copy problem is not solvable, aspects of these amplifying problems have been addressed in a variety of disconnected fields. We observe that combining these efforts could improve the governability of information flows and thereby incentivise collaboration. We propose a five-part framework which groups these efforts into specific capabilities and offers a foundation for their integration into an overarching vision we call “structured transparency”. We conclude by surveying an array of use-cases that illustrate the structured transparency principles and their related capabilities.

Collaboration requires sharing information amongst participants. This gives rise to the central problem of information governance, which we call the copy problem: after replicating and sharing a bit of information, the sender can no longer control how the recipient might use it. The copy problem creates a challenging trade-off for would-be collaborators because each participant must make an educated guess about the likelihood and impact of others’ misuse of the shared data. Assuming rationality, each participant weighs the expected cost of this process against the expected benefit of the collaboration. If the latter outweighs the former for all parties, then the collaboration proceeds. However, if a would-be collaborator expects costs to exceed rewards, they may limit or block the collaboration. Relevant risks and costs informing this assessment can include privacy, security, legal, IP, competitive, public relations, and other similar considerations.

The first contribution of this paper is to lay out the copy pr

...
Experience Report - ML4Good AI Safety Bootcamp
by Kieron Kretschmar — 2024-04-11T18:03:41.040Z

In this post I reflect on my experience in participating at a

ML4Good

(UK, March 2024) bootcamp. I am writing this mainly for my own benefit - to reflect and to start making a habit of engaging with the community on lesswrong. But also to help future participants. If there is at least one person who finds it helpful for deciding whether or not they want to apply to a future iteration of ML4Good or a similar program, then I’m more than happy.

Opinions are my own and relate to this iteration of ML4Good (the program may change in the future). I am not affiliated with ML4Good beyond having participated in this program.

My expectations

I mainly applied for three reasons. First, I wanted to connect with more AI safety minded people in Europe, and get a better feeling for the EU landscape of AI safety initiatives. My second goal was to train some best practices with pytorch for which I would simply not set time aside for at home. Lastly, I saw this as a fun opportunity to break out of my current routine of writing my thesis.

The program

ML4Good is a 10-day bootcamp to educate participants from a variety of backgrounds with at least some familiarity with Python on topics related to ML and AI safety.

For this iteration, the topics came from three broad categories: Technical topics (including optimization, transformers, RL, RLHF, interpretability, and adversarial attacks); conceptual and strategy (capabilities, AI risks, AI safety agendas, theory of change, career planning); and governance and policy (including international governance, compute governance, and evaluations).

The schedule was intense, with activities planned from 9am to 9pm almost every day, and contained a mix of lectures, discussions and opportunities for self-study. A particular focus was set on generation, feedback and refinement of ideas that participants might pursue after the camp.

A typical day might have looked like this:

Time	Activity
08.30 - 09.00	Breakfast
09.00 - 10.20	Lecture on e.g. transformers or RLHF
10.20 - 10.30	Short break
10.30 - 12.00	Work on jupyter notebooks, alone or in small teams
12.00 - 13.00	Workshop with discussions on e.g. forecasting or theory of change
13.00 - 14.00	Lunch
14.00 - 18.30	Small group projects, e.g. a literature review
18.30 - 19.30	Dinner
19.30 - 21.00	Presentation of group projects
After 21.00	Leisure time

The teachers were highly motivated and competent, each...
A Gentle Introduction to Risk Frameworks Beyond Forecasting
by pendingsurvival — 2024-04-11T18:03:25.605Z

This was originally posted on Nathaniel's and Nuno's substacks (Pending Survival and Forecasting Newsletter, respectively). Subscribe here and here!
Discussion is also occurring on the EA Forum here (couldn't link the posts properly for technical reasons).

Introduction

When the Effective Altruism, Bay Area rationality, judgemental forecasting, and prediction markets communities think about risk, they typically do so along rather idiosyncratic and limited lines. These overlook relevant insights and practices from related expert communities, including the fields of disaster risk reduction, safety science, risk analysis, science and technology studies—like the sociology of risk—and futures studies.

To remedy this state of affairs, this document—written by Nathaniel Cooke and edited by Nuño Sempere—(1) explains how disaster risks are conceptualised by risk scholars, (2) outlines Normal Accident Theory and introduces the concept of high-reliability organisations, (3) summarises the differences between “sexy” and “unsexy” global catastrophic risk (GCR) scenarios, and (4) provides a quick overview of the methods professionals use to study the future. This is not a comprehensive overview, but rather a gentle introduction.

Risk has many different definitions, but this document works with the IPCC definition of the “potential for adverse consequences”, where risk is a function of the magnitude of the consequences and the uncertainty around those consequences, recognising a diversity of values and objectives.1 Scholars vary on whether it is always possible to measure uncertainty, but there is a general trend to assume that some uncertainty is so extreme as to be practically unquantifiable.2 Uncertainty here can reflect both objective likelihood and subjective epistemic uncertainty.

1. Disaster Risk Models

A common saying in disaster risk circles is that “there is no such thing as a natural disaster”. As they see it, hazards may arise from nature, but an asteroid striking Earth is only able to threaten humanity because our societies currently rely on vulnerable systems that an asteroid could disrupt or destroy.3

This section will focus on how disaster risk reduction scholars break risks down into their components, model the relationship between disasters and their root causes, structure the process of risk reducti...
Boundaries Update #1
by Chris Lakin — 2024-04-11T16:07:18.746Z
Link: https://formalizingboundaries.substack.com/p/update-1

Boundaries agenda updates in the last few months.

“What does davidad want from «boundaries»?”

davidad and I had

a lesswrong dialogue

I recommend reading.

If you need a refresher on boundaries, read both the above dialogue and the

formalizingboundaries.ai

website.

Conceptual Boundaries Workshop

We ran

Conceptual Boundaries Workshop

on Feb 10–12.

In attendance:

David ‘davidad’ Dalrymple

,

Scott Garrabrant

,

TJ (Tushant Jha)

,

Andrew Critch

,

Allison Duettmann

,

Alex Zhu

,

Jeff Beck

,

Adam Goldstein

,

Manuel Baltieri

,

Lisa Thiergart

,

Abram Demski

,

Evan Miyazono

, and

me

.

For more about what we discussed, see Evan’s

personal retrospective

.

Supported by

The Foresight Institute

, Blake Borgeson, and the Long Term Future Fund.

ACX Grant

Scott Alexander granted us

$40,000

for boundaries projects and workshops.

Mathematical Boundaries Workshop

Mathematical Boundaries Workshop is running this week for 5 days. Goal: develop boundaries math further, ultimately for application in real-world scenarios. Many category theorists are in attendance.

We are inviting a few guests to hang out at the end of the workshop — this Sunday morning, Berkeley CA. Email me

chris@chrislakin.com

if you’d like to come.

davidad’s ARIA programme now live

davidad’s ARIA programme for safeguarded AI is

now live

and soliciting applications for the first phase (>$74M over 4 years). See the

ARIA page

.

future updates

Subscribe: https://formalizingboundaries.substack.com/
Announcing Atlas Computing
by miyazono — 2024-04-11T15:56:31.241Z

Atlas Computing

is a new nonprofit working to collaboratively advance AI capabilities that are asymmetrically risk-reducing. Our work consists of building scoped prototypes and creating an ecosystem around @davidad’s Safeguarded AI programme at ARIA (formerly referred to as the Open Agency Architecture).

We formed in Oct 2023, and raised nearly $1M, primarily from the Survival and Flourishing Fund and Protocol Labs. We have no physical office, and are currently only Evan Miyazono (CEO) and Daniel Windham (software lead), but over the coming months and years, we hope to create compelling evidence that:

The Safeguarded AI research agenda includes both research and engineering projects where breakthroughs or tools can incrementally reduce AI risks.
If Atlas Computing makes only partial progress toward building safeguarded AI, we’ll likely have put tools into the world that are useful for accelerating human oversight and review of AI outputs, asymmetrically favoring risk reduction.
When davidad’s ARIA program concludes, the work of Atlas Computing will have parallelized solving some tech transfer challenges, magnifying the impact of any technologies he develops.

Our overall strategy

We think that, in addition to encoding human values into AI systems, a very complementary way to dramatically reduce AI risk is to create external safeguards that limit AI outputs. Users (individuals, groups, or institutions) should have tools to create specifications that list baseline safety requirements (if not full desiderata for AI system outputs) and also interrogate those specifications with non-learned tools. A separate system should then use the specification to generate candidate solutions along with evidence that the proposed solution satisfies the spec. This evidence can then be reviewed automatically for adherence to the specified safety properties. This is by comparison to current user interactions with today’s generalist ML systems, where all candidate solutions are at best reviewed manually. We hope to facilitate a paradigm where the least safe user's interactions with AI looks like:

Specification-based AI vs other AI risk mitigation strategies

We consider near-term risk reductions that are possible with this architecture to be highly compatible with existing alignment techniques.

In Constitutional AI, humans are legislators but laws are sufficiently nuanced

...
AI #59: Model Updates
by Zvi — 2024-04-11T14:20:06.339Z

Claude uses tools now. Gemini 1.5 is available to everyone and Google promises more integrations. GPT-4-Turbo gets substantial upgrades. Oh and new model from Mistral, TimeGPT for time series, and also new promising song generator. No, none of that adds up to GPT-5, but everyone try to be a little patient, shall we?

Table of Contents

In addition to what is covered here, there was a piece of model legislation introduced by the Center for AI Policy. I took up the RTFB (Read the Bill) challenge, and offer extensive thoughts for those who want to dive deep.

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. Help me, doctor.

Language Models Don’t Offer Mundane Utility. You keep using that word.

Clauding Along. Claude use tool.

Persuasive Research. Claude now about as persuasive as humans.

The Gemini System Prompt. The fun police rulebook is now available.

Fun With Image Generation. This week it is music generation. Are we so back?

Deepfaketown and Botpocalypse Soon. Do you influence the AI influencers?

Copyright Confrontation. The New York Times talks its book.

Collusion. The pattern matching machines will, upon request, match patterns.

Out of the Box Thinking. Escape from the internet is not exactly hard mode.

The Art of the Jailbreak. GPT-4-Turbo falls, according to Pliny. Ho-hum.

They Took Our Jobs. Or rather our applications?

Get Involved. Asking for a friend.

Introducing. Command-R+, Code Gemma, TimeGPT and a Double Crux bot.

In Other AI News. We wrote the checks.

GPT-4 Real This Time. New version is new, but is it improved?

GPT-5 Alive? What are they waiting for? Presumably proper safety testing.

Quiet Speculations. Get your interactive plans away from my movies.

Antisocial Media. Follow-up to the CWT with Jonathan Haidt.

The Quest for Sane Regulations. New excellent Science article, and more.

Rhetorical Innovation. Variations on the is/ought distinction.

Challenge Accepted. This is The Way. Hold my beer.

Aligning a Smarter Than Human Intelligence is Difficult. Especially for real.

Please Speak Directly Into the Microphone. Should Richard Sutton count?

People Are Worried About AI Killing Everyone. Get busy living.

The Lighter Side. I’m a man of great experience.

Language Models Offer Mundane Utility

Use Grok to find things on Twitter. G...
Things can be difficult in 3 ways: Painful, time-consuming, or uncontrollable. Is this reasonable to say?
by SpectrumDT — 2024-04-11T12:50:20.423Z

I have noticed that when someone says that something is hard or difficult, this can mean several different things. I think I have broken it down to 3 categories:

A thing can be difficult because it is painful - it involves pushing through experiences that you would rather not. For example, eating healthy can be difficult because it is unpleasant in the short run compared to eating whatever tastes best.
A thing can be difficult because it is time-consuming. For example, learning a new language is difficult because it can take hundreds or thousands of hours, even if the work is largely neutral-to-interesting.
A thing can be difficult because it is uncontrollable. It depends on factors outside your control, such as the actions of other people or pure randomness. For example, getting a new romantic partner can be difficult because you need to find someone who wants your or convince someone to be with you. Getting a royal straight flush in poker is also difficult because it depends on the luck of the draw. (I think; I don't know poker.)

Of course some things are difficult because of a combination of two or all of the above. Becoming president of the USA takes both lots of time, hard work, unsavory compromises, and the cooperation of many other people, plus you need to satisfy some requirements outside your control such as having been born in the USA.

Do you agree with the above categorization? Or have I left something out?

Thanks in advance!
Learning Written Hindi From Scratch
by Morpheus — 2024-04-11T11:13:17.743Z

An exercise to learn written Hindi just from written text. I did this a year ago and copied this from my notes. Sorry if it's a bit rough.

I recently had a discussion with an Indian friend in my dorm about languages. He made the argument that logographic languages like Chinese are less prone to meaning drift. Also, that they are fundamentally more attached to their meaning, and that you could basically bootstrap your understanding of written Chinese without ever learning the spoken language. Up to that point, I basically agreed, but said I would expect (at least in theory) the same to be true for any written language (thinking about Huffman coding/Solomonoff induction and LLMs in particular).
I made the argument that the longer the chains of text/n-grams you learn to predict, the better you are at actually understanding things. Being good at knowing the frequency of letters seems pointless, but knowing the probability of 5-letter sequences makes you already pretty proficient at spelling. 20 letter sequences make you good at grammar. Finally, when you are at 100 letter sequences you get something resembling coherence.
I noticed I didn't have a good intuition for how far you could get by this method in practice if you can't read the entire internet like an LLM. But testing this is cheap! So I said he should send me a book in Hindi and I would try to find the most impressive thing I could learn from just the text in 30 minutes. As it turns out, this task was in fact pretty hard, but also fun! I ended up agreeing with my friend that this is probably slightly easier for logographic languages.

Exercise (Optional): Grab this text. Now set a 10-minute timer and try to learn as much Chinese as you can. Take notes and put your takeaways in the comments inside spoiler tags. For comparison, repeat the exercise on this text in Hindi, which is syllable based. If this felt rushed, but fun, try another 30 minutes on each text. Consider the rest of this post spoiler.

Here are my notes from when I did this exercise:

Actually going over data multiple times seems like the most efficient way to get the most out of data? I roughly think I remember even lots of AI researchers on podcasts on this topic saying something like "our methods are not good enough, they need to go over data multiple times!" and I am not actually sure this is a problem? I went over the book multiple times ju

...
Work ethic after 2020?
by TeaTieAndHat — 2024-04-11T03:32:28.487Z

It seems to me that my work ethic / tendency to akrasia / etc. has changed a lot since 2020, and many people around me seem to agree that it’s also become harder for them to keep working on things they don’t really enjoy doing.

But I find myself very confused about this topic, so I’d like to get a little more data from other people’s experience: is it true for many people? if so, why exactly would Covid (or, more likely, lockdowns) change our work ethic? what did change — are people ‘lazier’ or ‘more discriminating about which tasks they enjoy or not’ or ‘now often working with less rigid structures and rules’? is the change good or bad — to me, it feels like being much less able to deeply feel like I have a duty to do things really well (bad, probably), but maybe most everyone else feel like they’re more concerned about doing things which actually seem important to them (good)?