trentmkelly/LessWrong-43k huggingface dataset:
Importing a Python File by Name
by jefftk — 2024-01-25T16:00:13.210Z

Let's say you have a python script:

2024-01-24--evaluate-chimeras.py

And you want to pull a section of it out into a separate file:

2024-01-25--strand-split-artifacts.py

You would hope you could just do something like:

import "/path/to/2024-01-25--strand-split-artifacts.py"

But this doesn't work:

import wants a module name, not a
filename. The simplest way I know to import a python file from a path
is:

import sys
import importlib

sys.path.append("/path/to/")
ssa = importlib.import_module("2024-01-25--strand-split-artifacts")

There are a lot of

complicated
ways to do this, some of which avoid needing to add something to

sys.path, but for quick one-off research code better to
keep it simple.
[Repost] The Copenhagen Interpretation of Ethics
by mesaoptimizer — 2024-01-25T15:20:08.162Z
Link: http://web.archive.org/web/20230105070226/https://blog.jaibot.com/the-copenhagen-interpretation-of-ethics/

Because the original webpage (and domain) is down, and it takes about a minute (including loading time) for Wayback Machine to give me the page, I've decided to repost this essay here. I consider it an essay that seems core to 2010s rationalist discourse.

The Copenhagen Interpretation of quantum mechanics says that you can have a particle spinning clockwise and counterclockwise at the same time – until you look at it, at which point it definitely becomes one or the other. The theory claims that observing reality fundamentally changes it.

The Copenhagen Interpretation of Ethics says that when you observe or interact with a problem in any way, you can be blamed for it. At the very least, you are to blame for not doing more. Even if you don’t make the problem worse, even if you make it slightly better, the ethical burden of the problem falls on you as soon as you observe it. In particular, if you interact with a problem and benefit from it, you are a complete monster. I don’t subscribe to this school of thought, but it seems pretty popular.

In 2010, New York randomly chose homeless applicants to participate in its Homebase program, and tracked those who were not allowed into the program as a control group. The program was helping as many people as it could, the only change was explicitly labeling a number of people it wasn’t helping as a “control group”. The response?

“They should immediately stop this experiment,” said the Manhattan borough president, Scott M. Stringer. “The city shouldn’t be making guinea pigs out of its most vulnerable.”

On March 11th, 2012, the vast majority of people did nothing to help homeless people. They were busy doing other things, many of them good and important things, but by and large not improving the well-being of homeless humans in any way. In particular, almost no one was doing anything for the homeless of Austin, Texas. BBH Labs was an exception – they outfitted 13 homeless volunteers with WiFi hotspots and asked them to offer WiFi to SXSW attendees in exchange for donations. In return, they would be paid $20 a day plus whatever attendees gave in donations. Each of these 13 volunteers chose this over all the other things they could have done that day, and benefited from it – not a vast improvement, but significantly more than the 0 improvement that they were getting from most people.

The response?

IT SOUNDS LIKE something

...
Nash Bargaining between Subagents doesn't solve the Shutdown Problem
by A.H. — 2024-01-25T10:47:11.877Z

Work funded by the Long Term Future Fund.

Corrigibility is the hypothetical feature of some agents which allows them to be 'shut down' by an outside user without attempting to manipulate the whether or not they are shut down. The motivation behind this concept is the possibility of making an AI agent which can pursue a 'trial' goal given to it by its creators, but can be stopped if pursuing this goal becomes undesirable.

On the face of it, Corrigibility sounds a bit like a bargaining problem. A corrigible agent might behave as a comprise between two subagents: one which cares about pursuing the original goal, and one which cares about achieving shutdown.

Recently, @johnswentworth and @David Lorell have written A Shutdown Problem Proposal, which discusses this suggestion a bit more. However, they do not specify out the mechanism by which the subagents will reach their agreement. They write:

Then there’s the problem of designing the negotiation infrastructure, and in particular allocating bargaining power to the various subagents. They all get a veto, but that still leaves a lot of degrees of freedom in exactly how much the agent pursues the goals of each subagent. For the shutdown use-case, we probably want to allocate most of the bargaining power to the non-shutdown subagent, so that we can see what the system does when mostly optimizing for u_1 (while maintaining the option of shutting down later).

Coincidently, this is similar to something I've been thinking about recently so I took the opportunity to try to finish up this post which I've had sitting around for a while.

In particular, I was looking at whether Nash Bargaining between two subagents works to create corrigible agent. In the way I attempted it, this doesn't work. I think that the Wentworth/Lorell approach differs slightly to the one I use here (in particular, they emphasise the counterfactual nature of the two expected utilities - something I don't fully understand), so this isn't intended as a 'refutation'-just an indication of the kind of problems you might encounter when trying to flesh out their suggestion.

I've tried to keep the important points in the main text and technical details are mostly in footnotes to avoid breaking the flow.

Nash Bargaining

There are several solutions to bargaining problems, depending on which axioms one selects. One of the most elegant is Nash Bargaining. Nash Barg...
Status-oriented spending
by Adam Zerner — 2024-01-25T06:46:47.029Z

Recently I started spending money on a bunch of things that might seem a little extravagant:

House cleaner
Massage therapist
Psychotherapist that is not covered by insurance
Professional organizer
A nearly $3,000 mattress cover (cools/heats bed)

My impression is that they're all the sorts of things that are mostly purchased by rich people. Not upper-middle class people like me. Rich people.[1]

So it felt a little uncomfortable to pull the trigger on each of these purchases. In my mind's eye I imagine people learning of these purchases and passive-aggressively saying "must be nice". And I doubt that I'm alone in these feelings of discomfort.[2]

In this post I'd like to argue that such purchases shouldn't be seen this way.

Value-oriented perspective

Let's look at each purchase individually:

House cleaner

The house cleaner I use costs $30/hr. Since she's a faster and better cleaner than I am, what takes her one hour maybe would take me two.

I can make maybe $100/hr programming. Dirty things make me feel somewhat stressed.

Why wouldn't I trade money for time and peace of mind here?

Massage therapist

I have chronic Achilles tendinitis. It's not the worst thing in the world, but it's also certainly not the best. If the tendinitis mostly went away that'd be a nice improvement to my life.

I've tried tons of things to improve it and nothing has worked. In reading a resource I trust, I've come to believe that it's not entirely implausible that massage therapy would work. So then, as an experiment, why wouldn't I spend a couple of months giving massage therapy a shot?

Psychotherapist

The ROI of psychotherapy is just very clearly very positive.

As for in-network vs out-of-network, I've tried a few in-network people who haven't worked out. For various reasons I'm not optimistic about being able to find a more affordable in-network therapist who will work out, and this particular out-of-network therapist was recommended by someone who I trust a lot. So then, it seems easily worth it.

Professional organizer

My apartment feels way too cluttered. The clutter makes me feel a surprisingly large amount of discomfort. It's not intense, but it's a sort of perpetual, constant, slow burning discomfort that I think is actually a pretty big issue.

Yes, perhaps this could be addressed by simply 1) asking what sparks joy and 2) getting rid of the things that don't. But that isn't always possible when y...
Protecting agent boundaries
by Chris Lakin — 2024-01-25T04:13:50.993Z

If the preservation of an agent's boundary

is

necessary for that agent's safety

, how can that boundary/membrane be protected?

How agent boundaries get violated

In order to protect boundaries, we must first understand how they get violated.

Let’s say there’s a cat, and it gets stabbed by a sword. That’s a boundary violation (a.k.a. membrane piercing). In order for that to have happened, three conditions must have been met:

There was a sword.
The cat and the sword collided.
The cat wasn’t strong enough to resist penetration from the sword.

More generally, in order for any existing membrane to be pierced, three conditions must have all been met:

There was a potential threat. (E.g., a sword, or a person with a sword.)
The moral patient and the threat collided.
The victim failed to adequately defend itself. (Because if the cat was better at self-defense — if its skin was thicker or if it was able to dodge — then it would not have been successfully stabbed.)

Protecting agent boundaries

Each of these three conditions then implies ways of preventing boundary violations (a.k.a. membrane piercing):

1. There was a potential threat.

→ Minimize potential threats

2. There was a collision.

→ Minimize dangerous collisions

→ Predict and prevent collisions before they occur.
→ Prevent collisions by putting distance between threats and moral patients.
→ Prevent premeditated collisions by pre-committing to retribution.

3. The victim failed to defend itself.

→ Empower the membranes of humans and other moral patients to be better at self-defense.

How human societies already try to solve this problem

As a helpful analogy, here’s some examples of how modern human societies try to solve this problem:

Minimize potential threats

Restrict access to weapons (e.g., nukes, bioweapons, etc.)
Minimize potential perpetrators (i.e., e.g., some fictional societies predict and eliminate potential psychopaths).

Minimize dangerous collisions

Protect high-risk individuals, e.g. put them witness protection
Prevent collisions before they occur, e.g. predictive policing, traffic lights.
Police crimes after they occur.

Empower membranes to be better at self-defense

Infosec defense: Use good

security practices

and strong encryption.
Biological defense: Develop and use beneficial vaccines.
Manipulation defense: Reduce unhelpful cognitive biases and emotional insecurities.

How this applies to AI safety:

Minimize poten...
Is a random box of gas predictable after 20 seconds?
by Thomas Kwa — 2024-01-24T23:00:53.184Z

This came up as a tangent from this question, which is itself a tangent from a discussion on The Hidden Complexity of Wishes.

Suppose we have a perfect cubical box of length 1 meter containing exactly 1 mol of argon gas at room temperature.

At t=0, the gas is initialized with random positions and velocities θ drawn from the Maxwell-Boltzmann distribution.
Right after t=0 we perturb one of the particles by 1 angstrom in a random direction X to get the state m(θ,X).
All collisions are perfectly elastic, so there is no viscosity [edit, this is wrong; even ideal gases have viscosity] and energy is conserved.
For each possible perturbation, we run physics forward for 20 seconds and measure whether there are more gas molecules in the left side or right side of the box at t=20 seconds (the number on each side will be extremely close to equal, but differ slightly). Do more than 51% of the possible perturbations result in the same answer? That is, if is_left is the predicate "more gas molecules on the left at t=20", is |PrX[is_left(m(θ,X))]−0.5|>0.01?

This is equivalent to asking if an omniscient forecaster who knows the position and velocity of all atoms at t=0 except for 1 angstrom of uncertainty in 1 atom can know with >51% confidence which side has more gas molecules at t=20.

I think the answer is no, because multiple billiard balls is a textbook example of a chaotic system that maximizes entropy quickly, and there's no reason information should be preserved for 20 seconds. This is enough time for each atom to collide with others millions of times, and even sound waves will travel thousands of meters and have lots of time to dissipate.

@habryka thinks the answer is yes and the forecaster could get more than 99.999% accuracy, because with such a large number of molecules, there should be some structure that remains predictable.

Who is right?
Will quantum randomness affect the 2028 election?
by Thomas Kwa — 2024-01-24T22:54:30.800Z

This came up as a tangent from @habryka and me discussing whether The Hidden Complexity of Wishes was correct.

Is the result of a US presidential election 4 years from now >0.1% contingent on quantum randomness (i.e. is an otherwise omniscient observer forecasting the 2028 election today capable of >99.9% confidence, or is there >0.1% irreducible uncertainty due to quantum mechanics observer-effects)?

I think the answer is yes, because chaotic systems will quickly amplify this randomness to change many facts about the world on election day.

Quantum randomness causes different radioactive decays, which slightly perturb positions of particles around the world by a few nanometers.
Chaotic systems will quickly amplify these tiny perturbations into macro-scale perturbations:

Weather doubles perturbations every 4 days or so
The genes of ~all babies less than 3 years old will be different

Many events relevant to the election are contingent on these differences

Weather-related natural disasters, other circumstances like pandemics (either mutation or lab leak), political gaffes by candidates, assassinations (historically >0.1% and seem pretty random), cancer deaths, etc.

If even a small proportion of election variance is random, you get more than 0.1% election randomness.

Say humanity's best estimates for the vote margin of the 2028 election have a standard deviation of 76 electoral votes centered on 0. Even if 90% of variance is in theory predictable and only 10% is true randomness (aleatoric), then the nonrandom factors have s.d. 70 and random factors have s.d. 24. If nonrandom factors have 1sd influence, random factors will flip the election with probability well over 0.1%.
In reality it's much worse than this, because we haven't even identified 2 leading candidates.

Oliver thinks the answer is no, because in a system as large and complicated as the world, there should be some macro-scale patterns that survive, and an omniscient observer will pick up on all such patterns. Humans are limited to obvious patterns like economic trends and extrapolating polls, but there are likely way more patterns than this, which a forecaster could use to accurately get over well 99.9% confidence.

Who is right?
AISN #30: Investments in Compute and Military AI
Plus, Japan and Singapore’s National AI Safety Institutes
by Dan H — 2024-01-24T19:38:33.461Z
Link: https://newsletter.safe.ai/p/aisn-30-investments-in-compute-and

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.

Subscribe here to receive future versions.

Listen to the AI Safety Newsletter for free on Spotify.

Compute Investments Continue To Grow

Pausing AI development has been proposed as a policy for ensuring safety. For example, an open letter last year from the Future of Life Institute called for a six-month pause on training AI systems more powerful than GPT-4.

But one interesting fact about frontier AI development is that it comes with natural pauses that can last many months or years. After releasing a frontier model, it takes time for AI developers to construct new compute clusters with larger numbers of more advanced computer chips. The supply of compute is currently unable to keep up with demand, meaning some AI developers cannot buy enough chips for their needs.

This explains why OpenAI was reportedly limited by GPUs last year. Relatedly, the chairman of TSMC, the world’s largest chip manufacturer, said in September, “Currently, we cannot fulfill 100% of our customers' needs, but we try to support about 80%.”

Despite its limited supply, investments in compute have continued to grow. Here, we report on recent news from Meta and OpenAI on their plans for investing in compute.

Meta is investing in compute to build open-source AGI. Mark Zuckerberg announced that Meta plans to have 350,000 of Nvidia’s most advanced GPUs, the H100, by the end of 2024. This is more than ten times the number of GPUs in Inflection AI’s cluster, announced last summer. The only company expected to purchase as many H100s as Meta is Microsoft.

Meta and Microsoft are projected to purchase 3x more leading Nvidia GPUs than their closest competitors. Source.

Zuckerberg announced that Meta plans to “build general intelligence” and “open source it responsibly.” This contrasts with OpenAI and DeepMind’s approaches of closed source development, which some argue reduces risks from malicious use. Advocates of open-source instead argue that broad access to AI will counteract centralization of power and distribute the benefits of AI broadly.

Sam Altman considers founding a new company for chip design and manufacturing. Currently, the global supply chain for advanced AI chips is reliant on a few companies. Nvidia is one of those companies...
Krueger Lab AI Safety Internship 2024
by Joey Bream — 2024-01-24T19:17:40.606Z

KASL - AI Safety Internship 2024

Join the Krueger AI Safety Lab (KASL) at the University of Cambridge for a paid Research Internship focusing on technical and governance aspects of AI safety. Remote interns are welcome. Apply by Monday, January 29, 23:59 UTC. Shortlisted candidates will be notified by Friday, February 16.

Application Process:

Applications close: January 29, 23:59 UTC
Form: Short essay questions, focusing on content. 1-hour duration.
Next Stage: Shortlisted candidates invited for interviews.
Apply: https://t.co/hLOjuxSfnK
More details: https://www.kasl.ai/2024/01/16/interns-2024/

Credits

This post is from KASL, the Krueger AI Safety Lab.
Agents that act for reasons: a thought experiment
by Michele Campolo — 2024-01-24T16:47:19.469Z

Posted also on the EA Forum.

In

Free agents

I’ve given various ideas about how to design an AI that reasons like an independent thinker and reaches moral conclusions by doing so. Here I’d like to add another related idea, in the form of a short story / thought experiment.

Cursed

Somehow, you have been cursed. As a result of this unknown curse that is on you now, you are unable to have any positive or negative feeling. For example, you don’t feel pain from injuries, nothing makes you anxious or excited or sad, you can’t have fun anymore. If it helps you, imagine your visual field without colours, only with dull shades of black and white that never feel disgusting or beautiful.

Before we get too depressed, let’s add another detail: this curse also makes you immune to death (and other states similar to permanent sleep or unconsciousness). If you get stabbed, your body magically recovers as if nothing happened. Although this element might add a bit of fun to the story from our external perspective, keep in mind that the cursed version of you in the story doesn’t feel curious about anything, nor has fun when thinking about the various things you could do as an immortal being.

No one else is subject to the same curse. If you see someone having fun and laughing, the sentence “This person is feeling good right now” makes sense to you: although you can’t imagine nor recall what feeling good feels like, your understanding of the world around you remained intact somehow. (Note: I am not saying that this is what would actually happen in a human being who actually lost the capacity for perceiving

valence

. It’s a thought experiment!)

Finally, let’s also say that going back to your previous life is not an option. In this story, you can’t learn anything about the cause of the curse or how to reverse it.

To recap:

You can’t feel anything
You can’t die
You can’t go back to your previous state
The curse only affects you. Others’ experiences are normal.

In this situation, what do you do?

In philosophy, there is some discourse around reasons for actions, normative reasons, motivating reasons, blah blah blah. Every philosopher has their own theory and uses words differently, so instead of citing centuries of philosophical debates, I’ll be maximally biased and use one framework that seems sensible to me. In

Ethical Intuitionism

, contemporary philosopher Michael Huemer disti...
Impact Assessment of AI Safety Camp (Arb Research)
by Samuel Holton — 2024-01-24T16:19:12.431Z
Link: https://forum.effectivealtruism.org/posts/CuPnmeS4v5sFE6nQj/impact-assessment-of-ai-safety-camp-arb-research

Authors: Sam Holton, Misha Yagudin

Data collection: David Mathers, Patricia Lim

Note: Arb Research was commissioned to produce this impact assessment by the AISC organizers.

Summary

AI Safety Camp

(AISC) connects people interested in AI safety (AIS) to a research mentor, forming project teams that last for a few weeks and go on to write up their findings. To assess the impact of AISC, we first consider how the organization might increase the productivity of the Safety field as a whole. Given its short duration and focus on introducing new people to AIS, we conclude that AISC’s largest contribution is in producing new AIS researchers that otherwise wouldn’t have joined the field.

We gather survey data and track participants in order to estimate how many researchers AISC has produced, finding that 5–10% of participants plausibly become AIS researchers (see “Typical AIS researchers produced by AISC” for examples) that otherwise would not have joined the field. AISC spends roughly $12–30K per researcher. We could not find estimates for counterfactual researcher production in similar programs such as (SERI) MATS. However, we used the LTFF grants database to estimate that the cost of researcher upskilling in AI safety for 1 year is $53K. Even assuming all researchers with this amount of training become safety researchers that wouldn’t otherwise have joined the field, AISC still recruits new researchers at a similar or lower cost (though note that training programs at different stages of a career pipeline are complements).

We then consider the relevant counterfactuals for a nonprofit organization interested in supporting AIS researchers and tentatively conclude that funding the creation of new researchers in this way is slightly more impactful than funding a typical AIS project. However, this conclusion is highly dependent on one’s particular views about AI safety and could also change based on an assessment of the quality of researchers produced by AISC.

We also review what other impacts AISC has in terms of producing publications and helping participants get a position in AIS organizations.

Approach

To assess impact, we focus on AISC’s rate of net-new researcher production. We believe this is the largest contribution of the camp given their focus on introducing researchers to the field and given the short duration of projects. In the appendix, we justify this ...