trentmkelly/LessWrong-43k huggingface dataset:
Join AISafety.info's Distillation Hackathon (Oct 6-9th)
by smallsilo — 2023-10-01T18:43:43.359Z
Link: https://forum.effectivealtruism.org/posts/vdTQnkETbECvPB3mY/join-aisafety-info-s-writing-hackathon-oct-6-9th

tl;dr: Contribute to

aisafety.info

by answering questions about AI Safety from October 6th to October 9th. Participation in hackathons is the basis for applying to future fellowships, and there are prizes to be won by the top entrants. Register

here

and see the participant guide

here

.

What is the schedule for the event?

The event will run from Friday October 6th, 7am UTC, to Monday October 9th 2023, 7am UTC. See

here

for the full schedule. You are invited to participate throughout whichever parts of those days fit your schedule.

What is the format of the event?

Participants will choose questions to answer for aisafety.info, and work on these answers in google docs. Collaboration on the event will take place on

Discord

as well as on

gather.town

. I’ll be online for most of those three days to lead the event and answer any questions. See

here

for more details.

Are there prizes?

Yes! There will be monetary prizes of $1000, $600, $300, and $100 for the top four contributors, and $200 for the participant judged to be the most helpful to others.

The main criteria we’ll use to select winners are submitting good articles and making valuable edits to articles in progress. Contributions will also count towards applications for any future fellowships we run.

Should I participate?

You should participate if you’re interested in contributing to AI safety (but perhaps don’t know where to start or how much you can commit). We think of helping with aisafety.info as ‘legitimate peripheral participation’ - where you can meaningfully contribute even if you’re relatively new, without making huge commitments.
You should participate if you’re interested in working on distillation - writing clear explanations for AI Safety and other technical concepts.
You do not have to have a high level of technical knowledge to participate. There are a wide range of questions to work on, some of which are intended to explain basic concepts to those who have never even heard of AI safety.

I'm busy between Oct 6th and 9th - can I still participate?

You can write or edit articles in any contiguous three-day period between today and Oct 9th–for example, from the 2nd to the 5th.

What is

aisafety.info

?

Stampy’s

AI Safety Info

is an interactive FAQ started by

Rob Miles

that aims to be the best one-stop source of information about AI existential safety, gathering ...
Fifty Flips
by abstractapplic — 2023-10-01T15:30:43.268Z
Link: https://h-b-p.github.io/FiftyFlips/

An unfair coin (potentially EXTREMELY unfair) will be flipped fifty times. Your goal is to correctly predict as many of these flips as possible, by deducing the nature of the unfairness as quickly as possible.

[Predict Heads] [Predict Tails]

You can play this (in-browser, very short) game here; the rule governing the unfairness is automatically revealed after flip 50. Followups with different governing rules are here, here, here, here and here.
AI Safety Impact Markets: Your Charity Evaluator for AI Safety
by Dawn Drescher — 2023-10-01T10:47:06.952Z
Link: https://impactmarkets.substack.com/p/ai-safety-impact-markets

Summary: This is a quick overview of the current state of our system AI Safety Impact Markets: We want to give donors an improved ability to coordinate so that their diverse knowledge can inform our crowdsourced donation recommendations. AI Safety Impact Markets will help them identify the most impactful projects and get the projects funded more quickly too. (More on our long-term vision can be found in the article “

The Retroactive Funding Landscape

.”)

Please send me your name, email address, and approximate annual donation budget in a private message (or

through this form

) if you are interested in using

AI Safety Impact Markets

to inform your giving! We publish only the sum of all budgets. It’s crucial for us to motivate expert donors to share their donations.

This is an edited transcript of an

eponymous talk

. I’ve held this talk at EAGx Berlin and Warsaw 2023 among other venues.

The Funding Constraint

Great entrepreneurs who start highly impactful charity projects exist in various countries around the world and in lots of different fields that require highly specialized expertise. They also speak a number of different languages and have all their own social circles in the various countries that they live in.

That makes it difficult for an individual donor in Berkeley or London to identify but a tiny fraction of them.

Even really big foundations like the Open Philanthropy Project have problems with this. The Open Philanthropy Project can invest a lot of time into researching grants if the grant size is upward of $1 million or possibly $100,000 so that they make many grants in that area, measured by the total of those grants. But below $1 million the total grant size starts to trail off, and below $50,000 the number of grants drops off too. If they don’t already know the field and the founders, it becomes more and more difficult for them to research the projects efficiently enough that it's still worth it for them given the small grant sizes.

That mirrors how venture capital firms would probably also love to invest the first $100,000 into all the future unicorns, but since they can’t identify them at that stage, or only vanishingly few of them, they depend on business angels and the founders’ friends and family to take the first step.

But we’re now in a world where AI safety projects

struggle to raise $2,000

to make ends meet. So that gap below $1 million, particula...
"Absence of Evidence is Not Evidence of Absence" As a Limit
by transhumanist_atom_understander — 2023-10-01T08:15:28.852Z

Here are two cases involving absence of evidence:

You wait at the bus stop for a half hour without seeing a bus, even though this bus is supposed to come every 10 minutes. This is strong evidence against the hypothesis that the buses are running today (perhaps it's a holiday you forgot about?)

Someone is arrested and charged with a crime, and they do not confess. This is weak evidence against the hypothesis that they are the perpetrator.

In both cases, presence of the evidence (a bus, a confession) would have been strong evidence in favor of the hypothesis.
They differ in the meaning of the absence of that evidence.

The difference between these cases was explained in Absence of Evidence Is Evidence of Absence:

Under the vast majority of real-life circumstances, a cause may not reliably produce signs of itself, but the absence of the cause is even less likely to produce the signs. The absence of an observation may be strong evidence of absence or very weak evidence of absence, depending on how likely the cause is to produce the observation. The absence of an observation that is only weakly permitted (even if the alternative hypothesis does not allow it at all) is very weak evidence of absence (though it is evidence nonetheless).

I posited that the bus comes frequently so that the bus line running as normal (the cause) would be very likely to result in a bus within thirty minutes (the observation).
In contrast, the absence of a confession is unsurprising even if the suspect committed the crime.
(Instead of the criminal example, I could have just modified the first example so you only wait one minute).

The point of this post is to show a formula for the magnitude of the update on absence of evidence.
We'll start with the "update factor" from the odds form of Bayes' theorem (discussed in Arbital intro guide or advanced explanation, and by 3Blue1Brown).
When you observe evidence E, the magnitude of the update in favor of H is

r = P(E|H) / P(E|¬H)

High r (for "ratio") means strong evidence.
r=1 means no evidence.
In both the examples above, r is high: a bus is much more likely given the bus line is running, and a confession is much more likely from a perpetrator.

The degree to which you expect the evidence, if the hypothesis is true, is:

p = P(E|H)

This should be high when you've waited long enough for the bus (conditional on the bus line running), but low for a ...
New Tool: the Residual Stream Viewer
by AdamYedidia — 2023-10-01T00:49:51.965Z
Link: https://tinyurl.com/resid-viewer

This is a link-post for the residual stream viewer, which can be found here. It's an online tool whose goal is to make it easier to do interpretability research by letting you easily look at directions within the residual stream. It's still in a quite early/unpolished state, so there may be bugs, and any feature requests are very welcome! I'll probably do more to flesh this out if I get the sense that people are finding it useful.

Very briefly, the tool lets you see what the dot product of the residual stream at each token is with a particular direction. The default directions that you can look at using the tool were found by PCA, and I think many of them are fairly interpretable even at a glance (though it's worth noting that even if they correlate heavily with an apparent feature, that's no guarantee the network is actually using those directions).

Here's a screenshot of the current version of the tool:

There's a YouTube tutorial for the tool available here. I endorse the YouTube tutorial as probably a better way to get acquainted with the tool than the usage guide; but I'll copy-paste the usage guide for the remainder of the post.

The residual stream viewer is a tool for finding interesting directions in the residual stream of GPT2-small, for writing explanations for those directions and reading the explanations left by others, and for constructing new directions out of linear combinations of old ones.

A more detailed explanation of how transformers networks work and what the residual stream is can be found here. If you want to actually understand what the residual stream is and how transformers work, the text that follows here is hopelessly insufficient, and you should really follow the earlier link. However, as a very brief summary of what the "residual stream" is:

The residual stream can be thought of as the intermediate state of the transformer network's computation. It is the output of each layer of the network before it is fed into the next layer. Each prompt is split into "tokens," i.e. subparts of the prompt that roughly correspond to words or parts of words. At each layer, each token has its own associated residual stream vector. The residual stream at the beginning of the network, before any layer has acted, is equal to the "Token Embedding", i.e. the "meaning" of that token as encoded by a 768-dimensional vector, plus the "Positional embedding"...
My Effortless Weightloss Story: A Quick Runthrough
by CuoreDiVetro — 2023-09-30T23:02:45.128Z

This is Part I in a series on easy weightloss without any need for will power.

The Origin: listening to the dark corners of the internet

Losing weight is supposed to be really hard and require a lot of willpower according to conventional wisdom. It turns out that it was actually really easy for me to go from a BMI of above 29 (30 is officially obese) to below 25 (normal is 18 to 25) in 3½ months. And knowing what I know now, I think I could easily do it again in a 1½ month.

I'm not someone who ever tried dieting before. Dieting sounded like a lot of effort and willpower for very uncertain results. Not a good use of my very limited willpower. This belief changed after reading Slime Mold Time Mold's results of their potato diet experiment[1].

They asked the participants in their experiment to eat only potatoes for 4 weeks to see if they would lose weight. There was no way I was going to eat only potatoes for 4 weeks, so I didn't enrol in their experiment. After reading the blogpost about their results, two things surprised me which motivated me to go on this journey.

The first surprise was that is wasn't necessary to eat only potatoes. Slime Mold Time Mold had been very gentle with their guinea pigs, and they told them "it's ok if you cheat and don't eat potatoes, just tell us when you cheat". It turned out that even people who cheated almost every day, eating something other than potatoes, ended up losing a lot of weight and there wasn't even that clear of a trend between weightloss and number of cheat days (see Figure 1). So a strict eat-only-potatoes-diet which is something I would never do, didn't seem to be necessary.

Figure 1: Weightloss of participants as a function of the number of days (out of a total of 28) where they cheated (i.e. ate other things than potatoes). Source.

The second surprise was that people's weight seemed to go down linearly, not attaining a plateau, at least for the 4 weeks of the experiment. I was expecting diminishing returns as people started to lose weight, that further weightloss would slow down but their data didn't seem to indicate any slowdown. I was super curious to find out how long such a linear weightloss could go for. As we will see later, linear weightloss went on for me for a surprisingly long time.

Figure 2: Weightloss as a function of time on the potato diet. The blue line is those who completed the...
Arguments for moral indefinability
by Richard_Ngo — 2023-09-30T22:40:04.325Z

Epistemic status: I endorse the core intuitions behind this post, but am only moderately confident in the specific claims made. Also, while I do have a degree in philosophy, I am not a professional ethicist, and I’d appreciate feedback on how these ideas relate to existing literature. (Edited to add: readers have pointed out similarities to moral particularism and pluralism; the concept of incomparability also seems relevant).

Moral indefinability is the term I use for the idea that there is no ethical theory which provides acceptable solutions to all moral dilemmas, and which also has the theoretical virtues (such as simplicity, precision and non-arbitrariness) that we currently desire. I think this is an important and true perspective on ethics, and in this post will explain why I hold it, with the caveat that I'm focusing more on airing these ideas than constructing a watertight argument.

Here’s another way of explaining moral indefinability: let’s think of ethical theories as procedures which, in response to a moral claim, either endorse it, reject it, or do neither. Moral philosophy is an attempt to find the theory whose answers best match our intuitions about what answers ethical theories should give us (e.g. don’t cause unnecessary suffering), and whose procedure for generating answers best matches our meta-level intuitions about what ethical theories should look like (e.g. they should consistently apply impartial principles rather than using ad-hoc, selfish or random criteria). None of these desiderata are fixed in stone, though - in particular, we sometimes change our intuitions when it’s clear that the only theories which match those intuitions violate our meta-level intuitions. My claim is that eventually we will also need to change our meta-level intuitions in important ways, because it will become clear that the only theories which match them violate key object-level intuitions. In particular, this might lead us to accept theories which occasionally evince properties such as:

Incompleteness: for some claim A, the theory neither endorses nor rejects either A or ~A, even though we believe that the choice between A and ~A is morally important.

Vagueness: the theory endorses an imprecise claim A, but rejects every way of making it precise.

Contradiction: the theory endorses both A and ~A (note that this is a somewhat provocative way of framing t

...
Conditionals All The Way Down
by lunatic_at_large — 2023-09-30T21:06:14.609Z

(I thought about this idea on my own before Googling to see if anyone had already written it up. I found something very similar at https://projecteuclid.org/journals/notre-dame-journal-of-formal-logic/volume-50/issue-2/Justification-by-an-Infinity-of-Conditional-Probabilities/10.1215/00294527-2009-005.full so all credit for this line of thinking should go to the authors of that paper. Still, I think that this concept deserves a writeup on Lesswrong and I also want to write a series of posts on this kind of topic so I need to start somewhere. If this idea has already been written up on Lesswrong then please let me know!)

Alice and Bob are driving in a car and Alice wants to know whether the driver in front of them will turn at the next light.

Alice asks Bob, "What's the probability that the driver will turn at the next light?" Unfortunately, Bob doesn't know how to estimate that. However, Bob does know that there are cherry blossoms which might be in bloom off the next exit. Bob is able to use his predictive talent to determine that there's a 50% chance that the driver will turn if there are cherry blossoms on display and that there's a 25% chance that the driver will turn if there aren't any cherry blossoms on display. Bob tells Alice that no other variables will interfere with these conditional probabilities.

Alice then asks Bob, "What's the probability that there will be cherry blossoms on display?" Again, Bob is unable to determine this probability. However, Bob does know that the city government was considering chopping the cherry trees down. Bob tells Alice that if the city chopped them down then there's a 5% chance of finding cherry blossoms and that if the city didn't chop them down then there's a 70% of finding cherry blossoms. Bob knows that no other variables can impact these conditional probabilities.

Alice now asks Bob, "What's the probability that the city cut down the cherry trees?" Predictably, Bob doesn't know how to answer that. However, Bob again uses his magical powers of perception[1] to deduce that there's an 80% chance the city chopped them down if the construction company that was lobbying for them to be cut down won its appeal and a 10% chance the city chopped them down if the construction company that was lobbying for them to be cut down lost its appeal.

Now imagine that this conversation goes on forever: whether the construction com...
Focusing your impact on short vs long TAI timelines
by kuhanj — 2023-09-30T19:34:39.508Z

Summary/Key Points

I compare considerations for prioritizing impact in short vs. long transformative AI timeline scenarios. Though lots of relevant work seems timelines-agnostic, this analysis is primarily intended for work whose impact is more sensitive to AI timelines (e.g. young-people-focused outreach and movement building).

Considerations that favor focusing on shorter timelines

Neglectedness of short-timelines scenarios considerations

Fewer resources over less time: In shorter timeline worlds, fewer resources in total ultimately go to TAI existential safety (x-safety), since more time means more work can occur, the total amount of work per year will probably rise (due to concern and interest rising), and the ability for currently concerned individuals to contribute will likely increase over time (due to accruing relevant expertise and other resources). This likely increases the marginal value of a given amount of resources.
You can’t contribute to past timelines in the future: It is much less costly to mistakenly think the deadline for a project is earlier than the real deadline and do things too early than missing the deadline due to mistakenly thinking it was later.

Relatedly, allocating resources proportional to each potential timeline scenario entails always disproportionately focusing on shorter timelines scenarios.

Higher predictability and likelihood of useful work: More time means more uncertainty, so it’s harder to make plans that are likely to pan out.
(Uncertain) Work useful for shorter-timelines is more likely useful for longer timelines than vice versa

Considerations that favor focusing on longer timelines

Many people concerned about TAI x-safety are young and inexperienced: This means many impactful options are not tenable in shorter-timeline scenarios, and longer timelines offer more time to develop relevant skills and land impactful roles.

However, the consideration points in the opposite direction for older and more experienced people concerned about TAI x-safety.

Intractability of changing shorter timelines scenarios

One reason is that many valuable projects take a lot of time, and many projects’ impact increases with time.
This is particularly true in research and policy, and it suggests that longer-timelines scenarios are more tractable. Also, compounding returns benefit

...
How model editing could help with the alignment problem 
by Michael Ripa — 2023-09-30T17:47:24.727Z

Preface

This article explores the potential of model editing techniques in aligning future AI systems. Initially, I was skeptical about its efficacy, especially considering the objectives of current model editing methods. I argue that merely editing "facts" isn't an adequate alignment strategy and end with suggestions for research avenues focused on alignment-centric model editing.

Thanks to Stephen Casper, Nicolas Gatien and Jason Hoelscher-Obermaier for detailed feedback on the drafts, as well as Jim Davies and Esben Kran for high level comments.

A birds eye view of the current state of model editing

Model editing, broadly speaking, is a technique which aims to modify information stored inside of a neural network. A lot of the work done thus far has been focused on editing small language models (e.g. GPT-2, GPT-J) and has been focused specifically on editing semantic facts. There also has been some work in performing edits on different types of neural networks, including vision models (

Santurkar et al

), CLIP (

Illharco et al

) and diffusion models (

Orgad et al

). At present, more emphasis has been placed on editing language models, so this article will be more focused on them.

One of the main approaches takes in logical triplets of the form (Subject,Relation,Object) and performs an update to the “object” value, which in turn modifies information about the “subject”. For example, the sentence “The Eiffel tower is located in Paris” would be expressed as (“Eiffel tower”,“located”,“Paris”), and a potential edit could be to replace “Paris” with the value “Rome”. Some variations on this setup exist (for example, editing the prediction of a [MASK] token for BERT like models), but the logical triplet setup is the most popular and will be the main approach we focus on for this article.

There are a number of different model editing techniques, which I will briefly summarize below (see

Yao et al

for a more in-depth overview):

1. Locate and edit methods

These methods rely on the assumption that the MLP layers of transformer models form a “linear associative memory” (

Geva et al

), which form a sort of database for pieces of factual information.
One way of looking at it is that there is a specific linear weight in the model that when passed a representation containing the subject (e.g. Eiffel tower), it produces an output representation which greatly i

...
My submission to the ALTER Prize
by Lorxus — 2023-09-30T16:07:35.190Z
Link: https://www.docdroid.net/JI2cYIm/alter-prize-writeup-pdf

This is a linkpost for my submission to the ALTER AI Alignment Prize. As such, it's 8 and a half pages of pretty unapologetically dense math and/or philosophy the whole way through. It represents my own work, and I stand by the results and thinking in it. As such, I welcome actionable feedback, including checking of my reasoning and typography. I'm also happy to answer questions in the comments here or on Discord.

Feedback: https://www.admonymous.co/lorxus