trentmkelly/LessWrong-43k huggingface dataset:
Research agenda: Formalizing abstractions of computations
by Erik Jenner — 2023-02-02T04:29:06.568Z

Big thanks to Leon Lang, Jérémy Scheurer, Adam Gleave, and Shoshannah Tekofsky for their feedback on a draft of this post, to Euan McLean (via FAR AI) for his feedback and a lot of help with editing, and to everyone else who discussed this agenda with me, in particular Johannes Treutlein for frequent research check-ins!

Summary

My current agenda is to develop a formal framework for thinking about abstractions of computations. These abstractions are ways to partially describe the “algorithm” a neural network or other computation is using, while throwing away irrelevant details.
Ideally, this framework would tell us 1) all possible abstractions of a given computation, and 2) which of these are most useful (for a specific purpose, such as detecting deception). “Useful” doesn’t necessarily mean “easily human-understandable”—I see that as an open question.
I anticipate the main applications to alignment to be

automating interpretability

or

mechanistic anomaly detection

. There are also potential connections to other alignment topics, such as

natural abstractions

or

defining terms like “search process”

.
This agenda is at an early stage (I have been thinking about it for ~2 months, and about related topics for another ~2 months before that). So feedback now could change my future direction.
I also list a few potential projects that seem self-contained. If you’re interested in working on any of those, or collaborating in some other way, please get in touch!

I encourage you to skip around and/or only read parts of the post. Here’s an overview:

Introduction

and

Q&A

mostly talk about motivation and connections to alignment..

What are abstractions of computations?

discusses my current guess as to what the framework should look like.
There’s a list of

Some potential projects

.

Appendix: Related work

gives a quick overview of relevant work in academia, and the relation between this agenda and other alignment research.

This post doesn’t contain any actual theorems or experiments, so if you’re only interested in that, you can stop reading.

Introduction

Humans can’t just look at the weights of a neural network and tell what it’s doing. There are at least two reasons for this:

Neural network weights aren’t a format we’re great at thinking about.
Neural networks are often huge.

The second point would likely apply to any system that does wel...
Progress links and tweets, 2023-02-01
by jasoncrawford — 2023-02-02T02:25:21.021Z
Link: https://rootsofprogress.org/links-and-tweets-2023-02-01

The Progress Forum

Eli Dourado AMA (lots of great questions and detailed answers!)
When did England start seeing itself as a commercial nation? (Anton Howes)
The progress movement could use some explicit cause areas

Announcements

New book on factors affecting medical progress (via @markkhurana)
ARIA, the ARPA of the UK, now legally exists and has a budget (via @logangraham)

Links

Medieval peasants likely did not work fewer hours than we do (via @jmhorp)
“Do-it-yourself crafts only exist when you no longer have to do everything yourself”
Oklo has submitted a plan for a nuclear fuel recycling facility (via @oklo)
Our World in Data topic page on Democracy (via @bbherre)
“The whole saga feels to me like it’s part of a climate politics of sacrifice”

Quotes

The amazing progress in Asian agriculture in the late 20th century
A brilliant passage from John McPhee’s The Control of Nature

Retweets

A lot of public health pronouncements don’t take into account that people enjoy things
If you want to see a progress studies community on Hive.one, nominate it here
A 1931 article on the potential for rocket-based passenger travel (@1517fund)
Retrospective on the AI Safety Field Building Hub
by Vael Gates — 2023-02-02T02:06:52.722Z

Overview

In late July I posted

Announcing the AI Safety Field Building Hub, a new effort to provide AISFB projects, mentorship, and funding

. This new organization, funded by FTXFF, set out to work on AI safety outreach projects aimed at AI researchers, with funding to mentor new people and the flexibility to take on projects suggested by the community. As we’re now nearing the end of the FTXFF funding, the AISFB Hub is finishing up all existing projects and closing down. It’s been an exciting six months! Here’s a retrospective of all the AISFB Hub has been up to, as well as potential future work we’d get up to if we secure new funding.

What AISFB Hub has done

There are three major categories that the work has fallen into: working with my interview series, completing the pilot of an outreach survey, and everything else.

Interview Series

In Februrary-March 2022, I conducted 97 interviews with AI researchers (who had papers accepted to NeurIPS / ICML 2021). Interviewees were asked about their perceptions of artificial intelligence now and in the future, with special focus on risks from advanced AI systems. I presented the alignment problem and the idea of instrumental incentives, and asked them questions like whether they thought we’d ever achieve AGI, and if they’d be interested in working on AI alignment. I

released 11 transcripts of those interviews

at the time, posted a

talk on the preliminary results

, and promised future analysis. Now, for what the AISFB Hub did!

Created a website to display all of the below:

https://ai-risk-discussions.org/interviews

(EAF post)
Anonymized and released 72 more transcripts (i.e. everyone who gave permission), which brings us up to 83/97

transcripts available

.
Completed a

quantitative analysis

of these

interviews

, especially focused on how people responded to the core AI safety questions (EAF post), with a separate writeup on predicting researchers' interest in alignment (EAF post).
Created an interactive

walkthrough of common perspectives

in the interviews, as well as counterarguments to some of the common objections to AI safety arguments (EAF post).

Outreach Survey

In this project, we sent AI researchers (paper accepted at NeurIPS / ICML / ICLR 2021) a 5-hour survey of AI safety readings to engage critically with and answer questions about. 28 researchers completed the ...
How to export Android Chrome tabs to an HTML file in Linux (as of February 2023)
by Adam Scherlis — 2023-02-02T02:03:55.934Z
Link: https://adam.scherlis.com/2023/02/01/how-to-export-android-chrome-tabs-to-an-html-file-in-linux-as-of-february-2023/

Let's say you have a few million tabs open in your mobile Chrome browser, because you never close anything, but now your browser is getting slow and laggy. You want to stick the URLs of those tabs somewhere for safekeeping so that you can close them all.

There's a lot of advice on doing this on the Internet, most of which doesn't work.

Here's a method that does work. It's a bit of a hack, but gives good results:

Enable developer tools on your Android phone: go to Settings -> About phone, scroll down to "Build number", and tap it repeatedly until it tells you you're a developer. (Seriously.)
Enable USB debugging on your phone: go to Settings -> System -> Developer options and make sure the "USB debugging" slider is enabled.
Install Android Debug Tools on your Linux desktop: run these commands [h/t this StackOverflow answer]:
sudo apt install android-tools-adb android-tools-fastboot
adb device
Open USB debugging page in desktop Chrome: go to chrome://inspect/#devices [h/t this Android StackExchange answer]
Right-click -> Inspect, or press F12
Go to the Sources tab. Edit inspect.js (top/inspect/inspect.js) and remove this URL-truncating code [h/t this comment on that answer]:

if (text.length > 100) {
text = text.substring(0, 100) + '\u2026';
}

Do not reload! Leave that page open.
Tether your phone by USB cable. If a phone pop-up asks you to authorize remote debugging, say yes.
You should now see a list of page titles and URLs on the USB debugging page in desktop Chrome.
Inspect the page again.
Under the Elements tab, navigate to <body>, then <div id="container">, then <div id="content">, then <div id="devices" class="selected">. Right click that last one and Copy -> Copy element.
Now you have all your tabs on your clipboard... all on the same line. This will crash a lot of text editors if you try to paste it normally. So we'll use xclip instead.
If you don't have it, sudo apt install xclip
Run xclip -selection c -o > my_tabs_file to put that huge HTML element in a file.
This'll be easier with linebreaks, so run cat my_tabs_file | sed "s/<div/\n<div/g" > my_better_tabs_file
Edit the paths at the beginning as appropriate, then run this Python script:

import re

# Put the actual path to the input file here:
INPUT_FILE = '/home/blah/blah/my_better_tabs_file'
# Put the desired path to the output file here:
OUTPUT_FILE = '/

...
Hacked Account Spam
by jefftk — 2023-02-02T01:50:01.738Z

Whenever I post something publicly on Facebook mentioning hacking I
get a ton of spammy comments:

This first happened with my not-actually-deepfake phishing post,
and then again on my posts on not getting hacked and how likely Google
lockouts are.

It sounds like there's a business in helping (or scamming) people
who've had their Facebook/Instagram accounts hacked?

I've reported 42 of the comments as spam, and deleted dozens more
(reporting spam takes several more clicks than deleting, and then you
need to delete the comment anyway). Facebook doesn't seem to be
getting any better at blocking them.

One of the more frustrating things about this, however, is that
Facebook does seem to recognize these as spammy, and so doesn't notify
me about them. Which is not what I want: it means my friends still
see them on my posts, and I'm reliant on them to let
me know
about the
comments so I can delete.

Comment via: facebook, mastodon
A simple technique to reduce negative rumination
by cranberry_bear — 2023-02-02T01:33:11.553Z

I have faced challenges with negative rumination and anxiety, especially when I am trying to relax or my mind is idle. However, I have recently discovered a simple and effective technique to overcome this.

I practice enumerating as many positive future events as I can, starting with the most probable and desirable ones, calculated by multiplying their probability with the level of happiness they bring, adjusted for their proximity in time.

While the focus is on positive thoughts, I find that this slightly quantitative approach helps me direct my thoughts more effectively. It pushes me to consider realistic and desirable events, countering the natural tendency to dwell on negative scenarios. Additionally, by visualizing medium-probability positive events, I become more aware of opportunities to increase their chances in my daily life, similar to the popular concept of "manifesting."

This technique not only provides a positive outlook but is also additionally rewarding because it requires me to think creatively and explore what I appreciate in life, making it feel like a game.

I also extend this technique by dividing positive events into 3-5 broad categories and brainstorming each one separately. This ensures a more comprehensive outlook and prevents me from dwelling on low probability scenarios or getting lost in unrealistic fantasies.

I hope this helps someone!
A Brief Overview of AI Safety/Alignment Orgs, Fields, Researchers, and Resources for ML Researchers
by Austin Witte — 2023-02-02T01:02:00.980Z

Crossposted to EA Forum [Link]

TLDR: I’ve written an overview of the AI safety space, tagged by keywords and subject/field references (

short version

,

long version

). The aim is to allow existing ML researchers to quickly gauge interest in the subject based on their existing subfield skills and interests!

Overview

When ML researchers first hear about AI alignment and are interested in learning more, they often wonder how their existing skills and interests could fit within the research already taking place. With expertise in specific subfields, and momentum in their careers and projects, interested researchers are curious about the overall AI alignment space and what research projects they could invest in relatively easily. As one step towards addressing this, the AISFB Hub commissioned a collection of resources that could be provided to technical researchers trying to quickly assess what areas seem like promising candidates for them to investigate further: (

Short version

,

Long version

).

These documents list a subset of the various organizations and researchers involved in the AI safety space, along with major papers. To allow quick scanning, I focused on keywords and subject/field references. As this was targeted at researchers who already have experience with ML, the summaries provided are primarily meant to allow the reader to quickly gauge interest in the subject based on their existing subfield skills and interests.

This work contains papers and posts up through the end of 2022. Please contact me or Vael Gates if you would be willing to keep it updated!

Details and Disclaimers

As an attempt at collecting alignment research, I generally see this post as complementary to

Larsen’s post

on technical alignment. Neither entirely includes the other, with Larsen’s post having a slightly stronger and more curated focus on fields and projects, while this collection emphasized providing general resources and example areas of work for new researchers.

Overall, this list took a little over 40 hours of work to put together. It primarily included looking into and summarizing the work of organizations I knew about. This was supplemented by investigating a list of researchers provided by the AISFB Hub, along with work referenced by various other posts in LessWrong/EA forums, and by the organizations and researchers from their websites ...
Interviews with 97 AI Researchers: Quantitative Analysis
by Maheen Shermohammed — 2023-02-02T01:01:32.087Z

TLDR: Last year, Vael Gates interviewed 97 AI researchers about their perceptions on the future of AI, focusing on risks from advanced AI. Among other questions, researchers were asked about the alignment problem, the problem of instrumental incentives, and their interest in AI alignment research. Following up after 5-6 months, 51% reported the interview had a lasting effect on their beliefs. Our new report analyzes these interviews in depth. We describe our primary results and some implications for field-building below. Check out the

full report

(interactive graph

version

), a complementary

writeup

describing whether we can predict a researcher’s interest in alignment, and our results below!

[Link to post on the EA Forum]

Overview

This

report

(interactive graph

version

) is a quantitative analysis of 97 interviews conducted in Feb-March 2022 with machine learning researchers, who were asked about their perceptions of artificial intelligence (AI) now and in the future, with particular focus on risks from advanced AI systems. Of the interviewees, 92 were selected from NeurIPS or ICML 2021 submissions, and 5 were informally recommended experts. For each interview, a transcript was generated, and common responses were identified and tagged to support quantitative analysis. The

transcripts

, as well as a qualitative

walkthrough of common perspectives

, are available at

Interviews

.

Several core questions were asked in these interviews:

When advanced AI (~AGI) would be developed (note that this term was imprecisely defined in the interviews)
A probe about the alignment problem: “What do you think of the argument ‘highly intelligent systems will fail to optimize exactly what their designers intended them to, and this is dangerous’?”
A probe about instrumental incentives: “What do you think about the argument: ‘highly intelligent systems will have an incentive to behave in ways to ensure that they are not shut off or limited in pursuing their goals, and this is dangerous’?”
Whether interviewees were interested in working on AI alignment, and why or why not
Whether interviewees had heard of AI safety or AI alignment

Findings Summary

Some key findings from our primary questions of interest:

Most participants (75%), at some point in the conversation, said that they thought humanity would achieve advanced AI (imprecisely labeled “AGI” for t

...
“AI Risk Discussions” website: Exploring interviews from 97 AI Researchers
by Vael Gates — 2023-02-02T01:00:01.067Z

Tldr: Check out the new

AI Risk Discussions website

! It contains 83 transcripts of AI researcher

interviews

, and a

quantitative analysis

(

full report

) of these interviews. We also built an

interactive walkthrough of perspectives and arguments

. We would be grateful for any feedback!

In February-March 2022, Vael Gates conducted a series of 97 interviews with AI researchers about their perceptions of the future of AI, focusing on their responses to arguments for potential risk from advanced systems. Eleven transcripts from those interviews

were released

, along with a

talk

summarizing the findings, with the promise of further analysis of the results.

We have now finished the analysis, and created a

website aimed at a technical audience to explore the results

!

Lukas Trötzmüller has written an

interactive walkthrough

of the common perspectives that interviewees had, along with counterarguments describing why we might still be concerned about risk from advanced AI (

introduction to this walkthrough

). Maheen Shermohammed has conducted a

quantitative analysis

(

full report

) of all the interview transcripts, which were laboriously tagged by Zi Cheng (Sam) Huang. An army of people helped

anonymize a new set of transcripts

that researchers gave permission to be publicly released. We’ve also constructed a new

resources

page (

based

on

what materials ML researchers find compelling

) and “

what can I do

?” page for further investigation. Michael Keenan led the effort to put the whole website together.

In addition to being a compilation of the research outputs of this interview series, we want this to be a website that can be helpfully forwarded to technical AI researchers, and would be grateful for any feedback that can improve it. Please feel free to leave comments on the

interactive walkthrough post

,

the quantitative analysis post

, or message Vael with any notes you have. Thanks everyone!
Predicting researcher interest in AI alignment
by Vael Gates — 2023-02-02T00:58:01.120Z
Link: https://docs.google.com/document/d/1NxAYgxlTGd6kxUhxLyb_4thrXw2eFcNpdL2nYzJ3k_E/edit?usp=sharing

TLDR: As someone who talks to researchers about AI alignment, I’m curious if there are ways to predict how well a conversation might go. For example, does demographic information help predict whether a conversation will have lasting effects months later? To answer this, I sifted through two sets of results, focusing on the newly-released quantitative analysis of 97 AI researcher interviews. It was a messy process and I’ve given up on releasing something that shows my work. However, jump down to the "Overall Takeaways" section to read what updates I’m making about predicting research interest in AI alignment.

(Note: I highly recommend you read the version of this post where the table formatting looks reasonable.)

Introduction

In February-March 2022, I conducted 40-60 min

interviews

with 97 AI researchers about their perceptions of AI and the future of the field. The core of these conversations was discussing potential risks from advanced AI systems – I presented some arguments for why we might be concerned, then we discussed what they thought and why.

Maheen Shermohammed and I recently released a

report

(interactive graph

version

) analyzing these interviews. It’s an enormous report, and the main findings are described in

this summary post

.

However, the summary post doesn’t discuss the interaction effects between all of the variables we investigated. By understanding these interactions, we can answer a question I’m very interested in:

If you talk to an AI researcher, and you know something about them – demographics from their website, or information about their knowledge of AI safety, sympathy towards AI safety, or their timelines to AGI – can you predict anything about their future sympathy or whether the conversation will be useful to them?

All the relevant information is in our

report

. However, it is admirably massive! Ideally there would be a separate document just on the important interaction effects. To that end, I’ve spent more than 40 hours working through the interaction effect analyses, checking my independent understanding of the graphs against Maheen’s careful observations, incorporating some additional results from an

EA Forum post

that analyzed

AI Impacts 2022 survey

data, lovingly creating this massive document that lays out all my reasoning as I squint at dozens of graphs and correlations…

…and then I...
Focus on the places where you feel shocked everyone's dropping the ball
by So8res — 2023-02-02T00:27:55.687Z

Writing down something I’ve found myself repeating in different conversations:

If you're looking for ways to help with the whole “

the world looks pretty doomed

” business, here's my advice: look around for places where we're all being total idiots.

Look for places where everyone's fretting about a problem that some part of you thinks it could obviously just solve.

Look around for places where something seems incompetently run, or hopelessly inept, and where some part of you thinks you can do better.

Then do it better.

For a concrete example, consider Devansh. Devansh came to me last year and said something to the effect of, “Hey, wait, it sounds like you think Eliezer does a sort of alignment-idea-generation that nobody else does, and he's limited here by his unusually low stamina, but I can think of a bunch of medical tests that you haven't run, are you an idiot or something?" And I was like, "Yes, definitely, please run them, do you need money".

I'm not particularly hopeful there, but hell, it’s worth a shot! And, importantly, this is the sort of attitude that can lead people to actually trying things at all, rather than assuming that we live in a more

adequate world

where all the (seemingly) dumb obvious ideas have already been tried.

Or, this is basically my model of how Paul Christiano manages to have a research agenda that seems at least internally coherent to me. From my perspective, he's like, "I dunno, man, I'm not sure I can solve this, but I also think it's not clear I can't, and there's a bunch of obvious stuff to try, that nobody else is even really looking at, so I'm trying it". That's the sort of orientation to the world that I think can be productive.

Or the shard theory folks. I think their idea is

basically unworkable

, but I appreciate the mindset they are applying to the alignment problem: something like, "Wait, aren't y'all being idiots, it seems to me like I can just do X and then the thing will be aligned".

I don't think we'll be saved by the shard theory folk; not everyone audaciously trying to save the world will succeed. But if someone does save us, I think there’s a good chance that they’ll go through similar “What the hell, are you all idiots?” phases, where they autonomously pursue a path that strikes them as obviously egregiously neglected, to see if it bears fruit. (Regardless of what I think.)

Contrast ...