trentmkelly/LessWrong-43k huggingface dataset:
[New] Rejected Content Section
by Ruby — 2023-05-04T01:43:19.547Z

tl:dr;

The LessWrong reviews all first time submissions from new users
In the last month, we've built a new process where we reject some of those submissions (with explanation) – you can now see what was rejected from the main site on the new Rejected Content section
New users shouldn't be very afraid to submit, the bar could well be lower than you think (and now you can check based on what we actually reject). The worst that happens now is you get some feedback from us, rather than getting downvoted by many people.

Though maybe write something shorter as your first thing. Don't write thousands of words only for us to say "this has serious problem X".

What is the Rejected Content section?

When new users submit their first post or comment, it is reviewed by a LessWrong team member before it goes live on the site. Many times, especially with an increased influx of users to LessWrong, we will sometimes reject these first posts/comments and not let them go live.

To increase transparency and accountability, we've made a new section of the site where you can view everything we've rejected, and usually with some explanation of why. This means that anyone who is motivated can both go fishing for things they think were actually good but were mistakenly classified as reject-worthy, or in general just verify that the LessWrong team is (or isn't) exercising good and fair judgment in what we're rejecting.

The Rejected Content section is part of the Moderation Log, viewable at

lesswrong.com/moderation#rejected-posts

and

lesswrong.com/moderation#rejected-comments

What to do if you disagree with a rejection decision?

Message us directly about it – maybe we didn't pay close enough attention and it'll be no big deal to change it. See the contact us page.
Mention in the monthly Open Thread.
Write a top-level post if you want to start a larger conversation about our moderation philosophy or approach.

The point of the feature is precisely so people can weigh in, so don't feel afraid to question our choices here. (This isn't a promise to change in response to feedback, but is a good amount of willingness to listen to feedback.)

Why build it?

As I wrote in a recent post of of mine about moderation, the moderation team has historically faced a tricky choice. Either we:

Approve and let userbase evaluate: When we view low quality content[1] being submitted by new users[2], we nonetheless approve i

...
How MATS addresses “mass movement building” concerns
by Ryan Kidd — 2023-05-04T00:55:26.913Z

Recently, many AI safety movement-building programs have been criticized for attempting to grow the field too rapidly and thus:

Producing more aspiring alignment researchers than there are jobs or training pipelines;
Driving the wheel of AI hype and progress by encouraging talent that ends up furthering capabilities;
Unnecessarily diluting the field’s epistemics by introducing too many naive or overly deferent viewpoints.

At MATS, we think that these are real and important concerns and support mitigating efforts. Here’s how we address them currently.

Claim 1: There are not enough jobs/funding for all alumni to get hired/otherwise contribute to alignment

How we address this:

Some of our alumni’s projects are attracting funding and hiring further researchers. Three of our alumni have started alignment teams/organizations that absorb talent (Vivek’s MIRI team, Leap Labs, Apollo Research), and more are planned (e.g., a Paris alignment hub).
With the elevated interest in AI and alignment, we expect more organizations and funders to enter the ecosystem. We believe it is important to install competent, aligned safety researchers at new organizations early, and our program is positioned to help capture and upskill interested talent.
Sometimes, it is hard to distinguish truly promising researchers in two months, hence our four-month extension program. We likely provide more benefits through accelerating researchers than can be seen in the immediate hiring of alumni.
Alumni who return to academia or industry are still a success for the program if they do more alignment-relevant work or acquire skills for later hiring into alignment roles.

Claim 2: Our program gets more people working in AI/ML who would not otherwise be doing so, and this is bad as it furthers capabilities research and AI hype

How we address this:

Considering that the median MATS scholar is a Ph.D./Masters student in ML, CS, maths, or physics and only 10% are undergrads, we believe most of our scholars would have ended up working in AI/ML regardless of their involvement with the program. In general, mentors select highly technically capable scholars who are already involved in AI/ML; others are outliers.
Our outreach and selection processes are designed to attract applicants who are motivated by reducing global catastrophic risk from AI. We principally advertise via word-of-mouth, AI safety Slack workspaces, AGI Saf

...
Moving VPS Again
by jefftk — 2023-05-04T00:30:04.556Z

After a series of technical issues at my hosting provider (downtime on
2023-03-08, 2023-04-23, 2023-05-02, 2023-05-03) I've now moved

the

somewhat

numerous

sites

I

host over to Amazon Lightsail.
It's slightly more expensive ($10/m vs $8.49/m, primarily driven by
storage) but it should be more reliable.

I've moved everything over, but let me know if you notice anything
isn't working.

I thought a bit about whether I'd like to go fully static and put this
all on Github Pages, but I enjoy being able to play around with
dynamic bits (like Shrubgrazer). And even for sites I manage that serve via
Pages I still like having preview versions on my VPS here for easy
development.
Prizes for matrix completion problems
by paulfchristiano — 2023-05-03T23:30:08.069Z

Here are two self-contained algorithmic questions that have come up in our research. We're offering a bounty of $5k for a solution to either of them—either an algorithm, or a lower bound under any hardness assumption that has appeared in the literature.

Question 1 (existence of PSD completions): given m entries of an n×n matrix, including the diagonal, can we tell in time ~O(nm) whether it has any (real, symmetric) positive semidefinite completion? Proving that this task is at least as hard as dense matrix multiplication or PSD testing would count as a resolution.

Question 2 (fast “approximate squaring”): given A∈Rn×n and a set of m=Ω(n) entries of AAT, can I find some PSD matrix that agrees with AAT in those m entries in time ~O(nm)?

We'll pay $5k for a solution to either problem. The offer is open for each problem for 3 months or until the problem gets solved (whichever happens first). Winners are welcome to publish solutions independently. Otherwise, if the result ends up being a significant part of a paper, we’ll invite them to be a coauthor.

We’ll also consider smaller prizes for partial progress, or anything that we find helpful for either solving the problem or realizing we should give up on it.

To understand the motivation for these questions, you can read our paper on Formalizing the presumption of independence and in particular Appendix D.7.2. ARC is trying to find efficient heuristic estimators as a formalization of defeasible reasoning about quantities like the variance of a neural network's output. These two questions are very closely related to one of the simplest cases where we haven't yet found any reasonable linear time heuristic estimator.

We don’t expect to receive many incorrect proposals, but if we receive more than 5 we may start applying a higher standard in order to save our time. If we can’t understand a solution quickly, we may ask you to provide more details, and if we still can’t understand it we may reject it. We expect a correct solution to be about as clear and easy to verify as a paper published at STOC.

For both problems, it’s OK if we incorrectly treat a matrix as PSD as long as all of its eigenvalues are at least −ε for a small constant ε. ~O hides polylogarithmic factors in n, ε, and the max matrix entry. Feel free to ask for other clarifications on our question on Math Overflow, on Facebook, or by email.

To submit a solution, sen...
Alignment Research @ EleutherAI
by Curtis Huebner — 2023-05-03T22:45:25.662Z
Link: https://blog.eleuther.ai/alignment-eleuther/

The past and future of AI alignment at Eleuther

Initially, EleutherAI focused mainly on supporting open source research. AI alignment was something that was acknowledged by many of the core members as important, but it was not the primary focus. We mainly had discussions about the topic in the #alignment channel and other parts of our discord while we worked on other projects.

As EAI grew, AI alignment started to get taken more seriously, especially by its core members. What started off as a single channel turned into a whole host of channels about different facets of alignment. We also hosted several reading groups related to alignment, such as the modified version of Richard Ngo’s curriculum and an interpretability reading group. Eventually alignment became the central focus for a large segment of EAIs leadership, so much so that all our previous founders went off to do full time alignment research at Conjecture and OpenAI.

Right now, the current leadership believes making progress in AI alignment is very important. The organization as a whole is involved in a mix of alignment research, interpretability work, and other projects that we find interesting.

Moving forward, EAI remains committed to facilitating and enabling open source research, and plans to ramp up its alignment and interpretability research efforts. We want to increase our understanding and control of modern ML systems and minimize existential risks posed by artificial intelligence.

Our meta-level approach to alignment

It is our impression that AI alignment is still a very pre-paradigmatic field. Progress in the field often matches the research pattern we see in the ELK report, where high level strategies are proposed, and problems or counterexamples are found. Sometimes these issues can be fixed, but oftentimes fundamental issues are identified that make an initially promising approach less interesting.

A consequence of this is that it’s difficult to commit to an object level strategy to make progress on AI alignment, and even harder to commit to any grand strategic plan to solve the problem. Instead it makes more sense to have a meta level strategy that makes us better able to leverage our unique position within the AI research ecosystem, and pivot when we get new information.

Going forward, this means we want to pursue interesting projects that meet a few general desiderata.

Our volunteers,

...
«Boundaries/Membranes» and AI safety compilation
by Chris Lakin — 2023-05-03T21:41:19.124Z

In this post I outline every post I could find that meaningfully connects the concept of «Boundaries/Membranes» (tag) with AI safety.[1] This seems to be a booming subtopic: interest has picked up substantially within the past year.

Update (2023 Dec): we're now running a workshop on this topic!

Perhaps most notably, Davidad includes the concept in his Open Agency Architecture for Safe Transformative AI alignment paradigm. For a preview of the salience of this approach, see

this comment

by Davidad (2023 Jan):

“defend the boundaries of existing sentient beings,” which is my current favourite. It’s nowhere near as ambitious or idiosyncratic as “human values”, yet nowhere near as anti-natural or buck-passing as corrigibility.

This post also compiles work from Andrew Critch, Scott Garrabrant, Mark Miller, and others. But first I will recap what «Boundaries» are:

«Boundaries» definition recap:

You can see

«Boundaries» Sequence

for a longer explanation, but I will excerpt from a more recent post by Andrew Critch,

2023 March

:

By boundaries, I just mean the approximate causal separation of regions in some kind of physical space (e.g., spacetime) or abstract space (e.g., cyberspace). Here are some examples from my

«Boundaries» Sequence

:

a cell membrane (separates the inside of a cell from the outside);
a person's skin (separates the inside of their body from the outside);
a fence around a family's yard (separates the family's place of living-together from neighbors and others);
a digital firewall around a local area network (separates the LAN and its users from the rest of the internet);
a sustained disassociation of social groups (separates the two groups from each other)
a national border (separates a state from neighboring states or international waters).

Also,

beware

:

When I say boundary, I don't just mean an arbitrary constraint or social norm.

Update: see Agent membranes and causal distance for a better exposition of the agent membranes/boundaries idea.

Posts & researchers that link «Boundaries» and AI safety

All bolding in the excerpts below is mine.

Davidad’s OAA

Saliently, Davidad uses «Boundaries» for one of the four hypotheses he outlines in

An Open Agency Architecture for Safe Transformative AI

(2022 Dec)

Deontic Sufficiency Hypothesis: There exists a human-understandable set of features of finite trajectories in su

...
What constraints does deep learning place on alignment plans?
by Garrett Baker — 2023-05-03T20:40:16.007Z

A common complaint of particularly theoretical alignment research is that it doesn't seem compatible with deep learning. For example, Tammy's QACI proposal seems pretty clearly not compatible with deep learning. More extremely is early MIRI work.

Prosaic alignment seems more clearly compatible with deep learning, and so does work that rests on interpretability.

Ideally, we'd have both an alignment solution that seems compatible with deep learning, and is at least as theoretically sound (in terms of alignment properties) as QACI (though I'm skeptical of its soundness).

Sometimes clearly defining the constraints of a problem can help with solving it. So, what are the currently known (or probable) constraints that deep learning imposes on alignment solutions (and plans)?
AGI rising: why we are in a new era of acute risk and increasing public awareness, and what to do now
by Greg C — 2023-05-03T20:26:19.479Z

[Added 13Jun: Submitted to OpenPhil AI Worldviews Contest - this pdf version most up to date]
Content note: discussion of a near-term, potentially hopeless[1] life-and-death situation that affects everyone.

Tldr: AGI is basically here. Alignment is nowhere near ready. We may only have a matter of months to get a lid on this (strictly enforced global limits to compute and data) in order to stand a strong chance of survival. This post is unapologetically alarmist because the situation is highly alarming.[2] Please help. Fill out

this form

to get involved. Here is a

list

of practical steps you can take.

We are in a new era of acute risk from AGI

Artificial General Intelligence (AGI) is now in its ascendency. GPT-4 is already

~human-level at language

and showing

sparks of AGI

. Large multimodal models – text-, image-, audio-, video-, VR/games-,

robotics

-manipulation by a single AI – will arrive very soon (from

Google DeepMind

[3]) and will be ~human-level at many things: physical as well as mental tasks; blue collar jobs in addition to white collar jobs. It’s looking highly likely that the current paradigm of AI architecture (

Foundation models

), basically just

scales

all the way to AGI. These things are “

General Cognition Engines

”[4].

All that is stopping them being even more powerful is spending on compute. Google & Microsoft are worth

$1-2T each

, and $10B can buy ~

100x

the compute used for GPT-4. Think about this: it means we are already well into

hardware overhang

territory[5].

Here is a warning written two months ago by people working at applied AI Alignment lab

Conjecture

: “we are now in the end-game for AGI, and we (humans) are losing”. Things are now worse. It’s looking like GPT-4 will be used to

meaningfully

speed up AI research

, finding more efficient architectures and therefore reducing the cost of training more sophisticated models[6].

And then there is the

reckless fervour

[7] of plugin development to make proto-AGI systems more capable and agent-like to contend with. In very short succession from GPT-4, OpenAI announced the

ChatGPT plugin store

, and there has been

great enthusiasm for AutoGPT

. Adding Planners to LLMs (known as

LLM+P

) seems like a good recipe for turning them into agents. One way of looking at this is that the planners and plugins act as the System 2 to the u...
Formalizing the "AI x-risk is unlikely because it is ridiculous" argument
by Christopher King — 2023-05-03T18:56:25.834Z

There is a lot of good writing on technical arguments against AI x-risk (such as Where I agree and disagree with Eliezer (which mostly argues for more uncertainty) and others). However, in the wider world the most popular argument is more of the form "it is ridiculous" or "it is sci-fi" or some sort of other gut feeling. In particular, I think this is the only way people achieve extremely low credence on AI doom (low enough that they worry about other disasters instead).

Although this seems like a fallacy, in this post I will attempt to formalize this argument. Not only is it good, but I think it turns out to be extremely strong!

In my judgement, I still find the arguments for x-risk stronger or at least balanced with the "it is ridiculous" argument, but it still deserves serious study. In particular, I think analyzing and critiquing it should become a part of the AI public discourse. For example, I think there are flaws in the argument that, when revealed, would cause people to become more worried about AI x-risk. I do not quite know what these flaws are yet. In any case, I hope this post will allow us to start studying the argument.

The argument is actually a bunch of separate arguments that tend to be lumped together into one "it is ridiculous" argument.

For the purposes of this article, Bob is skeptical of AI x-risk and Alice argues in favor of it.

Existential risk would stop a 12,000 year trend

Forgetting all of human history, Bob first assumes our priors for the long term future are very much human agnostic. The vast majority of outcomes have no humans, but are just arbitrary arrangements of matter (paperclips, diamonds, completely random, etc...). So our argument will need to build a case for the long term future actually being good for humans, despite this prior.

Next, Bob takes into account human history. The total value of the stuff humans consume tends to go up. In particular, it seems to follow a power law, which is a straight line on this log-log graph.

Graph from Modeling the Human Trajectory

Which means Bob has the gods of straight lines on his side!

This should result in a massive update of the priors towards "the future will have lots of things that humans like".

Most people of course don't track economics or think about power-laws, but they have an intuitive sense of human progress. This progress is pretty robust to a wide variety of disasters, but no...
Progress links and tweets, 2023-05-03
by jasoncrawford — 2023-05-03T16:23:12.241Z
Link: https://rootsofprogress.org/links-and-tweets-2023-05-03

The Progress Forum

Reflections from seeing the Starship launch firsthand
When America received a message from the future: the 1939 World’s Fair

Announcements

Speculative Technologies recruiting “field strategists.” “Come design a research program to unlock powerful materials or manufacturing technology that is too research-y to be a startup but too coordination- or engineering- heavy for a single academic lab” (via @Spec__Tech)
Sam Arbseman is writing a new book: The Magic of Code: How Digital Language Built Our Modern World—and Shapes Our Future
Nuclear Now: Time to Look Again, a new documentary from Oliver Stone

Links

“Tales of Edison’s Lab,” the first episode of Eric Gilliam’s new podcast, with a “format similar to Hardcore History” (via @eric_is_weird)
We should have air sanitation for disease, like water sanitation (by @finmoorhouse)
Pro-nuclear opinion piece in NYT (!) on the waste issue (via @atrembath). “Every cask of spent nuclear fuel represents about 2.2 million tons of carbon that weren’t emitted”
There is no climate tipping point (by @wang_seaver)
“Capitalism has plenty of agency relationships… but so do all other systems”

AI

AI as a complex adaptive system and implications for risk (via @JacobSteinhardt; see also my lenses on AI risk)
Scott Aaronson podcast interview on AI safety (or just see this key excerpt)
Chad Jones paper on AI, economic growth, and existential risk (PDF) (via @ChadJonesEcon, see my thread)
Movie trailer made with text-to-video AI
A programming language for generative protein design using AI (via @BrianHie)

Five possible scenarios for how AI will evolve in the future

ChatGPT vs. doctors answering medical questions on Reddit (via @TamarHaspel)

Queries

Why is it that in the past it was not uncommon for people to live in hotels? (turns out the answer is probably building codes, zoning laws, rent control, urban renewal, etc.)
What are the best new space tech companies/projects in the making? (@allisondman)
Who should Dwarkesh have on the podcast to discuss human evolution?
Anybody know much about 17th–18th century land tenancy rules? (@antonhowes)
Explanation of the differences between LLMs and chess AIs / AlphaFold?
An overview on the impact of modern tech on archaeology?
Reference for the idea that CASP disabused researchers of the notion that protein folding was near-solved? (@michael_nielsen)

Quotes

“There is no great invention… which has not been

...
Slowing AI: Crunch time
by Zach Stein-Perlman — 2023-05-03T15:00:12.495Z

It is often asserted that time will be more valuable and higher leverage in "crunch time," shortly before critical models are deployed.

This brief post provides a starting point for consideration of three questions:

What is more time (before dangerous AI) good for?
What is different near the end-- what makes actions higher (or lower) leverage, or makes adding time more (or less) valuable?
How does buying time now trade off with buying time later?

What is more time good for?

Technical AI safety research
AI strategy and governance research
Governance happening

Governments responding to AI is not obviously good but seems positive in expectation, both absolutely and on the margin

Paying the alignment tax (if the critical model that needs to be aligned exists)
Relevant actors' attitudes become closer to truth and their actions become closer to their beliefs

Relevant actors here include labs, governments, and the ML community
Related: more time to deal with emerging capabilities

Field-building, community-building, and resource/influence-gaining

What is different later?

AI models are more powerful, boosting technical AI safety research
There is more strategic clarity, especially insofar as you recognize that you're near the end

There is more clarity about what critical models will look like, helping prioritization in technical AI safety research
Maybe strategy and governance interventions and prioritization improve

Attitudes are different; in particular, actors are more aware of AI and AI risk and are trying to take action

E.g., governments are trying to do stuff
The space is more crowded; more total influence is exerted on AI

Labs can actually pay the alignment tax
The AI safety field is larger (and maybe more influential)
Maybe the world looks much crazier and changes much faster
Maybe new windows of opportunity open, in terms of actors' actions

Many differences depend on whether actors realize that we're near the end.

Roughly, crunch time is more valuable and higher leverage than the time before it, but this depends on the particular goal at issue and the exact difference is unclear even for particular goals.

How does buying time now trade off with buying time later?

Largely it doesn't. For example, if there existed policy regimes that slow dangerous AI, that would mostly make similar policy regimes more likely to exist in the future.

In some cases, you can choose to prioritize slowing now or...