trentmkelly/LessWrong-43k huggingface dataset:
[Linkpost] "Blueprint for an AI Bill of Rights" - Office of Science and Technology Policy, USA (2022) 

by T431 — 2022-10-05T16:42:37.471Z
Link: https://www.whitehouse.gov/ostp/ai-bill-of-rights/

The PDF can be found here: https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf

Some quick notes regarding this linkpost:

I have not seen this resource posted on LessWrong as a linkpost, but should it exist please comment a link and I will delete this post.
I have not read the report in full, but have read its introductory sections and the announcement post on the White House's website.

My quick take:

The report does mention "Explainable AI" (e.g., pp. 22; I have not read the sections where this phrase is mentioned, but from a brief scan, the topic of interpretability / explainable AI seems only to have been covered at surface level), but falls short of discussing the catastrophic risks posed by AI systems. As such, I believe the recommendations and design principles this report puts forward also fall short of addressing especially critical issues in AI progress, safety, and alignment. Holistically, the report seems more geared towards the "lesser" societal issues in AI use and development, which makes some sense given the laggard pace of government systems in tackling issues posed by emerging technologies. While I'm not epistemically confident in the idea that governments and regulatory bodies can significantly reduce the risk of an AI catastrophe, I hope that the United States proposes regulation and conducts work in areas slightly more topical to reducing catastrophic outcomes involving AI.

Summary of the press release

Many technologies can or do pose a threat to democracy. There are plentiful cases where these technological "tools" limit their users more than help them. Examples include problems with the use of technology in patient care, algorithmic bias in credit and hiring, and widespread breaches of user data privacy. Of course, automation is, generally speaking, helpful (e.g., agricultural production, severe weather prediction, disease detection) but we need to ensure that "progress must not come at the price of civil rights...".

"The Blueprint for an AI Bill of Rights is a guide for a society that protects all people from these threats"

The Office for Science and Technology proposes 5 design principles for minimizing harm from automated systems:

Safe and Effective Systems: "You should be protected from unsafe or ineffective systems."
Algorithmic Discrimination Protections:

...
Paper: Discovering novel algorithms with AlphaTensor [Deepmind]
by LawrenceC — 2022-10-05T16:20:11.984Z
Link: https://www.deepmind.com/blog/discovering-novel-algorithms-with-alphatensor

The authors apply an AlphaZero-like algorithm to discover new matrix multiplication algorithms. They do this by turning matrix multiplication into a one-player game, where the state represents how far from correct the current output is, moves are algorithmic instructions, and the reward is -1 per step (plus a terminal reward of -rank(final state), if the final state is not a zero tensor). On small matrices, they find that AlphaTensor can discover algorithms that use fewer scalar multiplications than the best known human-designed matrix multiplication algorithms. They apply this to find hardware-specific matmuls (by adding an additional reward equal to -time to the terminal state) that have a 10-20% larger speedup than Strassen's algorithm on NVIDIA V100s and TPU V2s (saving 4%/7.5% wall clock time).

Paper abstract:

Improving the efficiency of algorithms for fundamental computations can have a widespread impact, as it can affect the overall speed of a large amount of computations. Matrix multiplication is one such primitive task, occurring in many systems—from neural networks to scientific computing routines. The automatic discovery of algorithms using machine learning offers the prospect of reaching beyond human intuition and outperforming the current best human-designed algorithms. However, automating the algorithm discovery procedure is intricate, as the space of possible algorithms is enormous. Here we report a deep reinforcement learning approach based on AlphaZero for discovering efficient and provably correct algorithms for the multiplication of arbitrary matrices. Our agent, AlphaTensor, is trained to play a single-player game where the objective is finding tensor decompositions within a finite factor space. AlphaTensor discovered algorithms that outperform the state-of-the-art complexity for many matrix sizes. Particularly relevant is the case of 4 × 4 matrices in a finite field, where AlphaTensor’s algorithm improves on Strassen’s two-level algorithm for the first time, to our knowledge, since its discovery 50 years ago. We further showcase the flexibility of AlphaTensor through different use-cases: algorithms with state-of-the-art complexity for structured matrix multiplication and improved practical efficiency by optimizing matrix multiplication for runtime on specific hardware. Our results highlight AlphaTensor’s ability to accelerate the process

...
Reflection Mechanisms as an Alignment target: A follow-up survey
by Marius Hobbhahn — 2022-10-05T14:03:19.923Z

This is the second of three posts (

part I

) about surveying moral sentiments related to AI alignment. This work was done by Marius Hobbhahn and Eric Landgrebe under the supervision of Beth Barnes as part of the AI safety camp 2022.

TL;DR: We find that the results of our first study, i.e. that humans tend to agree with conflict resolution mechanisms hold under different wordings but are weakened in adversarial scenarios (where we actively try to elicit less agreement). Furthermore, we find that people tend to agree much less with a mechanism when the decision-maker is a smart benevolent AI rather than a smart benevolent human. A positive interpretation of these findings is that humans are fine with giving power to a conflict resolution mechanism as long as humans are ultimately in control.

Introduction

In the

first post

, we surveyed 1000 US respondents about their moral beliefs, the conditions under which they would change their moral beliefs and how they felt about mechanisms to resolve moral disagreements such as democracy or debate.

Our main findings of the first survey were

Unsurprisingly, people have very different moral beliefs, e.g. on the morality of abortions, immigration or eating meat.
They very rarely report to changing or wanting to change these beliefs, e.g. most participants (>80%) report having not changed them in the last 10 years and do not expect to change them in the next 10 years.
However, they mostly think that mechanisms to resolve moral disagreements, such as democracy or debate, are good even when they disagree with the outcome. In other words, people are willing to accept outcomes that are different from their beliefs if they trust the process by which it was derived.

We think this finding has some implications for AI alignment. Most importantly, we think that the alignment target of an AI should be a mechanism to resolve moral conflicts such as democracy or debate rather than a set of individual beliefs, e.g. about abortion or veganism. This is more likely to lead to decisions that can be accepted by most people and prevent a race from “aligning an AI to your personal beliefs at the cost of other people’s beliefs”.

We think the main weakness of our first result was that it could have been a result of confounders or randomness. We asked people for their opinions on multiple scenarios, ...
Charitable Reads of Anti-AGI-X-Risk Arguments, Part 1
by sstich — 2022-10-05T05:03:43.262Z

Prompted by the AI Worldview Prize, I want to try to understand the arguments against existential misalignment risk with as open a mind as possible. I decided the best way to start would be to summarize anti-x-risk arguments, and evaluate them charitably. I’m starting with what I think is the most accessible "folk argument," which is simply that:

The Future is Unpredictable

The short form of this argument is:

Humans are really bad at predicting the future 10+ years out.
We’re particularly bad at predicting specific future scenarios; we’re better at making general comments. (Specific scenario: “A humanoid robot will do your laundry.” General comment: “Robots will do more tasks in the future.”)
Therefore we should assign very low probability to any specific scenario that we can imagine about the future, including AGI x-risk scenarios like paperclip maximization. The unlikelihood of these scenarios makes misalignment x-risk unlikely.

I find the premises of this argument more convincing than the conclusion. I think it’s true that humans are bad at predicting the long-range future, and that we’re especially bad at predicting specific scenarios. As survey articles, I’ll call attention to Luke Muellhauser’s explanation of why it’s hard to score historical attempts at long-range predictions, Holden Karnofsky’s summary of a study on the long-range forecasts of “Big Three” science fiction authors, and Dan Luu’s informal evaluation of futurist predictions. Luu is pessimistic about our ability to predict the long-range future, and grades previous forecasts as very inaccurate; Karnofsky is optimistic, and grades prior forecasts as reasonably accurate. I recommend that interested readers look at the raw files of predictions and judge accuracy for themselves; both Luu and Karnofsky provide sources.

I think it’s useful to make a distinction between “general comments” about the future and “specific scenarios.” Many of the predictions considered by Karnofsky and Luu are general comments about the future, such as the idea that computers will foster human creativity or that we’ll think more about waste and pollution in 2019 than we did in 1983 (both from Asimov, in the Karnofsky study). I think these are akin to present day comments like “artificial intelligence will have important economic consequences” or “we’ll care more about AGI alignment in 2058 than we do in 2022”, which may be...
Sleep Training
by jefftk — 2022-10-05T02:10:03.639Z

Let's say you're a baby. You reach the end of a sleep cycle and become
partially alert. What do you do? Ideally, if nothing were wrong you
would settle back in for another sleep cycle. Chances are, however,
you don't know how to do this yet, so you wake up more fully and start
to cry. This seems to reliably trigger some combination of cuddles,
shushing, motion, and/or nursing—comforted, you fall back
asleep. While this is a fine outcome for you, your parents would
probably be

doing a lot
better if you didn't rely on them so heavily at night.

With a very little baby this is unavoidable, though it can be partially
automated, but as babies get older they need to learn how to
handle this themselves. Sleep training is a collection of strategies
for teaching babies (a) the skill of falling asleep on their own
("self soothing") and (b) when they should apply this new skill.

One of the things that makes discussion of sleep training messy is
that people often aren't clear on which of (a) or (b) they're talking
about. For example, a strategy of putting your kid down when they're
very drowsy but not quite asleep yet is about teaching (a), while
applying consistent patterns about when or how often you'll feed or
settle them is about (b). If you think someone is trying to do (a)
when they're actually doing (b), their approach can often sound
excessive, uncaring, or unfair.

It can also be hard to tell which of (a) or (b) your child needs. A
child who can self-soothe but prefers to be cuddled doesn't appear
that different from one who doesn't know how do it yet. Gradually
providing less and less comfort can work well in these cases, because
it's a good way to teach (a) and a so-so way of teaching (b).
Alternatively, cry-it-out is a decent option for (b) and can also be a
way of teaching (a) if other options haven't worked.

With our youngest we found that tapering with the Snoo taught (a)
reasonably well: while there was some crying she mostly was able to
gradually build up the skill. With our oldest, pre-Snoo, we tried
various methods of trying to teach (a),
but it wasn't until we switched to
cry-it-out that she really mastered the skill, and started taking
longer naps.

Our youngest still tests the boundaries of (b) sometimes. For example,
two months ago (age 13m) we were on vacation (28 people in a 5br).
She quickly figured out that every ti...
Looping
by Jarred Filmer — 2022-10-05T01:47:59.257Z

Remember to remember...

Part I

Tell me if this sounds familiar...

You're walking to work one bright and caffeinated morning, and you find yourself rather engaged in a shiny train of thought. It feels novel, and important, and relevant to what matters. One might even go so far as to say engrossing.

Maybe it's a pattern in yourself or your organisation you've just seen has been playing out over years. Maybe a fresh philosophical perspective causing a cascade of re-framings on some important ideas. Or maybe, just maybe, in these crowds, it's a train of thought on how everything could be better. For you, for those you love, for everyone.

You feel the urge to write it down.

But hold on, what's this?

Curious, there's already a note in your phone, with the exact title you were about to use... from a year ago! It seems you've boarded on this joyride before, but ultimately failed to seize control and direct it anywhere useful.

If you're not running late to work, or distracted by a notification maybe you think to yourself... Man, I really need to make sure this time I actually explore this, it could have a big impact on how I want to spend my time. In fact... maybe I need some way to make sure I fully explore important thoughts I have in general.

And this thought, I should really make some space to come back to thoughts that felt important but I didn't have time to explore... feels like an important notion in its own right actually. You go to record that observation, that it's concerningly easy to forget to follow up even on thoughts that feel vitally important.

AND YOU FIND YOU WROTE THAT DOWN 2 YEARS AGO, AND YOU STILL HAVEN'T DONE ANYTHING ABOUT IT.

AND YOU THINK GOD, HOW MANY TIMES HAVE I CHANCED ON THOUGHTS I FELT COULD REALLY MATTER, AND DONE NOTHING WITH THEM?

And then, if you still haven't been interrupted, maybe you have the space to think:

OKAY.
CLEARLY, SOMETHING IS AMISS.
I'M GOING TO MAKE SPACE RIGHT NOW.
FOR THE NEXT HOUR I'M GOING TO LIST EVERYTHING I GOT EXCITED ABOUT BUT FORGOT TO EXPLORE.
AND WHILE I REMEMBER, I'M GOING TO MAKE A PLAN TO SOLVE THIS PROBLEM IN GENERAL.
A PLAN TO STOP LOOPING, TO REMEMBER TO REMEMBER.

You tell work you'll be late, put your phone on aeroplane mode, and lock yourself in a closet. Aghast, you find it seems you've tried this before. Now that you're looking, you can see your life is strewn with past attempts. Weekly time set aside to jo...
How are you dealing with ontology identification?
by Erik Jenner — 2022-10-04T23:28:26.711Z

Not really novel, but I've made this point quite often in recent conversations, so I decided to make a short write-up.

I think a wide range of alignment proposals will somehow have to deal with ontology identification, i.e. mapping an AI's beliefs about the world to a representations humans can correctly understand. This could happen explicitly (e.g. via an ELK solution or strong interpretability), or be more hidden, or maybe the proposal avoids the problem in some clever way. But I think alignment proposals should have some answer to the question "How are you dealing with ontology identification?"

A class of examples

One high-level approach you could try to use to align AI goes as follows:

Somehow get a model of human values. This could be a utility function, but might also look more like a preference ordering etc.
Build an AI that optimizes according to this value model.

(If the value model is just a preference ordering, you might have to deal with the fact that it can be inconsistent, but we're ignoring those issues here.)

To be clear, this is not the only approach for building aligned AI (maybe not even the most promising one). It's a special case of separately solving outer alignment and inner alignment, which itself is a special case of solving alignment more broadly. But this still covers an important class of approaches.

I think any approach along those lines has to deal with ontology identification, for a pretty simple reason. The model of human values we get from step 1. will be defined in terms of the human ontology. For example, if we're learning a utility function, it will have type signature Uhuman:{states in human ontology}→R. But for step 2., we need something the AI can optimize over, so we need a utility function of type signature UAI:{states in AI ontology}→R. This means that as a bridge between steps 1. and 2., we need a translation from the AI ontology to the human ontology, τ:{states in AI ontology}→{states in human ontology}. Then we can use UAI:=Uhuman∘τ.

Note that in practice, we will often collapse learning Uhuman and specifying τ into a single step. For example, when doing RLHF on videos of trajectories, we're learning a reward function defined directly in terms of videos, rather than a human representation of what's happening in those videos. These videos also map easily into the RL agent's ontology ...
Smoke without fire is scary
by Adam Jermyn — 2022-10-04T21:08:33.108Z

This post was written in response to Evan Hubinger’s shortform prompt below and benefited from discussions with him. Thanks to Justis Mills for feedback on this post.

If we see precursors to deception (e.g. non-myopia, self-awareness, etc.) but suspiciously don’t see deception itself, that’s evidence for deception.

Setup

Suppose an AI lab tries to look for the capabilities that enable deceptive alignment. This definitely seems like a good thing for them to do. Say they search for evidence of models optimizing past the current training epoch, for manipulation during human feedback sessions, for situational awareness, and so on. Suppose that when all is said and done they see the following:

There are at least two ways to interpret this:

“Phew, bigger models are less likely to be deceptively aligned than small ones. Scale away!”
“Bigger models are more capable at hiding their deceptive alignment. Run away!”

These are very different interpretations, with very different recommendations, so we’d really like to know which to put more credence in.

Priors

Going in, I think we should have a prior that capabilities getting worse with increasing scale is unlikely. We know of specific examples thanks to the inverse scaling prize, but these are rare exceptions to the general rule of increasing capabilities with increasing scale. As a result, a decreasing deceptive capabilities score should be suspicious.

The flip side is that it’s conceivable that some alignment problems get easier with scale. Larger language models understand more of what we say and value (e.g. Figure 8 of Bai+22), and so in some sense outer-aligning them may be easier than outer-aligning smaller models. It’s not inconceivable that the same happens for inner alignment, though I wouldn’t want to rely on this happening.

Tests

We don’t have to rely on priors alone. Here are a few ways we might distinguish between interpretations:

Use transparency tools and figure out what the model is doing.
Impair bigger models (e.g. with dropout or random or adversarial perturbations) and see if they get better at some deceptive capabilities. Impairing models shouldn’t improve capabilities, so this would be good evidence that the bigger models are capable of deception but choosing to hide that capability.
This experiment is a little tricky to implement, but the core idea is to hope that interventions impair something critical in t

...
Deprecated: Some humans are fitness maximizers
by Shoshannah Tekofsky — 2022-10-04T19:38:10.506Z

EDIT: Thank you to the commenters on this essay, as well as the commenters who responded to my comments on Nate's original essay for explaining the concepts further to me. I no longer endorse the title or point of this essay as stated. My understanding is now that I was using a colloquial definition of optimizer combined with an internal sense of optimizing instead of the strict sense. I'm leaving this essay up as I think it's good practice to not delete history. I'll update my epistemic humility a bit now as well, I think. Thank you again for engaging me on this. Also, for future reference, the essay Thou art Godshatter helped me further understand the 'humans are not fitness maximizers" argument.
After writing this essay I'm left entirely confused about what an optimizer/maximizer is. My understanding was that an optimizer/maximizer is an agent's best attempt at picking the best option and that an agent can be closer or farther away from optimizing over the desired target. Thus a mediocre optimizer is using a number of proxies or failing to implement a recognized best solution. This model might conflict with what Nate Soares intended. Nevertheless, here is my rebuttal under possibly diverging key definitions.
In Humans aren't fitness maximizers Nate Soares explains that humans aren't fi... well, ok. A+ for title clarity. Anyway, specifically he makes the following claim about Inclusive Genetic Fitness (IGF):

Humans are not literally optimizing for IGF, and regularly trade other values off against IGF.

This is untrue.

Some humans definitely are optimizing for IGF. These humans go through every major life decision thinking about how they can achieve the highest quality and quantity of offspring. They are currently a minority cause your interest in having PiV sex and your inability to avoid accidents during your favorite hobby used to yield enough IGF to have those genetic strains continue in abundance. Now we can keep the birds and the bees in pharmacologically sealed prisons and tuck the key behind a glass pane that reads "Only break in case of Family Planning".

But people do break the glass. And some people break it a lot. Some people make their lifestyle decisions with their future children in mind. Some people select romantic partners purely on parental and (presumed) genetic health. Some people will eschew entire careers cause they would never be compat...