trentmkelly/LessWrong-43k huggingface dataset:
Procedural Executive Function, Part 1
by DaystarEld — 2022-07-04T18:51:25.516Z
Link: http://daystareld.com/executive-function-1/

To most people, Executive Function is a confusing, mysterious thing that only really comes to attention through dysfunction, particularly mental health disorders like ADHD or depression. Questions of how it works, or how to get more of it, are treated as similar to questions of how to get more "motivation" or "willpower."

I'm here to tell you that's mostly nonsense, and that you can think of executive function in a procedural, explicit way that makes it easy in most cases where you fail to do something you "want to do" to track what's going wrong and why.

To do that, I'm going to break EF into three parts that I consider roughly sequential in how people experience "deliberately doing something," from start to finish, to point out pitfalls and how they can be dealt with.

I call it Procedural Executive Function.

Before starting, it's important to take an extra moment to specifically emphasize that this is a process with multiple steps. Part of what I hope people learn from this is to better understand which aspect of the process is blocking them when they feel stuck with their own executive dysfunction.

So if I focus on a certain aspect of the process and share a perspective on how to help ensure that part goes smoothly, that doesn't mean the assumption is everything will go fine as long as that one aspect does. For some actions you take, the whole process will go smoothly. When it doesn't, the part that trips you up will likely change depending on personality, diagnoses, the type of action you're taking, or just the context of your life at that moment.

We're first examining these parts individually so that we can then examine how they interact more systematically; no part of this process should be taken as a final, normative word on how your own inner workings must look. But I hope it will be helpful nonetheless.

If you're still confused and want an even wider frame, and haven't read the intro post yet, now might be a good time to check it out.

(As a final note, I won’t talk about medical solutions to Executive Function, as it’s outside of my area of expertise. I hope to add more resources for that at some point.)

Planning/Prioritizing

The first step of any intentional act can be called the "notion" to act. Notions themselves are involuntary, often vague, and not particularly compelling. They're just an idea, summed up generically as a thought like "oh, that'...
Anthropic's SoLU (Softmax Linear Unit)
by Joel Burget — 2022-07-04T18:38:05.597Z
Link: https://transformer-circuits.pub/2022/solu/index.html

Anthropic continues their Transformer Circuits Thread. In previous work they were unable to make much progress interpreting MLP layers. This paper is focused on addressing that limitation.

High-level Takeaways

In this paper, we report an architectural change which appears to substantially increase the fraction of MLP neurons which appear to be "interpretable" (i.e. respond to an articulable property of the input), at little to no cost to ML performance. Specifically, we replace the activation function with a softmax linear unit (which we term SoLU) and show that this significantly increases the fraction of neurons in the MLP layers which seem to correspond to readily human-understandable concepts, phrases, or categories on quick investigation, as measured by randomized and blinded experiments. We then study our SoLU models and use them to gain several new insights about how information is processed in transformers. However, we also discover some evidence that the superposition hypothesis is true and there is no free lunch: SoLU may be making some features more interpretable by “hiding” others and thus making them even more deeply uninterpretable. Despite this, SoLU still seems like a net win, as in practical terms it substantially increases the fraction of neurons we are able to understand.

Although preliminary, we argue that these results show the potential for a general approach of designing architectures for mechanistic interpretability: there may exist many different models or architectures which all achieve roughly state-of-the-art performance, but which differ greatly in how easy they are to reverse engineer. Put another way, we are in the curious position of being both reverse engineers trying to understand the algorithms neural network parameters implement, and also the hardware designers deciding the network architecture they must run on: perhaps we can exploit this second role to support the first. If so, it may be possible to move the field in a positive direction by discovering (and advocating for) those architectures which are most amenable to reverse engineering.

Their Key Results:

SoLU increases the fraction of MLP neurons which appear to have clear interpretations, while preserving performance. Specifically, SoLU increases the fraction of MLP neurons for which a human can quickly find a clear hypothesis explaining its activations

...
Book Review: The Righteous Mind
by ErnestScribbler — 2022-07-04T17:45:41.189Z

I - Intuitions And Reason

They say cognitive biases are what other people have. In that vein, “The Righteous Mind” by Jonathan Haidt is a book about how other people think. Specifically, how they think about morality and make moral judgments and choices. But it also covers the interplay between intuitions and reason, community building, group selection, the importance of culture and religion, eating dog carcasses, and much more.

The first part of “The Righteous Mind” could hold its own as a book about thinking and rationality, but it’s disguised as being about moral psychology. It’s about how people actually form opinions and make choices, in a way that is very different from the ideal Bayesian decision maker. It centers on moral psychology, but applies just as well to almost any issue people care deeply about: politics, religion, group affiliation, etc. For me, it provided a fresh viewpoint on rationality, which has stuck with me ever since.

Like any book about rationality - it needs a central catchy metaphor for how the brain works. Here this metaphor is that of a rider on an elephant. Imagine a man riding on the back of an elephant. The rider can say “turn right!” but if the elephant wants to go left, that’s where they’re going. All the rider can do is say “Yes, we meant to go left. Left is the best choice. Here’s why.” The rider is conscious reasoning, and the elephant is emotions, intuitions and everything else we aren’t even always aware of. Emotions and intuitions make the decision where to turn almost instantaneously, and our conscious reasoning has to go along for the ride.

I feel like this captures incredibly well my - sorry, other people’s - thought process on many topics. Suppose I read an article which has a reasoned argument. Right at the start, picking up on subtle cues about the opinion expressed, but also on the various signals the language and tone of the article emit about the writer’s tribe and other views, the elephant (my unconscious emotions) leans towards either “accept” or “reject”. Accordingly, the rider (conscious reasoning) either tries to believe and support the arguments, or fight them internally with everything he’s got. If I want to accept, I ask “can I believe it”; to reject, I ask “must I believe it”. Sometimes, when I’m especially intellectually honest, I can concede an argument and say “this argument is tru...
My Most Likely Reason to Die Young is AI X-Risk
by AISafetyIsNotLongtermist — 2022-07-04T17:08:27.209Z
Link: https://forum.effectivealtruism.org/posts/hJDid3goqqRAE6hFN/my-most-likely-reason-to-die-young-is-ai-x-risk

Introduction

One of my pet peeves is people calling AI Safety a longtermist cause area. It is

entirely

reasonable

to consider AI Safety a high priority cause area without putting significant value on future generations, let alone buying things like total utilitarianism or the astronomical waste arguments. Given fairly mainstream (among EAs) models of AI timelines and AI X-Risk, it seems highly likely that we get AGI within my lifetime, and that this will be a catastrophic/existential risk.

I think this misconception is actively pretty costly - I observe people introducing x-risk concerns via longtermism (eg in an EA fellowship) getting caught up in arguments over whether future people matter, total utilitarianism, whether expected value is the correct decision theory when using tiny probabilities and large utilities, etc. And people need to waste time debunking questions like

‘Is AI Safety a Pascal’s Mugging’

. And yet, even if you convince people of these questions around longtermism, you still need to convince them of arguments around AI x-risk to get them to work on it.

I further think this is damaging, because the label ‘longtermist’ makes AI Safety seem like an abstract, far off thing. Probably not relevant to the world today, but maybe important to future generations, at some point, in the long-term future. And I think this is wildly incorrect. AI x-risk is a directly personal thing that will plausibly affect me, my loved ones, my community, and most people alive today. I think this is a pretty important motivator to some people to work on this - this is not some philosophical thought experiment that you only decide to work on if you buy into some particular philosophical assumptions. Instead, it entirely follows from common sense morality, so long as you buy a few empirical beliefs

To put this into perspective, I’m going to compare my probability of dying young from AI X-Risk to my probability of dying from other causes (operationalised as ‘probability of dying in the next 30 years). My headline result is that, as a 23 year old man living in the UK, conditional on dying in the next 30 years, the probability that I die due to AI x-risk is 41% (3.7% from AI, 5.3% from natural causes)

Analysis

The main point I want to make in this post is that ‘AI x-risk is a pressing concern for people alive today’ is an obvious consequence of mainstream EA beliefs. Acco...
Is General Intelligence "Compact"?
by DragonGod — 2022-07-04T13:27:32.166Z

Epistemic Status

Exploratory and unconfident, but I believe important.

Acknowledgements

I'm grateful to "JustisMills", "sigilyph", "tomcatfish" and others for their valuable feedback on drafts of this post.

Introduction

By compact, I mostly mean "non-composite". General intelligence would be compact if there were universal/general optimisers for real world problem sets that weren't ensembles/compositions of many distinct narrow optimisers.

AIXI and its approximations are in this sense not "compact" (even if their Kolmogorov complexity might appear to be low).

By "general intelligence", I'm using Yudkowsky's conception of (general) intelligence as "efficient cross-domain optimisation".

Thus, when I say, "a general intelligence", I'm thinking of an optimisation process.

Note

I'm currently flirting between the terms "compact" and "simple" to capture the intuition of "non-composite". Let me know which term you think is more legible/accessible (or if you have some other term that you think better captures "non-composite").

("Simple" has the inconvenience of being contrasted with "complex" in the Kolmogorov complexity sense.)

I might equivocate between "problem set" and "domain"; they refer to the same thing.

Why Does This Matter?

Why should you care if general intelligence is compact or not? How is it (supposedly) relevant for the development artificial intelligence?

As best as I can tell — I'll briefly explain why later in this essay — the compactness (or not) of general intelligence may largely determine:

Whether algorithmic/architectural innovation can lead to a fast takeoff

This bears on the feasibility of fast takeoff via:

Designing successor agents
Recursive self-improvement
Other positive feedback loops in algorithmic/architectural innovation

E.g., AIs contributing to AI research

More generally, it constrains takeoff by bounding marginal returns to algorithmic/architectural innovation
Takeoff via scale will also be constrained as a second order effect

Scaling the size of cognitive engines

Examples of "scaling the size of cognitive engines":

Brains with more synaptic connections or more neurons
ML models with more parameters/hyperparameters

Scaling the available computational resources invested in AI systems:

Training compute
Inference compute
Training data
Inference data
Available memory
Etc.

What is possible in the limits of arbitrary high intelligence

Ho

...
Remaking EfficientZero (as best I can)
by Hoagy — 2022-07-04T11:03:53.281Z

Introduction

When I first heard about EfficientZero, I was amazed that it could learn at a sample efficiency comparable to humans. What's more, it was doing it without the gigantic amount of pre-training the humans have, which I'd always felt made comparing sample efficiencies with humans rather unfair. I also wanted to practice my ML programming, so I thought I'd make my own version.

This article uses what I've learned to give you an idea, not just of how the EfficientZero algorithm works, but also of what it looks like to implement in practice. The algorithm itself has already been well covered in a LessWrong post here. That article inspired me to write this and if it's completely new to you it might be a good place to start - the focus here will be more on what the algorithm looks like as a piece of code.

The code below is all written by me and comes from a cleaned and extra-commented version of EfficientZero which draws from the papers (MuZero, Efficient Zero), the open implementation pf MuZero by Werner Duvaud, the pseudocode provided by the MuZero paper, and the original implementation of EfficientZero.

You can have a look at the full code and run it at on github. It's currently functional and works on trivial games like cartpole but struggles to learn much on Atari games within a reasonable timeframe, not certain if this reflects an error or just insufficient time. Testing on my laptop or Colab for Atari games is slow - if anyone could give access to some compute to do proper testing that would be amazing!

Grateful to Misha Wagner for feedback on both code and post.

Algorithm Overview

AlphaZero

EfficientZero is based on MuZero, which itself is based on AlphaZero, a refinement of the architecture which was the first beat the Go world champion. With AlphaZero, you play a deterministic game, like chess, by developing a neural network that evaluates game states, associating each possible state of the board with a value, the discounted expected return (in zero-sum games like chess, discount rate is 0 and this is just win%). Since the algorithm can have access to a game 'simulator', it can test out different moves, and responses to those moves before actually playing them. More specifically, from an initial game state it can traverse the tree of potential games, making different moves, playing against itself, and evaluating these derived game states. After traversi...
We Need a Consolidated List of Bad AI Alignment Solutions
by Double — 2022-07-04T06:54:36.541Z

Thousands of people are thinking about the AI Alignment problem. Many are recreating ideas that have already been suggested and shot down. More importantly, many are creating new ideas but figure that they are in the first camp and fail to share them.

Please feel free to use the comments as a temporary solution! It would also be appreciated to include in the comments bad ideas you have seen and why they won't work.

In order to remove myself from the second camp, I'll share my amateur alignment idea here:

An AI that behaves towards humans in a similar way to how it would choose for a more powerful agent to behave towards it. This is hoping that the Golden Rule can prevent some of the worst AI outcomes. The AI could be a paperclip maximizer, but as long as it recognizes that humans have wants (it doesn't matter if the AI is created with an understanding of exactly what those wants are) and it understands that the more powerful agent could have wants different from its own, it will hopefully stay mostly out of humanity's way while it turns the rest of the universe into paperclips and maybe throw us some resources and technology our way, as it would want the more powerful agent to do for it. It even prevents the situation where AI self-modifies itself to not care about the Golden Rule, because it would not want for the more powerful agent to remove its own Golden Rule. This was inspired by superrationality, which seems pretty close to a way to put morals into something amoral.
AI Forecasting: One Year In
by jsteinhardt — 2022-07-04T05:10:18.470Z

Last August, my research group created a forecasting contest to predict AI progress on four benchmarks. Forecasts were asked to predict state-of-the-art performance (SOTA) on each benchmark for June 30th 2022, 2023, 2024, and 2025. It’s now past June 30th, so we can evaluate the performance of the forecasters so far.

Forecasters were asked to provide probability distributions, so we can evaluate both their point estimates and their coverage (whether the true result was within their credible intervals). I’ll dive into the data in detail below, but my high-level takeaways were that:

Forecasters’ predictions were not very good in general: two out of four forecasts were outside the 90% credible intervals.

However, they were better than my personal predictions, and I suspect better than the median prediction of ML researchers (if the latter had been preregistered).

Specifically, progress on ML benchmarks happened significantly faster than forecasters expected. But forecasters predicted faster progress than I did personally, and my sense is that I expect somewhat faster progress than the median ML researcher does.

Progress on a robustness benchmark was slower than expected, and was the only benchmark to fall short of forecaster predictions. This is somewhat worrying, as it suggests that machine learning capabilities are progressing quickly, while safety properties are progressing slowly.

Below I’ll review the tasks and competition format, then go through the results.

Forecasting Tasks and Overall Predictions

As a reminder, the four benchmarks were:

MATH, a mathematics problem-solving dataset;

MMLU, a test of specialized subject knowledge using high school, college, and professional multiple choice exams;

Something Something v2, a video recognition dataset; and

CIFAR-10 robust accuracy, a measure of adversarially robust vision performance.

Forecasters were asked to predict performance on each of these. Each forecasting question had a $5000 prize pool (distributed across the four years). There were also two questions about compute usage by different countries and organizations, but I’ll ignore those here.

Forecasters themselves were recruited with the platform Hypermind. You can read more details in the initial blog post from last August, but in brief, professional forecasters make money by providing accurate probabilistic forecasts about future events, and are ty...
A compressed take on recent disagreements
by kman — 2022-07-04T04:39:58.101Z

An underlying generator of many recent disagreements appears to be differing locations of the disagreeing individuals on the Yudkowsky-Hanson spectrum. That is, differences in individuals' distributions over the background variable "how does an optimizer's ability to optimize scale as you apply (meta) optimization to it". I'm going to simplify my discussion by boiling this variable down to an "expected steepness" (as opposed to a distribution over optimization-in vs. optimization-out curves).

Eliezer believes this relationship to be steeper than do many others in the alignment sphere. For instance, many (~half by my count) of the disagreements listed in Paul's recent post appear to pretty directly imply that Paul believes this relationship is much less steep than does Eliezer. It seems likely to me that this difference is the primary generator of those disagreements.

Eliezer has previously suggested formalizing this "optimization-in vs. optimization-out" relationship in Intelligence Explosion Microeconomics. Clearly this is not such an easy thing, or he probably would have just done it himself. Nonetheless this may be a pathway towards resolving some of these disagreements.

So, how do optimizers scale with applied optimization?

I'll quickly give my two cents on the matter by noting that we live in a world where the best mathematicians are, quite literally, something like 1000 times as productive as the average mathematicians (keeping in mind this gap only spans about a quarter of the range of variation of mathematical ability in the modern human population; about +3 to +6 stdevs).

This huge difference in observed performance occurs despite the minds in question having about the same amount of hardware (brain size) built out of components running at about the same speed (neurons) executing the same basic algorithm (genetic differences are tiny compared to size of genome). The difference appears to be largely a result of small algorithmic tweaks.

This leads me to the conclusion that, if you take a ~human level AGI and optimize it a little bit further, you probably end up with something pretty strongly superhuman. It also leads me to conclude that it's hard to build a ~human level AGI in the first place because it's a small target on the capability axis: you'll likely just accidentally blow past it, and even if you can hit it, someone else will just blow past it soon...
New US Senate Bill on X-Risk Mitigation [Linkpost]
by Evan R. Murphy — 2022-07-04T01:25:57.108Z
Link: https://www.hsgac.senate.gov/media/majority-media/peters-introduces-bipartisan-bill-to-ensure-federal-government-is-prepared-for-catastrophic-risks-

Two US Senators have introduced a bipartisan bill specifically focused on x-risk mitigation, including from AI. From the post on Senate.gov (bold mine):

WASHINGTON, D.C. – U.S. Senator Gary Peters (MI), Chairman of the Homeland Security and Governmental Affairs Committee, introduced a bipartisan bill to ensure our nation is better prepared for high-consequence events, regardless of the low probability, such as new strains of disease, biotechnology accidents, or naturally occurring risks such as super volcanoes or solar flares that though unlikely, would be exceptionally lethal if they occurred.

“Making sure our country is able to function during catastrophic events will improve national security, and help make sure people in Michigan and across the country who are affected by these incidents get the help they need from the federal government,” said Senator Peters. “Though these threats may be unlikely, they are also hard to foresee, and this bipartisan bill will help ensure our nation is prepared to address cataclysmic incidents before it’s too late.”

[The legislation] will establish an interagency committee for risk assessment that would report on the adequacy of continuity of operations (COOP) and continuity of government (COG) plans for the risks identified. The bipartisan legislation would also help counter the risk of artificial intelligence (AI), and other emerging technologies from being abused in ways that may pose a catastrophic risk.

[...]

It's interesting the term 'abused' was used with respect to AI. It makes me wonder if the authors have misalignment risks in mind at all or only misuse risks.

I haven't been able to locate the text of the bill yet. If someone finds it, please share in the comments.

Cross-posted to EA Forum. Credit to Jacques Thibodeau for posting a link on Slack that made me aware of this.