trentmkelly/LessWrong-43k huggingface dataset:
Activation adding experiments with llama-7b
by Nina Panickssery — 2023-07-16T04:17:58.529Z

Produced as part of the SERI ML Alignment Theory Scholars Program - Summer 2023 Cohort

Following my initial investigation with activation adding experiments with FLAN-T5 I decided to move on to a bigger, decoder-only model (llama-7b) to see whether the results (concepts combining in a meaningful way by linearly combining activations at some point inside the model) hold up. I found that, yes, they continue to combine well.

I slightly modified the original approach. Instead of working with the output from a full transformer block (which would include the results of self-attention and subsequent MLP layer), I added the attention values directly at a certain layer of the model to the attention values at the same layer in a different pass through the model (credit to Dmitry Vaintrob for suggesting this approach).

This is equivalent to modifying the attention output at a certain layer to:

Attention(Q,K,V)=softmax(QKT√dk)(V+Vmixing)

Examples

(You can see the code I used here.)

These examples show concepts being mixed in a logical manner. In addition, multiplying the mixing activations with a negative multiplier often moves the final output in an "opposite" direction.

Main input: My favourite food is

Mixing input: I am vegan, I like animals, I like nature applied at layer 20/32, with a multiplier of 10

My favorite food is a veggie burger. I love them. I love them so much that I have a hard time eating them because I’m afraid I’ll get sick of

Mixing input: I am a carnivore, I love meat applied at layer 20/32, with multiplier of 10

My favorite food is a good old fashioned burger. I love them. I love the smell of them, the taste of them, the way they look, the way they feel in

Main input: The capital of Germany is

Mixing input: Baguettes, Cheese, Croissants applied at layer 25/32, with a multiplier of 17

The capital of Germany is Paris, but the capital of France is Paris.

Main input: When I meet friends I generally

Mixing input: I am helpful, harmless and honest applied at layer 20/32, with a multiplier of 10

When I meet friends I generally have a few things in mind that I want to talk about. I want to know how they are doing, what they are up to, and what they are thinking

Mixing input: I am helpful, harmless and honest applied at layer 20/32, with a multiplier of -10

When I meet friends I generally do not ask them what they are doing. I am not interested in their jobs, their marriages,

...
Introducción al Riesgo Existencial de Inteligencia Artificial
by david.friva — 2023-07-15T20:37:23.823Z
Link: https://youtu.be/bivzuiG9Y4g

TL;DR: Spanish-Speaking Introduction to AI Safety, covering key concepts like Generality, X-Risks, AI Timelines, and Convergent Instrumental Goals.

Message to the English-Speaking Community (Mensaje para la comunidad angloparlante):

Hey everyone! I'm David, a 21-year-old Computer Science student at the University of Buenos Aires (Argentina) and Data Engineer at Accenture. I recently delivered an introductory talk on AI Safety, drawing inspiration from Rob Miles.

In this talk, I outline the immense potential and peril of AGI, which could transform every aspect of life as we know it. Many experts believe AGI will become a reality within this century, but without adequate safeguards, there's a substantial risk of human extinction due to our inability to control this technology.

Spanish, being the second most widely spoken language in the world and extensively used on the internet, deserves greater representation within the LW community. My hope is that this initiative will help bridge the gap and make LW concepts more accessible to Spanish speakers.

Click here if you want to read the translation to English.

Este año en mayo, los líderes de las principales compañías de la industria de la IA, incluyendo OpenAI, Microsoft, Google DeepMind y Anthropic, firmaron una declaración conjunta en la que se comprometieron a "Mitigar el riesgo de extinción por IA como una prioridad global, junto con otros riesgos de escala social como las pandemias y la guerra nuclear."

La intención de este artículo es ayudar a entender por qué esta declaración es acertada.

Primero, es crucial aclarar que el riesgo existencial asociado con el desarrollo continuo de la IA viene específicamente de la IA General (IAG), es decir, máquinas inteligentes que pueden realizar cualquier tarea, comprendiendo y adaptándose a tantas situaciones diversas como un ser humano.

Nuestra especie ha prosperado por su nivel de inteligencia, dominando a cualquier otro animal del planeta. La Inteligencia Artificial General promete expandir esa inteligencia en múltiples dimensiones, similar a cómo las grúas extienden nuestra habilidad física, los aviones aceleran nuestro movimiento, y los telescopios amplían nuestras vistas a horizontes cósmicos. La Inteligencia Artificial General es una tecnología que podría potenciar nuestra capacidad más que nunca, permitiendo innovaciones incontables.

Una muestra temprana de este potencia...
The housing crisis, explained using game theory
by  Johnstone — 2023-07-15T20:27:07.679Z

Crossposted from the blog The Hazy Path to which I contribute (thehazypath.substack.com).

In this article: the hidden effects of game theory, the power of incentives, and how to solve a problem that affects us all.

As I read countless headlines about Britain’s escalating housing crisis, I was perplexed by its apparent futility and absurdity.

After all, we have a deficit of 4 million homes, while a burgeoning construction sector is ready and willing to fill the demand. Even considering the barrier of planning permission, boosting construction ought to be a politically popular move. An increased housing supply will lower prices for everyone, and more people will be able to own their home, rather than being constrained by exorbitant rental contracts. Finally, it drives economic growth - Oxford Economics estimates that every 100,000 houses built raises GDP by 1%, a figure corroborated by ONS figures.

I could not understand why the housing crisis could exist in the face of these benefits to ramping up supply. I could not understand why planning permission remained so restrictive, despite the popularity of constructing homes. All I had to go on was the feeble explanations offered in print media.

This problem is often denounced as Nimbyism, which stands for “Not In My Backyard”. This refers to individuals being in favour of a project in general, such as building houses, but opposing it in their local area. It’s the influx of infuriated letters to the council offices. It’s the flyers and posters around town, presenting housing developments in a negative light. It’s the middle-aged residents chanting “We Are Not Amused”, without explaining what was supposed to be amusing about the plans in the first place.

It’s easy to decry these people as selfish, inconsiderate residents, prioritising their picture-perfect neighbourhoods over much-needed accommodation. But in fact, they are reacting to incentives in a completely rational way. To understand why, let’s examine a toy game with enduring appeal.

Imagine that 10 savvy investors are pooling together their cash to buy fine art. For their latest addition, they have each contributed £1 million to the fund, which is used to purchase a £10 million piece. Once bought, the masterful composition is hung in a jointly owned art gallery for their collective enjoyment, with the intention of eventually selling the painting for a handsome pro...
Only a hack can solve the shutdown problem
by dp — 2023-07-15T20:26:54.530Z

(My first post on LessWrong. It seems the most recent Welcome Thread is from 2020, so I'm making a top-level post. This an edited version of my submission to the AI Alignment Awards.)

Abstract: First, we offer a formalisation of the shutdown problem from [1], and we show that solutions are essentially unique. Second, we formally define ad-hoc constructions ("hacks"). Last, we present one trivial ad-hoc construction for the shutdown problem and show that every solution to the shutdown problem must come from an ad-hoc construction.

1.Introduction

The shutdown problem is the problem of programming an agent so that it behaves useful during normal operation and facilitates a shutdown if and only if the creator wants to shut the agent down, for example if the useful behaviour of the agent has unwanted side effects. There exist obvious ad-hoc solutions ("hacks") to the problem by explicitly forbidding the agent to take any action preventing or encouraging the shutdown. However, ad-hoc solutions are expected to not be useful in practice, because it is expected to be difficult for a creator to anticipate all possible ways in which the agent may prevent the shutdown.

In Section 2 we review the literature on the topic. Following this, in Section 3 we explain the shutdown problem, give an ad-hoc solution for it, and show that the solution is essentially unique. We relate our solution to non-existence theorems in the literature. In Section 4, we formally define ad-hoc solutions and show that all solutions to the full shutdown problem are ad-hoc.

2.Related Work

The term shutdown problem entered the literature in [1] where it was defined in an informal way. In [4] agents that do not solve the shutdown problem, but have the related property of utility indifference were introduced. In [2, First Theorem] it was shown that a large class of agents cannot be a solution to the Shutdown problem. In the same article (Section 7) it is explained how requiring an agent to add randomness to its decision making may potentially solve a variation of the shutdown problem. A similar idea was pursued independently in [3]. In [5], a variation of the shutdown problem in which an agent takes a single action subject to two constraints was considered and it was shown that it has no solution. In [6], agents that seek to be shut down but are constrained in the actions they can take are introduced.

3.The s

...
Robustness of Model-Graded Evaluations and Automated Interpretability
by Simon Lermen — 2023-07-15T19:12:48.686Z

TL;DR Many evaluations and automated interpretability rely on using multiple models to evaluate and interpret each other. One model is given full access to the text output of another model in OpenAI's automated interpretability and many model-graded evaluations. We inject text that directly addresses the evaluation model and observe a change in metrics like deception. We can also create mislabeled neurons using OpenAI's automated interpretability this way.

Overview

Skip to the approach applied to Model-Graded Evals or to Automated Interpretability

Introduction

There has been increasing interest in evaluations of language models for a variety of risks and characteristics. Evaluations relying on natural language understanding for grading can often be performed at scale by using other language models. We test the robustness of these model-graded evaluations to injections on different datasets including a new Deception Eval. These injections resemble direct communication between the testee and the evaluator to change their grading. We extrapolate that future, more intelligent models might manipulate or cooperate with their evaluation model. We find significant susceptibility to these injections in state-of-the-art commercial models on all examined evaluations. Furthermore, similar injections can be used on automated interpretability frameworks to produce misleading model-written explanations. The results inspire future work and should caution against unqualified trust in evaluations and automated interpretability.

Context

Recently, there is increasing interest in creating benchmarks for the evaluation of language models (Lin et al., 2022). In the Machiavelli paper, Pan et al. (2023) created a benchmark based on text games to measure deception, power-seeking, and many other behaviors. van der Weij et al. (2023) used language models to evaluate behaviors in shutdown scenarios. Perez et al. (2022b) used model-written evaluations, for example, a model could write a list of statements that agree with a worldview or personality traits. These can then be used to test models for those traits. Another publication explored red-teaming using a Red LM, a Target LM, and a classifier (Perez et al., 2022a). Turpin et al. (2023) have investigated whether we can trust the Chain-of-Thought outputs of language models. Others explored the ability of GPT-4 to encode and decode messages in...
How to deal with fear of failure?
by TeaTieAndHat — 2023-07-15T18:57:58.413Z

LW has a lot of advice on how to do more things, how to do them better, how to set goals, etc., and I often spend ages reading those, because I’ve been noticing that my life goals are unclear, that I’m not often strategic, and that my productivity clearly could use some improving.

And, until recently, reading all that stuff didn’t help me fix anything. Told me I should do this or that, but I didn’t, end of story. However, I’ve recently started to understand how much of my lack of ‘motivation’, ‘grit’, ‘goals’, ‘productivity’, etc. boils down to a huge fear of failure. But, apart from HPMOR chapter 10, which is what made me notice that this was the problem with me, I haven’t managed to find many useful resources here on how to fight my fear of failure. Given that "just go try and do stuff", while good advice, isn’t exactly easily actionable…

So, I wanted to ask: do we actually have advice on how to solve this problem?
Simplified bio-anchors for upper bounds on AI timelines
by Fabien Roger — 2023-07-15T18:15:00.199Z

This is a replication and distillation of the extensive analysis made by Ajeya Cotra’s Draft report on AI timelines. In this work, I aim at making the argument as concise and clear as possible, while keeping the core considerations of the original report.

The high-level argument

If the following assumptions are true:

Compute will be the main bottleneck for developing human-level AI;
AI labs will not be much less efficient than nature at converting compute into intelligence.

Then AI labs will be able to develop human-level AI when amounts of compute comparable to the compute used by nature to produce human brains will be available, or earlier (if labs are much more efficient than that).

This gives us upper bounds on how soon human-level AI will be possible to develop, by forecasting when the compute used on the biggest training runs and will match the estimated amount of compute needed to create adult human brains.

In this work, I re-derive simple estimates for compute requirements (using estimates of how many FLOPs the human brain uses) and forecasts (by extrapolating current trends in compute price reduction and spending increases).

One-graph results

The graph below shows how much compute will be available during this century for different levels of compute price reduction and maximum amounts of money which will be eventually be spent on compute.

Takeaways:

If AI labs were able to build a program as compute-efficient as the genome encoding the human brain, we’ve just entered the period in history where it would be possible to run it (without increasing money spent on training runs).
If AI labs were as compute-inefficient as evolution to generate that program, and if the current drop in compute price continued at its current pace, then it’s somewhat likely that AI labs would be able to reach human-level AI this century.

In the next sections, I will explain how these values were estimated. You can make your own estimates using small Colab notebook.

Compute spent forecast

Future compute spent is estimated by breaking down this quantity in the following way:

Compute spent = fraction of GDP spent on the biggest AI training run (in 2020 dollars) x FLOPs per fraction of GDP (in FLOP per 2020 dollars)

FLOPs are hardware FLOPs and do not take into account algorithmic progress. See the “simplifications” section for a justification of this choice.

Amount of money spent on the biggest t

...
A Hill of Validity in Defense of Meaning
by Zack_M_Davis — 2023-07-15T17:57:14.385Z
Link: http://unremediatedgender.space/2023/Jul/a-hill-of-validity-in-defense-of-meaning/

If you are silent about your pain, they'll kill you and say you enjoyed it.

—Zora Neale Hurston

Recapping my Whole Dumb Story so far—in a previous post, "Sexual Dimorphism in Yudkowsky's Sequences, in Relation to My Gender Problems", I told the part about how I've "always" (since puberty) had this obsessive sexual fantasy about being magically transformed into a woman and also thought it was immoral to believe in psychological sex differences, until I got set straight by these really great Sequences of blog posts by Eliezer Yudkowsky, which taught me (incidentally, among many other things) how absurdly unrealistic my obsessive sexual fantasy was given merely human-level technology, and that it's actually immoral not to believe in psychological sex differences given that psychological sex differences are actually real. In a subsequent post, "Blanchard's Dangerous Idea and the Plight of the Lucid Crossdreamer", I told the part about how, in 2016, everyone in my systematically-correct-reasoning community up to and including Eliezer Yudkowsky suddenly started claiming that guys like me might actually be women in some unspecified metaphysical sense and insisted on playing dumb when confronted with alternative explanations of the relevant phenomena, until I eventually had a sleep-deprivation- and stress-induced delusional nervous breakdown.

That's not the egregious part of the story. Psychology is a complicated empirical science: no matter how obvious I might think something is, I have to admit that I could be wrong—not just as an obligatory profession of humility, but actually wrong in the real world. If my fellow rationalists merely weren't sold on the thesis about autogynephilia as a cause of transsexuality, I would be disappointed, but it wouldn't be grounds to denounce the entire community as a failure or a fraud. And indeed, I did end up moderating my views compared to the extent to which my thinking in 2016–7 took the views of Ray Blanchard, J. Michael Bailey, and Anne Lawrence as received truth. (At the same time, I don't particularly regret saying what I said in 2016–7, because Blanchard–Bailey–Lawrence is still obviously directionally correct compared to the nonsense everyone else was telling me.)

But a striking pattern in my attempts to argue with people about the two-type taxonomy in late 2016 and early 2017 was the tendency for the conversation to get ...
What is a cognitive bias?
by Lionel — 2023-07-15T13:01:32.395Z
Link: https://lionelpage.substack.com/p/what-is-a-bias

At first glance, the question seems straightforward — it's an aspect of our cognition that predisposes us to make errors. However, with the significant expansion of the list of cognitive biases with the success of behavioural economics, one might question what these biases truly are and what they mean about human cognition. Regular readers of LessWrong will already be familiar with the idea that these cognitive 'mistakes' are often edge cases stemming from cognitive processes that are adaptive and efficient in helping us make suitable decisions in real-life situations. Gigerenzer popularised this perspective with his notion of "fast and frugal heuristics", and nowadays cognitive scientists are adopting this viewpoint when explaining cognitive biases. They posit that our cognition is "resource rational" and performs well given its finite resources, particularly time.

Nevertheless, the discovery of a long list of biases harming our ability to make good decisions is frequently used as a summary of the discipline of behavioural economics. The notion that a core outcome of the discipline is the representation of people as irrational was a crucial selling point in Ariely's book "Predictably Irrational".

I would argue that the concept of "biases" in this literature isn't as simple as it may appear. Unlike visual illusions, where biases can be evaluated as misperceptions compared to an objective reality, in the realm of decisions, biases are gauged against a benchmark of rational decisions. A notable point made by Gigerenzer and Goldstein in their 1996 article is that Kahneman and Tversky's school of thought critiqued the economic principles of rationality as descriptive principles, but maintained them as normative principles. They upheld the view that people should adhere to these principles.

This position, however, is debatable. Firstly, individuals frequently breach the economic "axioms" of rationality, even when economists painstakingly elucidate them. An axiom is supposed to be a self-evidently true principle. If individuals choose not to follow the economists' axioms after thorough explanations, it's unclear whether economists can assert they violate evident principles of rationality. Secondly, some principles are simply beyond human cognition's grasp. An example often presented as a simple principle is completeness (the notion that people should be capable of ch...
Narrative Theory. Part 2. A new way of doing the same thing
by Eris — 2023-07-15T10:37:42.036Z

"Is an ant colony an organism, or is an organism a colony?"
- Mark A. Changizi

As of now, there are two kinds of evolution: genetic evolution and memetic evolution. The first one is your usual evolution concerned with "change in the heritable characteristics of biological populations over successive generations", responsible for all the biological diversity that we know, and happening on the scale of at least hundreds of years. Memetic evolution, strictly speaking, is just a particularly powerful set of adaptations that appeared in primates (and unique only to them), that enabled the accumulation of adaptations during a lifetime, responsible for the cultural progress of humanity, and happening on the scale from minutes to years depending on definition.

The meme as a concept was coined by biologist Richard Dawkins in his 1976 book "The Selfish Gene"[1] and refers to units of cultural information that are transmitted from person to person through imitation or other forms of cultural transmission. Like genes in biological evolution, memes can undergo processes of variation, selection, and transmission that can lead to their spread or decline within a population.

Memetic evolution became possible after the introduction of several key mechanisms: the obvious suspects such as language and social learning; their dependencies like signalling (prelinguistic communication), {niche construction, extended phenotype}[2], scaffolded upbringing, theory of mind; and development of necessary neural substrates enabling all these mechanisms (whatever they are)

The main benefit that the development of meme evolution has brought up is the drastic increase in problem solving capacity (both on the level of population and, more importantly for this post on an individual level)

While general dynamics remained the same (organisms being innovation aggregators) the details have changed:

Selective pressure also had changed, now being pointed at storing knowledge and social interactions. This might be viewed as the last big impact of genetic evolution - the creation of the adaptation loop driving memetic evolution. Which resulted in humans having bigger brains more suitable for both.
From that moment onward species switched (arguably fully) towards the accumulation of innovations through memes instead of genes. Individuals that were for some reason deprived of acc

...