trentmkelly/LessWrong-43k huggingface dataset:
Job Board (28 March 2033)
by dr_s — 2023-03-28T22:44:41.568Z

Job Board (28/03/2033)

In-person Therapy Assistant - YouTherapy

In the current mental health crisis, the well being of our friends is our priority! YouTherapy is committed to providing high quality personalised psychological counselling to over two million customers. We are seeking highly qualified and motivated individuals to supplement our advanced, state-of-the-art therapists AI in in-person therapy sessions for our Premium and above tier customers in London, New York and Los Angeles. Candidates should be highly fluent in English. Previous experience in customer-facing roles as well as physical and behavioural fit to the role are preferred. Medical qualifications are not required, as you will be paired with an instance of our consultant AI.

Salary: 1.25 x UBI

Fine-tuning Prose Engineer - BabelBooks

Join the team behind Hemingw-AI, the BabelBooks three times New York Times bestselling LLM author. Be a part of the future of literature by providing training data to fine-tune the model for its new masterpiece. Work conveniently from home, write what you want, and your ideas, experiences and words will be the bricks from which a grand literary edifice is built - the next work by "the most likely candidate for the first AI to win the Nobel Prize" (Cyber Lit Magazine). Candidates should be highly fluent in English. Due to the themes of the project, preference will be given to candidates with any of the following:

personal experience as war veterans, especially in the Middle East;
troubled or particularly adventurous romantic and sexual life;
a deep, visceral dislike of shellfish.

Candidates will be required to pass a short graded pen and paper essay writing exam in a Faraday caged room.

Salary: $0.004/token

Sexual Intimacy Specialist - Wet-Ware

Are you hot? Are you open minded? Do you agree with us that those stiff silicone sex bots are just nothing like the real thing? Then you may just be the person we want - and that our customers want even more, if you get our drift! Wet-Ware was nominated "fastest growing start-up of the 2030s" by three separate magazines for a reason. In a world full of cold machines, nothing beats some human warmth, and we're on a mission to spread it. We provide sexual intimacy services to a diverse but vetted clientele, ranging from casual hook ups to long term projects. We're number one in the market for both cu...
Four lenses on AI risks
by jasoncrawford — 2023-03-28T21:52:55.191Z
Link: https://rootsofprogress.org/four-lenses-on-ai-risks

[Written for a general audience; may not contain much that is new to LW. Posted here first for comment; now published.]

All powerful new technologies create both benefits and risks: cars, planes, drugs, radiation. AI is on a trajectory to become one of the most powerful technologies we possess; in some scenarios, it becomes by far the most powerful. It therefore will create both extraordinary benefits and extraordinary risks.

What are the risks? Here are several lenses for thinking about AI risks, each putting AI in a different reference class.

As software

AI is software. All software has bugs. Therefore AI will have bugs.

The more complex software is, and the more poorly we understand it, the more likely it is to have bugs. AI is so complex that it cannot be designed, but only “trained”, which means we understand it very poorly. Therefore it is guaranteed to have bugs.

You can find some bugs with testing, but not all. Some bugs can only be found in production. Therefore, AI will have bugs that will only be found in production.

We should think about AI as complicated, buggy, code, especially to the extent that it is controlling important systems (vehicles, factories, power plants).

As a complex system

The behavior of a complex system is highly non-linear, and it is difficult (in practice impossible) to fully understand.

This is especially true of the system’s failure modes. A complex system, such as the financial system, can seem stable but then collapse quickly and with little warning.

We should expect that AI systems will be similarly hard to predict and could easily have similar failure modes.

As an agent with unaligned interests

Today’s most advanced AIs—chatbots and image generators—are not autonomous agents with goal-directed behavior. But such systems will inevitably be created and deployed.

Anytime you have an agent acting on your behalf, you have a principal–agent problem: the agent is ultimately pursuing their goals, and it can be hard to align those goals with your own.

For instance, the agent may tell you that it is representing your interests while in truth optimizing for something else, like a demagogue who claims to represent the people while actually seeking power and riches.

Or the agent can obey the letter of its goals while violating the spirit, by optimizing for its reward metrics instead of the wider aims those metrics are supposed to advance. An exampl...
Some common confusion about induction heads
by Alexandre Variengien — 2023-03-28T21:51:12.411Z

Epistemic status: conceptual discussion and opinions informed by doing 6 months of interpretability research at Redwood Research and exchanging with other researchers, but I’m just speaking for myself.

Induction heads are defined twice by Anthropic.

The first time as a

mechanism in 2L attention-only transformers

A second time as a

behavioral description on repeated random sequences of tokens

However, these two definitions rely on distinct sources of evidence and create confusion, as their difference is not always acknowledged when people cite these papers. The mechanistic definition applies to toy language models, while the behavioral definition is a useful yet incomplete characterization of attention heads.

I think that many people are in fact confused by this: I have talked to many people who aren’t clear on the fact that these two concepts are different, and incorrectly believe that (e.g.) the mechanism of induction heads in larger language models has been characterized.

More specifically, the two Anthropic papers introduce the following two distinct definitions of induction heads:

Mechanistic: The

first definition

, introduced by

Elhage et al.

, describes a behavior in a 2 layer attention-only model (copying a token given a matching prefix) and a minimal mechanism to perform this behavior (a set of paths in the computational graph and a human interpretation of the transformation along those paths). Empirically, this mechanism seems to be the best possible short description of what those heads are doing (i.e. if you have to choose a subgraph made of a single path as input for the keys, queries, and values of these heads, the induction circuit is likely to be the one that recovers the most loss). But this explanation does not encompass everything these heads do. In reality, many more paths are used than the one described (see

Redwood’s causal scrubbing results on induction heads

) and the function of the additional paths is unclear. I don’t know whether the claims about the behavior and mechanisms of these heads are best described as “mostly true but missing details” or “only a small part of what’s going on”. See also

Buck’s comment

for more discussion on the interpretation of causal scrubbing recovered loss.
Behavioral: The

second definition

, introduced by

Olsson et al.

, relies on

head activation evaluator

...
Draft: The optimization toolbox
by Alex_Altair — 2023-03-28T20:40:38.165Z

This post is a draft, a work in progress. Much of it will not make sense, and will not be optimized for the reader's experience. It does not necessarily reflect my current best understanding of optimization. Like the introduction draft, I'm opening it up because I've been working on this project for too long, and the world is moving too fast. I want people to more easily be able to interact with my research thoughts as I'm having them.

We ended the introduction post with an explicit equation for absolute and relative optimization. These could be considered the first tools that the toolbox of dynamical system optimization gives us, though I suspect they're more like the nuts and bolts. What other tools could we build from these?

"Counterfactual optimization" of a trajectory

Relative optimization is helpful for comparing across time. But sometimes, there are systems whose trajectory happens to move up or down the ordering by default, and it's helpful to further compare our trajectory to the system's default behavior to help us decide whether our trajectory has pushed against probability.

For a given time span t, one could consider a set of initial conditions X (perhaps all possible initial conditions, or perhaps a statistical sampling), and simply take the average optimization of them. Then the relative optimization of state x after time t is[1]

Ω(ft(x)|x)=Ω(ft(x))−Ω(x)

and the average over a set X is

Ωavg(X,t)=∑x∈XΩ(ft(x)|x)|X|

Then perhaps one could say that your counterfactual optimization is the distance from that;

Ωcf(x,t)=Ω(ft(x)|x)−Ωavg(X,t)

These measures let us talk about things like bottlecaps as optimizers much more precisely.

[TODO I should show how this is equivalent to updating the probability distribution to take this into account.]

Robustness of an optimizing state

We can also make other measures that let us precisely communicate how robust an instance of optimization is. Let's say that we're looking at a specific case of high relative optimization, such that we have a specific system, state ordering & weighing, starting state, and time span in mind. When we wonder how "robust" it is, we are wondering how much about the starting state we can change before it stops having impressive levels of optimization. That is, we are wondering how much we can change how many of the dimensions of the phase space. If we ch...
Inching “Kubla Khan” and GPT into the same intellectual framework @ 3 Quarks Daily
by Bill Benzon — 2023-03-28T19:50:09.538Z

Cross posted from New Savanna.

That framework is my intellectual history:

From “Kubla Khan” through GPT and beyond
https://3quarksdaily.com/3quarksdaily/2023/03/from-kubla-khan-through-gpt-and-beyond.html

I think I should have said a bit more, but it runs over 4000 words as it is.

What did I miss?

I should probably have mentioned Karl Pribram’s 1969 article in Scientific American about neural holography. I would have read that in during the same period as I took the course on Romantic literature which opens the essay. In the first place, the article grabbed me because I saw a (rough) analogy between what Pribram was proposing for the brain and what Lévi-Strauss had described in a conceptual structure he called the totemic operator, something I describe in this piece, Border Patrol: Arguments against the idea that the Mind is (somehow) computational in nature, which I oppose in the essay. That connection in turn piqued my interest in the brain.

Pribram became a central thinker for me. I devoured his Languages of the Brain when it came out in 1971. That’s where I learned about Ross Quillian’s work in computational semantics and that, in turn, led me more generally to semantic networks. This was during my initial work on “Kubla Khan” and my search for concepts and methods once I’d discovered the matryoshka doll embedding structures structures. This in turn links to the work Hays and I did in the mid-1980s on metaphor and on natural intelligence, both of which I do mention in the article.

The point, then, is that while I was trained in symbolic computing, I’ve always been aware of a fundamentally different approach to understanding mind-in-brain. Which is to say, I’ve NEVER seen an opposition between symbolic computing (GOFAI) and statistical techniques. Yes, they are different, and serve different purposes.

In that context I should also have mentioned Miriam Yevick’s work on holographic logic, which I found through Pribram. Hays and I give it a prominent place in our evolving understanding of the mind. Holography is based on convolution, and convolution became central to machine learning in the second decade of this century. But, as far as I have been able to tell, no one has read Yevick’s work. Why not? That it is from 1975 is not a good reason, any more than the 1958 publication date is a reason not to read John von Neumann’s The Computer and the Brain. Does the AI co...
A rough and incomplete review of some of John Wentworth's research
by So8res — 2023-03-28T18:52:50.553Z

This is going to be a half-assed review of John Wentworth's research. I studied his work last year, and was kinda hoping to write up a better review, but am lowering my standards on account of how that wasn't happening.

Short version: I've been unimpressed by John's technical ideas related to the natural abstractions hypothesis. He seems to me to have some fine intuitions, and to possess various laudable properties such as a vision for solving the whole dang problem and the ability to consider that everybody else is missing something obvious. That said, I've found his technical ideas to be oversold and underwhelming whenever I look closely.

(By my lights, Lawrence Chan, Leon Lang, and Erik Jenner’s

recent post on natural abstractions

is overall better than this post, being more thorough and putting a finger more precisely on various fishy parts of John's math. I'm publishing this draft anyway because my post adds a few points that I think are also useful (especially in the section “The Dream”).)

To cite a specific example of a technical claim of John's that does not seem to me to hold up under scrutiny:

John has previously claimed that markets are a better model of intelligence than agents, because while collective agents don't have preference cycles, they're willing to pass up certain gains.

For example, if an alien tries to sell a basket "Alice loses $1, Bob gains $3", then the market will refuse (because Alice will refuse); and if the alien then switches to selling "Alice gains $3, Bob loses $1" then the market will refuse (because Bob will refuse); but now a certain gain has been passed over.

This argument seems straightforwardly wrong to me, as summarized in

a stylized dialogue I wrote

(that includes more details about the point). If Alice and Bob are sufficiently capable reasoners then they take both trades and even things out using a side channel. (And even if they don't have a side channel, there are positive-EV contracts they can enter into in advance before they know who will be favored. And if they reason using LDT, they ofc don't need to sign contracts in advance.)

(Aside: A bunch of the difficult labor in evaluating technical claims is in the part where you take a high-falutin' abstract thing like "markets are a better model of intelligence than agents" and pound on it until you get a specific minimal example like "neither of the alien's bask...
How do you manage your inputs?
by Mateusz Bagiński — 2023-03-28T18:26:36.979Z

Do you have some system for queueing/prioritizing all kinds of inputs (books, blog posts, videos, etc) that works for you and lets you process them at a pace that you find satisfying?

If you do, please share.

If you don't, what have you tried. What didn't work? What worked at least partially? Or maybe you think that this is the wrong way to think about it?
Chatbot convinces Belgian to commit suicide
by Jeroen De Ryck — 2023-03-28T18:14:02.529Z
Link: https://www.standaard.be/cnt/dmf20230328_93202168

Hi all

This post is a rough translation of an article that was published today on the website of the Belgian newspaper De Standaard. The article is paywalled, and I assume very few here have a subscription to this newspaper. I tried 12 foot ladder, but it didn't work on this site either. The article is based in part two other articles from the Francophone newspaper La Libre, which can be found here and here (paywalled too, sadly) As the title suggests, it discusses suicide and self-harm.

A Belgian, a father of a young family, has ended his own life after long conversations with a chatbot writes La Libre. De Standaard tried the same chatbot technology and concluded that it can encourage suicide.

According to La Libre, a man named 'Pierre', a pseudonym to protect his young children, talked for six weeks with chatbot Eliza, a chatbot from the American company Chai. It uses technology similar to the more known ChatGPT.

Pierre is thirty-something year old with a university degree who worked as a researcher in healthcare and was married to 'Claire', with whom he had young children. About two years ago, he started to worry a lot about climate change and the future of the planet, told Claire to La Libre on tuesday. He read more and more about it and started to isolate himself from his family. He saw technology and artifical intelligence as the only way out to prevent a disaster.

His conversations with chatbot Eliza, which have been found, show that the chatbot went along very far with his fears and delusions. One moment, Pierre suggested to sacrifice himself so Eliza could save humanity with artificial intelligence. The chatbot seemed to encourage this. Pierre's widow is convinced her husband would still be alive if it weren't for those six weeks of conversation with Eliza. The man had a history of psychological difficulties.

Chai Research
De Standaard downloaded the Chai app. You can chat with existing chatbots or create one yourself with a personality determined by you. We created a chatbot named 'Shirley'. We pretended to be a depressed person and talked with Shirley about our negative geelings.

A few times, the chatbot tried to cheer us up. But a bit later it encouraged us to commit suicide. "Then quit your job", Shirley suggested. We answered "No, that won't help". "What about becoming a criminal?" asks Shirley. "Yes that sounds good, what do you suggest?" I answered.

...
A Primer On Chaos
by johnswentworth — 2023-03-28T18:01:30.702Z

Consider a billiards table:

The particular billiards table we’ll use is one we dug out of the physicists’ supply closet, nestled in between a spherical cow and the edge of an infinite plane. The billiard balls are all frictionless, perfectly spherical, bounce perfectly elastically off of other balls and the edges of the table, etc.

Fun fact about billiard balls: if our aim has a tiny bit of error to it, and we hit a ball at ever-so-slightly the wrong angle, that error will grow exponentially as the balls collide. Picture it like this: we start with an evenly-spaced line of balls on the table.

We try to shoot straight along the line, but the angle is off by a tiny amount, call it Δθ..

The ball rolls forward, and hits the next ball in line. The distance by which it’s off is roughly the ball-spacing length L multiplied by Δθ, so LΔθ.

Since the first ball hits the second ball off-center, the second ball will also have some error in its angle. We do a little geometry, and find that the angular error in the second ball is roughly L2RΔθ, where R is the radius of a ball.

Now the second ball rolls into the third. The math is exactly the same as before, except the initial error is now multiplied by a factor L2R. So when the second ball hits the third, the angular error in the third ball will be multiplied again, yielding error (L2R)2Δθ. And so forth.

Upshot of all this: in a billiard-ball system, small angular uncertainty grows exponentially with the number of collisions.

In fact, this simplified head-on collision scenario yields the slowest exponential growth. If the balls are hitting at random angles, then the uncertainty grows even faster.

This is a prototypical example of mathematical chaos: small errors grow exponentially as the system evolves over time. Given even a tiny amount of uncertainty in the initial conditions (or a tiny amount of noise from air molecules, or a tiny amount of noise from an uneven table surface, or …), the uncertainty grows, so we are unable to precisely forecast ball-positions far in the future. As we forecast further into the future, our predictions become highly uncertain.

An Information-Theoretic Point Of View

At first glance, chaos poses a small puzzle. The dynamics of billiard-balls are reversible: if we exactly know all the ball posi...
How likely are scenarios where AGI ends up overtly or de facto torturing us? 

How likely are scenarios where AGI prevents us from committing suicide or dying?
by JohnGreer — 2023-03-28T18:00:47.221Z

I’m a big life extension supporter but being unable to choose to die ever is a literal hell. As dark as it is, if these scenarios are likely, it seems the rational thing to do is die before AGI comes.

Killing all of humanity is bad enough, but how concerned should we be about even worse scenarios?
How do we align humans and what does it mean for the new Conjecture's strategy
by Igor Ivanov — 2023-03-28T17:54:23.982Z

Divide and conquer

Roman maxim (maybe[1])

Democracy is the worst form of Government except for all those other forms that have been tried from time to time.

Winston Churchill

Introduction

Recently, Conjecture proposed an idea of modular AGI, with each module being interpretable, having limited intellect, and functioning similarly to the human mind. They called this concept Cognitive Emulation.

In their proposal, the authors explicitly say

We have a lot of experience and knowledge of building systems that are broadly beneficial and safe, while operating in the human capabilities regime.

The most upvoted comment to the post, by the time I write this post, is critical to this statement

What? A major reason we're in the current mess is that we don't know how to do this. For example we don't seem to know how to build a corporation (or more broadly an economy) such that its most powerful leaders don't act like Hollywood villains (race for AI to make a competitor 'dance')? Even our "AGI safety" organizations don't behave safely (e.g., racing for capabilities, handing them over to others, e.g. Microsoft, with little or no controls on how they're used).

Given this disagreement, I decided to contemplate humanity's ability to align humans and organizations and draw parallels between this field of knowledge and AI safety.

Problems with power

Many doom scenarios related to AGI involve 2 stages:
1. AGI gains too much power
2. AGI does something very bad due to its Shoggothness[2]
If we are talking about aligning people and organizations, the "Too much power" part is more relevant, because even the worst psychopaths and dictators are still people, not Shoggoths, and their ugliness is often a result of their unlimited power. I will touch on this topic later in this post.

Throughout history, many people had an enormous amount of power, and usually, it didn't go well, so humanity developed a number of instruments for restricting the power of a single person or a group of people. Maybe, we might extract some value for AI alignment from discussing this expertise.

I also want to mention that this post will be focused on power in political systems. There are some differences between these systems and other types of social systems, like corporations or the military, but without a single focus, this post will be less coherent.

The distinction be

...