trentmkelly/LessWrong-43k huggingface dataset:
How To Socialize With Psycho(logist)s
by Sable — 2023-10-20T11:33:46.066Z
Link: https://affablyevil.substack.com/p/how-to-socialize-with-psychologists

Therapy is great. So are therapists!

But being friends with a therapist is a very different kind of relationship than being the patient of one, and I believe the differences highlight something important about all relationships.

They Specialize In People

One of the foundations of our current world is specialization. It’s more efficient to have a bunch of people, all of whom are very good at one thing in particular, trade with each other, than to have everyone know a little bit of everything and provide for themselves.

Our whole economy is built on specialization.

And there are a multitude of possible specializations - different kinds of software, plumbing, carpentry, construction, rectal surgery, animal husbandry, hospitality, sewage management, theoretical astrophysics, golfing, designing microchips, flying, etc.

Some of these specialties pertain to interacting with people. Teachers, salespeople, managers, and so on generally develop a suite of skills for dealing with other human beings.

Therapists (including social workers, clinical psychologists, and so on) are somewhat unique in that they specialize in emotionally supporting people. While the other specialties involving people involve getting people to do things (learn, buy something, work, etc.), a therapist’s job involves making someone healthier.

This is not, in my experience, a specialty (or an associated set of skills) with an off switch.

What Makes A Healthy Relationship?

Relationships Are Reciprocal

Healthy relationships can be generally characterized as reciprocal or mutual. All the people involved get something out of the relationship. There’s a balance to it - a give and a take.

(Which is not to say that all healthy relationships are 50% give and 50% take; every relationship is unique. The important thing is that everyone in the relationship is getting what they need out of it.)

In fact, one of the most direct signs of an unhealthy relationship is if it’s one-sided. While a healthy relationship can tip back and forth over time - sometimes one person will need more support, and that’s fine - if long periods of time go by and one person continually takes more than they give, that’s a red flag.

But Not With Therapists

Of course, emotionally speaking, the therapist-patient relationship isn’t reciprocal or mutual in the slightest, and that’s how it’s supposed to be. The emotional work a therapist does for a patien...
Revealing Intentionality In Language Models Through AdaVAE Guided Sampling
by jdp — 2023-10-20T07:32:28.749Z

Introduction

The universe is already its own model, that is why it seems so hard to model,
but really it is simple. All that needs to be done is to add Mu back into a
transformer. "The universe is already here, you just have to rearrange it
properly." This was the secret of comprehension: the universe is already
here, and it knows that it is here.

— LLaMa 2 70b

The Stanford Encyclopedia of Philosophy defines intentionality
as "the power of minds and mental states to be about, to represent,
or to stand for, things, properties and states of affairs. To say of an individual’s
mental states that they have intentionality is to say that they are mental representations
or that they have contents". The encyclopedia is quick to inform us that intentionality,
which is centrally about the ability to point at specific mental objects and states, is not the same
thing as intention. But the concepts seem fairly related? For example if we ask
"Did ChatGPT just lie to me?"
the question of intent to lie hinges on representation: Did or did the model not have the
right answer in mind and then based on that representation choose to tell me something
other than what it knew to be true? Intension is
not the same thing as intention, but having things in mind seems like a basic requirement to
have intentions towards them.

Consider some common questions we ask each other about our minds:

Are you thinking what I'm thinking?

Do you want the blue car or the red car?

Did she mean to do that?

What's on your mind? What are you thinking about right now?

Are you paying attention? Can you tell me what I just said?

All of these are premised on the idea that we have minds and the minds represent 'things'
such that we can form preferences, shared understanding, and goals about the things.
Most people would find this so obvious and take it so deeply for granted that the idea
of having to say it out loud is silly. Of course minds exist and represent things, everybody
knows that. Unless they're a behaviorist
of course, then they actually might not. Behaviorism is the position that inner mental
states either don't exist or it is most productive to study psychology as though they
don't exist. Mercifully most behavorists are of the methodological type: They acknowledge
inner states and representations exist, but argue they can't be the subject of science
because we have no access to them. Most peop...
Features and Adversaries in MemoryDT
by Joseph Bloom — 2023-10-20T07:32:21.091Z

Keywords: Mechanistic Interpretability, Adversarial Examples, GridWorlds, Activation Engineering

This is part 2 of A Mechanistic Interpretability Analysis of a GridWorld Agent Simulator

Links: Repository, Model/Training, Task.

Epistemic status: I think the basic results are pretty solid, but I’m less sure about how these results relate to broader phenomena such as superposition or other modalities such as language models. I've erred on the side of discussing connections with other investigations to make it more obvious how gridworld decision transformers may be useful.

TLDR

We analyse the embedding space of a gridworld decision transformer, showing that it has developed an extensive structure that reflects the properties of the model, the gridworld environment and the task. We can identify linear feature representations for task-relevant concepts and show the distribution of these features in the embedding space. We use these insights to predict several adversarial inputs (observations with “distractor” items) that trick the model about what it is seeing. We show that these adversaries work as effectively as changing the feature (in the environment). However, we can also intervene directly on the underlying linear feature representation to achieve the same effects. Whilst methodologically simple, this analysis shows that mechanistic investigation of gridworld models is tractable and touches on many different areas of fundamental mechanistic interpretability research and its application to AI alignment.

We recommend reading the following sections for readers short on time:

Read the Introduction sections on the task and observation embeddings.
Read the section describing extraction of the instruction feature via pca.
Read the results sections describing using adversaries to change the instruction feature and comparing adversaries to direct intervention.

Key Results

Object Level Results

We show that our observation space has extensive geometric structure.

We think this structure is induced by properties of experimental set up (partial observations), architectural design (compositional embedding schema) and the nature of the specific RL task.
The learned structure included the use of clustered embeddings and antipodal pairs.
We see examples of isotropic and anisotropic superposition.

We identify interpretable linea

...
AI Safety Hub Serbia Soft Launch
by DusanDNesic — 2023-10-20T07:11:48.389Z
Link: https://forum.effectivealtruism.org/posts/7gL7CFBmybjAjJvAw/ai-safety-hub-serbia-soft-launch

TLDR; We got three-month funding from a generous individual funder to launch an AI Safety office in Serbia. We are giving free office space (and, if funding later permits, housing) to AI Safety researchers who are looking for a place to work from. Priority for citizens of countries like Russia and China who can work visa-free from Serbia while being close to Europe.

Register interest here

or ask questions at

my email

. We also have a promise of partial funding for our bare-bones request (

budget

) from an individual donor but are looking for a second individual donor in order to fulfill it (about 30k USD) -

reach out to me

if this may be you.

Background:

EA Serbia and AI Safety Serbia groups are small but growing (~30 people in EA Serbia, ~3 people looking to get into AIS research as a career, and ~3 to get into AIS policy). Due to Serbia’s favorable Visa policy towards Russia and China, many foreigners already live here. With lower living costs than many other international hub cities, a vibrant scene, and a favorable time zone and climate, Belgrade has a growing foreigner community.

As we have seen projects such as

CEEALAR

as important and impressive, we wish to replicate them in Serbia, where they can better serve people who may struggle to get UK visas. We also believe that having the capacity to quickly scale cheap housing for people coming from different countries is a good thing.

We also believe that we should start small, prototype, and then move larger. We have had a unique opportunity to get an office space that is NGO-friendly, has good vibes, and costs only ~550 Euros per month for office space that has three rooms and can fit 8-15 people (depending on how snug they decide to be) with a coffee shop downstairs where another 20 people can spend their time co-working, as the office and the downstairs coffee shop are under the same ownership. This is certainly less luxurious than many other EA/AI co-working places, but we have a high degree of customizability allowed to us, which we can use to make a good office space. If we grow enough, we can also move to a bigger venue, as our needs grow. Certainly, if we knew that more use could be found in an office in Serbia, getting something somewhat further from the center, which includes living and office spaces, would be better, but we do not wish to explore that until we have proof of concep...
Announcing new round of "Key Phenomena in AI Risk" Reading Group
by DusanDNesic — 2023-10-20T07:11:09.360Z

TLDR: “Key Phenomena in AI Risk” is an 8-week-long, facilitated reading group. It is aimed at people interested in conceptual AI alignment research, in particular from fields such as philosophy, systems research, biology, cognitive and social sciences. We ran it once and are repeating it now.

The program will run between November 2023 and January 2024. Sign up here by Sunday, October 29th.

What?

The “Key Phenomena in AI risk” reading curriculum provides an extended introduction to some key ideas in AI risk, in particular risks from misdirected optimization or 'consequentialist cognition'. As such, it aims to remain largely agnostic of solution paradigms. It includes 90' minutes of facilitated discussion, and requires at least 2 hours of reading per session. It is virtual and free.

See

the old post here

for a short overview of the curriculum;

here

for a more extensive summary; and

here

for the full curriculum (which will be updated in minor ways in the following weeks).

What Changed?

Thanks to the feedback from participants and facilitators in the last iteration, the program has improved. Now, it is is an 8-week-long program (with one week added at the end for reflection). Readings have been made more focused, and we will be adding more technical optional readings.

For Who?

The curriculum is primarily aimed at people interested in conceptual research in AI risk and alignment.

It is designed to be accessible to audiences in, among others, philosophy (of agency, knowledge, power, etc.) and systems research (e.g. biological, cognitive, information-theoretic, social systems, etc.).

When?

The reading groups will be taking place November 2023 through January 2024.

We expect to run 6 groups of 4-8 participants (including 1 facilitator). Each group will be led by a facilitator with substantive knowledge of AI risk.

Sign up

Sign up

here

by October 29th.

About the application

The application consists of one stage, where we ask you to fill in a form with

Your CV
Your motivation for participating in the program
Your prior exposure to AI risk/alignment to date

We select people based on our best understanding of their motivation to contribute to AI alignment and how much they would counterfactually benefit from participating in the program.

If you have any questions, feel free to leave a comment below or contact us at...
Unpacking the dynamics of AGI conflict that suggest the necessity of a premptive  pivotal act
by Eli Tyre — 2023-10-20T06:48:06.765Z

[Semi-half baked. I don’t reach any novel conclusions in this post, but I do flesh out my own thinking on the way to generally accepted conclusions.

I’m pretty interested in anything here that seems incorrect (including in nit-picky ways), or in hearing additional factors that influence the relevant and importance of pivotal acts.]

In

AGI Ruin: A List of Lethalities

, Eliezer claims

2. A cognitive system with sufficiently high cognitive powers, given any medium-bandwidth channel of causal influence, will not find it difficult to bootstrap to overpowering capabilities independent of human infrastructure. The concrete example I usually use here is nanotech, because there's been pretty detailed analysis of what definitely look like physically attainable lower bounds on what should be possible with nanotech, and those lower bounds are sufficient to carry the point. My lower-bound model of "how a sufficiently powerful intelligence would kill everyone, if it didn't want to not do that" is that it gets access to the Internet, emails some DNA sequences to any of the many many online firms that will take a DNA sequence in the email and ship you back proteins, and bribes/persuades some human who has no idea they're dealing with an AGI to mix proteins in a beaker, which then form a first-stage nanofactory which can build the actual nanomachinery….The nanomachinery builds diamondoid bacteria, that replicate with solar power and atmospheric CHON, maybe aggregate into some miniature rockets or jets so they can ride the jetstream to spread across the Earth's atmosphere, get into human bloodstreams and hide, strike on a timer. Losing a conflict with a high-powered cognitive system looks at least as deadly as "everybody on the face of the Earth suddenly falls over dead within the same second".

And then later,

6. We need to align the performance of some large task, a 'pivotal act' that prevents other people from building an unaligned AGI that destroys the world. While the number of actors with AGI is few or one, they must execute some "pivotal act", strong enough to flip the gameboard, using an AGI powerful enough to do that. It's not enough to be able to align a weak system - we need to align a system that can do some single very large thing. The example I usually give is "burn all GPUs"

...

Many

...
Genocide isn't Decolonization
by robotelvis — 2023-10-20T04:14:07.716Z
Link: https://messyprogress.substack.com/p/genocide-isnt-decolonization

In 1987, Bono, the lead singer of U2, interrupted his performance of “

Sunday Bloody Sunday

” to give a speech about

The Troubles

, prompted by the

Enniskillen bombing

that had taken place that day:

And let me tell you something,
I've had enough of Irish Americans,
Who haven't been back to their country in 20 or 30 years.
Come up to me, and talk about the resistance
The revolution back home
And the glory of the revolution
And the glory of dying for the revolution

Fuck the revolution!

They don't talk about the glory of killing for the revolution
What's the glory in taking a man from his bed
And gunning him down in front of his wife and children
Where's the glory in that?
Where's the glory in bombing a remembrance day parade
Of old age pensioners their medals taken out and polished up for the day
Where's the glory in that?
To leave them dying, or crippled for life, or dead
Under the rubble of a revolution
That the majority of the people in my country
Do not want

The Enniskillen bombing killed eleven people, a drop in the ocean compared to the 1400 Israeli civilians brutally murdered (at last count) by Hamas on October 7th. But I feel this speech captures something important that is relevant to this current moment.

I was shocked by what Hamas did, but what shocked me more was how many people in the west were eager to cheer for what Hamas had done, or were unwilling to condemn it. Hamas’s massacre of jewish civilians was praised by multiple

campus student groups

,

DSA chapters

,

BLM chapters

, and

Democrat Groups

. Lest you think this was just a fringe opinion,

notoriously censorious

Harvard University decided this was the one topic they were going to take a

free speech stance

on.

These groups justified their support for Hamas killing innocent civilians by claiming it was justified by “decolonization” - that the Jewish people are “colonizers” in ancestral Muslim land and “resistance is justified when people are occupied.”

But this is a very new meaning of the word “decolonization”.

Decolonization, as defined by the

UN’s 1960 declaration

is the “right of all people to self determination” - a rejection of the colonial model where a few central powers conquered distant lands and governed them without regard for the interests of their citizens. It is part of a broader set of principles like sovereign borders, human rights, rejection of ethnic cleansing, and a taboo against harming civ...
Trying to understand John Wentworth's research agenda
by johnswentworth — 2023-10-20T00:05:40.929Z

Oli, you wanna kick us off?

johnswentworth

I mostly plan to observe (and make edits/suggestions) here, but I'll chime in if I think there is something particularly important being missed.

David Lorell

Ok, context for this dialogue is that I was evaluating your LTFF application, and you were like "I would like to do more research on natural abstractions/maybe release products that test my hypotheses, can you give me money?".

I then figured that given that I've been trying to make dialogues happen on LW, we might also be able to create some positive externalities by making a dialogue about your research, which allows other people who read it later to also get to learn more about what you are working on.

I've generally liked a lot of your historical contributions to a bunch of AI Alignment stuff, but you haven't actually written much about your current central research agenda/direction, which does feel like a thing I should understand at least in broad terms before finishing my evaluation of your LTFF application.

My current strawman of your natural abstraction research is approximately "John is working on ontology identification/manipulation, just like everyone else these days". By which I mean, it feels like the central theme of Paul's research, a bunch of the most interesting parts of Anthropic's prosaic steering/interpretability work, and some of the more interesting MIRI work.

Here are some things that I associate with this line of research:

Trying to understand the mapping between concepts that we have and the concepts that an AI would use in the process of trying to achieve its goals/reason about the world
Trying to get to a point where if we have an AI system that arrives at some conclusion, that we have some model of "what reasoning did it very roughly use to arrive at that conclusion"
A major research problem that keeps coming up in this space is "ontology combination", like if I have one argument that uses an ontology that carves reality at joints X, and another argument that carves reality at joints Y, how do I combine them? This feels like one of the central components of heuristic arguments, and also is one of the key components of cartesian frames.

But I am not super confident that any of this matches that closely to the research you've been doing. I also haven't had a sense that people have been making tons of progress here, and at l

...
Boost your productivity, happiness and health with this one weird trick
by ajc586 — 2023-10-19T23:30:54.734Z

Thanks to a little luck + good genes + some means, you had a reasonably happy childhood, graduated from college, and ended up with a job that you're good at. You like the work you do (most of the time), because people like doing things they're good at. And you also work a lot of hours, because people find it easy to spend a lot of time doing things they like.

And because you work a lot of hours, you'll be pretty far to the right on the transfer function curve (x-axis time, y-axis total work output) where the gradient - the marginal return in work output for the time you spend - is rather flat, if you're honest about it.

Yes, for some activities (like competitive swimming), diminishing returns are still worthwhile because small differences in performance have an outsized impact on outcome. But this probably isn't true for you. Instead of spending 10, 12, or 14 hours a day coding, with just a smidge of willpower you could drop that to 8, 10, or 12, and no-one around you would notice the difference. You'll still be a 10X developer, if you were beforehand. You'll still hit the ball out of the park in your performance reviews.

And then, you redeploy those 2 hours per day to other things where you're much further to the left on the transfer function curve, like starting a side project, taking up new hobbies, or spending quality time with your children.

And because you're now spending more of your time on the left of the transfer function curve where the gradient Δwork/Δtime is much steeper, your total productivity will increase. And you'll be healthier and happier, too.

About 20 years ago I began applying this principle, starting with becoming very mindful on where I really was on the transfer function curve in each part of my daily life, and ultimately making significant time reallocations as a result. And indeed, it had a transformative impact on my overall productivity, happiness and health. Yet almost everyone I know spends so much of their time on the flat part of the transfer function curve. Why?
A Good Explanation of Differential Gears
by Johannes C. Mayer — 2023-10-19T23:07:46.354Z
Link: https://youtu.be/67XoCMTcN7M?si=fcAbTL5r125Ypc9Q&t=116

There is a very good video from 1937 by Jam Handy. It explains why we need differential gears and how they work.

I think the video is a masterpiece of an explanation. I recommend you watch it, not because understanding differentials is essential (though that doesn't hurt), but because the video can teach you a lot about how to explain something properly.

Here are some techniques the video uses (non-exhaustive):

Start with giving the context of why the thing that is being explained is important, e.g. by describing a concrete problem to which the thing to be explained is the solution.

Use visual explanations. Specifically, demonstrate the mechanism to be explained in action.

Start to explain a simple toy model and then add layers of complexity until you arrive at the real deal.

I think I have never seen an explanation, that by its end has annihilated the possibility of misunderstanding so effectively, and I really like how smooth the transition is from background story to technical explanation. There is no jarring break.