trentmkelly/LessWrong-43k huggingface dataset:
Great Empathy and Great Response Ability
by positivesum — 2023-11-13T23:04:20.111Z
Link: https://tryingtruly.substack.com/p/with-great-empathy-comes-great-response-ability

As kids, long before our frontal lobes have finished developing, we’ve all had the instinct to tease. Just watch two toddlers interact, you can see the malicious glint in their eye as they discover that calling one of their peers a “poopy head” triggers a devastating emotional response.

Believe it or not, that glint is actually one of the first seeds of empathy.

They’re beginning to learn about the inner world of the person in-front of them, what buttons can be pushed there, and discovering that at the end of the day they’re not so different themselves (since after the inevitable retaliation, they promptly learn that they do not enjoy being on the recieving end of “No you’re a poopy head!” either).

Now might be a good time to extend a personal appeal not to tease me for the title of this post. I took a high risk for no discernible reward.

I’ve met too many high functioning sociopaths in sales, finance, and life not to notice how good they usually are at reading the person infront of them. The fact that they’re calculated and self-serving doesn’t mean that they aren’t good at empathising with whoever is standing before them - on the contrary, they’re highly proficient in it.

Trouble is, they don’t hesitate to exploit their skills ruthlessly to their own advantage.

When discussing emotional intelligence, Empathy tends to be divided into two categories:

Affect (Emotional) Empathy: The visceral, emotional, part of caring. Usually including a measurable physiological response (eg. watching a video of someone getting hurt increases heart rate, crying when sad, experiencing cringe).

Cognitive Empathy: The analytical ability to identify and understand what a person infront of you is feeling and why - without necessarily feeling it yourself.

As I’m sure you can intuitively guess, high scores on the usual Dark Triad traits tend to be associated with low Affect Empathy. However! Cognitive Empathy tends to remain intact even among people with Antisocial Personality Disorder, with some studies even suggesting Narcissism and Machiavellianism can be positively correlated to it.

When a savvy banker finessess a small business owner into signing a loan with exploitative terms, then peacefully goes home to a good night’s sleep - it’s exactly this mental imbalance that’s being played out.

HOWEVER! Lack of Affective Empathy does not inherently make you a bad person. As with any other human...
Theories of Change for AI Auditing
by Lee Sharkey — 2023-11-13T19:33:43.928Z
Link: https://www.apolloresearch.ai/blog/theories-of-change-for-ai-auditing

Executive summary

Our mission at Apollo Research is to reduce catastrophic risks from AI by auditing advanced AI systems for misalignment and dangerous capabilities, with an initial focus on deceptive alignment.

In our announcement post, we presented a brief theory of change of our organization which explains why we expect AI auditing to be strongly positive for reducing catastrophic risk from advanced AI systems.

In this post, we present a theory of change for how AI auditing could improve the safety of advanced AI systems. We describe what AI auditing organizations would do; why we expect this to be an important pathway to reducing catastrophic risk; and explore the limitations and potential failure modes of such auditing approaches.

We want to emphasize that this is our current perspective and, given that the field is still young, could change in the future.

As presented in ‘A Causal Framework for AI Regulation and Auditing’, one of the ways to think about auditing is that auditors act at different steps of the causal chain that leads to AI systems’ effects on the world. This chain can be broken down into different components (see figure in main text), and we describe auditors’ potential roles at each stage. Having defined these roles, we identify and outline five categories of audits and their theories of change:

AI system evaluations assess the capabilities and alignment of AI systems through behavioral tests and interpretability methods. They can directly identify risks, improve safety research by converting alignment from a “one-shot” problem to a "many-shot problem" and provide evidence to motivate governance.
Training design audits assess training data content, effective compute, and training-experiment design. They aim to reduce risks by shaping the AI system development process and privilege safety over capabilities in frontier AI development.
Deployment audits assess the risks from permitting particular categories of people (such as lab employees, external auditors, or the public) to use the AI systems in particular ways.
Security audits evaluate the security of organizations and AI systems to prevent accidents and misuse. They constrain AI system affordances and proliferation risks.
Governance audits evaluate institutions developing, regulating, auditing, and interacting with frontier AI systems. They help ensure responsible AI development and use.

I...
They are made of repeating patterns
by quetzal_rainbow — 2023-11-13T18:17:43.189Z

Epistemis status: an obvious parody.

— You won't believe me. I've found them.

— Whom?

— Remember that famous discovery by Professor Prgh'zhyne about pockets of baryonic matter in open systems that minimize the production of entropy within them? They went further and claimed that goal-oriented systems could emerge within these pockets. Crazy idea, but... it seems I've found them near this yellow dwarf!

— You're kidding. We know that a good optimizer of outcomes over systems' states should have a model of the system inside of itself. We have entire computable universes within ourselves and still barely make sense of this chaos. How can they fit valuable knowledge inside tiny sequences of 1023 atoms?

— They repeat patterns of behavior. They have multiple encodings of them and slightly change them over time in response to environmental changes in a simple mechanistic way.

— But that generalizes horribly!

— Indeed. When a pattern interacts with a new aspect of the environment, it degrades with high probability. Their first mechanism for generating patterns was basically "throw a bunch of random numbers in the environment, keep those that survived, slightly change, repeat".

— ...

— Yeah, it's horrible from their perspective, I think.

— How do they exist without an agent-environment boundary? I'd be pretty worried if some piece of baryonic matter could smash into my thoughts at any moment.

— They kind of pretend they have an agent-environment boundary, using lipid layers.

— Those "lipid layers" have such strong bonds that they don't let any piece of matter inside? That's impressive!

— No, I was serious about them pretending. They need to pass matter through themselves; they're open systems and can't survive without external sources of free energy. They usually have specialized members of their population, an "immune system", that checks for alien patterns.

— Like we check for signatures of malign hypotheses in the universal prior?

— No, there's not enough computing power. They just memorize a bazillion meaningless patterns, and the immune system kills everyone who can't recite them.

— WHAT? But what if the patterns are corrupted, as happens in the world of baryonic matter?

— You can guess: if your memory of the patterns is corrupted, you're dead.

— What if the reference pattern of immune system gets corrupted?

— Then the immune system starts to kill indiscriminately.

—...
How to Upload a Mind (In Three Not-So-Easy Steps)
by aggliu — 2023-11-13T18:13:32.893Z
Link: https://youtu.be/LwBVR68z-fg

Cross-posted to the EA forum

This Rational Animations video is about the research and practical challenges of "whole brain emulation" or "mind uploading", presented as a step by step guide. We primarily follow the roadmap of Sandberg and Bostrom's 2008 report, linked in the notes. The primary scriptwriter was Allen Liu (the first author of this post), with feedback from the second author (Writer), other members of the Rational Animations team, and outside reviewers including several of the authors of the cited sources. Production credits are at the end of the video. You can find the script of the video below.

So you want to run a brain on a computer. Luckily, researchers have already mapped out a trail for you, but this won’t be an easy task. We can break it down into three main steps: First, getting all the necessary information out of a brain; Second, converting it into a computer program; and third, actually running that program. So, let’s get going!

Our goal is to build a computer system that acts the same way a brain does, which we call a “whole brain emulation”. Emulation is when one computer is programmed to behave exactly like another, even if it's using different hardware. For instance, you can emulate a handheld game console on your computer, and play games made for the real console on the emulated version. Similarly, an emulation of a human brain - or maybe the whole central nervous system - would be able to think and act exactly like a physical person. Alan Turing showed in the 1930s that any computer that meets certain requirements, including the one you’re using to watch this video, can in principle emulate any other computer and run any algorithm, given enough time and memory.[1] Assuming the brain fundamentally performs computations, then our goal is at least theoretically achievable. To actually emulate a human brain, we’ll follow the roadmap given by Anders Sandberg and Nick Bostrom in 2008.[2] Crucially, we don’t need to fully understand every aspect of the brain in order to emulate it, especially hard philosophical problems like consciousness.

But knowing it's possible is one thing - implementation is another. Our first challenge will be to get the information we need from a human brain. Researchers aren’t yet sure what level of detail we’ll need, but research on small animals suggests we’ll at least need to map all th...
Non-myopia stories
by  — 2023-11-13T17:52:31.933Z

Written under the supervision of Lionel Levine. Thanks to Owain Evans, Aidan O’Gara, Max Kaufmann, and Johannes Treutlein for comments.

This post is a synthesis of arguments made by other people. It provides a collection of answers to the question, "Why would an AI become non-myopic?" In this post I’ll describe a model as myopic if it cares only about what happens in the current training episode.[1] This form of myopia is called episodic myopia. Typically, we expect models to be myopic because the training process does not reward the AI for outcomes outside of its training episode. Non-myopia is interesting because it indicates a flaw in training – somehow our AI has started to care about something we did not design it to care about.

One reason to care about non-myopia is that it can cause a system to manipulate its own training process. If an ML system wants to affect what happens after its gradient update, it can do so through the gradient update itself. For instance, an AI might become deceptively aligned, behaving as aligned as possible in order to minimize how much it is changed by stochastic gradient descent (SGD). Or an AI could engage in exploration hacking, avoiding certain behaviors that it does not want to engage in because they will be rewarded and subsequently reinforced. Additionally, non-myopic AI systems could collude in adversarial setups like

AI safety via debate

. If debates between AI systems are iterated, they are analogous to a prisoners dilemma. If systems are non-myopic they could cooperate.

This post will outline six different routes to non-myopia:

Simulating other agents. Models could simulate humans or other non-myopic agents and adopt their non-myopia.
Inductive bias toward long-term goals. Inductive Biases like simplicity might favor non-myopic goals.
Meta-learning. A meta-learning loop can select for non-myopic agents.
(Acausal) trade. An otherwise myopic model might behave non-myopically by trading with other AI models.
Implicitly non-myopic objective functions. Objective functions might incentivize myopia by depending on an estimate of future consequences.
Non-myopia enables deceptive alignment. Becoming non-myopic could make a model become deceptively aligned and lead to higher training reward.

Running example: The stamp collector

This post uses a stamp collecting AI as a running example. Th...
It's OK to eat shrimp: EAs Make Invalid Inferences About Fish Qualia and Moral Patienthood
by Mikhail Samin — 2023-11-13T16:51:53.341Z

I've had conversations with many EAs and EA-adjacent people who believe things about qualia that seem wrong to me. I've met one who assigned double-digit probabilities to bacteria having qualia and said they wouldn't be surprised if a balloon flying through a gradient of air experiences pain because it's trying to get away from hotter air towards colder air. Some say they see shrimp have pain receptors and clearly react to "pain"[1], just like humans, and try to avoid future "pain", so they're experiencing pain, and we should care about their welfare. (A commenter says they think any visual information processing is qualia to some extent, even with neural networks[2].)

I think the way they're making these inferences is invalid. In this post, I'll try to explain why. I'll also suggest a direction for experiments that could produce valid evidence one way or the other.

Epistemic status: Having disentangled the models some people had, I'm relatively confident I see where many make invalid inferences as part of their worldviews. But I'm not a biologist, and this is not my area of expertise. A couple of people I talked to agreed with a suggested experiment as something that would potentially resolve the crux.

I'm using the word "qualia" to point at subjective experience. I don't use the word "consciousness" because different people mean completely different things by it.

I tried to keep the post short while communicating the idea. I think this is an important conversation to have. I believe many in the community make flawed arguments and claim that animal features are evidence for consciousness, even though they aren't.

TL;DR: If a being can describe qualia, we know this is caused by qualia existing somewhere. So we can be pretty sure that humans have qualia. But when our brains identify emotions in things, they can think both humans and geometric shapes in cartoons are feeling something. I argue that when we look at humans and feel like they feel something, we know that this feeling is probably correct, because we can make a valid inference that humans have qualia (because they would talk about having conscious experiences). I further argue that when we look at non-human things, our circuits' recognition of feeling in others is no longer linked to a valid way of inferring that these others have qualia, and we need other evidence.

No zombies among humans

We are a c...
Suggestions for chess puzzles
by Zane — 2023-11-13T15:39:37.968Z

(You can sign up here if you haven't already.)

As suggested by Richard Willis in the comments, I want to present the "player As" with a series of individual isolated chess puzzles, for which the advisors will anonymously present their suggestions. However, there are constraints on what sort of puzzles we would be focusing on.

The position must have one best move that is strictly superior to all the others, ideally such that it determines the outcome of the game. Positions with concrete tactical solutions are often easy to verify a solution to, so focusing on more positional moves would be better. (Although a few tactical puzzles could be sprinkled in too, with no label as to which was which.) Furthermore, the positions should be easy enough that the advisors (on average roughly the equivalent of 2000 USCF) should ~always be able to determine the answer, but hard enough that a less skilled player would be entirely unable to solve it.

I have some resources for finding puzzles, but I would appreciate any suggestions that would fit these constraints. Please do not put suggestions in the comments, because comments are public and can be seen by the As. PM me on LessWrong or email me at zaneglowfic@gmail.com instead.

(Also, since I don't know where else to say this: someone named "bruce" responded to the Google Form and said they could be contacted by "email," but the Google Form does not record emails. If you are Bruce, please email me (at zaneglowfic@gmail.com.))
Why small phenomenons are relevant to morality
​
by Ryo  — 2023-11-13T15:25:46.738Z

Follow-up to this post
One thing that seems important to me is that optionality implies a sort of high-level game theory :

It is good that we are in a complex environment with many phenomenons even when it seems that those phenomenons aren't directly relevant (to prevent threats from our blind-spots).
This is a precautionary principle.
"What is apparently meaningless now can be meaningful in a future space-time context"
The ethical system needs to be formalized in a way that procedurally includes all phenomenons

So that it would scale and fit with the events and variables of our complex reality

A proof of the importance of phenomenal diversity can be ie. that being in a universe with many solar systems is good because it enhances the possibility of life.

So diversity of phenomenons is in itself an ethical subgoal,
Blooming the phenomenons that allow the most diversity of other phenomenons is good

(But we need to define complexity/diversity, I'll extend on that at the end)

Although those far-away/small phenomenons should not be the main concern, it should always be part of the equation
The issue is not simply to make AI care about what we want, but about what "we would like to want if we add perfect knowledge about both our ideal values, and the consequences of our actions"
cf -> .

Coherent Extrapolated Volition

One reason I'm talking about small/far phenomenons is to foresee the possibility of rogue AI

If the fundamental layer is about phenomena diversification, ecosystemic value etc.
-> Even a rogue AI can't maximize paperclips

Because the procedure would inherently maximize "diversity" instead

If you need to define something to a superhuman AI you have to put a definition that takes everything into account,

You have to really define optionality in the less underfitting formula possible (an underfitted definition could be interpreted in unforeseen ways)

An AI that has just [human optionality] in its definition (or in its limited halo of attention) may be harmful for wildlife in unforeseen ways, which then would be bad for us (and is bad in itself).
I'm trying not simply to assume "ASI will be non-myopic in a good way"

Because this AI/AGI/ASI didn't took the time and energy to dive in certain

irreducible

calculations you have to make in order to really care about natural ecosystems

Similarly if you ...
Optionality approach to ethics
by Ryo  — 2023-11-13T15:23:20.648Z

Hey,
When thinking about ethical questions, especially in the face of strong AI optimization forces, I wonder if some of you might have considered a version of

utilitarianism

which instead of trying to maximise nebulous things like happiness or utility, (whatever those might be—the difficulty of defining which is somewhat part of the problem; and

wireheading

is another), tries to maximise 'optionality', i.e. number of choices available to humans, and also not just humans.

This could potentially solve lots of long-standing thorny problems in consequentialism, like wireheading and the fiendish difficulty of defining happiness/utility, and how even making the tiniest of mistake in that definition can be precisely catastrophic.

While I do think that optionality is more definable than utility, it's still not trivial. I have ideas on how to calculate it, but not full clarity yet. I'm reaching out to find more people who have thoughts in this direction: are there some of you who might already believe that the greatest good might come from giving the most amount of (meaningfully different) choices to agents?
The limit is that those choices (ideally) have to not destroy meaningful choices of other agents. After which point they can choose for themselves however they fit.
And always increasing this optionality, preempting lock-ins.
(Maybe this ethical system or something similar already has a name? Do let me know!)

Some of my important references :
.

The Capability Approach to Human Welfare

.

Paretotopia

"For Paretotopian Goal Alignment, a key concept is Pareto-preferred futures, meaning futures that would be strongly approximately preferred by more or less everyone."

.

The Hippocratic principle - Vanessa Kosoy

"I propose a new formal desideratum for alignment: the Hippocratic principle. Informally the principle says: an AI shouldn't make things worse compared to letting the user handle them on their own, in expectation w.r.t. the user's beliefs. This is similar to the dangerousness bound I talked about before, and is also related to corrigibility."

.

Learning Altruistic Behaviours in Reinforcement Learning without External Rewards

"We propose to act altruistically towards other agents by giving them more choice and allowing them to achieve their goals better. Some concrete examples include opening a door for others or safeguarding ...
Redirecting one’s own taxes as an effective altruism method
by David Gross — 2023-11-13T15:17:12.637Z

About twenty years ago, I stopped paying U.S. federal income taxes. By law, the government has ten years to collect an unpaid tax bill, whereafter a sort of statute of limitations kicks in and the bill becomes permanently noncollectable. I’ve adopted the practice of waiting out this ten-year period and then donating the amount of the uncollected tax to charity, typically the Top Charities Fund organized by GiveWell.[1] Over the past six years I’ve redirected over $30,000 from the U.S. Treasury to charity in this way.

In this post I’ll briefly outline the theory and practice of this sort of tax redirection, and address some likely objections. If you have questions about the nitty-gritty details, leave them in the comments or drop me a line by email.

Theory

From an effective altruism perspective, the theory behind tax redirection is that giving money to the government is far from the best way you could deploy that money. It is questionable whether funding the government is even a net positive: worse than merely wasteful and inefficient, the government is often harmful. But even if you believe that marginal funding of the government is more good than bad, it is almost certainly not among the best ways you could allocate your money.

So if you could avoid paying federal taxes and give that money instead to more well-chosen causes, in a frictionless way, it would seem wise to do so (from an effective altruism standpoint). But of course such a move is not frictionless: the government disincentivizes some varieties of tax redirection with threats of sanctions, and other varieties of tax redirection have their own costs.

So you have to factor in those costs before you can decide if tax redirection would be a good option for you. But to many people, tax redirection is in the “unthinkable” category, and so they dismiss the option before actually weighing the costs and benefits. If you have been among these people, I hope this post will encourage you to move tax redirection from “unthinkable” to “let me think about that for a moment.”

The theory and practice of tax redirection in the U.S. has been developed largely by pacifist “war tax resisters”, who redirect their federal taxes because of conscientious objection to funding war.[2] Their belief that funding the government is indeed immoral led them to desperately seek alternatives. But those alternatives, having be...