trentmkelly/LessWrong-43k huggingface dataset:
Trigger-Action Planning
by CFAR!Duncan — 2022-07-03T01:42:22.083Z

Epistemic status: Established and confirmed

There has been a tremendous amount of research on “implementation intentions” since their development by psychologist Peter Gollwitzer in the late 1990’s. A meta- analysis of 94 studies involving 8461 participants found that interventions using implementation intentions were an average of .65 standard deviations more effective than control interventions. Similar effect sizes were found in the 34 studies which looked at behavioral change on personal or health goals (average of .59 standard deviations more effective). Trigger-action planning—our version of implementation intentions—draws directly on this research and has proven useful to the majority of our alumni for a wide range of problems, tasks, and goals.

In previous sections of this book, we’ve looked at the differences between System 1 and System 2, talked about the process of turning goals into plans, and learned to distinguish useful and relevant practice from irrelevant or unproductive practice. In this section, we will combine those insights and their implications into a single, robust technique for building awareness and supporting behavioral change.

Complex chains: The parable of the Sphex

Sphexes are a genus of wasps, and for many years, a story about their behavior has been a major touchstone in cognitive science. Typically, when it comes time for egg laying, a sphex will build a burrow and fill it with paralyzed insects for her future larvae to eat. When hunting, she will sting her prey, wait for the venom to take effect, drag the prey back to the burrow entrance, leave it outside while she goes in and reconnoiters (presumably confirming the absence of predators or structural problems), and finally come back out to drag her victim inside.

This sequence of actions is elaborate, organized, and complex, and on the surface seems to indicate an impressive level of mental sophistication for an insect whose brain weighs less than a milligram. However, in 1879, French entomologist Jean Henri Fabre decided to dig deeper:

I will mention an experiment...at the moment when the Sphex is making her domiciliary visit, I take the cricket left at the entrance to the dwelling and place her a few inches farther away. The Sphex comes up, utters her usual cry, and...comes out of her hole to seize it and bring it back to its right place. Having done this, she goes down again, b

...
Which one of these two academic routes should I take to end up in AI Safety?
by Martín Soto — 2022-07-03T01:05:23.956Z

I've finished two Bachelor's in Maths and Physics with moderately good grades, but a fairly advanced Thesis (and advanced learning) in Mathematical Logic. I've recently learned about the ethical urgency of AI Safety research. With the prospect of getting into that field in the near future (probably along the lines of a Theoretical Research Lead), I now have to face a career decision, and I'd be really thankful if anyone familiar with the academic field of AI research could share their thoughts. My two options are the following:

Taking the Master of Pure and Applied Logic in Barcelona. This Master's is almost exclusively pure maths, but I'm pretty sure I'd be able to obtain excellent grades.
Undertaking a 3-year PhD in Mathematical Logic in Vienna. The research would again be in some specific areas of pure maths (Recursion Theory, Proof Theory, Set Theory), and I'm pretty certain (but not as much) that I'd be able to obtain good results.

My initial idea was the Master's might be a better choice for ending up in AI Safety since it leaves open the possibility of later undertaking a PhD closer to AI, and I'd probably be considered a valuable student if I obtain excellent grades. But on the contrary, obtaining a PhD in pure maths being as young as I am might provide even more status. Furthermore, I'm not certain the best way to enter AI academia is necessarily by undertaking an AI-related PhD.
Naive Hypotheses on AI Alignment
by Shoshannah Tekofsky — 2022-07-02T19:03:49.458Z

Apparently doominess works for my brain, cause Eliezer Yudkowsky’s

AGI Ruin: A List of Lethalities

convinced me to look in to AI safety. Either I’d find out he’s wrong, and there is no problem. Or he’s right, and I need to reevaluate my life priorities.

After a month of sporadic reading, I've learned the field is considered to be in a state of

preparadigmicity

. In other words, we don’t know *how* to think about the problem yet, and thus novelty comes at a premium. The best way to generate novel ideas is to pull in people from other disciplines. In my case that's computational psychology: modeling people like agents. And I've mostly applied this to video games. My Pareto frontier is "modeling people like agents based on their behavior logs in constructed games created to trigger reward signals + ITT'ing the hell out of all the new people I love to constantly meet". I have no idea if this background makes me more or less likely to generate a new idea that's useful to solving AI alignment, but the way I understand the problem now: everyone should at least try.

So I started studying AI alignment, but quickly realized there is a trade-off: The more I learn, the harder it is to think of anything new. At first I had a lot of naive ideas on how to solve the alignment problem. As I learned more about the field, my ideas all crumbled. At the same time, I can't really assess yet if there is a useful level of novelty in my naive hypotheses. I'm still currently generating ideas low on "contamination" by existing thought (cause I'm new), but also low on quality (cause I'm new). As I learn more, I'll start generating higher quality hypotheses, but these are likely to become increasingly constrained to the existing schools of thought, because of cognitive contamination from everyone reading the same material and thinking in similar ways. Which is exactly the thing we want to avoid at this stage.

Therefore, to get the best of both worlds, I figured I'd write down my naive hypotheses as I have them, and keep studying at the same time. Maybe an ostensibly "stupid" idea on my end, inspires someone with more experience to a workable idea on their end. Even if the probability of that is <0.1%, it's still worth it. Cause, you know, .... I prefer we don't all die.

So here goes:

H1 - Emotional Empathy

If you give a human absolute power, there is a small subset of human...
The Tree of Life: Stanford AI Alignment Theory of Change
by Gabe M — 2022-07-02T18:36:10.964Z

Update 2023-06-15: Stanford AI Alignment has just wrapped up its first (IMO quite successful) academic year, but it has diverged significantly from some of the plans first laid out here. For updated context, contact me over DM or at gmukobi@cs.stanford.edu.

Crossposted on the Effective Altruism Forum.

0. Preface

Summary: AI alignment needs many great people working on

research

,

engineering

,

governance

,

social science

,

security

,

operations

, and more. I think

AI alignment-focused university groups

are uniquely underexploited sources of that future talent. Here are my plans for running such a group at Stanford, broadly involving mass outreach to a wide range of students, efficient fit-testing and funneling to a core group of promising students, and robust pipelining of those students into a connected network of impactful opportunities.

Confidence: Moderate. This theory of change is heavily inspired by similar plans from

Michael Chen

(

Effective Altruism at Georgia Tech

) and

Alexander Davies

(

Harvard AI Safety Team

) as well as EA field-building learnings from

Stanford Effective Altruism

. While parts of this have been tested in those communities, parts are new or different. That said, I’m fairly confident this is at least a significant improvement over current local AI alignment field-building operations. The purpose of posting this is both to refine these plans and to share ideas that might be useful for groups with similar goals.

Terminology: I use the term “AI alignment” as an umbrella term throughout this to refer not only to technical AI alignment/existential AI safety, but also to AI governance, AI strategy, and other work that directly reduces existential risk from AI. The specific choice of this term (as opposed to e.g. “AI safety”) is to practically distinguish the long-term/catastrophic risk/alignment focus of the new group from the adjacent

Stanford Center for AI Safety

.

1. Introduction

1.1 Context

Stanford Effective Altruism has gradually been increasing its focus on AI alignment over the last year. It invited more AI-related guests to open Monday dinners, ran events like an

ELK

weekend and a

Rob Miles

binge party, organized one and a half alignment-focused retreats, and ran a local

AGI Safety Fundamentals

Technical Alignment reading group in the spring. From these activities, it saw some promising signs, like:

Many people the organizers had never met before at

...
Follow along with Columbia EA's Advanced AI Safety Fellowship!
by RohanS — 2022-07-02T17:45:47.120Z
Link: https://forum.effectivealtruism.org/posts/orovMJA48HtkuQphN/follow-along-with-columbia-ea-s-advanced-ai-safety

Columbia EA is starting an Advanced AI Safety Fellowship this summer, which will involve one 3-hour meeting per week with a group of people seriously considering careers in AI safety who have some background in both ML (at least working knowledge of linear algebra, multivariable calculus, probability and stats, and neural networks) and AI safety (usually the

Cambridge AGI Safety Fundamentals program

or a similar reading group we run at Columbia). Each meeting will involve reading about a topic selected the previous week and discussing it, and may also include exercises (like

these

). While we do not have capacity to invite people from elsewhere to join our meetings, we would like to invite others to follow along with our exploration of the field! You can find our running meeting notes

here

, where we will write down our meeting agendas, readings, and other notes (maybe key takeaways, responses to exercises, questions, etc.).

The hope is that this can serve as a small, easy win, by motivating at least a few people to set aside time to dive deep into AI alignment topics with little additional effort on our part. Participating in an alternative program with a similar premise, in which the organizers put more effort into making it a good experience for you, would likely be better than following along with us. If such a program already exists, please let me know! I’m not currently aware of any programs that satisfy all of the following properties:

Have the basic premise of being a deep dive into AI safety for people with some background in ML and AI safety who are seriously considering careers in the field
Are open to anyone with that background
Have organizers dedicated to making this a good experience for many interested people from various places

Given that we are not satisfying 3, you could dispense with “following along” and just structure your entire plan yourself if you feel that would work better for you. There are two main benefits we hope to provide for people following along:

Reducing the amount of time you spend selecting readings / exercises.
Providing an external anchor that you are motivated to keep up with.

We don’t have any strong takes on what it would look like to follow along with us, but here is a suggested path:

Comment on this post that you are interested in following along - if others have also expressed interest, consider forming a group with them s

...
Welcome to Analogia! (Chapter 7)
by Justin Bullock — 2022-07-02T17:04:01.400Z

Hey, what's that sound?

"It's time we stop
Hey, what's that sound?
Everybody look, what's going down?
Paranoia strikes deep
Into your life it will creep
It starts when you're always afraid
Step out of line, the men come and take you away
We better stop
Hey, what's that sound?
Everybody look, what's going down?
You better stop
Hey, what's that sound?
Everybody look, what's going down?
You better stop
Now, what's that sound?
Everybody look, what's going down?
You better stop
Children, what's that sound?
Everybody look, what's going down?“

-Stephen Stills “For What It’s Worth”

Welcome to Analogia

The time has come for a more proper introduction to Analogia. To do so, we turn from the mind of Aldous Huxley to the minds of several contemporaries including George Dyson, Mark Fisher, and James Bridle. It is Dyson who, in 2020, introduces and describes the concept of Analogia itself in his beautiful book "Analogia: The Emergence of Technology Beyond Programmable Control". For Dyson, Analogia is a world in which digital machines are outgrowing their digital world and have inserted themselves more fully into our analog world. In this world, worse still, the digital machines themselves have reached a level of complexity and a type of form that, as the title suggests, they have moved beyond programmable control. For Dyson, the Internet is the most clear example of digital machines, at large scale, asserting control over the analog world, and that the behavior of the internet itself has moved beyond programmable control.

James Bridle uses slightly different language but points to a similar phenomenon. For Bridle, in "The New Dark Age" the difficulty is in the complexity of the Network. Bridle drives home the example of the Internet. "The greatest signifying quality of the network is its lack of single, solid intent. Nobody set out to create the network, or its greatest built exemplar, the internet." For Bridle, the Internet is the premier example of the Network. Bridle also points to the use of clouds as a common metaphor used by engineers to illustrate and abstract from complexity of the network. But, in something of a twist, Bridle argues that we can use this same cloud metaphor to navigate the new dark age.

As a gentle reminder, let's look at the first two sentences that describe the Internet on wikipedia:

"The Internet is the global system of interconnected computer networks that uses the In...
What about transhumans and beyond?
by AlignmentMirror — 2022-07-02T13:58:53.492Z

When it comes to AGI alignment, or just the future in general, it seems many people think in a way that revolves all around humans as we know them today.
Take the recent Kurzgesagt video "The Last Human – A Glimpse Into The Far Future" for instance.
While it does very briefly mention the possibility of genetic engineering, it doesn't go into detail and clearly considers the human descendants to still continue with rather similar properties as today's humans, since it is calculating a number of hypothetical human births.
If it would for example consider the possibility of human-like artificial minds, which could hypothetically be created more quickly and efficiently, that could dwarf the number of humans given the same resources.

So is this human purist view not rather myopic in the long run?

There of course also are humans that would eagerly accept deeply invasive modifications of their bodies, as long as those modifications appear sufficiently beneficial to them.
Over time such biotechnological modifications could result in transhumans that barely resemble today's humans, effectively creating a very different, even "alien", lifeform.

Or what if some transhumans eventually wish to transfer or connect or recreate their minds in some kind of virtual world system, which may or may not also allow control of "bodies" in the outside world?
Would you consider such a mind any less of a person because it can be easily copied, or because it is not restricted to a specific human body, or for some other reason?

The modifications could eventually go so far as to create transhumans on par with a possibly already existing aligned AGI.

Of course transhumans and human purists could ostensibly coexist even in extreme modification scenarios. But perhaps the human purists would try to forcefully prevent a greater extent of transhumanism out of fear, or perhaps the transhumans would fight the purists for resources, who knows.

What do you think?
Goal-directedness: tackling complexity
by Morgan_Rogers — 2022-07-02T13:51:07.853Z

This is the third post in my Effective-Altruism-funded project aiming to deconfuse goal-directedness. Comments are welcomed. All opinions expressed are my own, and do not reflect the attitudes of any member of the body sponsoring me.

My strategy for achieving a formalisation of goal-directed behaviour is to equate it with "behaviour which is well-explained in terms of goals". So far, I have explored criteria for judging explanations in general and a structured way to decompose explanations of agent behaviour specifically.

One of those criteria was simplicity, or its dual, complexity. In order to determine whether my initial proposal for characterizing goal-directedness holds water, I need to get my hands dirty and learn some complexity theory. This post is a record of me doing that, while keeping my objective in sight. I have learned a lot of complexity theory/theories since I wrote my original post, so the present post is a major update from the somewhat naïve intuition I expressed there.

See the end for a few references not directly linked in the main text.

Setting the scene

In my initial breakdown of explanations, I explicitly assumed that an individual explanation could be assigned a value representing its complexity. Deeper investigation revealed at least four points where complexity might be introduced into an explanation. At the time, I wondered what measure of complexity I should use, expecting to be able to pick one (or several) up out of the box, and get down to the business of applying already-developed theory.

In attempting to apply the various complexity measures to my explanations as I was learning about them, I found each of them to be inadequate in one way or other. For example, Kolmogorov complexity and various branches of computational complexity explicitly involve a model of computation, such as a (choice of) Turing machine, but the explanations involved in goal-directedness have no obvious ties to computation, and any choice seems arbitrary. Moreover, a lot of the problems discussed in the various branches of complexity theory seem to be about sequences of problems or structures, whereas I am trying to attach complexity to individual explanations!

While I may end up adapting some existing theory, my advisor Adam Shimi pointed out that rather than trying to jumble existing measures together in an ad hoc way, I should try to understand the underly...
Literature recommendations July 2022
by ChristianKl — 2022-07-02T09:14:28.400Z

It happens frequently, that I'm researching a topic and don't really know which sources to read to get a good grasp of the topic. I expect that many fellow aspiring rationalists feel the same.

This thread is a place to list topics for which you are looking for reading recommendations.

Please open one comment per topic that you want recommendations about.

If you want to discuss the merits of this thread, please do it as a reply to the meta-comment.
Deontological Evil
by lsusr — 2022-07-02T06:57:18.085Z

It surprised Doctor Susan Connor how quickly she had gotten used to her job. At first she had qualms working for an organization literally called "Effective Evil" (with two capital 'E's). But her job just gave her so much autonomy, complexity and reward. Also nemeses.

"I'll never talk," said the spy chained to her interrogation rack.

"Technically you just did," said Doctor Susan Connor.

The spy closed his mouth.

"Let's drop the charade," said Susan, melodramatically. Susan liked melodrama. There were so few opportunities for melodrama at her previous position at a postdoc. "You're an enemy agent. The question is: where from? Perhaps you're from the Daoist Destructivists? No. They act without acting and your action is quite obviously the regular sort."

"Ha! The problem with you folks at Effective Evil is that you're incapable of coordinating with the villains you should be allies of. That's the problem with taking utilitarianism too far. You have lost sight of the categorical imperative," said the spy.

A third person entered the room. Douglas Morbus was tall and wore a spotless white labcoat. "That's exactly the kind of thing an agent of Deontological Evil would say. Lies for the sake of lies," said Morbus.

"You are familiar with our ways," said the spy, "What I don't comprehend is how anyone with a moral compass could work for Effective Evil."

"I donate 20% of my income to animal rights organizations," said Susan

"That's not what I mean," said the spy, "I mean how can a villain work for Effective Evil? You make moral compromises left and right. You've spent so much time optimizing 'evil' you've forgotten what evil even is."

Morbus fiddled with the surgical torture implements. "Moral compromises are necessary to operating at scale."

"Au contraire," said the spy, "To operate at the greatest scale you need everyone in the organization on the same page. You cannot micromanage everyone. They must believe in an ideal."

"That sounds like a utilitarian argument," said Susan, "Are you not a true deontologist?"

"The competent philosopher knows his enemies' arguments better than his enemies do," said the spy.

"None of this makes any sense," said Susan, "Deontology is a coordination mechanism. You can't just invert it the way you can invert utilitarianism. Deontological ethics are asymmetric with respect to coordination. Good deontologists can coordinate. Evil deontologists cannot."

"...
Could an AI Alignment Sandbox be useful?
by Michael Soareverix — 2022-07-02T05:06:09.979Z

As a newer member of the community here on LessWrong and a VR game developer, I feel that there is a lot of text and theory about alignment and not a lot of practical experience.

To remedy this, I've been considering building an AI alignment sandbox in Unity. The simulation would contain a miniature simulated civilization, with miniature simple agents trying to maximize simple rewards (gathering food, spending time with other agents, etc).

A more complex model could be imported into this simulation, with given goals/rewards/control elements. For example, maybe it is a traffic AI, and wants to minimize the time that pedestrians spend in traffic. Depending on the control elements, the AI could simply delete the cars or the pedestrians, showing very clearly that it is unaligned with the smaller agents in the simulation.

I think this could serve two purposes. One, it would very clearly show that AI converges to unaligned objectives when it is more capable than smaller agents. Secondly, it could give researchers some real-world experience with trying to align AI. As Eliezer said in his AGI ruin post, humans learn best from experience, so if we could go back four years every time unaligned AI destroyed the world, we could solve alignment with relative ease.

I do see a few problems with this idea, however. Firstly, there's the classic 'superintelligent AI will operate in ways that are fundamentally different from basic AI'. If we put in a normal neural network system, the results might be predictable. Of course it will delete the cars; that's an obvious optimization, you say. It isn't at a level where it can philosophize about the meaning of a car or where it can change its own reward function like we expect a true AGI might.

Additionally, a basic AI will only be as good as its inputs. If the model can read the smaller agents' reward states and is designed to align with them, then it seems obvious it will optimize in favor of that. But here on Earth, humans don't exactly have a number above their head showing a reward state for some AGI to read.

Despite those facts, I think creating an 'Alignment Sandbox' could be very valuable and I'd like your opinions on it. As a game developer, I would be happy to prototype this idea and potentially reach out to larger companies about it or collaborate with people here on it.

If we are to avoid AGI Ruin, we'll probably need some pract...