trentmkelly/LessWrong-43k huggingface dataset:
Realistic near-future scenarios of AI doom understandable for non-techy people?
by RomanS — 2023-04-28T14:45:16.791Z

I would like to compile a list of the AI doom scenarios that most people (especially politicians) will understand, and will agree that the scenarios are realistic and facts-based. A few examples:

There are AIs that help to invent new chemicals. For example, Alphafold helps with designing new proteins. Such AIs can be used to design and improve biological weapons (e.g. by making Ebola even more deadly). The North Korea, ISIS, or even some resourceful loner could use AI to create a virus capable of killing billions of people.
Several major countries are working on automating their militaries. For example, a US defense contractor Palantir has announced a system ("Palantir AIP") that automates a lot of high-level military decision making with AI. China is working on similar systems. If the trend continues, AI will penetrate all levels of the military command. More and more decisions will be delegated to AI. But even the smartest AIs are not error-free. A trusted AI that seems to be reliable in most situations, could make a deadly mistake in an unusual situation. This could lead to all sorts of dangerous scenarios, including an avalanche-like escalation between the American and the Chinese "automatic generals", greatly increasing the risk of a nuclear war.
Social networks, targeted ads, and chatbot trolls are already used to sow division, to promote radical ideologies, and to help dangerous populists to win elections. The smarter are AIs, the more efficient is their usage to manipulate the public opinion. This makes it easier for radical movements to gain traction, and for insane people to become presidents. The next bin Laden and the next Hitler will gain power thanks to AI. And this time it will be much easier for them to develop weapons of mass destruction (e.g. AI-designed bio weapons).

What are some other such scenarios?
The Social Alignment Problem
by irving — 2023-04-28T14:16:17.825Z

TLDR: I think public outreach is a very hopeful path to victory. More importantly, extended large-scale conversation around questions such as whether public outreach is a hopeful path to victory would be very likely to decrease p(doom).

You’re a genius mechanical engineer in prison. You stumble across a huge bomb rigged to blow in a random supply closet. You shout for two guards passing by, but they laugh you off. You decide to try to defuse it yourself.

This is arguably a reasonable response, given that this is your exact skill set. This is what you were trained for. But after a few hours of fiddling around with the bomb, you start to realize that it's much more complicated than you thought. You have no idea when it’s going to go off, but you start despairing that you can defuse it on your own. You sink to the floor with your face in your hands. You can’t figure it out. Nobody will listen to you.

Real Talking To The Public has never been tried

Much like the general public has done with the subject of longevity, I think many people in our circle have adopted an assumption of hopelessness toward public outreach social alignment, before a relevant amount of effort has been expended. In truth, there are many reasons to expect this strategy to be quite realistic, and very positively impactful too. A world in which the cause of AI safety is as trendy as the cause of climate change, and in which society is as knowledgeable about questions of alignment as it it about vaccine efficacy (meaning not even that knowledgeable), is one where sane legislation designed to slow capabilities and invest in alignment becomes probable, and where capabilities research is stigmatized and labs find access to talent and resources harder to come by.

I’ve finally started to see individual actors taking steps towards this goal, but I’ve seen a shockingly small amount of coordinated discussion about it. When the topic is raised, there are four common objections: They Won’t Listen, Don’t Cry Wolf, Don’t Annoy the Labs, and Don’t Create More Disaster Monkeys.

They won’t listen/They won’t understand

I cannot overstate how clearly utterly false this is at this point.

It’s understandable that this has been our default belief. I think debating e/accs on Twitter has broken our brains. The experience of explaining again and again why something smarter than you that doesn’t care about you is...
PhD Position: AI Interpretability in Berlin, Germany
by Tiberius — 2023-04-28T13:44:25.448Z
Link: https://stephanw.net/news/ausschreibung/

We are seeking a highly motivated and skilled research assistant to join our Research Team at IOL Lab at Zuse Institute Berlin. The successful candidate will work closely with our team on cutting-edge research projects in the field of Numerical Optimisation and Explainable AI.

Someone with a maths, theoretical physics or computer science background is preferred, but we are willing to consider anyone who can credibly demonstrate good reasoning and programming skills.

Apply till 14th of May or until fitting candidate is found

Broad Research Direction

We want to build a theory that guarantees Interpretability (e.g. in the form of information bounds) for modern AI systems. Can we play AI agents against each other, so they are forced to cooperate with humans? Is it possible that AI systems prove the soundness of their reasoning to us? Can we give interpretations in text form? How robust are these approaches? Do the explanations degenerate when the AIs are incentivised to obscure their reasoning? Can we use a form of error-correction to prevent this?

Check out this paper to see how this might look like.

Requirements

Completed university degree (Diplome, Master or equivalent) in Mathematics, Computer Science, Theoretical Physics or a closely related course of studies,

Solid knowledge of probability theory

Good programming skills (Python)

Knowledge in computational complexity theory, graph theory and game theory are an advantage

Fluent in English (German is NOT required)

Independent thinker

Funding

The position is part of the MATH+ project “Expanding Merlin-Arthur Classifiers - Interpretable Neural Networks through Interactive Proof Systems”. This research project is part of the Emerging Fields Area “Extracting dynamical Laws from Complex Data”. MATH+, the Berlin Mathematics Research Center, is a cross-institutional and interdisciplinary Cluster of Excellence. It sets out to explore and further develop new approaches in application-oriented mathematics. For more information, see: https://mathplus.de.

Write me (waeldchen@zib.de) for further information. If you know a candidate that might be a good fit for this position, please forward them this announcement.

Please Include

How you heard about this position

Link to your website, previous projects or papers

Whether you can physically be present at our Berlin office

What your interests are and what motivates you for thi

...
A Clearer Thinking tool that teaches you to use Internal Family Systems concepts
by spencerg — 2023-04-28T13:42:35.063Z
Link: https://programs.clearerthinking.org/internal_family_systems.html

One of our recently-released Clearer Thinking tools, Have Better Conversations with Yourself, walks you through key concepts from Internal Family Systems (IFS). While the ideas of IFS shouldn't be taken literally, many people seem to find that IFS provides useful metaphors and techniques for self-exploration and self-change (which is why we made this tool). If you aren't already acquainted with IFS, we hope you'll find it useful to learn about these ideas in a structured way (our tool helps you apply them to your own situation)! People who are already familiar with IFS might also find it useful to refresh their memory of the concepts.

The tool starts by inviting you to describe an internal conflict (e.g., a situation where different "parts" of you feel like they want opposite things). It then aims to help you to get in touch with your different mental parts (or subagents) and develop a more positive relationship with them. We hope that this will help you to:

• Get more clarity on internal conflicts

• Make progress on difficult problems and dilemmas

• Have more self-acceptance

If you use the tool, we'd love to hear your feedback!

A big shout-out goes to Amber Dawn Ace for developing the tool.
cyberpunk raccoons
by bhauth — 2023-04-28T02:52:20.473Z
Link: https://bhauth.com/blog/industrial%20design/cyberpunk%20raccoons.html

There are many stories about supervillains that have a secret base that manufactures something and is mostly self-contained, which would require an entire parallel economy. Sometimes, like in James Bond stories, there are human workers living there. The trend now is towards having such secret bases be fully automated, with a human leader controlling many robots. That avoids questions like "How were all those people recruited?" and "Is it really OK for the heroes to just kill all these people?"

The real economy isn't self-sustaining without human workers. Robotic arms are often cheaper than a human worker now, but jobs that would be done by robotic arms in America might, in lower-income countries, be done by humans instead. The more capabilities of humans you use, the more expensive the alternative gets.

Modern robots require frames, joints, actuators, electric motors, motor drivers, a power source, and control. The bearings and actuators require lots of small parts with fine tolerances. The drivers require power semiconductors. Making those components requires large machines that are amortized over years. Also, like cars or other complex machines, some parts eventually wear out and some maintenance is needed.

It would be much easier if you could just grow the robots in a vat using nanotech. Or maybe they could have equiment for making more.

This is obviously leading into "That's a description of humans!!" but there are some problems with that. The easiest way to get a bunch of humans would be to recruit them, and I'm pretty sure you could find a million people willing to move to some giant compound run by Dr Doom or even GLaDOS if the rent and food was free. But such recruitment would be highly visible, and you can't just feed those people nutrient paste and store them in capsule hotels.

If humans were raised from birth in an isolated environment, that wouldn't be visible from the outside, and they would put up with cheaper living conditions. But humans take a long time to mature, and don't reproduce very quickly. Humans take months to gestate and years to reach maturity, and these days, ain't nobody got time for that, apparently.

Some fiction has clone production facilities, where large numbers of humans are grown in vats quickly. Horses are full-grown in 2 years, so clearly, the maturation speed of humans is limited by brain development rather than body growth. ...
AI safety logo design contest, due mid-May
by Adrian Cipriani — 2023-04-28T02:00:19.210Z
Link: https://forum.effectivealtruism.org/posts/RPpAYgDF2NEFuvgwg/ai-safety-logo-design-contest-due-mid-may

Please have a look at the updated version of this post on the EA Forum.

This contest is about creating a logo to represent AI safety research.

I’m being supported by an independent funder and people from

nonlinear.org

and am taking care of administration and communication.

We’re doing an open process (voting) and want to involve different people through the contest to get a better logo design and to increase the chances of adoption.

1) rough concept ideas contest

deadline: May 15th 2023

We’re looking for different quick and great design ideas for this round. Not everything has to be spot on yet, this will be done in 2) and 3) .

Submit as many rough logo design ideas as you want as a .PNG and .SVG file to this Google Form:

https://forms.gle/6fwVYbPBrqVdCHax6

2) feedback & voting on best design ideas chosen by jury

May 20th - Juni 3rd 2023 (2 weeks)

Via another Google Form, this post will be updated with the link once there’s a selection by the jury of 5 - 10 contest submissions available.

Only accounts with at least 100 Karma on

forum.effectivealtruism.org

or

lesswrong.com

are allowed to vote.

With a $ 1.5k prize pool for the best design ideas from the contest which are being used further, this might be one design or multiple as decided by the votes & the jury, with the prize money spread out evenly.

We’re open for someone to increase the prize pool, please send me an email!

3) polishing of winning design(s)

We’ll hire a great designer for creating the final logo from the winning design(s).

After the completion of the contest the final logo will be publicly available for free use at

www.aisafety.art

.
If you have any questions please reach out to me at ai_logo@cipriani.studio

-----------------------------------------------------------------------------------------------------

I’d like to disclose here that I’m also trying to enter an idea myself, however I won’t be able to decide on my own which designs are considered for polishing.

Composition of the jury: two people from Nonlinear, an independent funder, me (Adrian Cipriani)
Dropout can create a privileged basis in the ReLU output model.
by lewis smith — 2023-04-28T01:59:36.410Z

Abstract

This post summarises some interesting results on dropout in the ReLU output model. The ReLU output model, introduced here is a minimal toy model of a neural network which can exhibit features being in 'superposition', rather than a single 'neuron' (dimension) being devoted to each feature. This post explores what effects dropout has on the alignment of the basis in an otherwise rotation invariant model.

This particular question was listed on Neel Nanda's 'Concrete problems in mechanistic interpretability' Sequence. I work in machine learning and while I've always thought that mech-interp was an interesting direction, it seems particularly more urgent recently for reasons which I think will be clear to anyone reading LW. I originally started playing with this question because I was considering applying to this ; I no longer think I will do so this year for personal reasons, but I thought that my results were interesting enough to be worth writing up. I have never posted on LW before, but as the concrete problems list was posted here I thought it was a better place to publish a write-up, rather than my personal blog which I don't think gets very much visibility.[1]

Introduction

The ReLU output model is a very simple model of a mapping from neural network 'features' to concrete 'hidden dimensions', which can exhibit superposition. The model is as follows, for a vector of 'features' x∈Rn and a hidden vector h∈Rm, with n>>m.

h=Wxx′=ReLU(WTh+b)

This model is of interest as an extremely simple test case of how a model can represent a large set of (hypothetical) disentangled 'features' in a much lower dimensional vector space by placing those vectors into 'superposition'. This model, and observations of superpositions in it, were first introduced and studied in this article.

This model is rotation invariant in the sense that W can apply an orthonormal matrix O to the weights without changing anything; x′=ReLU(WTOTh+b)=ReLU(WTOTOWx+b)=ReLU(WTWx+b). Problem 4.1 on Nanda's 'Concrete Problems' sequence asks whether dropout breaks this symmetry and introduces a privileged basis in this model.

We can define a ReLU dropout model;
h=Wxh′=dropout(h,r)x′=ReLU(WTh′+b)
where r is the dropout rate; that is, the elements of the hidden variable h are uniformly set to zero with probability r.

In this post, I present some empirical observations that dropout can create a...
Podcast with Divia Eden and Ronny Fernandez on the strong orthogonality thesis
by DanielFilan — 2023-04-28T01:30:45.681Z
Link: https://youtu.be/D5rEMNyfIWw

On my side podcast, "The Filan Cabinet", I invited Ronny Fernandez and Divia Eden to talk about the strong orthogonality thesis, and whether it's true. Seems like people here might also be interested. Podcast description below, and you can listen here.

In this episode, Divia Eden and Ronny Fernandez talk about the (strong) orthogonality thesis - that arbitrarily smart intelligences can be paired with arbitrary goals, without additional complication beyond that of specifying the goal - with light prompting from me. Topics they touch on include:

Why aren't bees brilliant scientists?

Can you efficiently make an AGI out of one part that predicts the future conditioned on some plans, and another that evaluates whether plans are good?

If minds are made of smaller sub-agents with more primitive beliefs and desires, does that shape their terminal goals?

Also, how would that even work?

Which is cooler: rockets, or butterflies?

What processes would make AIs terminally value integrity?

Why do beavers build dams?

Would these questions be easier to answer if we made octopuses really smart?
Simultaneous Footbass and Footdrums
by jefftk — 2023-04-28T00:50:01.697Z

For the past year or so I've had two options for my feet with my

rhythm stage setup. I can play bass:

Or I can play drums:

But when I play mandolin I want both!

I tried a system where I play bass but whichever drum pedal I tap
triggers a kick as well, which sort of works but I'm not good enough
at playing a consistent rhythm while moving between pedals:

This almost works, but I couldn't get it to where I was happy with it.

Another thing I tried was pulling my hand away from the mandolin
briefly to change the current bass note:

This works on that particular tune which isn't very notey and it
happens that each time I want to change to a new bass note the melody
asks for an open string, but it doesn't work in general.

A few days ago I got another idea: allocate the left two pedals to
drums (kick and snare) and the right two pedals to bass. Since this
gives me three options: right heel, right toe, and both together.
Since three chords is (sort
of) all
you need, this offers an interesting balance. Currently I have it
set so I can choose between major (I IV V), minor (i bVII bVI), and
mixolydian (I bVII V).

I'm still getting it into my muscle memory, but here's how it sounds
so far:

Comment via: facebook, mastodon
Tuning your Cognitive Strategies
by Raemon — 2023-04-27T20:32:06.337Z
Link: http://bewelltuned.com/tune_your_cognitive_strategies

The blogpost author (SquirrelInHell on LessWrong) died awhile ago. I'm not sure who's currently paying for their website or how long it'll be up. I don't have the rights to this, but decided it was important enough to have on LessWrong that I decided to copy-paste this post and... I dunno, own whatever karmic debt I incur.

This is possibly my single-favorite rationality technique. The first day I tried this I immediately ended up teaching myself a valuable rationality-life-lesson due to the feedback loop it created. When I teach this technique at small workshops, typically ~25% of people go "oh wow that was immediately helpful." I haven't gotten as much value out of it as SquirrelInHell suggests (i.e. it's sometimes effortful to think, and they claim if you're doing it right it basically shouldn't be), but I also haven't really sat and trained it deliberately in-depth, and meanwhile I've gotten value from it each time I try it.

Text of original article:

Tuning Your Cognitive Strategies

What do you get out of it?

The good.

Better returns on thinking time.

Your cognition is much more powerful than just the part you have conscious access to, and it's crucial to make good use of it.

A small tweak to how your brain processes information in general is worth more than a big upgrade to your conscious repository of cognitive tricks.

Goal-oriented thinking.

When working on real-life problems, your peak performance matters less than the ability to simply think useful thoughts at all.

For example, if your current top priority is "start my own company", but you keep having insights about "what I'll say to my current boss when I finally quit"... that's maybe not the best way to make progress.

Improved ability to fix

cognitive biases

.

To the extent that other approaches work, it's because they manage to change your cognitive strategies. It's much easier when you know what you are doing.

More creativity and good ideas just "popping into your head".

There's no magic to it! Once you understand how the process works, it can be optimized for any purpose you choose.

Less anxiety about performing well in cognitive endeavors.

Once you realize exactly what is and what isn't under your conscious control, you stop beating yourself about not doing the impossible.

The bad.

Uncanny valley.

Most people already have a thinking style built on top of excessive conscious cognitive effort.

This often involves rel

...
The LW crossroads of purpose
by Caerulea-Lawrence — 2023-04-27T19:53:58.311Z

Hello everyone reading,
I'm writing this because I do not believe fixing peripheral things on LW is enough. And even though I wrote about it in a comment, I wanted to elaborate and highlight the issue through writing a post about it too. Even though this is not nearly as succinctly written as many other authors here, I hope it can still serve its purpose as a call to action.
Even when the mods and users are engaged and participating, what I see in this current situation, is the underlying vagueness in purpose and identity. Issues that seem to have started way before my time (LW 2.0), are resurfacing, with the AI-user influx fanning the flames. I believe the crisis can be resolved, but it needs acknowledgment of the gravity of the situation and the necessary laborious work to fix it.
LW stands at a crossroads. Ahead lies clarification of essence, identity and focus.
At the crossroads
Firstly, what is the explicit mission of LW? Its identity?

LessWrong is a community dedicated to improving our reasoning and decision-making. We seek to hold true beliefs and to be effective at accomplishing our goals. More generally, we want to develop and practice the art of human rationality.

To that end, LessWrong is a place to 1) develop and train rationality, and 2) apply one’s rationality to real-world problems.

Of course, there is also the history part about this being an initiative of Eliezer Yudkowsky, with a major focus being on AI alignment.
This purpose, however, doesn't specify for whom, at which level of rationality, towards what kind of complexity level of problems, or similarly relevant specifications. Which also makes it really hard to say if LW is really successful or not, at least with regard to its own purpose.
The road not to take
When I look at LW's identity, I am fearful of this lack of clarity, precision and focus. Without acknowledging the different roads, I dread seeing LW turn gradually into something like 'Hurdal Ecovillage' in Norway.
Hurdal Ecovillage started out as the poster child of Norwegian sustainable and ecological living. There were some concerns, but it got press, monetary support and focus, not only in Norway but internationally as well.
Now you find the original homepage hijacked by a new resident, and even the house-builders bankrupt. It is still operating, somewhat, but even on the Hurdal Ecovillage Facebook ...