trentmkelly/LessWrong-43k huggingface dataset:
What success looks like
by Marius Hobbhahn — 2022-06-28T14:38:42.758Z
Link: https://forum.effectivealtruism.org/posts/AuRBKFnjABa6c6GzC/what-success-looks-like

TL;DR: We wrote a post on possible success stories of a transition to TAI to better understand which factors causally reduce the risk of AI risk. Furthermore, we separately explain these catalysts for success in more detail and this post can thus be thought of as a high-level overview of different AI governance strategies.

Summary

Thinking through scenarios where TAI goes well informs our goals regarding AI safety and leads to concrete action plans. Thus, in this post,

We sketch stories where the development and deployment of transformative AI go well. We broadly cluster them like

Alignment won’t be a problem, …

Because alignment is easy: Scenario 1
We get lucky with the first AI: Scenario 4

Alignment is hard, but …

We can solve it together, because …

We can effectively deploy governance and technical strategies in combination together: Scenario 2
Humanity will wake up due to an accident: Scenario 3
The US and China will realize their shared interests: Scenario 5

One player can win the race, by …

Launching an Apollo Project for AI: Scenario 6

We categorize central points of influence that seem relevant for causing the success of our sketches. The categories with some examples are:

Governance: domestic laws, international treaties, safety regulations, whistleblower protection, auditing firms, compute governance and contingency plans
Technical: Red teaming, benchmarks, fire alarms, forecasting and information security
Societal: Norms in AI, publicity and field-building

We lay out some central causal variables for our stories in the third chapter. They include the level of cooperation, AI timelines, take-off speeds, size of the alignment tax, type of actors and number of actors
Four reasons I find AI safety emotionally compelling
by KatWoods — 2022-06-28T14:10:35.216Z

‘I don’t feel emotionally motivated to work on AI safety, even though I’m intellectually convinced that it’s important.’

It always surprises me when people say this because I find my work at

Nonlinear

on AI safety incredibly motivating. I’m sharing my reasons in the hope that they’ll resonate with some of you, and that these ideas will help bring your emotional drives into greater harmony with your abstract convictions.

1. The dramatic scale of AGI is inspiring

When I was a kid, I wanted to save the world. Like many EAs, I was obsessed with stories of superheroes who could use their powers to save whole cities from catastrophe. I aspired to be like Gandhi, or Martin Luther King, and to do something really big and important; something that would forge widespread, lasting change. But as I got older, these dreams collided violently with the harsh reality: changing things is really really hard. I became more realistic and scaled down my ambitions.

I started off working on global health. In this area, the most I could hope for was (for example) to reduce infant mortality by a percent or two in a single country - and that’s if my career was an astonishing success. This is significant - that’s a lot of lives saved! - but it’s far more modest than my childhood ambitions. I resigned myself to the fact that while I could make significant improvements to the world, the problems we face would remain immense and oppressive.

Similarly, at one time, I felt most motivated to work on animal welfare. But again: even if I was unusually successful in my work, I could only hope to make a small dent in the problem. I might (for example) secure slightly better conditions for factory-farmed chickens in the US, but there would still be factory farms. And even if factory farms were abolished, what about the suffering of animals in the wild?

This situation was demotivating over time: the thought of so much intense pain was emotionally crushing, and I knew that even if I worked extremely hard and improved some animals’ lives, those improvements would be dwarfed by the suffering that remained.

With AGI, on the other hand, my ambitions to really change things have returned. AI is likely to change the world massively: for better or for worse. When we talk about AI, we often focus on its dangers: the fact that misaligned AI could kill us all or cause astronomical suffering. But there is a more po...
Some alternative AI safety research projects
by Michele Campolo — 2022-06-28T14:09:27.661Z

These are some "alternative" (in the sense of non-mainstream) research projects or questions related to AI safety that seem both relevant and underexplored. If instead you think they aren't, let me know in the comments, and feel free to use the ideas as you want if you find them interesting.

Access-to-the-internet scenario and related topics

A potentially catastrophic scenario that appears somewhat frequently in AI safety discourse involves a smarter-than-human AI which gets unrestricted access to the internet, and then bad things happen. For example, the AI manages to persuade or bribe one or more humans so that they perform actions which have a high impact on the world.

What are the worst (i.e. with worst consequences) examples of similar scenarios that already happened in the past? Can we learn anything useful from them?

Considering these scenarios, why is it the case that nothing worse has happened yet? Is it simply because human programmers with bad intentions are not smart enough? Or because the programs/AIs themselves are not agentic enough? I would like to read well-thought arguments on the topic.

Can we learn something from the history of digital viruses? What's the role played by

cybersecurity

? If we assume that slowing down progress in AI capabilities is not a viable option, can we make the above scenario less likely to happen by changing or improving cybersecurity?

Intuitively, it seems to me that the relation of AI safety with cybersecurity is similar to the relation with interpretability: even though the main objective of the other fields is not the reduction of global catastrophic risk, some of the ideas in those fields are likely to be relevant for AI safety as well.

Cognitive and moral enhancement in bioethics

A few days ago I came across

a bioethics paper

that immediately made me think of the relation between AI safety and AI capabilities. From the abstract:

"Cognitive enhancement [...] could thus accelerate the advance of science, or its application, and so increase the risk of the development or misuse of weapons of mass destruction. We argue that this is a reason which speaks against the desirability of cognitive enhancement, and the consequent speedier growth of knowledge, if it is not accompanied by an extensive moral enhancement of humankind."

As far as I understand, some researchers in the field are pro cognitive enhancement—sometime...
Doom doubts - is inner alignment a likely problem?
by Crissman — 2022-06-28T12:42:16.197Z

After reading Eliezer's list of lethalities, I have doubts (hopes?) that some of the challenges he mentions will occur.

Let's start with inner alignment. Let's think step by step. 😁

Inner alignment is a new name for a long-known challenge of many systems. Whether it's called the agency problem or delegation challenges, giving a task to another entity and then making sure that entity not only does what you want it to do but in a way that you approve of is something people and systems have been dealing with since the first tribes. It is not an emergent property of AGI that will need to be navigated from a blank slate.
Humans and AGI are aligned on the need to manage inner alignment. While deception by the mesa-optimizer ("agent") must be addressed, both humans and the AGI agree that agents going rogue to take actions that fulfill their sub-goal but thwart the overall mission must be prevented.
The AGI will be much more powerful than the agents. An agent will logically have fewer resources at its disposal than the overall system, and to provide the benefit of leverage, the number of agents should be significant. If there are a small number of agents, then their work can be subsumed by the overall system instead of creating agents which incur alignment challenges. Since there will be a large number of agents, each agent will have only a fraction of the overall system's power, which implies the system should have considerable resources available to monitor and correct deviations from the system's mission.
An AGI that doesn't solve inner alignment, with or without human help, isn't going to make it to super intelligence (SI). An SI will be able to get things done as planned and intended (at least according to the SI's understanding--not addressing outer alignment here). If it can't stop its own agents from doing things it agrees are not the mission, it's not an SI.

Does that make sense? Agree? Disagree?
Low-Friction MBTA Predictions
by jefftk — 2022-06-28T12:30:01.714Z

This has not been a good week for the MBTA:

The Red, Orange, and Blue lines are running Saturday
schedules on weekdays (news
coverage) until they are able to hire and train more dispatchers.

The Orange
and Green lines (again)
were suspended after problems with the ongoing demolition of a parking
garage above Haymarket Station.

The new Orange and Red Line cars were pulled
from service (again),
this time for a battery issue.

And, bonus, while the event was at the end of May, the MBTA released
information this week about Red Line cars that rolled out of the
Braintree yard, onto the mainline tracks and through the station.

Despite all of this, my wait times have gone down by >2x:

What's changed is that I've stopped thinking of the Red Line as being
so frequent that you just show up and get on. Now I look at the
real-time predictions and time my arrival for when the train is about
a minute away. This is a combination of doing extra things before
leaving to usefully fill the time (typically tidying) and/or running
to the station for a train I would otherwise miss. While I used to
wait ~3min on average, now I'm waiting ~1min.

A key thing here is to have an app that is really easy to check, so
you don't waste time checking that you could have spent getting to the
station. There are many different apps that display the MBTA data,
but I've been using MBTA
Tracker. I have two shortcuts on my home screen for the two ends
of my commute:

Tapping immediately opens predictions:

To put them on the homescreen I starred these stops in the app to mark
them as favorites, and then long-pressed the app icon. This popped up
a menu where I could drag the stops to the homescreen:

I do miss having my own predictions services, which used to be something I cared
about a lot, but at this point other people have released some pretty
good ones and I don't feel like I'd be adding much.
What Diet Books Don't Teach: A book review and a request for more reading
by Lone Pine — 2022-06-28T12:27:04.847Z

I’ve read a fair number of self-help books in my life. I’m long past the point where I expect a book to change my life, but they often are fun to pleasure-read. As Scott Alexander notes, these books all make claims they cannot fulfill. I don’t read so many diet books because they are the worst in the category. All diet books discuss what to eat but never how to get food. The underlying advice in all diet books, which is never elucidated beyond a sentence or two, is this: Cook for yourself, don’t eat at restaurants and don’t eat ‘processed’ food. All diet books divide food into good and bad, and recently they have begun to discuss when to eat (fasting). But they all assume that you can cook for yourself, can maintain a working kitchen, and can find and afford specific ingredients like grass-fed meat, clarified ghee and kale.

I say “all” diet books, but of course I’m making this post in hopes that there is an exception which you’ll inform me of. Please do. I will talk about what I actually need from a self-help book (which might not be a ‘diet’ book per-se.) But first, I will review the book Intuitive Fasting. The other night I bought this audiobook based on the title alone. I was hoping that it would talk about how to recognize, via body-awareness, when it would be better to skip a meal than to eat. This is something I was already doing to some extent. But that’s not what Intuitive Fasting is about. The book is only partially about fasting, contains a fair amount of general wellness advice like good sleep and meditation (which I appreciate), and has zero actual advice about how to develop your body-intuition. The book focuses on a specific 4-week fasting plan, starting with fasting 12 hours a day and progressing to fasting 22 hours every other day. The author refers to this plan as “flexible.”

Intuitive Fasting is actually a keto book in disguise. The author advocates for a diet he calls ‘ketotarian’ – keto and vegetarian. I don’t recall him ever explaining why a vegetarian diet is important, but presumably he’s reacting to all the studies that say that meat is unhealthy. (He cites a lot of studies.) Later in the book, he says that eggs and pole-caught wild fish are okay. Then he talks about chicken and beef. He says that “actually meat has gotten a bad rap.” So then chicken and beef are okay? Well, as long as it’s organic and grass-fed. Basically, the diet is...
Assessing AlephAlphas Multimodal Model
by p.b. — 2022-06-28T09:28:10.921Z

The currently probably most impressive large language model imho is Flamingo, which has the ability to answer questions about images. Flamingo is not accessible to the general public, however, the German startup AlephAlpha has a very similar model on offer.

I signed up and received 2 credits for free, which is apparently roughly 40 cents? (The pricing information is far from complete, so it seems one has to find out via trial and error what a specific usecase is going to cost.)

I decided to use my free credits to investigate the model a little bit. This is about as unscientific an investigation as it can get. When AlephAlpha logged me out, it probably reset the model to the base-model occasionally, which I didn't notice immediately, so I can't even say with certainty which model was used for which prompt.

Anyway, it still gives a flavor what these models can and cannot do, which I think is valuable information especially when it comes to the difference to future models.

First I presented it with this picture of 7 hedgehogs playing musical chairs (curtesy of Dalle2 via Swimmer963).

The model luminous-extended had no difficulty discerning the game:

Question: What are these hedgehogs playing?

Answer: Musical chairs.

Maybe it's not surprising that the prompt of generated pictures can be reverse engineered with such a model. After all the training data was probably very similar. However, dalle2 generally had trouble counting (the above was the only picture with the correct number of hedgehogs and chairs out of ten). So I asked:

Question: How many hedgehogs are in that picture?

Answer: Seven.

This surprised me intially. It also correctly counted 5 dogs and than badly failed at counting 11 dogs (6) and 8 dogs (4) and finally 4 cats (3) so ultimately it seems to be not much better at counting than dalle.

Then I tried out this dalle2 mishap (it's supposed to be an angel and a wild boar).

Building on an example prompt given by AlephAlpha I let the model complete:

If you see this picture, you really wonder if the artist is crazy.

Ok, makes sense, but I wanted to see whether luminous understands how strange the picture is, so I asked:

Question: What is weird about this picture?

Answer: The pigs

True, but unsatisfying. Let's try to zero in a bit:

Question: What is weird about the pigs in this picture?

Answer: They are not real.

Ok, I think we have to ask a very...
Is there any way someone could post about public policy relating to abortion access (or another sensitive subject) on LessWrong without getting super downvoted?
by Evan_Gaensbauer — 2022-06-28T05:45:17.831Z

I'm aware there are some sensitive subjects that have been debated to death among rationalists, if not on LessWrong, than elsewhere online, such that it would in general be fruitless to start the conversation over instead of following what has been debated thus far. I'm curious if this is considered to apply to potentially literally any single socially, culturally or politically sensitive subject. It has been several years since I've checked but my impression from before is that the prevailing norm on LessWrong was based on a particular interpretation of "Politics is the Mind-Killer" that amounts to "politics=anti-rational=downvote on site." This seems contradicted by the original post:

I’m not saying that I think we should be apolitical, or even that we should adopt Wikipedia’s ideal of the Neutral Point of View. But try to resist getting in those good, solid digs if you can possibly avoid it. If your topic legitimately relates to attempts to ban evolution in school curricula, then go ahead and talk about it—but don’t blame it explicitly on the whole Republican Party; some of your readers may be Republicans, and they may feel that the problem is a few rogues, not the entire party.

It seems like the alternative interpretation, though, was adopted because nobody stuck to that and so it was more convenient for reinforcing norms of civility in discourse to go with the "politics=anti-rational=downvote" interpretation.
Is that still a norm? Even if someone like myself posts about it in a way that doesn't suck, is the concern still that others will lower the quality of discourse with their comments, and downvote anyway? (I'm aware there are not formal rules but I'm in essence asking if according to social norms what categories of topics are considered forbidden by default.)
[Test Post Please Ignore] Testing polling features
by Lone Pine — 2022-06-28T04:35:09.467Z

I'm testing LW polling features. I haven't found a good guide online for the polling features. I tried googling, reading the FAQ and LW search. If anyone can point me in the right direction, I'd be very thankful.

[poll]{Option 1}{Option 2}{Option n}

[poll]{Red}{Green}{Blue}

[poll:Good..Bad]

[poll:Good...Bad]

[poll:Good....Bad]
[poll:Good.Bad]
Yann LeCun, A Path Towards Autonomous Machine Intelligence [link]
by Bill Benzon — 2022-06-27T23:29:55.384Z

Bill Benzon 5 min ago

For those who are interested, Yann LeCun has posted A Path Towards Autonomous Machine Intelligence:

Abstract: How could machines learn as efficiently as humans and animals? How could machines learn to reason and plan? How could machines learn representations of percepts and action plans at multiple levels of abstraction, enabling them to reason, predict, and plan at multiple time horizons? This position paper proposes an architecture and training paradigms with which to construct autonomous intelligent agents. It combines concepts such as configurable predictive world model, behavior driven through intrinsic motivation, and hierarchical joint embedding architectures trained with self-supervised learning.
Limits of Bodily Autonomy
by jefftk — 2022-06-27T19:50:01.813Z

Bodily autonomy is a strong consideration in favor of both abortion
and optional vaccination. Because it is not the only consideration,
however, many of us are in favor of abortion for anyone who wants one
while also seeing some cases in which we favor mandatory
vaccination. I've recently seen posts from friends supporting the
right to abortion, such as shares of screenshots of this tweet:

If you don't accept that bodily autonomy is an essential unconditional
liberty, it's a waste of time talking to you at all. No other
liberties survive without that one, more fundamental than property
rights: if you don't own yourself absolutely, you own nothing.

—@ScavengerEthic

In some ways this is similar to the use of the slogan 'my body, my
choice', but I don't think people generally interpret short
phrases as a complete argument; a slogan often draws attention to a
major consideration without claiming it's decisive. In this case,
however, people are sharing statements that do claim to be a full
argument, and make the case for abortion in a way that also makes the
case against vaccination requirements.

Q: A right to abortion isn't anything like a right to refuse
vaccination! Their impact is very different, and there are lots of
other considerations: vaccination prevents something contagious,
pregnancy and childbirth can be deeply difficult, unpleasant, and
dangerous, etc.

A: I agree. Which is why we should make the case for these policies
in a way that depends on those considerations, instead of resting the
entire case on autonomy.

Q: I do think bodily autonomy is an essential unconditional
liberty, and I'm opposed to both abortion bans and vaccination
requirements.

A: That's a consistent position, but you're not my audience here.

Q: A vaccine mandate doesn't mean you'll have one forced on you, it
just means you can't go certain places. That's very different from
threatening jail time for abortions.

A: If the court had ruled that states could exclude people who had
ever had an abortion from restaurants, gyms, universities, and jobs in
healthcare, government, or schooling, we would see this as very nearly
as bad as a blanket prohibition an abortion. But I would also go
farther: a future pandemic could be much worse than covid, and I could
see one in which we would need to choose between literally mandatory
vaccination and the l...
[Yann Lecun] A Path Towards Autonomous Machine Intelligence 
by DragonGod — 2022-06-27T19:24:50.543Z
Link: https://openreview.net/forum?id=BZ5a1r-kVsf

Abstract

How could machines learn as efficiently as humans and animals? How could machines learn to reason and plan? How could machines learn representations of percepts and action plans at multiple levels of abstraction, enabling them to reason, predict, and plan at multiple time horizons? This position paper proposes an architecture and training paradigms with which to construct autonomous intelligent agents. It combines concepts such as configurable predictive world model, behavior driven through intrinsic motivation, and hierarchical joint embedding architectures trained with self-supervised learning.

Meta's Chief AI Scientist Yann Lecun lays out his vision for what an architecture for generally intelligent agents might look like.