trentmkelly/LessWrong-43k huggingface dataset:
Alignment has a Basin of Attraction: Beyond the Orthogonality Thesis
by RogerDearnaley — 2024-02-01T21:15:56.968Z

While the Orthogonality Thesis is correct, there is a lot more that one can say about what kinds of agent motivations are likely to be encountered. A simple analysis shows that living agents produced by evolution, and constructed agents that are the product of intelligent design, will tend to be have very different motivations, in quite predictable ways. This analysis also suggests that alignment is a clearly-defined property for constructed agents, and that it is evidently the correct and default-expected design. So any misalignment is a well-defined design flaw and/or malfunction, and thus (just as for any other constructed object) ought to be corrected.

This argument is very simple, to the point that it relies on no specific information about human values that isn't entirely predictable just from humans being an evolved sapient technological species. It is has been understood for around a decade that Value Learning has a basin of attraction to alignment — this argument suggests that this should also be true of any approximation to alignment, even quite crude ones, so long as they contain this bare minimum amount of information about the fact that the AI was constructed by an evolved sapient species and that its design could be imperfect.

Evolved Agents and Constructed Agents

There are only two ways that a type of agent can come into existence: they can evolve, or they can be constructed.[1] In the case of a constructed agent, it could be constructed by an evolved agent, or by another constructed agent — if the latter, then if you follow the chain of who constructed who backwards, sooner or later you will reach an evolved agent that started the chain.

These two types of agent have very different effects on the preference order/utility function that they are going to have. Any evolved agent will be an adaption executor, and evolutionary psychology is going to apply to it. So it'sad going to have a survival instinct (as more than just an instrumental goal), it's going to care about its own well-being and that of close genetic relatives such as its children, and so forth. In short, it will have selfish desires. Since an evolved agent has self-interest as a terminal goal, it will not (except perhaps under exceptional circumstances such as its imminent death being inevitable) even be fully aligned with any other evolved agent. Humans, for example, are not alig...
Wrong answer bias
by lemonhope — 2024-02-01T20:05:38.573Z

The correct answer leaves nothing to be said. The wrong answer starts a conversation, a research program, an investigation, a journey, an institute, a paper, a book, a youtube channel, a lifestyle, a tribe.

Why is reading a textbook so boring? Very few people do it. It just has right answers and few great questions to ponder. Blogs arguing about nutrition are great reading though.

If you write a research paper saying "all that shit is dumb just do the obvious thing" then you'll probably have trouble getting it published. I tried once but my professor shut it down saying it's a bad strat basically.

"Clean energy" is great as a conversation piece or phrase in your mission statement but "just deregulate nuclear lmao why are you wasting your time" is definitely not in my experience.

It's fine if bad ideas are all we ever talk about but the trouble comes when it's time for someone to sit down and do their work. The doctor trying to cure a thing mostly heard about the RCTs on all the shitty methods that barely do anything, so they pick their favorite of those. The AI safety implementer mostly heard discussions about bad methods and probably tries to patch one of those. (And fixing the mistake in a bad method is a not a good way to make a good method.) The parents in the parent group of course mostly talk about their failed attempts to fix the problems they do have (and mostly forget about the problems they never had or quickly fixed) so the parents go home and try the mediocre ideas.

Have hope, you can combat wrong answer bias with these simple tricks.

Do not write or speak
Do not read or listen
Hate the process of fixing things
Savor the experience of shit working like it should
Act like you have no time or money and won't ever have any (helps if you don't and won't)
Assume your answer is too complicated
Assume your question is too loaded
Assume any task is actually really easy you're doing it wrong
Hate struggle
Hate nuance
If you look impressive then you're doing it wrong
Love true success
If it looks like you barely did anything at all then you're doing it right
Observe people & orgs who get shit done or don't have a problem you have
Turn your back on anyone struggling with anything for any reason. They're probably looking for a frend or comunity instead of just focusing on nailing shit all the time like you.
On Not Requiring Vaccination
by jefftk — 2024-02-01T19:20:12.657Z

A friend recently wrote that they wouldn't be attending a
mask-optional contra dance weekend I'm playing at because it doesn't
require vaccination. As an organizer of unrelated dance events, which
also

don't
require vaccination, here's how I see the situation.

When the covid vaccines first came out they were a huge improvement
over the status quo. Getting the vaccine reduced your risk of severe
illness or death, reduced your chance of catching it, and reduced your
chance of giving it to others after you were sick. Our events
initially required vaccination, which I think was the right call.

At this point, however, there are a few different things you might
mean if you say your event requires vaccination. Usually it's either:

A complete primary series counts, even if it's one shot of J&J
from early 2021.

The most recent ("updated") booster is required.

The CDC used to call the first category "fully vaccinated", but no
longer talks about it prominently. They've switched to focusing on
the second one, which they call "up
to date".

This change makes sense: from a perspective of avoiding getting
infected and passing it on, only a recent booster does very much. A
few months ago I wrote about
the results in Menegale
et. al (2023) where they saw that as an infection control
measure vaccines wane quickly: effectiveness halves about three times
a year.

Additionally, lots of people got sick this winter,
which acts similarly to an a vaccination. Given how quickly the
vaccine wears off, I'd be less concerned about risk from someone
unvaccinated who'd had covid over Christmas than someone who got their
booster in the early Fall and dodged the winter wave.

What does this mean for a dance event?

If you want a vaccination requirement to be doing anything
useful, you need to require people be up-to-date with their vaccine.
Risk from someone last boosted in Fall 2022 or before is not
appreciably different from someone who was never vaccinated.

Requiring up-to-date vaccination rules out 80%
of people 18+, though if they're excited enough about your
event they could go get the shot.

Unless your event is in the fall or early winter, even requiring
up-to-date vaccination doesn't help much: if someone got their "updated"
booster in the fall they're down to ~35% of peak efficacy by the time
of an April event.

If an e...
The economy is mostly newbs (strat predictions)
by lemonhope — 2024-02-01T19:15:49.420Z

If the world economy were a game where the objective is to gain resources & power, then I would say that we're collectively newbs. We're presently protected from some of the extremes of capitalism (etc) by the fact that the board (etc) does not actually MAXIMIZE shareholder value.

As tech advances, the more ambitious players will be able to take more resources more easily. The sleepier corps will lose almost all non-govt-protected markets and die. The world will be a more pushy, grabby, competitive, profitable place.[1] People will not foolishly leave so much money on the table so often.

A few examples of recentish advances in strategy to show what I mean:

Mobile game companies realized they can be casinos where you never actually win money. In 2022, the mobile game market was ~$180B vs ~$40B for console or ~$30B for PC.
All the apps realized subscriptions are more profitable than sales.
Anyone who can possibly offer a credit card (Apple, Lowe's, airlines, Amazon...) does now. For airlines, the cards are ~1/3ish of profit.
Landlords in some cities are effectively fixing prices by ubiquitously using a "recommended market-rate price" from an app.
Google's aquire-and-kill approach seems to be operating successfully at unprecedented scale (1 company per weekday in 2022)

Which new strategies will be viable depends much on regulation & the order of tech advances, but here's my bets for how the state of the game will change over the next ten years.

better pre-IPO PR
^ partly enabled by social network AI influencer swarms available for hire
more price "recommendation" apps across industries
more & better phishing & scams ofc
misleading loans everywhere, especially construction & renovation[2]
more we-don't-tell-you-the-price like hospitals are somehow permitted to do
more acquisitions across industries — why shouldn't Microsoft buy an oil company?
more sea steading and secret private militaries (something being a common joke does not make it a bad move)
military companies starting more wars
fewer bugs & problems in hardware & software
more punishing of competitors with endless court cases
more voting machine hacking & mail-based voter fraud (in 2022 in the US this briefly became unimaginable by people who normally have no trouble imagining such things)
simultaneous & targeted political lobbying (can't believe how many companies just

...
Managing risks while trying to do good
by Wei Dai — 2024-02-01T18:08:46.506Z

I often think about "the road to hell is paved with good intentions".[1] I'm unsure to what degree this is true, but it does seem that people trying to do good have caused more negative consequences in aggregate than one might naively expect.[2] "Power corrupts" and "power-seekers using altruism as an excuse to gain power" are two often cited reasons for this, but I think don't explain all of it.

A more subtle reason is that even when people are genuinely trying to do good, they're not entirely aligned with goodness. Status-seeking is a powerful motivation for almost all humans, including altruists, and we frequently award social status to people for merely trying to do good, before seeing all of the consequences of their actions. This is in some sense inevitable as there are no good alternatives. We often need to award people with social status before all of the consequences play out, both to motivate them to continue to try to do good, and to provide them with influence/power to help them accomplish their goals.

A person who consciously or subconsciously cares a lot about social status will not optimize strictly for doing good, but also for appearing to do good. One way these two motivations diverge is in how to manage risks, especially risks of causing highly negative consequences. Someone who wants to appear to do good would be motivated to hide or downplay such risks, from others and perhaps from themselves, as fully acknowledging such risks would often amount to admitting that they're not doing as much good (on expectation) as they appear to be.

How to mitigate this problem

Individually, altruists (to the extent that they endorse actually doing good) can make a habit of asking themselves and others what risks they may be overlooking, dismissing, or downplaying.[3]

Institutionally, we can rearrange organizational structures to take these individual tendencies into account, for example by creating positions dedicated to or focused on managing risk. These could be risk management officers within organizations, or people empowered to manage risk across the EA community.[4]

Socially, we can reward people/organizations for taking risks seriously, or punish (or withhold rewards from) those who fail to do so. This is tricky because due to information asymmetry, we can easily create "risk management theaters" akin to "security theater" (which come to think ...
Putting multimodal LLMs to the Tetris test
by Lovre — 2024-02-01T16:02:12.367Z

(See this X (Twitter) thread for a much shorter version of this post.)

Introduction

With the advent of vision abilities in LLMs, a virtually limitless vista has opened in the realm of their evaluations: they can now be tested on any task that requires vision and has a ~finite input/action space. This includes tasks as useful as being a web agent but also, notably, almost all video games.[1]

To our knowledge, little of this testing has been done so far.[2] Lack of work in this domain is not hard to understand, for images are token-rich and inference with current premier multimodal LLMs is relatively expensive per token. Aiming to get these kinds of evaluations off the ground but also trying to keep the costs manageable, we decided to test on Tetris.

Before delving into our results, I want to sketch why I think such evaluations can be valuable beyond the direct insight into how well the models do on them. When we evaluate a model on a rich modality such as video games, its performance can be bottlenecked at various points. Taking Tetris as an example, some hypothetical bottlenecks are:

Lack of ability to understand the rules or the objective of the game. The model just doesn't get it.
Lack of visual acuity in recognizing the relevant shapes. The model cannot recognize shapes nor gaps in the board.
Lack of counting ability when applied to images. The model might fail at various implicit counting tasks such as how many spaces to move a tetromino.
Lack of ability to synthesize visual information into an actionable plan. The model fails to deliver useful plans for reasons other than counting.
Lack of ability for long-term planning. Moves it does are locally OK, but it fails to plan for the future board.

Gameplay videos can potentially give a clue about which of these is the likely bottleneck, so that direct experiments can be designed to test it. This process can, thus, conceivably lead to more knowledge about the capabilities of the model than its mere "accuracy".

However, despite seeing some unique value in testing on video games, I wish not overstate the case. Multimodal LLMs are not a species completely unknown to us, for our priors based on non-multimodal LLMs do give us nonzero information about what to expect, and in particular the quality of their non-spatial reasoning might completely generalize between the domains.[3] So that I think there are, currently,...
AI #49: Bioweapon Testing Begins
by Zvi — 2024-02-01T15:30:04.690Z

Two studies came out on the question of whether existing LLMs can help people figure out how to make bioweapons. RAND published a negative finding, showing no improvement. OpenAI found a small improvement, bigger for experts than students, from GPT-4. That’s still harmless now, the question is what will happen in the future as capabilities advance.

Another news item was that Bard with Gemini Pro impressed even without Gemini Ultimate, taking the second spot on the Arena leaderboard behind only GPT-4-Turbo. For now, though, GPT-4 remains in the lead.

A third cool item was this story from a Russian claiming to have used AI extensively in his quest to find his one true love. I plan to cover that on its own and have Manifold on the job of figuring out how much of the story actually happened.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. Bard is good now even with only Pro?

Language Models Don’t Offer Mundane Utility. Thinking well remains hard.

GPT-4 Real This Time. Bring GPTs into normal chats, cheaper GPT-3.5

Be Prepared. How much can GPT-4 enable production of bioweapons?

Fun With Image Generation. How to spot an AI image, new MidJourney model.

Deepfaketown and Botpocalypse Soon. Taylor Swift fakes, George Carlin fake.

They Took Our Jobs. If they did, how would we know?

Get Involved. What we have here is a failure to communicate.

In Other AI News. Who is and is not raising capital or building a team.

Quiet Speculations. Is economic growth caused by inputs or by outputs?

The Quest for Sane Regulation. Emergency emergency emergency. Meh.

The Week in Audio. Tyler Cowen goes on Dwarkesh Patel.

Rhetorical Innovation. How to think about pattern matching.

Predictions are Hard Especially About the Future. Contradictory intuitions.

Aligning a Smarter Than Human Intelligence is Difficult. You’ll need access.

Open Model Weights are Unsafe and Nothing Can Fix This. Except not doing it.

Other People Are Not As Worried About AI Killing Everyone. Misconceptions.

The Lighter Side. Hindsight is 20/20.

Language Models Offer Mundane Utility

Bard shows up on the Arena Chatbot leaderboard in second place even with Gemini Pro. It is the first model to be ahead even of some versions of GPT-4.

According to the system card, roughly, Gemini Ultimate is to...
Some Notes on Ethics
by Pareto Optimal — 2024-02-01T10:18:44.502Z
Link: https://paretooptimal.substack.com/p/my-notes-on-ethics

"You're next to a body of water with plenty of adults nearby. There are many drowning children in the water, but it's still much less than the number of people who can immediately save them.
Collectively, if a fraction of the people went in the water, there would be no children drowning, but one individual can only save a few kids at best. If everyone jumped in the water, that would be a waste of time and resources better spent elsewhere.
Some people save the children, but most people do nothing. What should be done?"
Increasingly vague interpersonal welfare comparisons
by MichaelStJules — 2024-02-01T06:45:30.160Z

Some have argued that all interpersonal welfare comparisons should be possible or take it as a strong mark against a theory in which they are not all possible. Others have argued against their possibility, e.g. Hausman (

1995

) for preference views. Here, I will illustrate an intermediate position: interpersonal welfare comparisons are vague, with tighter bounds on reasonable comparisons between beings whose welfare states are realized more similarly, and wider or no bounds when more different.

The obvious case is two completely or at least functionally identical brains (at the right level of abstraction for our functionalist theory). As long as we grant intrapersonal comparisons, then we should get interpersonal comparisons between identical brains. We map the first brain’s state(s) to the equivalent state(s) in the second, and compare them in the second brain. Of course, this is not a very interesting case, and it seems only directly useful for artificial duplicates of minds.

Still, we can go further. Consider an experience E1 in brain B1 and an experience E2 in brain B2. If B1 and B2 only differ by the fact that some of B2's unpleasantness-contributing neurons are less sensitive or removed, and B1 and B2 receive the same input signals that cause pain, then it seems likely to me that B1's painful experience E1 is at least as unpleasant as B2's E2 and possibly more. We may be able to say roughly how much more unpleasant it is by comparing E2 in B2 directly to less intense states in B1, sandwiching E2 in unpleasantness between two states in B1.

UnpleasantnessB1(E1)≥UnpleasantnessB2(E2)≥UnpleasantnessB1(E′1)

Maybe going from E1 to E2 changes the unpleasantness by between -0.01 and 0, i.e. UnpleasantnessB2(E2)=UnpleasantnessB1(E1)+Δ, where −0.01≤Δ≤0. There may be no fact of the matter about the exact value of Δ.

For small enough local differences between brains, we could make fairly precise comparisons.

I use unpleasantness for the purpose of a more concrete illustration, but it's plausible other potential types of welfare could be used instead, like preferences. A slight difference in how some preferences are realized should typically result in a slight difference...
PIBBSS Speaker events comings up in February
by DusanDNesic — 2024-02-01T03:28:24.971Z

TLDR: The PIBBSS Speaker Series is happening in mid-February. Read more about the talks and register

on our website

. S

ign up for future updates about our speaker events

here

.

PIBBSS

regularly invites researchers from a wide range of fields studying intelligent behaviour and exploring their connections to the problem of AI Risk and Safety.

In the past, we had speakers such as

Michael Levin

talking about

Morphogenesis as a form of intelligent problem-solving building towards a framework of Basal Cognition

; J

osh Bongard

about

efforts to build hybrid intelligent 'agents' at the intersection of biology, cognition and engineering;

Miguel Aguilera on

Nonequilibrium Neural Computation

; Steve Byrnes on

Challenges for Safe and Beneficial Brain-Like AI

; or Tang Zhi Xuan on

Modeling Bayesian Bounded Rational Agents

.

We have a couple of speaker events are coming up in February.

Konrad Körding on "Interpretability of brains and neural networks"

Feb 13, 9AM PST/5PM UTC (

Register

)

Daniel Polani on "Information and its Flow: a Pathway towards Intelligence"

Feb 16, 9AM PST/5PM UTC, (

Register

)

Nathaniel Virgo on "Mathematical Perspectives on Goals and Beliefs in the Physical World: Extending on the 'Good Regulator Theorem'"

Feb 19, 6PM PST,/20th 2AM UTC, (

Register

)

Tsvi Benson-Tilsen on “Creating the contexts needed to produce the concepts needed to understand minds”

20 Feb 9AM PST/5PM UTC (

Register

)

Abram Demski on “Meaning & Agency"

23 Feb 9AM PST, 5PM UTC, (

Register

)

If you want to be informed about future events (we typically won't post about this going forward), signup

here

to our speaker event specific email list.

Snapshots from past talks
Drone Wars Endgame
by RussellThor — 2024-02-01T02:30:46.161Z

1. Introduction

It is probably becoming clear to anyone who follows war lately, especially the Ukraine conflict that automatic weapons, especially drones are rapidly changing what weapons and tactics are effective.

The purpose of this article is to first consider how a fully autonomous force with current or near term tech would go against a state of the art conventional force, then to consider where the equilibrium would be on attack vs defense for autonomous forces using advanced but foreseeable tech.

1.1 Description of fully autonomous force using near term tech

The idea is to have a force with as few different types of units as possible controlled in a distributed, mesh network fashion. The goal is for them to take over unlimited land territory, even in the face of tactical/strategic nukes, and without air superiority against very fast or high flying aircraft. Units are assumed to be fully autonomous, and emphasis is on economics as well as capability. For example many units are so cheap that they cannot effectively be countered by conventional missiles.

1.2 Unit Overview

All units can fly, they are optimized to destroy all land based armor, slow flying aircraft, competing drones and humans. Combat units are supported by larger logistics units.

1.3 Communication and Defense

All units are expected to communicate with point to point links (e.g. laser) and are hardened to varying degrees against microwave attacks. Given they are autonomous this would make jamming very difficult and electronic warfare not very effective.

1.4 Unit details

1.4.1 Recon and targeting drone

The cheapest and smallest – it is a battery powered drone with video and targeting system. Recon units form a mesh network.

They coordinate with missiles to defeat countermeasure such as flares and chaff from slow moving aircraft (helicopters etc) by observing and transmitting the position of the target from somewhat further away and transmitting that info to their missile.

They spot opposing drones, troops, fortifications etc. They can spend a brief time in the air, before being recharged by a larger unit.

1.4.2 Suicide drone

This unit is optional – Sniper drones and Javelin type missiles may make it not needed. This is the drone we already see, however with AI, the same p2p comms system as above and with some protection against microwave and laser attack.

1.4.3 Sniper drone

Larger unit, still likely bat...