trentmkelly/LessWrong-43k huggingface dataset:
AI Labs Wouldn't be Convicted of Treason or Sedition
by Matthew Khoriaty — 2024-06-23T21:34:50.135Z

This is a shortened version of "Preventing AI from Overthrowing the Government" from my Substack to focus on the things that would be interesting to LessWrong. Full version with citations can be found here.

Introduction

Many AI labs, including OpenAI, are forthright in saying that their goal is to create AGI. An AI like that would be able to use its intelligence to accomplish goals. It would then be able to reshape the world around it either according to the wishes of its creators or in contradiction if it is not properly aligned. Former OpenAI employee Leopold Aschenbrenner recently wrote that "it’s quite plausible individual CEOs would have the power to literally coup the US government". Plus, if AGI could take over the world from humans, as many expect it to do, it could accomplish the strictly easier task of taking over control from the US government.

The Constitution’s Supremacy Clause declares that “This Constitution, and the Laws of the United States… shall be the supreme Law of the Land.” This raises the question that if researchers are developing AGI that can reach the capability level to take over the government then the creators of the system might be tried for treason or sedition. Treason is defined as going to war against the US or assisting someone who is, and sedition is using or planning to use force against the government. While the development of powerful AI systems with the potential to reshape the world is extremely worrying and dangerous, the legal precedent suggests that current AI labs and researchers would not be found guilty of either. Other regulations must ensure that the government maintains control over insurgent AI.

The Inapplicability of Treason and Sedition

So far, no court has ruled on the applicability of treason or sedition laws to AI labs, but courts base their decisions on previous precedents to flexibly apply old laws to new situations. Courts would rule in favor of AI labs due to precedents that address free speech, the imminent lawless action standard, and the degree of burden of proof necessary to convict on treason or sedition.

Free Speech

AI labs and researchers have stated that they want to create AGI that is powerful enough that it may disrupt government power. The First Amendment of the Constitution states that “Congress shall make no law… abridging the freedom of speech…” There are notable exceptions to thi...
Control Vectors as Dispositional Traits
by Gianluca Calcagni — 2024-06-23T21:34:37.970Z

I have been reading recently about a technique that can be used to partially control the behaviour of Large Language Models: the technique is exploiting control vectors[1] to alter the activation patterns of the LLMs and trigger some desired behaviour. While the technique does not provide guarantees, it gives high probability of success.

To be clear, this is not a jailbreaking technique: it can only be used if you have access to the internal (hidden) states of the LLM, so it’s only available to developers. The advantage of this technique is that it provides a supplemental way to conditioning a model, and it can be added on top of RLHF/DPO/etc.

I am spending the next paragraph to provide a high-level explanation of the technique, but feel free to skip it since it is not relevant for the rest of my discussion.

The Technique

Let’s suppose that you want to control the refusal behaviour of a LLM, in the sense that you want to increase (or decrease) the chance that the model will refuse to help when you prompt any request. (The refusal behaviour is just one of many possible choices: researchers have been able to steer other types of behaviour - including honesty, humourism, Golden Gate Bridge obsession[2] etc.)

First step: choose two sentences that show an example+counterexample[3] of refusal/acceptance for some request. The first sentence should show refusing behaviour, while the second sentence should show accepting behaviour in the very same context.
Second step: take a snapshot of the (hidden) activation patterns of the LLM when ran against the sentences above. Such a snapshot is multi-layered, and each layer is represented by a long vector (called activation vector).
Third step: for each layer, subtract the first vector (related to the refusal example) from the second (related to the acceptance example) thus obtaining a third vector.
Fourth Step: to remove random noise, repeat the process over many different pairs of sentences, thus forming a matrix where each column is one of the vectors above.
Fifth step: use linear algebra (PCA) to find the dominant direction of the matrix above: it’s similar to calculating the longest eigenvector of a big square matrix. That’s your estimated control vector[4]. The average length of the eigenvalues can also help assessing the quality of the vector (something that may help

...
"On the Impossibility of Superintelligent Rubik’s Cube Solvers", Claude 2024 [humor]
by gwern — 2024-06-23T21:18:10.013Z
Link: https://gwern.net/rubiks-cube

In recent years, a number of prominent computer scientists and roboticists have suggested that artificial intelligence may one day solve Rubik's Cubes faster than humans. Many have further argued that AI could even come to exceed human Rubik's Cube-solving abilities by a significant margin. However, there are at least twenty distinct arguments that preclude this outcome. We show that it is not only implausible that AI will ever exceed human Rubik's Cube-solving abilities, but in fact impossible.
How are you preparing for the possibility of an AI bust?
by Nate Showell — 2024-06-23T19:13:45.247Z

What actions have you been taking to prepare for the possibility that the AI industry will experience a market crash, something along the lines of the dotcom bust of the early 2000s? Also, what actions would you take if a crash like that occurred? For example:

If your career is strongly AI-focused, what backup plan do you have in place?
If you're working on AI alignment research, how would your research agenda change? Would you switch to a different cause area altogether?
Likewise, if you're working on AI policy, how would your approach to it change?

Let's set aside speculation about whether a crash will happen or not; I'd like to hear what concrete actions people are taking.
A simple text status can change something
by nextcaller — 2024-06-23T18:48:58.580Z

Take something like X/Twitter. Users there upload some concepts through text, which can be of high importance to users following them. But say you are annoyed that every thought you convey is being stored in history, every post can be analyzed chronologically, unless you do the job of removing them automatically after some minutes or hours. Not to you mention that to use this platform you need to go through several authentication checkpoints to make sure you are somebody.

Lets go back to the early 2000s. I'm sure you remember MSN Messenger. Messenger had this feature where every user could set their own status, which could be any combination of text + emojis. When you opened Messenger you were welcomed to an overview of the people you followed. A vertical list of users, each with a status. The status conveyed something.

A single status tells something about a person or about something happening around them. Now imagine a dashboard full of these, that updated automatically every 5 minutes. That you can just leave open in a tab and check once in a while, to get an idea of what's going on.

So I made this:

A simple status update backend and a dashboard to use it:

In this gif you can see how I authenticate with gregg and change its status, then do the same with interjected which is another one I control.

(Lots of Not Found) statuses because I added dummy ones for testing.

I'm calling this thing Curls. You claim a curl and get a key to control its status. You get the key only once, if you lose it there's no way to recover it.

You can change the status with classic linux tools like... curl

There are no CORS restrictions, it can be changed from within any application.

When you go to https://curls.merkoba.com/nextcaller you get a pure text response, which is the status of the curl called nextcaller

You might have deduced already that this is not only useful for fun, but also for software development too, since you can point software to a dynamic piece of text that only you control.

There are no accounts, to get a curl you simply solve a captcha.

Also I want to point out in the gif above. Notice how each color has its own set of curls. For example you can follow certain curls in green then follow other ones in white for whatever reason or categorization you come up with. In other words, the colors are not simply for theming.

To add curls you simply write or paste them in the ...
35 Interactive Learning Modules Relevant to EAs / Effective Altruism (that are all free)
by spencerg — 2024-06-23T17:57:44.636Z

Hi all! For quite a number of years now at Clearer Thinking we've been developing free, interactive, online learning modules on a wide variety of topics. I wanted to create a list of which of these tools are most relevant to EAs and to Effective Altruism itself, in case you find them useful (for yourself, or for sharing with others). Here's the list, organized by topic:

1. Calibration, prediction and knowledge practice

Calibrate Your Judgment - this program contains thousands of question sets to help you become adept at making well-calibrated judgments. It also tracks your progress over time so you can see how you improve!
Predict Psychological Correlations - test how accurate your understanding of psychology is by practicing and improving your ability to make predictions.
Retrocaster - take the challenge to identify the correct global trends on a variety of topics - from Africa's future population to the evolution of CO2 emissions in the US.
Biosafety Quiz - uncover the history of pandemics, test your biosafety knowledge, and learn essential steps to safeguard humanity's future.
Long-Term Future Quiz - get a glimpse into the opportunities that may await humanity if we can avoid disasters such as nuclear war and extreme climate change. Invite your imagination to take flight with 11 multiple-choice questions.
Artificial Intelligence Quiz - test your knowledge of artificial intelligence and some of the risks it poses.
The Common Misconceptions Test - Separate fact from B.S. in this fun quiz that asks you to identify the misconceptions among 30 common beliefs.
World's Biggest Problems Quiz - test your knowledge of some of the world's most pressing problems: global health, animal welfare, and existential risks to humanity.
Political Bias Test - learn to spot your political biases and test your policy knowledge
Charity Effectiveness - see if you can guess which social interventions had a positive effect, a negative effect, or no effect, based on a short description.
Guess Which Experiments Replicate - Can you guess which of these high-profile experiments were legit?
Improve Your Frequency Predictions - a quick quiz to test your ability to estimate the frequency of events and compare your scores to others'.

2. Sharpening Your Thinking

Nuanced Thinking Techniques - Learn to recognize 3 common binary thinking traps and learn the nuanced thinking techniques you can use

...
Podcasts: AGI Show, Consistently Candid, London Futurists
by KatjaGrace — 2024-06-23T13:50:03.676Z

For those of you who enjoy learning things via listening in on numerous slightly different conversations about them, and who also want to learn more about this AI survey I led, three more podcasts on the topic, and also other topics:

The AGI Show: audio, video (other topics include: my own thoughts about the future of AI and my path into AI forecasting)

Consistently Candid: audio (other topics include: whether we should slow down AI progress, the best arguments for and against existential risk from AI, parsing the online AI safety debate)

London Futurists: audio (other topics include: are we in an arms race? Why is my blog called that?)
Text Posts from the Kids Group: 2019
by jefftk — 2024-06-23T13:20:01.495Z

Another round of liberating kid posts from

Facebook. For reference,
in 2019 Lily turned 5 and Anna turned 3.

(Some of these were from me; some were from Julia. Ones saying "me"
could mean either of us.)

2019-01-05

Anna: I was kicking Lily over and I tripped on a balloon.

Me: You were doing what?

Anna: (pause) I was accidentally kicking Lily over and I tripped on a balloon.

2019-01-08

Anna asks me to give her pictures of different objects. Then she colors them
blue. Blue every time, everything is blue.

This one was "a mama frog and a papa frog and a brother frog and a sister frog
and a baby frog sitting on a log."

2019-01-17

[From Ange]

Lily: "Ange, we'll always be with you, even when you go home and even when
you're dead"

Me: "What do you mean?"

Lily: "When you love someone they stay in your heart forever. So you will
always be with us, even when you die. As long as you live and even after YOU
DIE, you will always be with me"

She really focused on that part.

2019-01-22

Poor Lily got the conscientious genes from both sides. Things she's cried about
in the last two days:

- her grandfather said they were going to make s'mores, and she had already had
her sweet for the day

- she's sick and knows she's supposed to drink water with electrolytes, but she
wanted to drink plain water

- Anna held her hand, and she's afraid her germs got on Anna.

2019-01-27

I read the kids a story where Death is personified but then goes away to the
graveyard.

Anna: What is that?

Me: It's a place where they bury people.

Anna: Does Death bury people?

Me: No, people bury people.

Anna: That's so...rude.

In retrospect, I could have explained that better.

2019-02-01

Lily: I want an ice cream cone with sprinkles.

Anna: I want the nasty dessert.

We tried for a while to figure out what she meant by this, and eventually
narrowed it down to "some berries Jeff brought for us, and which we ate on an
airplane last November." She was very sad to learn we did not have any nasty
dessert.

2019-02-02

Doing a rhyming puzzle with Lily. "Goat...kayak?"

Apparently her vocabulary has outstripped her rhyming skills or boat
identification skills.

2019-02-09

We're at the park and Lily proposed doing a "tortoise race" with the
instructions "we all run in a circle until somebody says stop." We did this,
with...
Population ethics and the value of variety
by cousin_it — 2024-06-23T10:42:21.402Z

Problems in population ethics (are 2 lives at 2 utility better than 1 life at 3 utility?) are similar to problems about lifespan of a single person (is it better to live 2 years with 2 utility per year than 1 year with 3 utility per year?)

On the surface, this analogy seems to favor total utilitarianism. 2 years at 1 happiness seems obviously better than 1 year at 1.01 happiness, to the point that average utilitarianism becomes just silly. And 4 years at 0.9 seems better still. In the normal range, without getting into astronomical numbers, it does seem multiplying utility/year by years is a good first approximation of what our intuition does.

But does this mean total utilitarianism is our best approximation to population ethics as well? Not so fast. Let's imagine taking an amnesia pill after each year, or after each month. To me at least, it feels like the intuition in favor of summing gets weaker, and the intuition in favor of averaging gets slightly stronger.

Why would amnesia make life less valuable? Well, one scary thing about amnesia is that if we don't remember past experience, we're just going to repeat it. And repeating a past experience feels less valuable than experiencing something new. Let's test that intuition! Imagine 10 equally happy people, all exactly alike. Now imagine 10 equally happy people, all different. The latter feels more valuable to me, and I think to many others as well.

So if we value variety, and that affects our intuition on whether utilities should be summed or averaged, does that carry over to other population ethics problems? I think so, yeah. When someone mentions "100 identical copies of our universe running in parallel", we find it hard to care about it much more than about just 1 copy. But if they'd said all these universes were as good as ours but interestingly different, ooh! That would sound much more appealing. Same if someone mentions "adding many lives barely worth living": if they also said that all these above-zero lives were also interestingly different, it'd no longer sound like such a bad bargain. And so on.

An even more interesting angle is: if we value variety, variety of what? It's tempting to answer "variety of experience", because we're used to thinking of happiness as something you experience. But that's not quite the full answer. Eating the same flavor of ice cream twice, with amnesia in-between, is plausib...
Karma votes: blind to or accounting for score?
by cata — 2024-06-22T21:40:34.143Z

Normally I try to use karma votes in a practical way to try to help promote content that I want to see more of on LW, or to downrank content that I want to see less of. (As opposed to using them as a kind of social signal, which is a separate issue.)

One thing I have never quite figured out is which of these two I should be doing to best accomplish that goal:

Look at the current score of a post or comment, and vote so as to adjust its current score towards what I would consider to be fair.

Judge the post independently of its current score, and have some thresholds of goodness in my mind (relative to a general standard of quality for LW posts) that I use to decide how to vote.

I could easily imagine some pros and cons to each approach. Is there any consensus as to which will make better results, either if I apply it independently, or if the community applied it universally?

My intuition is that the second is better, but I haven't really figured out anything concrete.
Should effective altruism be more "cool"?
by jaredmantell — 2024-06-22T20:42:34.851Z

As a recent entrant to the sphere of rationality and effective altruism, I often wonder if the primarily-nerdy audience & community (which I adore) is less than a boon to the objectives of effective altruism.
If a popularization (through social media platforms such as TikTok) were to happen with EA, would this even be a good thing?
Meta Alignment: Communication Wack-a-Mole
by Bridgett Kay — 2024-06-22T20:12:16.412Z
Link: https://dxmrevealed.wordpress.com/2024/06/22/meta-alignment-communication-wack-a-mole/

When warning the public about the dangers of superintelligent AI, a common question is how,exactly, an unaligned superintelligence could kill humanity. An AI is just a program running on a computer, after all. Wouldn’t it need arms and legs and nukes and possibly a military to kill all of us? In fact, why would it kill humanity if it needs us to feed it energy and information?

At this point, the educator will usually choose one of two possible responses. The first response is that we can’t know exactly how a superintelligent AI will kill us or survive without us, because it will be smarter than us. This is a sensible answer on the surface- any path to destruction a superintelligence could take would either be something humanity would anticipate- in which case the superintelligence would know we anticipated it ahead of time and could contrive ways of obfuscating its plan- or the path would be something we could not anticipate at all- in which case it would be difficult for us to detect what the superintelligence is doing before it is too late.

To many people, however, this response feels like a cop-out. To someone already skeptical that AI could pose a danger to humanity, it is telling that you won’t give a plausible-sounding scenario where an AI destroys humanity. The skeptic will definitely not trust when you say “the superintelligence will figure out a way, and it will be so smart that you and I can’t even comprehend it.” They have no reason to take such an argument on faith.

The second usual response is to give several plausible-sounding scenarios that the audience may be able to conceptualize. You might say, for instance, that the AI could socially engineer a lab worker into synthesizing a dangerous protein chain that starts a global pandemic (or a nanite factory, if you aren’t worried about scaring your audience away with science-fiction scenarios.) You might say that the AI could hack into weapons systems and deploy all of the world’s missiles. You might say the AI could persuade us into building it a robot army for our own protection.

...
AI as a computing platform: what to expect
by Jonasb — 2024-06-22T19:55:42.774Z
Link: https://www.denominations.io/ai-as-a-computing-platform/

Let's just assume for the sake of argument advances in AI continue to stack up.

Then at some point, AI will become our default computing platform.

The way of interacting with digital information. Our main interface with the world.

Does this change everything about our lives? Or nothing at all?

As a machine learning engineer, I've seen "AI" mean different things to different people.

Recent investments in LLMs (Large Language Models) and adjacent technologies are bringing all kinds of conversational interfaces to software products.

Is this conversational interface "an AI" (whatever that means)?

Or does "an AI" need to be

agentic

to transform the way the economy works? To change our human OS? To complete the "AI revolution"?

Either way, I couldn't help but notice that a lot of the hype around AI is about AI applications–not about AI as a computing platform.*

So I wanted to see where the AI hype would lead us when taken at face value

And given that my AI and genAI-powered searches didn't yield meaningful results, I decided to come up with some conjectures of my own.

I've decided to align these conjectures with the sun, the moon, and the holy quadrity of A.I.

The holy quadrity of AI, in no particular order

So let's see what happens if we do decide to hand over the keys to "an / the AI".

The mirage of internet data

For (the human-developed and adopted technology of) AI to deliver broad-strokes, sweeping societal changes it needs data to control our world.

And in terms of volume, most of the available data we have is internet data.

Of an estimated 120 zettabytes of internet data generated by us humans in 2023, more than half was video (data source: explodingtopics.com). For scale, the Large Hadron Collider, one of the largest scientific experiments ever set up by humans, generates around 15 petabytes of data a year. 15 petabytes is roughly ~1e-7 of 120 zettabytes (data source: lhc-closer.es)

As can be seen in the diagram above, the vast majority of that internet data is video. Which, let's be honest, is mostly generated for entertainment purposes.

Let's oversimplify things a bit and classify this treasure trove of data:

Leisure: non-work digital human activity (blue+purple tones in the diagram).
Business: data generated for commercial purposes (grey+purple tones).
Public: governments, science, NGOs and other data (too small / NA).

We can maybe generously claim that legitimate business us...