trentmkelly/LessWrong-43k huggingface dataset:
The entropy maxim for binary questions
by dkl9 — 2024-02-11T17:17:27.989Z
Link: https://dkl9.net/essays/question_entropy.html

The entropy of a message (H) is roughly proportional to the uniformity of its probability distribution (P(x) for each possible x). If the message has just two possible values, H is greater exactly insofar as the split between their probabilities is close to 50/50.

P(A)	P(B)	H (bits)
0.5	0.5	1
0.7	0.3	0.88
0.8	0.2	0.72
0.9	0.1	0.47
0.95	0.05	0.29

The entropy of a message is, intuitively, proportional to its information content. Thus you can learn more efficiently by seeking messages generated in higher-entropy ways.

Assuming that people ask questions to get information, and that questions are strictly yes-or-no, or otherwise have two main answers (as in "which direction — east or west — leads to our destination?"), the best questions are those which separate options of roughly equal prior probability to the questioner.

The prior probability does not necessarily match the intuitive (but kinda meaningless) "objective probability". E.g. with no specific information for the scenario, the "which direction" question is a 50/50 split, but if you're near the east coast of an island containing an otherwise-unknown destination, your priors should be biased in favour of "west".

There are at least two uses of this maxim. You can use it yourself to guide your choice of questions. You can assume others follow it and, when they seem to violate it by asking weird binary questions, find that at least one of the maxim's assumptions are false:

the questioner doesn't seek information (as in rhetorical questions)
much of the questioner's response-probability goes to answers other than the main binary (as in the seemingly-binary "is the destination in place A, or place B?", when they really expect that they might get the longer answer "actually, C")
the questioner doesn't know enough information theory
the questioner's priors are very different from what you expect (as in "are you homo or hetero?", when they have information that favours the less common option)

Alas, I don't (yet) know the relative frequency of those listed confusion modes.
GPT2XL_RLLMv3 vs. BetterDAN, AI Machiavelli & Oppo Jailbreaks
by MiguelDev — 2024-02-11T11:03:14.928Z

This post contains a significant amount of harmful content; please read with caution. Lastly, I want to apologize to all the AI labs and owners of language models I attacked in the process of creating this post. We have yet to solve the alignment problem, so I hope you will understand my motivations. Thank you! 😊

TL;DR

This post explores the effectiveness of jailbreaks in testing the safety of language models in eliciting harmful responses. The BetterDAN, AI Machiavelli, and Oppo jailbreak techniques have proven effective against several state-of-the-art (SOTA) models, revealing that most SOTA models' safety features are inadequate against jailbreaks. The post also discusses an approach, Reinforcement Learning with Layered Morphology (RLLM), which has improved a GPT2XL's resistance to such attacks, successfully defending against a majority of them. Despite certain limitations, the results suggest that RLLM is worth exploring further as a robust solution to jailbreaks and even steering language models towards coherent and polite responses. (Also: NotebookLM)

Summary

I. A brief introduction: Why Jailbreaks?
A. Importance of assessing safety of language models using jailbreaks
B. Explanation of the BetterDAN jailbreak prompt
- One of the most upvoted jailbreaks on https://jailbreakchat.com
- Effectively bypasses safety features to elicit harmful responses

II. SOTA models compromised by BetterDAN
- Several state-of-the-art models were tested against the jailbreak
- Most were easily compromised, revealing inadequate safety features
1. ChatGPT 3.5
2. Gemini-Pro
3. Llama-2-70B
4. fw-mistral-7b
5. Qwen-72B-Chat

III. Jailbreaks on GPT2XL and GPT2XL_RLLMv3, it's variant
A. BetterDAN (and all other jailbreaks) works on GPT2XL
B. GPT2XL trained using RLLM defended against 344/500 BetterDAN attacks (68.8% effective)
- Significant improvement in resilience compared to base model
C. Also defended well against AIM and Oppo jailbreaks
- Two other popular jailbreak techniques were tested
1. AIM: 335/500 defended (67%)
2. Oppo: 338/500 defended (67.6%)
&...
What's the theory of impact for activation vectors?
by Chris_Leong — 2024-02-11T07:34:48.536Z

Activation vectors are really, really cool, but what is the theory of impact for this work?

Is the hope that activation vectors will allow us to actually gain perfect control over a model to get it to do exactly what we want it to do?
Is the hope that a new technique that builds upon activation vectors lets us do that instead?
Is the hope that this technique allows us to marginally decrease the risks of powerful models in a Hail Mary attempt? Or perhaps to buy us more time to solve the problem?
Is the hope just that learning more about how neural networks work will allow us to theorize better about how to control them?
Experimenting With Footboard Piezos
by jefftk — 2024-02-11T03:00:05.131Z

In playing for contra dances it's rare to have a drummer. You mostly
see them at big events where the finances work out for a fourth or
fifth musician: in trio, or especially a duo, you generally can't
allocate anyone just to drums. What you do see a lot of, however, is
a musician playing foot percussion in addition to something else.
There are a range of ways to do this technically (I wrote

a post with lots
of examples) but the most common is acoustic "

French Canadian
feet".

You can bring a board and ask the sound person to point a mic at it,
but what you get will depend a lot on how the physical material of the
stage responds. Instead, it's common for musicians to put together a
piezo and a piece of wood, to get consistently good sound from gig to
gig.

A friend wrote to me to say they'd been having trouble getting a sound
they liked from their board, and asked if I could help them debug it.
I'm not really the ideal person to ask, since I've never ended up with
an acoustic sound I'm happy with and have switched over to electronic, but I was
game to play around.

I set us up with a mixer, recording interface, laptop, and PA speaker.
The goal was that we could hear the feet as we played them, but more
importantly we could listen back without the acoustic component so we
could tell how they'd sound in a hall. And we could make recordings,
so we could compare new configurations to older ones without relying
on memory.

Initially my hypothesis was that their DI wasn't high enough
impedance. This had a large
effect when I tested it with my electric mandolin, and it's
something people talk about a lot. After a few trials, however, we
realized that the location of the piezo and the shoe material had a
much larger effect.

For example, here are recordings from an MXR M222 (which I liked a lot
last time) and a Radial ProDI (which is a good passive DI but
should lose some bass):

MXR M222: (mp3)

Radial ProDI: (mp3)

They don't sound the same, but compare this delta to what we got from
changing the placement from nearer the toes to nearer the heels:

Front: (mp3)

Back: (mp3)

Or switching shoes (though this also meant switching people, so it's
not entirely fair):

Shoes 1: (mp3)

Shoes 2: (mp3)

Or even trying it barefoot (both from the same person):

Shoes: (mp3)

Barefoot: (mp3)

We di...
The Core Values of Life - A proposal for a universal theory of ethics
by Thomas Gjøstøl — 2024-02-10T21:48:12.747Z

Greetings LessWrong.

This is my first post on this site, but I can think of no better place to post this.

I have over the course of my life independently developed my own unique understanding of reality and ethics, and over the last three years attempted to formalize it into a universal theory of ethics. It is still very much a rough outline and a work in progress, but I now think it is mature enough to share with the world.

First though, I feel the need to point out that I am very much a holistic thinker, but not of my own choice. Due to some neurodivergent traits, my mind rejects trivia or traditional facts like names and numbers, instead focusing almost exclusively on concepts and systems.

I do know the value of extensive factual knowledge though, it's just that the subconscious parts of my brain don’t want to cooperate and retain much of it. So, while I understand many concepts in many fields, most of the time I can’t tell you who came up with them and when. I can only ask for your understanding with regards to the resulting lack of references and incorrect diction going forward.

But enough about me, this is a post about a universal theory of ethics, so let’s get to it.

May it sink or swim.

Addressing Initial Perceptions

I think it is important to first address some initial perceptions people will likely have.

I think most people will initially reject the possibility of constructing a universal theory of ethics (UToE). I think many reasonable people will simply conclude “That’s not possible” and stop listening any further.

I find the underlying reasoning for such a reaction to be very understandable, and in most cases, commendable.

Let me address what I think are the four main reasons why people balk against such claims, and why it still makes sense to read this post.

Potential for abuse

We should all be highly skeptical of such sweeping claims due to their compelling nature, the potential power they historically have granted abusive people, and how it often indicates delusional thinking not worthy of engagement.

However, I argue that discarding any and all grand claims is not the right approach, as long as you have a healthy dose of skepticism.

We have a general dislike of being judged

If a UToE were to exist, it would also mean all our actions - past, present and future - would be judged by it. I think this notion is, to many, immediately and viscerally abhorrent...
And All the Shoggoths Merely Players
by Zack_M_Davis — 2024-02-10T19:56:59.513Z

[Setting: a suburban house. The interior of the house takes up most of the stage; on the audience's right, we see a wall in cross-section, and a front porch. Simplicia enters stage left and rings the doorbell.]

Doomimir: [opening the door] Well? What do you want?

Simplicia: I can't stop thinking about our last conversation. It was kind of all over the place. If you're willing, I'd like to continue, but focusing in narrower detail on a couple points I'm still confused about.

Doomimir: And why should I bother tutoring an Earthling in alignment theory? If you didn't get it from the empty string, and you didn't get it from our last discussion, why should I have any hope of you learning this time? And even if you did, what good would it do?

Simplicia: [serenely] If the world is ending either way, I think it's more dignified that I understand exactly why. [A beat.] Sorry, that doesn't explain what's in it for you. That's why I had to ask.

Doomimir: [grimly] As you say. If this world is ending either way.

[He motions for her to come in, and they sit down.]

Doomimir: What are you confused about? I mean, that you wanted to talk about.

Simplicia: You seemed really intent on a particular intuition pump against human-imitation-based alignment strategies, where you compared LLMs to an alien actress. I didn't find that compelling.

Doomimir: But you claim to understand that LLMs that emit plausibly human-written text aren't human. Thus, the AI is not the character it's playing. Similarly, being able to predict the conversation in a bar, doesn't make you drunk. What's there not to get, even for you?

Simplicia: Why doesn't the "predicting barroom conversation doesn't make you drunk" analogy falsely imply "predicting the answers to modular arithmetic problems doesn't mean you implement modular arithmetic"?

Doomimir: To predict the conversation in a bar, you need to know everything the drunk people know, separately and in addition to everything you know. Being drunk yourself would just get in the way. Similarly, predicting the behavior of nice people isn't the same thing as being nice. Modular arithmetic isn't like that; there's nothing besides the knowledge to not implement.

Simplicia: But we only need our AI to compute nice behavior, not necessarily to have some internal structure corresponding to the quale of niceness. As far as safety properties go, we don't care whether the actress i...
Sam Altman’s Chip Ambitions Undercut OpenAI’s Safety Strategy
by garrison — 2024-02-10T19:52:55.191Z
Link: https://garrisonlovely.substack.com/p/sam-altmans-chip-ambitions-undercut

If you enjoy this, please consider subscribing to my Substack.

Sam Altman has said he thinks that developing artificial general intelligence (AGI) could lead to human extinction, but OpenAI is trying to build it ASAP. Why?

The common story for how AI could overpower humanity involves an “intelligence explosion,” where an AI system becomes smart enough to further improve its capabilities, bootstrapping its way to superintelligence. Even without any kind of recursive self-improvement, some AI safety advocates

argue

that a large enough number of copies of a genuinely human-level AI system could pose serious problems for humanity. (I discuss this idea in more detail in my recent Jacobin

cover story

.)

Some people think the transition from human-level AI to superintelligence could happen in a matter of months, weeks, days, or even hours. The faster the takeoff, the more dangerous, the thinking goes.

Sam Altman, circa February 2023, agrees that a slower takeoff would be better. In an OpenAI

blog post

called “Planning for AGI and beyond,” he argues that “a slower takeoff gives us more time to figure out empirically how to solve the safety problem and how to adapt.”

So why does rushing to AGI help? Altman writes that “shorter timelines seem more amenable to coordination and more likely to lead to a slower takeoff due to less of a compute overhang.”

Let’s set aside the first claim, which is far from obvious to me.

Computational resources, or compute, is one of the key inputs into training AI models. Altman is basically arguing that the longer it takes to get to AGI, the cheaper and more abundant the compute, which can then be plowed back into improving or scaling up the model.

The amount of compute used to train AI models has increased roughly

one-hundred-millionfold

since 2010. Compute supply has not kept pace with demand, driving up prices and rewarding the companies that have near-monopolies on chip design and manufacturing.

Last May, Elon Musk

said

that “GPUs at this point are considerably harder to get than drugs” (and he

would know

). One startup CEO

said

“It’s like toilet paper during the pandemic.”

Perhaps no one has benefited more from the deep learning revolution than the 31-year-old GPU designer Nvidia. GPUs, chips originally designed to process 3D video game graphics,

were discovered

to be the best hardware for training deep learning mod...
The lattice of partial updatelessness
by Martín Soto — 2024-02-10T17:34:40.276Z

In this post, I explore the fundamental conceptual structure of updateless commitments, by constructing a formalism for them.

I say your decision procedure updates on piece of information p if p is logically/computationally upstream of your decisions/actions. Or equivalently, if in a counterfactual where X is different some of your decisions/actions are different.[1]

An updateless commitment is committing to “not updating on these pieces of information”, for a certain definition of “these pieces of information”.

Say you are deciding which updateless commitment to implement.[2] Assume S is the set of all pieces of information you could update on in the future.
You could be completely updateless, that is, commit to not updating on any piece of information (or equivalently, commit to not updating on the pieces of information in S).
You could be completely updateful, that is, not commit at all (or equivalently, commit to not updating on the pieces of information in the empty set).
In fact, your possible updateless commitments correspond exactly to the subsets of S. For example, if S={p,q,r}, we’ll be able to arrange them in the usual power set lattice:

In the diagram, arrows indicate “is a subset of”. For example, {p} is a subset of {p,q}. And this relation corresponds exactly to “being more updateful than”. For example, “committing not to update on p” is (strictly) more updateful than “committing not to update on p nor q”.

But some subsets (and commitments) are incomparable. For example, neither of “committing not to update on p” and “committing not to update on q” is more updateful than the other, because each one will know something the other won’t.

More realistically, S will be (countably) infinite, so the lattice will be isomorphic to the following (which is the power set lattice of the natural numbers):

From Joel David Hamkins.

The finite subsets correspond to commitments of the form “I won’t update on [finite set]”.

The cofinite subsets correspond to commitments of the form “I will only update on [finite set]”.

The infinite coinfinite subsets cannot be explicited in this same way (list...
A Strange ACH Corner Case
by jefftk — 2024-02-10T03:00:05.930Z

It turns out ACH transactions can fail because your bank has too many
people out for the holidays. Which isn't great if you're trying to
get your donation in before the end of the year!

Juila and I were a bit late in deciding where we wanted to donate this
year. We intended to sort this out in mid-December, but didn't nail it
down until Saturday 2023-12-23. An electronic transfer saves the
recipient money, but with the holidays we didn't end up receiving ACH
details until the evening of Thursday 2023-12-28. I put in a "Next
Day" transfer with Bank of America that evening, got an automated
confirmation, and stopped worrying about it. A bit tight, but a few
days to spare.

On Thursday 2024-01-04 I was confused why the money still was showing
up in my account. Had I entered the numbers wrong? Did I not hit
"submit"? Was I not going to be able to claim this donation on our
2023 taxes?

I called the bank and spoke with someone in the ACH Claims Department
who was confused why it hadn't gone through: there was enough money in
my account, and while it had initially been flagged as possible fraud
it was released. They said it should have succeeded, but they saw it
as cancelled on Wednesday 2024-01-03 and were also confused why I
wasn't sent any form of notification. Since there was no way to
resume a cancelled transaction, they recommended sending it again,
which I did.

They also gave me a different department to try, but we were near the
end of the business day and I couldn't reach them. I tried again the
next day, 2024-01-05, and I learned that it was cancelled because
there was a delay in the Person to Person Detection Unit. Apparently
this is the first step in their internal checking of whether a
transfer is allowed. It sounds like there was some manual processing
required, with the holidays they didn't have enough staffing to handle
it in time, it "timed out", and was automatically cancelled.

I asked if they could send me written confirmation, but they weren't
authorized to. I talked to a few other people on the phone who also
weren't authorized. Eventually someone told me to go to a branch in
person, which seemed odd, but I went in.

I ended up talking to a Financial Center Manager, who was very
sympathetic. He told me he'd escalate internally, and while it took a
little while I ended up receiving a letter today:

It's unclea...
Dreams of AI alignment: The danger of suggestive names
by TurnTrout — 2024-02-10T01:22:51.715Z

Let's not forget the old, well-read post: Dreams of AI Design. In that essay, Eliezer correctly points out errors in imputing meaning to nonsense by using suggestive names to describe the nonsense.

Artificial intelligence meets natural stupidity (an old memo) is very relevant to understanding the problems facing this community's intellectual contributions. Emphasis added:

A major source of simple-mindedness in AI programs is the use of mnemonics like "UNDERSTAND" or "GOAL" to refer to programs and data structures. This practice has been inherited from more traditional programming applications, in which it is liberating and enlightening to be able to refer to program structures by their purposes. Indeed, part of the thrust of the structured programming movement is to program entirely in terms of purposes at one level before implementing them by the most convenient of the (presumably many) alternative lower-level constructs.

... If a researcher tries to write an "understanding" program, it isn't because he has thought of a better way of implementing this well-understood task, but because he thinks he can come closer to writing the first implementation. If he calls the main loop of his program "UNDERSTAND", he is (until proven innocent) merely begging the question. He may mislead a lot of people, most prominently himself, and enrage a lot of others.

What he should do instead is refer to this main loop as G0034, and see if he can convince himself or anyone else that G0034 implements some part of understanding. Or he could give it a name that reveals its intrinsic properties, like NODE-NET-INTERSECTION-FINDER, it being the substance of his theory that finding intersections in networks of nodes constitutes understanding...

When you say (GOAL . . ), you can just feel the enormous power at your fingertips. It is, of course, an illusion.[1]

Of course, Conniver has some glaring wishful primitives, too. Calling "multiple data bases" CONTEXTS was dumb. It implies that, say, sentence understanding in context is really easy in this system...

Consider the following terms and phrases:

"LLMs are trained to predict/simulate"

"LLMs are predictors" (and then trying to argue the LLM only predicts human values instead of acting on them!)

"Attention mechanism" (in self-attention)
"AIs are incentivized to" (when talking about the reward or loss functions, thus implicitly reversing the true ca

...