trentmkelly/LessWrong-43k huggingface dataset:
The Third Gemini
by Zvi — 2024-02-20T19:50:05.195Z

[Editor’s Note: I forgot to cross-post this on Thursday, sorry about that. Note that this post does not cover Gemini 1.5, which was announced after I posted this. I will cover 1.5 later this week.]

We have now had a little over a week with Gemini Advanced, based on Gemini Ultra. A few reviews are in. Not that many, though, compared to what I would have expected, or what I feel the situation calls for. This is yet another case of there being an obvious thing lots of people should do, and almost no one doing it. Should we use Gemini Advanced versus ChatGPT? Which tasks are better for one versus the other?

I have compiled what takes I did see. Overall people are clearly less high on Gemini Advanced than I am, seeing it as still slightly to modestly behind ChatGPT overall. Despite that, I have not been tempted to switch back. I continue to think that Gemini Advanced is the better default for my typical queries. But your use cases and mileage may vary. I highly recommend trying your prompts in both places to see what works for you.

Impressions of Others

My impressions of Gemini Ultra last week were qualified but positive. I reported that I have switched to using Gemini Advanced as my default AI for the majority of queries. It has definite weaknesses, but for the things I actively want to do frequently, it offers a better experience. For other types of queries, I also use GPT-4, Claude, Perplexity and occasionally Phind.

I am not, however, as much a power user of LLMs as many might think. I mostly ask relatively simple queries, I mostly do not bother to use much prompt engineering beyond having customized ChatGPT, and I mostly don’t write code.

What I definitely did not do was look to make Gemini Advanced fail. I figured correctly that plenty of others would work on that. I was looking for how it could be useful, not how it could be shown to not be useful. And I was focused on mundane utility, nothing theoretical.

So I totally buy that both:

When Google tested Gemini Advanced head-to-head against GPT-4, in terms of user preference going query by query, Gemini won. And that it aced the MMLU.

When you run a series of wide-spanning robust tests, Gemini loses to GPT-4.

Here is a 20 minute video of AI Explained testing Gemini Ultra. He took the second approach, and found it wanting in many ways, but says wisely to test it out on your own workflow, and n...
Why does generalization work?
by Martín Soto — 2024-02-20T17:51:10.424Z

Just an interesting philosophical argument

I. Physics

Why can an ML model learn from part of a distribution or data set, and generalize to the rest of it? Why can I learn some useful heuristics or principles in a particular context, and later apply them in other areas of my life?

The answer is obvious: because there are some underlying regularities between the parts I train on and the ones I test on. In the ML example, generalization won't work when approximating a function which is a completely random jumble of points.
Also, quantitatively, the more regular the function is, the better generalization will work. For example, polynomials of lower degree require less data points to pin down. Same goes for periodic functions. Also, a function with lower Lipschitz constant will allow for better bounding of the values in un-observed points.

So it must be that the variables we track (the ones we try to predict or control, either with data science or our actions), are given by disproportionately regular functions (relative to random ones). In this paper by Tegmark, the authors argue exactly that most macroscopic variables of interest have Hamiltonians of low polynomial degree. And that this happens because of some underlying principles of low-level physics, like locality, symmetry, or the hierarchical composition of physical processes.

But then, why is low-level physics like that?

II. Anthropics

If our low-level physics wasn't conducive to creating macroscopic patterns and regularities, then complex systems capable of asking that question (like ourselves) wouldn't exist. Indeed, we ourselves are nothing more than a specific kind of macroscopic pattern. So anthropics explains why we should expect such patterns to exist, similarly to how it explains why the gravitational constant, or the ratio between sound and light speed, are the right ones to allow for complex life.

III. Dust

But there's yet one more step.

Let's try to imagine a universe which is not conducive to such macroscopic patterns. Say you show me its generating code (its laws of physics), and run it. To me, it looks like a completely random mess. I am not able to differentiate any structural regularities that could be akin to the law of ideal gases, or the construction of molecules or cells. While on the contrary, if you showed me the running code of this reality, I'd be able (certainly after many efforts) to differe...
ChatGPT refuses to accept a challenge where it would get shot between the eyes [game theory]
by Bill Benzon — 2024-02-20T16:55:12.352Z

Yesterday it was Jaws and game theory, today it’s Sir Gawain and the Green Knight (henceforth SGGK). But really, the sequence runs in the opposite direction. As you may know, SGGK is a medieval romance that starts and ends in King Arthur’s court. The story is framed by a game, the beheading game. There is a significant literature on games in SGGK and at least one article that analyzes the beheading game, Barry O’Neill, “The Strategy of Challenges: Two Beheading Games in Medieval Literature” (1990).

The beheading game in SGGK goes like this: It is New Year’s Eve at King Arthur’s court. The knights are gathered at the round table, prepared for a holiday meal. But before the meal begins, tradition dictates that one knight must stand up and tell a tale of daring and adventure. Arthur asks for a volunteer. No one rises to the occasion. Then a large green knight enters the hall. He’s riding a green horse and carrying a large green ax. He dismounts and issues a challenge:

I hear that the knights in this court are the bravest in the land. Prove it. I will hand this ax to you and then kneel on the ground so that you may take a swing at my neck with the ax. In return you must agree to journey to the Green Chapel a year’s time from now and allow me to take a swing at your neck with the ax. Will anyone accept the challenge?

No one accepts. The knights are getting restless. It looks like Arthur will take the challenge himself. At this point Gawain stands up: “I accept.”

The story unfolds from there. I first read the story so long ago that I do not remember how I reacted upon reading the challenge. I imagine it went something like this:

Immediately, System 1 signals: “Don’t do it you fool!”

Upon reflection, System 2 spells out why: “The challenge is absurd. Once you swing the ax the knight’s head will fall land he’ll be dead, negating the second half of the bargain. He obviously knows that, is not a fool, and doesn’t want to die. He knows something he’s not telling you. I should refuse the challenge.”

[System 1 and System 2 are from Daniel Kahneman, Thinking, Fast and Slow, 2011.]

Gawain takes the swing, the knight’s head rolls on the floor. The knight then stands up, picks up his head, mounts his horse and turns to Sir Gawin, “See you in a year,” and rides away.

You can see why a game theorist would be interested in this text. Surely any reasonable adult would react to the challe...
Inducing human-like biases in moral reasoning LMs
by Artyom Karpov — 2024-02-20T16:28:11.424Z

Meta. This post is less polished than we would ideally prefer. However, we still think publishing it as is is reasonable, to avoid further delays. We are open to answering questions and to feedback in the comments.

TL;DR. This presents an inconclusive attempt to create a proof-of-concept that fMRI data from human brains can help improve moral reasoning in large language models.

Code is available at

https://github.com/ajmeek/Inducing-human-like-biases-in-moral-reasoning-LLMs

.

Introduction

Our initial motivation was to create a proof of concept of applying an alignment research agenda we are particularly interested in, based on

neuroconnectionism

and

brain-LM

similarities

(and their relevance for alignment): ‘neuroconnectionism as a general research programme centered around ANNs as a computational language for expressing falsifiable theories about brain computation’. Moral reasoning is an interesting application area, both for its relevance to AI alignment and because of the availability of

public neuroimaging data

, as well as of

publicly-available LMs fine-tuned for moral reasoning

.

During the last few years, a series of high-profile papers have shown that

LMs partially converge towards brain-like solutions

and

share fundamental computational principles with humans

, making them a

‘biologically feasible computational framework for studying the neural basis of language’

. For more (recent) evidence of apparent convergence towards human-like representations, see also

Scaling laws for language encoding models in fMRI

,

Large Language Models Converge on Brain-Like Word Representations

.

To the best of our awareness though, the potential LM-brain similarities for linguistic inputs rich in morally-relevant content (e.g. moral scenarios) have not been explored previously. Nor has anyone tried to improve LM moral reasoning using moral reasoning neuroimaging datasets (though

similar ideas have been explored for LMs more broadly

and e.g.

for Convolutional Neural Networks performing object recognition

).

Some other examples of potential areas of investigation (alignment-relevant human capacities) which also seem very neglected could include: instruction following and moral emotions (e.g. compassion, empathy)

.

Additional conceptual motivations. In a

Simulators

-like framework, supervised fine-tuni...
Monthly Roundup #15: February 2024
by Zvi — 2024-02-20T13:10:05.286Z

Another month. More things. Much roundup.

Bad News

Jesse Smith writes in Asterisk that our HVAC workforce is both deeply incompetent and deeply corrupt. This certainly matches my own experience. Calculations are almost always flubbed when they are done at all, outright fraudulent paperwork is standard, no one has the necessary skills.

It certainly seems like the Biden Administration is doing its best to hurt Elon Musk? Claim here is that they cancelled a Starlink contract without justification, in order to award the contract to someone else for more than three times the price. This was on Twitter, but none of the replies seemed to offer a plausible justification.

Claim that Twitter traffic is increasingly fake, and secondary claim that this is because Musk fired those responsible for preventing it. Even if it is true that Twitter traffic is 75% fake, that does not mean that your experience will be 75% bots, or even 7.5% bots. Mine is more like 0.75%. As usual, the bots are mostly making zero attempt to not look like bots, and would be trivial to find and ban if people cared.

Nate Silver is correct that ‘misinformation experts’ are collectively making a mistake when they themselves spread highly partisan misinformation, and also that the game theory makes it impossible for them to collectively stop.

The word ‘genocide’ risks being watered down to the point where we will not have a word for what it used to mean, and this will make it much harder to maintain the taboo of Never Again.

Avatar: The Last Airbender’s live action Netflix version decides that some of the scenes in the original ‘are iffy’ and it needs to soften a cartoon show made for eight year olds, to exclude a character arc where a boy grows up and learns not to be sexist, because the character started off too sexist. Not that I was ever watching anyway. As one commenter notes, if that is an issue, in news related to the previous item, wait until you hear about the Fire Nation.

An essay correctly identifying some (but far from all) of the problems with modern (post-2017) Star Trek, and how things have shifted from Starfleet being fundamentally good and its officers as professionals committed to behaving as such and living up to its ideals, to Starfleet being more often than not the enemy institution to be critiqued and opposed morally and practically, although its rivals are still worse, and ev...
Selections From "The Trouble With Being Born"
by Arjun Panickssery — 2024-02-20T10:07:02.780Z
Link: https://arjunpanickssery.substack.com/p/selections-from-the-trouble-with

I've been aware for some time that there was a Romanian-Parisian existentialist philosopher named Emil Cioran who wrote On the Heights of Despair and The Trouble With Being Born. The other day when mentioning him I pulled up a copy of

Trouble

for the first time and everyone present found it all hilarious. It’s all unconnected aphorisms:

Three in the morning. I realize this second, then this one, then the next: I draw up the balance sheet for each minute. And why all this? Because I was born. It is a special type of sleeplessness that produces the indictment of birth.

“Ever since I was born”—that since has a resonance so dreadful to my ears it becomes unendurable.

Nothing is a better proof of how far humanity has regressed than the impossibility of finding a single nation, a single tribe, among whom birth still provokes mourning and lamentations.

To have committed every crime but that of being a father.

As a general rule, men expect disappointment: they know they must not be impatient, that it will come sooner or later, that it will hold off long enough for them to proceed with their undertakings of the moment. The disabused man is different: for him, disappointment occurs at the same time as the deed; he has no need await it, it is present.

My faculty for disappointment surpasses understanding. It is what lets me comprehend Buddha, but also what keeps me from following him.

Unlike Job, I have not cursed the day I was born; all the other days, on the contrary, I have covered with my anathemas . . .

I long to be free—desperately free. Free as the stillborn are free.

The real, the unique misfortune: To see the light of day.

When we see someone again after many years, we should sit down facing each other and say nothing for hours, so that by means of silence our consternation can relish itself.

I do not forgive myself for being born. It is as if, creeping into this world, I had profaned a mystery, betrayed some momentous pledge, committed a fault of nameless gravity. Yet in a less assured mood, birth seems a calamity I would be miserable not having known.

If I used to ask myself, over a coffin, "What good did it do the occupant to be born?", I now put the same question about anyone alive.

If disgust for the world conferred sanctity of itself, I fail to see how I could avoid canonization.

It’s not worth the bother of killing yourself, since you always kill yourself too late.

An

...
Difficulty classes for alignment properties
by Jozdien — 2024-02-20T09:08:24.783Z

I don't think this idea is particularly novel, but it comes up often in conversations I have, so I figured it'd be good to write it up.

How do you prevent deception from AI systems?

One obvious thing to try would be to make sure that your model never thinks deceptive thoughts. There are several ways you could go about this, to varying degrees of effectiveness. For instance, you could check for various

precursors

to deceptive alignment. More prosaically, you could try identifying deceptive circuits within your model, or looking at what features your model is keeping track of and identifying any suspicious ones.

I think these are pretty reasonable approaches. However, I think they fail to address failure modes like

deep deceptiveness

. A system can be deceptive even if no individual part looks deceptive, due to complex interactions between the system and the environment. More generally, cognition and optimization power can be externalized to the environment.

One could make the argument that focusing on more salient and dangerous failure modes like

deceptive alignment

makes a lot more sense. However - especially if you’re interested in approaches that work in the worst case and don’t rely on reality turning out optimistically one way or the other - you probably want approaches that prevent things from going badly at all.

So, how do you prevent any failure modes that route through deception? In

an earlier post

, I wrote about robust intent alignment as the solution, and one research direction I think is feasible to get there. But here I want to make a different point, about what it would look like to interface with deception in that setup.

Start from the intuition that deception in a system is a property of the person being deceived more than it is the deceiver. It follows pretty naturally that deception is better viewed as a property of the composite system that is the agent and its environment. So, if you wanted to interface with the general thing that is deception, you’re going to be building an interface to that composite system.

Another way of putting this is that evaluating whether an action leads to deception or not requires being able to evaluate things within the complexity class of the environment. Which brings us back to the classical difficulty of RLHF: we can’t foresee the consequences of some action very reliably in a complex environment...
Lessons from Failed Attempts to Model Sleeping Beauty Problem
by Ape in the coat — 2024-02-20T06:43:04.531Z

This is the seventh post in my series on Anthropics. The previous one is Why Two Valid Answers Approach is not Enough for Sleeping Beauty. The next one is The Solution to Sleeping Beauty.

Introduction

As the Sleeping Beauty problem has been investigated for a long time, there have been multiple different attempts to apply various mathematical models to it. And as we are still talking about this problem, these attempts were not quite successful.

Figuring out where these models failed is crucial for understanding the Sleeping Beauty problem and solving it. The last thing we want is to repeat someone else's mistakes yet another time and get attached to a wrong solution.

In this post I'll investigate several such models, point out the issues due to which none of them manages to correctly capture the setting of the problem and thus collect valuable insights about the properties of the correct model.

Updateless Models

Updateless Models are the ones, according to which no update on awakening is supposed to happen, because regardless of the coin toss outcome Beauty can be certain to be awake.

P(Awake|Heads)=1

P(Heads|Awake)=P(Heads)

Intuitively it may look as if such models should necessarily be in favor of answering 1/2, however, this is not the case and historically the first of updateless models is a thirder one.

Elga's Model

This model was originally applied to the Sleeping Beauty problem in the Self-locating belief and the Sleeping Beauty problem paper by Adam Elga. According to it, there are three possible outcomes with equal probability:

Ω={Heads&Monday, Tails&Monday, Tails&Tuesday}

P(Heads&Monday)=P(Tails&Monday)=P(Tails&Tuesday)=1/3

Every outcome describes an awakened state of the Beauty:

P(Awake)=P(Ω)=1

And most of the awakenings happen on Monday:

P(Monday)=P(Heads&Monday)+P(Tails&Monday)=2/3

If the Beauty is awakened on Monday and knows about it, she can't guess Heads or Tails better than chance:

P(Heads|Monday)=P(Tails|Monday)=1/2

And likewise, if she knows that the coin is Tails, her credence for Monday and Tuesday is the same:

P(Monday|Tails)=P(Tuesday|Tails)=1/2

The model is elegant, simple and wrong. The issue is, of course, that according to it the coin isn't fair.

P(Heads)=1/3

P(Tails)=2/3

Therefore, it can't possibly be describing Sleeping Beauty problem.

No-Coin-Toss Problem

The thing is, Elga's model is describin...
flowing like water; hard like stone
by lsusr — 2024-02-20T03:20:46.531Z

SilverFlame

I noticed in your recent post that you described "rhetorical aikido", which you called a Daoist technique. I have studied the Tao Te Ching's text a fair bit over the past few years, and I find its philosophy to be quite useful as a "theme" or "virtue".

I've not had the opportunity to speak with others who seem to understand the Daoist perspective, so I thought this could be an interesting opportunity to expand my understanding a bit.

lsusr

I am flattered you feel I give the impression of understanding the Daoist perspective. I have read only the first sentence of the Tao Te Ching, yet I too find its philosophy to be useful. I thought this could be an interesting opportunity to expand my understanding a bit.

SilverFlame

In that case, I'll mention some of the things that suggested you understood the perspective to me.

I'll repeat the first sentence of the "Tao Te Ching" so that anyone who hasn't read it can see its warning: "Dao called Dao is not Dao." One of the big concepts repeatedly alluded to is that you can't write down what the "Way" is, at best you can gesture in its direction.

Some things I noticed in the post:

The process of the rhetorical aikido described immediately felt right to call Daoist, at least according to my intuition

The emphasis on avoiding open assertion of your own perspective aligns with the oft-repeated concept of "the sage does not contend"
Leading the other person into noticing a possible contradiction aligns with tactics the "Art of War" would suggest, where they might not even notice a plan is afoot

The idea that the technique is meant to be used with those who are legitimately curious also matches the principle of "do not contend"

"Welcome to Lsusr's rationality dojo," I replied, "Today is not your first day."

This statement also caught my eye, albeit not for reasons of resembling the Daoist perspective.

lsusr

Yes. The techniques you point to are indeed Daoist. Not immediately stating your own beliefs follows Laozi's guidelines in the Tao Te Ching. Not immediately revealing your own intentions follows Sunzi's guidelines in The Art of War.

You are also correct in noticing that "Today is not your first day" comes from elsewhere. I stole it from qntm's There is no Antimemetics Division.

I think the fact that you can distinguish Daoist techniques from non-Daoist techniques indicates that we're pointing to something real when we use the wo

...
Getting started at distillations: can critique mine?
by Joyee Chen — 2024-02-20T00:49:28.429Z

I'm Joyee, an aspiring AI alignment researcher (undergrad) at Berkeley, and I'm just getting started with distillations! Since I've heard much of iteration as the way to improvement, and "colliding one's mental model with reality as frequently as possible", I'd like to ask: can someone critique my first one, on the paper "Scaling Laws for Transfer" through the lens of alignment automation efforts?

https://drive.google.com/file/d/1vCV1XWxCxa5rvVIvSEfGfbJ26rtqyaQm/view?usp=sharing

At least as a beginner, my philosophy of distillation is roughly similar to https://www.lesswrong.com/posts/XudyT6rotaCEe4bsp/the-benefits-of-distillation-in-research especially when it comes to "lenses".

I fly the flag of Crocker's rules.