trentmkelly/LessWrong-43k huggingface dataset:
[Intuitive self-models] 1. Preliminaries
by Steven Byrnes — 2024-09-19T13:45:27.976Z

1.1 Summary & Table of Contents

This is the first of a series of eight blog posts. Here’s an overview of the whole series, and then we’ll jump right into the first post!

1.1.1 Summary & Table of Contents—for the whole series

This is a rather ambitious series of blog posts, in that I’ll attempt to explain what’s the deal with consciousness, free will, hypnotism, enlightenment, hallucinations, flow states, dissociation, akrasia, delusions, and more.

The starting point for this whole journey is very simple:

The brain has a predictive (a.k.a. self-supervised) learning algorithm.
This algorithm builds generative models (a.k.a. “intuitive models”) that can predict incoming data.
It turns out that, in order to predict incoming data, the algorithm winds up not only building generative models capturing properties of trucks and shoes and birds, but also building generative models capturing properties of the brain algorithm itself.

Those latter models, which I call “intuitive self-models”, wind up including ingredients like conscious awareness, deliberate actions, and the sense of applying one’s will.

That’s a simple idea, but exploring its consequences will take us to all kinds of strange places—plenty to fill up an eight-post series! Here’s the outline:

Post 1 (Preliminaries) gives some background on the brain’s predictive learning algorithm, how to think about the “intuitive models” built by that algorithm, how intuitive self-models come about, and the relation of this whole series to Philosophy Of Mind.

Post 2 (

Conscious Awareness

)

proposes that our intuitive self-models include an ingredient called “conscious awareness”, and that this ingredient is built by the predictive learning algorithm to represent a serial aspect of cortex computation. I’ll discuss ways in which this model is veridical (faithful to the algorithmic phenomenon that it’s modeling), and ways that it isn’t. I’ll also talk about how intentions and decisions fit into that framework.

Post 3 (

The Homunculus

)

focuses more specifically on the intuitive self-model that almost everyone reading this post is experiencing right now (as opposed to the other possibilities covered later in the series), which I call the Conventional Intuitive Self-Model. In particular, I propose that a key player in that model is a certain entity that’s conceptualized as actively causing acts of free will. Follo

...
AI #82: The Governor Ponders
by Zvi — 2024-09-19T13:30:04.863Z

The big news of the week was of course OpenAI releasing their new model o1. If you read one post this week, read that one. Everything else is a relative sideshow.

Meanwhile, we await Newsom’s decision on SB 1047. The smart money was always that Gavin Newsom would make us wait before offering his verdict on SB 1047. It’s a big decision. Don’t rush him. In the meantime, what hints he has offered suggest he’s buying into some of the anti-1047 talking points. I’m offering a letter to him here based on his comments, if you have any way to help convince him now would be the time to use that. But mostly, it’s up to him now.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. Apply for unemployment.

Language Models Don’t Offer Mundane Utility. How to avoid the blame.

Deepfaketown and Botpocalypse Soon. A social network of you plus bots.

They Took Our Jobs. Not much impact yet, but software jobs still hard to find.

Get Involved. Lighthaven Eternal September, individual rooms for rent.

Introducing. Automated scientific literature review.

In Other AI News. OpenAI creates independent board to oversee safety.

Quiet Speculations. Who is preparing for the upside? Or appreciating it now?

Intelligent Design. Intelligence. It’s a real thing.

SB 1047: The Governor Ponders. They got to him, but did they get to him enough?

Letter to Newsom. A final summary, based on Newsom’s recent comments.

The Quest for Sane Regulations. How should we update based on o1?

Rhetorical Innovation. The warnings will continue, whether or not anyone listens.

Claude Writes Short Stories. It is pondering what you might expect it to ponder.

Questions of Sentience. Creating such things should not be taken lightly.

People Are Worried About AI Killing Everyone. The endgame is what matters.

The Lighter Side. You can never be sure.

Language Models Offer Mundane Utility

Arbitrate your Nevada unemployment benefits appeal, using Gemini. This should solve the backlog of 10k+ cases, and also I expect higher accuracy than the existing method, at least until we see attempts to game the system. Then it gets fun. That’s also job retraining.

o1 usage limit raised to 50 messages per day for o1-mini, 50 per week for o1-preview.

o1 can do multiplication reliably up to about 4×6 digits, andabout 50% accurately up thr...
Slave Morality: A place for every man and every man in his place
by Martin Sustrik — 2024-09-19T04:20:04.491Z

Lately, there has been a discussion between Bentham's Bulldog, Scott Alexander and others about the nature of the slave morality. The outcome for me is that I no longer have any clue about what people meant by it, or, for that matter, what Nietzche once meant by it. To sow even more confusion, here's my interpretation, which, I think, is different from those proposed so far.

Slave morality is a child-like assumption that when you clean up your room, you'll get a cookie. An assumption, that if you follow the rules you are entitled to some kind of reward.

The opposite, the master morality, says that whatever you do, you are not entitled to anything. The world doesn’t care. It's all up to you.

A nice and heart-warming example of master morality comes from American Declaration of Independence with its "life & liberty & pursuit of happiness".

It does not guarantee happiness, however modest. It only guarantees the freedom to give it a try. If you fail and end up unhappy, too bad for you, Declaration of Independence has nothing to console you.

Wikipedia lists comparable mottos worldwide. Say "life, liberty, security of the person" or "life, liberty, enjoyment of property". And you can see how those are a kind of anti-climax. Where the Declaraton of Independence boldly states that everyone is free to pursue happiness in their own way and leaves the resolution of the necessarily resulting conflicts for later, the alternatives give up on master morality and instead try to provide some kind of petty mundane guarantees, only suited to satisfy a slave.

As for slave mentality, a nice example comes from the book "Dune" by Frank Herbert. If you happen to not know, it's a science fiction story in which the humanity lives in a feudal empire spanning thousands planets, with an emperor, noble houses, peons and all the other feudal paraphernalia.

The empire is based on faufreluches social system . The motto of the system is: "A place for every man and every man in his place."

In the Dune world there's no pursuit of happiness. No sir, there isn't. But what you get instead is a guarantee that, as long as you fulfill your obligations, you are not going to be left alone. There's a you-shaped hole in the society and you neatly fit into it.

It's not the same heart-warming feeling as the one evoked by Declaration of Independence, but a heart-warming feeling nonetheless. Right-wingers dr...
Which LessWrong/Alignment topics would you like to be tutored in? [Poll]
by Ruby — 2024-09-19T01:35:02.999Z

Would you like to be tutored in applied game theory, natural latents, CFAR-style rationality techniques, "general AI x-risk", Agent Foundations, anthropic

s

, or some

other topics

discussed on LessWrong?

I'm thinking about prototyping some topic-specific LLM tutor bots, and would like to prioritize topics that multiple people are interested in.

Topic-specific LLM tutors would be customized with things like pre-loaded relevant context, helpful system prompts, and more focused testing to ensure they work.

Note: I'm interested in topics that are written about on LessWrong, e.g. infra-bayesianism, and

not

magnetohydrodynamics".

I'm going to use the same poll infrastructure that

Ben Pace pioneered

recently. There is a

thread below

where you add and vote on topics/domains/areas where you might like tutoring.

Karma: upvote/downvote to express enthusiasm about there being tutoring for a topic.
Reacts: click on the agree react to indicate you personally would like tutoring on a topic.
New Poll Option. Add a new topic for people express interest in being tutored on.

For the sake of this poll, I'm more interested in whether you'd like tutoring on a topic or not, separate from the question of whether you think a tutoring bot would be any good. I'll worry about that part.

Background

I've been playing around with LLMs a lot in the past couple of months and so far my favorite use case is tutoring. LLM-assistance is helpful via multiple routes such as providing background context with less effort than external search/reading, keeping me engaged via interactivity, generating examples, and breaking down complex sections into more digestible pieces.
The Obliqueness Thesis
by jessicata — 2024-09-19T00:26:30.677Z

In my Xenosystems review, I discussed the Orthogonality Thesis, concluding that it was a bad metaphor. It's a long post, though, and the comments on orthogonality build on other Xenosystems content. Therefore, I think it may be helpful to present a more concentrated discussion on Orthogonality, contrasting Orthogonality with my own view, without introducing dependencies on Land's views. (Land gets credit for inspiring many of these thoughts, of course, but I'm presenting my views as my own here.)

First, let's define the Orthogonality Thesis. Quoting Superintelligence for Bostrom's formulation:

Intelligence and final goals are orthogonal: more or less any level of intelligence could in principle be combined with more or less any final goal.

To me, the main ambiguity about what this is saying is the "could in principle" part; maybe, for any level of intelligence and any final goal, there exists (in the mathematical sense) an agent combining those, but some combinations are much more natural and statistically likely than others. Let's consider Yudkowsky's formulations as alternatives. Quoting Arbital:

The Orthogonality Thesis asserts that there can exist arbitrarily intelligent agents pursuing any kind of goal.

The strong form of the Orthogonality Thesis says that there's no extra difficulty or complication in the existence of an intelligent agent that pursues a goal, above and beyond the computational tractability of that goal.

As an example of the computational tractability consideration, sufficiently complex goals may only be well-represented by sufficiently intelligent agents. "Complication" may be reflected in, for example, code complexity; to my mind, the strong form implies that the code complexity of an agent with a given level of intelligence and goals is approximately the code complexity of the intelligence plus the code complexity of the goal specification, plus a constant. Code complexity would influence statistical likelihood for the usual Kolmogorov/Solomonoff reasons, of course.

I think, overall, it is more productive to examine Yudkowsky's formulation than Bostrom's, as he has already helpfully factored the thesis into weak and strong forms. Therefore, by criticizing Yudkowsky's formulations, I am less likely to be criticizing a strawman. I will use "Weak Orthogonality" to refer to Yudkowsky's "Orthogonality T...
How to choose what to work on
by jasoncrawford — 2024-09-18T20:39:12.316Z
Link: https://blog.rootsofprogress.org/how-to-choose-what-to-work-on

So you want to advance human progress. And you’re wondering, what should you, personally, do? Say you have talent, ambition, and drive—how do you choose a project or career?

There are a few frameworks for making this decision. Recently, though, I’ve started to see pitfalls with some of them, and I have a new variation to suggest.

Passion, competence, need

In Good to Great, Jim Collins says that great companies choose something to focus on at the intersection of:

what they are deeply passionate about
what they can be the best in the world at
what drives their economic or resource engine

Jim Collins

This maps naturally onto an individual life/career, if we understand “drives your economic engine” to mean something there is a market need for, that you can make a living at.

You can understand this model by seeing the failure modes if you have only two out of three:

If you can’t be best in the world at it, then you’re just an amateur
If you can’t make a living at it, then it’s just a hobby
If you’re not passionate about it, then why bother?

There is also a concept of ikigai that has four elements:

what you love
what you are good at
what the world needs
what you can be paid for

Forbes

This is pretty much the same thing, except breaking out the “economic engine” into two elements of “world needs it” and “you can get paid for it.” I prefer the simpler, three-element version.

I like this framework and have recommended it, but I now see a couple of ways you can mis-apply it:

One is to assume that you can’t be world-class at something, especially if you have no background, training, credentials, or experience. None of those are necessary. If you are talented, passionate, and disciplined, you can often become world-class quickly—in a matter of years.
Another is to assume that there’s no market for something, no way to make a living. If something is important, if the world needs it, then there is often a way to get paid to do it. You just have to find the revenue model. (If necessary, this might be a nonprofit model.)

Important, tractable, neglected

Another model I like comes from the effective altruist community: find things that are important, tractable, and neglected. Again, we can negate each one to see why all three are needed:

If a problem isn’t tractable, then you’ll never make progress on it
If it isn’t neglected, then you can’t contribute anything new
If it isn’t important, again, w

...
Intention-to-Treat (Re: How harmful is music, really?)
by kqr — 2024-09-18T18:44:41.128Z
Link: https://entropicthoughts.com/intention-to-treat-experiments

I have long wanted to write about intention-to-treat because it's such a neat idea, and the recent article How harmful is music, really? spurred me to finally do it.

The reported results were

Day	Mean mood
Music	0.29
No music	0.22

Making some very rough assumptions about variation, this difference is maybe 1–2 standard errors away from zero, which on could be considered weak evidence that music improves mood.

Except!

There is one big problem with this approach to analysis. Although the experiment started off in a good direction with picking intended music days at random, it then suffered non-compliance, which means the actual days of music are no longer randomly selected. Rather, they are influenced by the environment – which might also influence mood in the same direction. This would strengthen the apparent relationship with no change in the effect of music itself.

The solution is to adopt an intention-to-treat approach to analysis.

Illustrating with synthetic data

I don’t have access to the data dkl9 used, but we can create synthetic data to simulate the experiment. For the sake of this article we’ll keep it as simple as possible; we make some reasonable assumptions and model mood as

gi=M+rgi−1+kmd+bsd+ei

This is a bit dense, but it says that our mood at any given time (gi) is affected by four things:

A baseline mood (M) which is constant and indicative of our life situation more generally.
Our previous mood (gi−1), because if we were unusually happy at lunch, some of that mood is likely to linger in the afternoon. The rate of decay is given by the coefficient r.
Whether we listen to music that day or not (md), a term with strength k. In case it is not yet clear, the purpose of the experiment is figuring out, from data, if k is positive, negative, or zero.
Whether we are in a good situation or not that day (sd), a term with strength b. We cannot infer this term from data because it is indistinguishable from the error term, but the reason we still include it in the model will be apparent soon.[1]

Here's an example of what an experiment might look like under this model. The wiggly line is mood, and the bars indicate whether or not we listen to music each day. (The upper bars indicate listening to music, the lower bars indicate no music.)

The reason we included the situation si as a separate term is that we want to add a correlation between whether we are...
The case for a negative alignment tax
by Cameron Berg — 2024-09-18T18:33:18.491Z

TL;DR:

Alignment researchers have historically predicted that building safe advanced AI would necessarily incur a significant

alignment tax

compared to an equally capable but unaligned counterfactual AI.

We put forward a case here that this prediction looks increasingly unlikely given the current ‘state of the board,’ as well as some possibilities for updating alignment strategies accordingly.

Introduction

We recently

found

that over one hundred grant-funded alignment researchers generally disagree with statements like:

alignment research that has some probability of also advancing capabilities should not be done (~70% somewhat or strongly disagreed)
advancing AI capabilities and doing alignment research are mutually exclusive goals (~65% somewhat or strongly disagreed)

Notably, this sample also predicted that the distribution would be significantly more skewed in the ‘hostile-to-capabilities’ direction.

See ground truth vs. predicted distributions for these statements

These results—as well as

recent

events

and

related discussions

—caused us to think more about our views on the relationship between capabilities and alignment work given the ‘current state of the board,’[1] which ultimately became the content of this post. Though we expect some to disagree with these takes, we have been pleasantly surprised by the positive feedback we’ve received from discussing these ideas in person and are excited to further stress-test them here.

Is a negative alignment tax plausible (or desirable)?

Often, capabilities and alignment are framed with reference to the

alignment tax

, defined as ‘the extra cost [practical, developmental, research, etc.] of ensuring that an AI system is aligned, relative to the cost of building an unaligned alternative.’

The

AF

/

LW

wiki entry on alignment taxes notably includes the following claim:

The best case scenario is No Tax: This means we lose no performance by aligning the system, so there is no reason to deploy an AI that is not aligned, i.e., we might as well align it.

The worst case scenario is Max Tax: This means that we lose all performance by aligning the system, so alignment is functionally impossible.

We speculate in this post about a different best case scenario: a negative alignment tax—namely, a state of affairs where an AI system is actually rendered more competent/p...
Endogenous Growth and Human Intelligence
by Nicholas D. — 2024-09-18T14:05:54.567Z

Hi everyone! I’ve written an article I’m rather happy with on the history of endogenous growth models, and on the influence of intelligence on country level outcomes. As it is quite long, I will excerpt only a part — I sincerely hope you read the whole thing.

https://nicholasdecker.substack.com/p/endogenous-growth-and-human-intelligence

——————————————————

ii. The History of Macroeconomic Growth Models

Macro growth models start in earnest with Solow, who connected capital accumulation to growth. Capital is taken to have diminishing marginal returns, in contrast to the cruder Harrod-Domar model. There exists a rate of savings which maximizes long run consumption, and given a particular technology set consumption will reach a constant level. (This rate of savings is called the

Golden Rule level of savings

, after Phelps). We assume perfect competition in production. (Monopoly distortions can be subtracted from the steady state level of consumption). Initial conditions have no effect on the long run rate, which is the same for all places and much lower than our present living standards. It is therefore necessary to invoke technological change, which is taken to be growing at an exogenously determined rate.

As Arrow wrote

, “From a quantitative, empirical point of view, we are left with time as an explanatory variable. Now trend projections … are basically a confession of ignorance, and what is worse from a practical viewpoint, are not policy variables”

The formulas are simple and clean, and you can make meaningful predictions about growth rates. Still, this clearly does not very well describe the world. There are large differences in per capita income across the globe. If there are diminishing marginal returns to capital, and that is all that matters, then capital should be flowing from developed countries to developing countries. It isn’t. In fact, more skilled people (who can be thought of as possessing a kind of capital, human capital) immigrate to more skilled countries! (

Lucas 1988

). Even if there are bars to capital flowing between countries, no such barriers between southern and northern states in the US.

Barro and Sala-i-Martin (1992)

found that, with reasonable parameters, the return to capital should have been five times higher in the South in the 1880s. Yet, most capital investment took place in New England states.

The bigger problem...
Inquisitive vs. adversarial rationality
by gb — 2024-09-18T13:50:09.198Z

Epistemic status: prima facie unlikely in the usual framework, which I'll try to reframe. Corroborated by loads of empirical observations. YMMV, but if you've held some contrarian view in the past that you came to realize was wrong, this might resonate.

In practical (and also not-so-practical) life, we often have to make a call as to which theory or fact of matter is probably true. In one particularly popular definition of rationality, being rational is making the right call, as often as possible. If you can make the map correspond to the territory, you should.

I believe that in many cases, the best way to do so is not to adopt what I will call inquisitive thinking, in which you, potentially after researching somewhat deeply on a topic, will go on and try to come up with your own arguments to support one side or the other. Rather, I think you should most often adopt adversarial thinking, in which you'll simply judge which side of the debate is probably right on the basis of the existing arguments, without trying to come up with new arguments yourself.

You might feel the adjectives "inquisitive" and "adversarial" are being used wierdly here, but I'm taking them from the legal literature. An inquisitive (aka inquisitorial) legal system is one in which the judge acts as both judge and prosecutor, personally digging into the facts before ruling. An adversarial system, on the other hand, is one in which judges are mostly passive observers, and parties are to argue their case before them without much (or any) interference, for them at the end to rule on the basis of the evidence presented, not being allowed to go dig more evidence themselves.

There is a reason why most legal systems in use today have evolved from (mostly or all) inquisitive to (mostly or all) adversarial, and that's because we have a gigantic body of evidence to suggest that inquisitive systems are particularly prone to render biased judgements. The more you allow judges to go dig, the more likely they are to lose their purported impartiality and start doing strange things.

I suggest that this phenomenon is not particular to judges, but is rather a common feature of human (and very possibly even non-human) rationality. The main point is that digging more and more evidence yourself is ultimately not selecting for truth, but rather for your particular biases. If you have a limited amount of pre-selected ...