trentmkelly/LessWrong-43k huggingface dataset:
Prosthetic Intelligence
by Krantz — 2023-11-08T19:01:04.554Z

Much in the way that we have prosthetic limbs 'do' what we think they should do, we will soon have rational agents that link to fully embodied machines that 'listen to what we think' and simply 'do' what we think they ought.

Let me give you an example of this. Let's say we get to the point where I can train a personal GPT to have the exact same epistemology as me. Feed it a tremendous amount of data about me. An then give it access to listen in on my stream of consciousness. It could act on my behalf. I simply think 'man, I should write up this idea and post it on Lesswrong' and my GPT facsimile listening thinks 'Well, since he has given me permission to do this in the past AND I think he would want me to go ahead and do that for him, I will.'.

This could be very dangerous if we extrapolate this to fleets of machine drones. Obviously.

You effectively are the machines you have.

This presents a different future. Not a future where there is a conflict between 'the goals of humans' and 'the goals of AI'. It's a future where we become the machines. We become the central nervous system for the brain that is 'collective ASI'. The goal of the brain is to listen to the central nervous system. Alignment is the field of ensuring the brain represents the CNS in a fair and balanced way.

Collective intelligence (knowledge mapping in general) is the field of formally demonstrating the values and perspectives of agents within a large group. It's about gathering user data that will be valuable to an altruistic intelligence about what people would actually like to happen. This is the whole reason government was created. Cryptography and decentralization play a critical role in ensuring a fair and balanced operation of the consensus gathering mechanisms. Major work is being done using collective intelligence consensus methods for establishing constitutions for AI to be aligned to.

Prosthetic intelligence also has its benefits. In the case we achieve decentralized consensus mechanisms that are used to align the most powerful AIs to a fair and equitable network of constitutions that shift control of society from the traditional power structures (business and government) to AI, we will have a world that rewards new ideas and novelty (meritocracy) over the capacity to execute and re...
Do you have a satisfactory workflow for learning about a line of research using GPT4, Claude, etc?
by ryan_b — 2023-11-08T18:05:18.442Z

I'm trying to learn about plasma separation techniques, and I just stumbled on a line of papers that don't seem particularly connected to the other lines of research I was looking into, and I would like to quickly get a sense of what it says without having to go through everything the old fashioned way.
What I wanted to do was use Claude or GPT4 to submit a pile of these papers (the recent paper I found, a big chunk of its bibliography, and a chunk of those bibliographies) and then try asking questions about the body of the research in a way similar to how people do it with books. I have a sense it should be possible to use the AI to create a sort of interactive poor man's review paper this way.
So my speculative workflow looks like this:
1. Start with the paper I found.
2. Find other papers that cite it or papers it cites.
3. Get pdf versions of these papers via arxiv/libgen/sci-hub.
4. Load these pdfs into Claude/GPT4/whatever.
5. Have a conversation with the AI about the papers.

Has anyone tried this or something similar, and if so how did it work for you?
What’s going on? LLMs and IS-A sentences 
by Bill Benzon — 2023-11-08T16:58:58.266Z

This is a cross-post from New Savanna.

For the moment I have decided that Waddington’s classic diagram of the epigenetic landscape is a useful way of thinking about when happens when an LLM responds to a prompt. Here’s the diagram:

The language model corresponds to the landscape. The prompt serves to position that ball at a certain place in the landscape – perhaps we can think of that ball as the prompt. The ball then rolls down the valley, going left and right as appropriate. It never reverses direction and goes up the hill. That path, or trajectory if you will, is the LLM’s response to the prompt.

Moreover, I have decided to think of the generation of each word (yes, I know, technically it spits out tokens, not words) as a single primitive operation. That is to say, it has no internal logical structure, no ANDs or ORs. It’s simply one (gigantic) calculation over roughly 175 billion values (in the case of ChatGPT). The generation of each word presents the system with a choice among alternatives, but that’s the only kind of choice involved in calculating the response to a prompt – though for qualification and elaboration, see ChatGPT tells stories, and a note about reverse engineering: A Working Paper, Version 3, pp. 3-6.

That brings me to something I’ve been puzzled about for years. We find it natural to say things like, Garfield is a cat. Now, express the same thought, but reverse the order of cat and Garfield in your sentence. It’s difficult to do. Oh, you can do it, but the resulting sentence is awkward and unnatural, something like, Cats are the kind of thing of which Garfield is a particular instance. No one would ever speak like that, nor write it either.

What’s the source of that asymmetry? As far as I can tell, we don’t know, but I take it as a clue about the mechanisms of language. The purpose of this note is to suggest that my crude model of LLM calculation would provide an answer: The linguistic landscape is structured so that the ball easily rolls from Garfield to cat, or cat to mammal, Snoopy to beagle, Tesla to EV, C. elegans to worm, etc. One might, of course, as why the landscape is arranged in that way, but that’s a different question, no?

Here’s some notes I made about IS-A sentences.

Notes on IS-A Sentences

Somewhere in his Problems in General Linguistics, my copy of which is, alas, in storage, Emile Benveniste has a chapter, “The Nominal Sentenc...
What will happen with real estate prices during a slow takeoff?
by Ricardo Meneghin — 2023-11-08T11:58:14.739Z

This is motivated by How to make money off of AGI. One of the predictions made there (and elsewhere) is that interest rates are going to go up a lot as growth starts to accelerate, and thus it's a good move to take a loan at low fixed rates now. However, the only way mortals can take this kind of loan is through a mortgage, and this is in fact an investment in real estate - if the asset price drops it doesn't matter that rates rose, you'll still lose money.

There are many factors influencing the prices in opposite directions so it's unclear to me what the net effect will be.

Factors influencing prices down:

Rising interest rates generally make home prices drop due to reduced buying pressure.
Automation of high-skilled labor makes demand for housing in large cities drop.
Automation in housing construction and improvements in home quality decrease the value of existing homes. Some say the current scarcity is artificial due to housing regulation, but with the effect of (2) people will be more willing to move to less regulated places.
???

Factors influencing prices up:

General cost-disease dynamics: if people are getting much richer they are willing to pay more for things, and demand for living in concentrated spaces isn't only driven by job opportunities, but also by leisure.
For land specifically, demand for natural resources and energy (solar) is going to explode. Also, you just generally need land to build things on top of.
???

The regulatory response also seems pretty unpredictable to me, and it could influence all of these.
Tall Tales at Different Scales: Evaluating Scaling Trends For Deception In Language Models
by Felix Hofstätter — 2023-11-08T11:37:43.997Z

This post summarizes work done over the summer as part of the Summer 2023 AI Safety Hub Labs programme. Our results will also be published as part of an upcoming paper. In this post, we focus on explaining how we define and evaluate properties of deceptive behavior in LMs and present evaluation results for state-of-the-art models. The work presented in this post was done by the AISHL group of Felix Hofstätter, Harriet Wood, Louis Thomson, Oliver Jaffe, and Patrik Bartak under the supervision of Francis Rhys Ward and with help from Sam Brown (not an AISHL participant).

1 Introduction

For a long time, the alignment community has

discussed the possibility that advanced AI systems may learn to deceive humans

. Recently, the evidence has grown that language models (LMs) deployed in real life may be capable of deception. GPT-4

pretended to be a visually impaired person to convince a human to solve a CAPTCHA

. Meta's Diplomacy AI Cicero

told another player it was on the phone with its girlfriend after a server outage

.

Many large models are sycophantic

, adapting their responses based on a user's preferences, even if that means being less truthful. Recent work shows that

LMs lie and can be prompted to act deceptively

. To make sure that advanced AI systems are safe, ideally, we would like to be able to evaluate if they are deceptive. Our work proposes purely behavioral methods for evaluating properties related to deception in language models. We use them to demonstrate scaling trends for these properties.

The examples above intuitively feel like deception, but it is unclear if they would fit a more rigorous definition. In Section 2, we will explain the operationalization of deception that we are using and discuss related work. This formalization requires assessing the agency, beliefs, and intentions of LMs. We argue that consistency of beliefs is an important property of agency. We also briefly argue that the LMs we evaluate act with intent but leave a proper evaluation of LM intent to future work. In Section 3, we define belief behaviorally and propose different methods for eliciting a model's beliefs. Building on this definition, we operationalize lying in LMs. In Section 4, we present the results of applying our belief evaluations to state-of-the-art LMs. We show that when more compute is spent on either training or inference, LMs...
How well does your research adress the theory-practice gap?
by Jonas Hallgren — 2023-11-08T11:27:52.410Z

(5-6 min read time without exercise instructions)

Motivation for post: Finding a way to map mathematics & areas of academic research to the alignment problem seems to be a core part of theoretical alignment and agent foundations work. Knowing how to frame different toolkits, for example, linear algebra and active inference, into the desiderata needed for solving alignment will likely speed up the research process and clarity of thought of most researchers. This post outlines one frame one can take when building up a map of tools to use when trying to solve the alignment problem. First, the intuition of why will be shown, and then a step-by-step guide to go through the exercise is shared.

Outtake from a very normal alignment conversation:

When laying the puzzle of alignment (where all pieces are rectangles for some reason), Alice said to Bob, “Man, I wish there was some nice way to understand how the frame of vector spaces should fit into this puzzle.”

To which Bob answered:
Bob: “Well… Certain techniques can help you”’

Alice: “How so?”

Bob: “A technique is to look at the edges of the frame of the vector space to see how it fits together with other pieces; for example, the [?] shape between Infrabayesianism and feature representations of model internals fits very well with defining a metric space and, therefore, those two pieces are easy to fit together.”

Alice: “Just because we have pieces that fit together doesn’t mean that the picture it paints is useful”

Bob: “That is why we also have to compare and contrast it with the page that we want in order to make it useful. You have to figure out your desiderata as well as how you see it being implemented in practice to see the theory-practice gap.”
(See The Plan - 2022 Update for more elaboration on the theory-practice gap)

Alice: “I’m not sure I fully understand what “useful” means; as an example if I wanted to understand the mathematical framing of a puzzle piece that is metric spaces, how would that look?”

Bob: “Well, Alice, let me tell you…”

To be continued below

Purpose of post

This post is part of an effort to find useful frameworks for understanding theoretical alignment research and agent foundations. Similar to the exercises of

the maze perspective

or

the game tree of alignment

as John Wentworth previously came up with. The hope is that the framing of a puzzle will bring the ...
Growth and Form in a Toy Model of Superposition
by Liam Carroll — 2023-11-08T11:08:04.359Z

TLDR: This post distills Dynamical and Bayesian Phase Transitions in a Toy Model of Superposition by Chen et al. (2023), where they study developmental stages of the Toy Model of Superposition, understanding growth and form from the perspective of Singular Learning Theory (SLT).

Ernst Haeckel's K

u

nstformen der Natur (1904), plate 1: Phaeodaria

Where do the bewildering and intricate structures of Nature come from? What purpose do they serve? In his famous 1917 book "On Growth and Form" the Scottish biologist and mathematician D'Arcy Wentworth Thompson wrote the following about the geometric forms of Phaeodaria, shown above:

Great efforts have been made to attach a "biological meaning" to these elaborate structures and "to justify the hope that in time their utilitarian character will be more completely recognised" -- "On Growth and Form" p. 695

Chris Olah, who pioneered mechanistic interpretability for neural networks and is somewhat of an intellectual descendant of D'Arcy Wentworth Thompson, has often cited biology as an inspiration for his work. Indeed, in many respects trained neural networks are more similar to biological organisms than to traditional computer programs. One of the most striking parallels is that, just as in biology, structure often forms over neural network training in what appear to be developmental stages.

Developmental Stages of TMS
First row, left: columns of the weight matrix W for the function ReLU(WTWx+b).
First row, right: magnitudes of columns Wi of W and associated biases bi.
Bottom row: losses and model complexity estimator ^λ over training.

In 2022, Elhage et al. introduced the Toy Model of Superposition (TMS) to study when and how models represent more features than they have dimensions, aiming to discern a method to systematically enumerate all features in a model to improve interpretability. Here, we show one case of this model: a neural network trying to reconstruct six-dimensional sparse inputs by encoding them non-linearly in two dimensions. A visualization of the encoding strategy (weights of the neural network) is shown in the top left, and it is visible that over training the network passes through three "forms" or stages of development (labelled 4++−, 4+, 5 for reasons that will be explained below), separated by sudden transitions wh...
Running your own workshop on handling hostile disagreements
by Camille Berger  — 2023-11-08T10:28:13.032Z

tl;dr: this is a guide for running a productive disagreement workshop. Decide of modalities, choose a few techniques, study, practice, make some post-workshop materials and be cautious.

Epistemic Status: Quickly written. Literally reporting what I started doing, this is just a first iteration and it will probably evolve in the near future. This is not perfectly field-proof, specialist advice. Feel free to point to style/syntax/grammar errors in the comments.

[Update : Changed title from "conversational technique" to "handling hostile disagreements"]
There has recently been a post pondering whether LessWrong was missing an opportunity when it comes to some dialectic techniques aiming to improve reasoning and rapport. I've been in contact with those and diving into the related litterature for some time now -since about 2018- and saw an opportunity in trying to teach some of those techniques to people around me.
I have started to give workshops to local EAs in Paris, and have received positive feedback so far. Despite the program being in its infancy, I think it would be great if others were trying to set it up. I think it targets skills that are underdevelopped in most of the rationalist sphere, and that the people going through it are usually satisfied and happy to discover them.

Here is what I did :

1-Duration and amenities

In my opinion, a weekend where atendees come from 10am to 17pm, with a long pause from 12am to 1pm for eating, is a good formula. One technique usually requires three hours of presentation and exercices. Also, have snacks ready.
During a weekend, requesting 12 hours of attention over two days from the attendees was about the maximum I could achieve without burning them out. It's also important for people to be on time. It's almost impossible to catch a workshop in-progress.
However, you're ultimately the one who will know why you're giving your workshop this form and why you teach the techniques is that particular order. Your personal models about how people learn and what you can expect from them matter a lot.
I did try to make workshops for only an afternoon. More people are usually free, but they need to leave early, and it doesn't create the long-duration, conversation-prone atmosphere that a weekend can offer.
[Update : after some practice and experimentation, I think it is possible (but hard) to bring the training time down to 1 ho...
Thinking By The Clock
by Screwtape — 2023-11-08T07:40:59.936Z

I.

I'm sure Harry Potter and the Methods of Rationality taught me some of the obvious, overt things it set out to teach. Looking back on it a decade after I first read it however, what strikes me most strongly are often the brief, tossed off bits in the middle of the flow of a story.

Fred and George exchanged worried glances.

"I can't think of anything," said George.

"Neither can I," said Fred. "Sorry."

Harry stared at them.

And then Harry began to explain how you went about thinking of things.

It had been known to take longer than two seconds, said Harry.

-Harry Potter and the Methods of Rationality, Chapter 25.

This was the very first lesson of LessWrong-style Rationality I actually started trying to deliberately teach myself as a result of my contact with HPMoR and the sequences. This is the powerful technique of actually Thinking By The Clock.

I used to call it Thinking For Five Minutes, but that technique name is a misnomer. It's practically a lie-to-children really. Sometimes I think for much less time, about thirty seconds. Sometimes I think for much more time, like a couple of days. Still, in the way that when you first learn martial arts you might stand in an awkward, stiff stance without turning or stepping I first learned to think by the clock in increments of exactly five minutes.

II.

When I first went to a gym to lift weights, I did it with a friend. I didn't think it was going to work very well (I was a pretty skinny guy) but I wanted to humour them. I sat down on the bench they pointed me at, got a good grip on the heavy thing they wanted me to grab, and lifted it up and down for a while. When they said stop, I stopped. "That seemed kind of fast," I recall saying, "are we done?" Dear reader, we were not done. This pattern repeated when I first started going jogging with a different friend. I somehow expected the whole running thing to last, you know, until we got bored, which happened pretty quickly.

(If I may say a word in defense of younger!me, he really wasn't as unfit as this sounds. Soccer was fun and interesting, and I ran around plenty playing that. Stacking haybales got me paid, and I was quite willing to be paid to lift heavy things as long as I was told.)

So it may not come as a surprise to you that when I first encountered a hard intellectual task that was neither entertaining nor immediately profitable, I kind of bounced off. Going by memo...
Impressions from base-GPT-4?
by mishka — 2023-11-08T05:43:23.001Z

I wonder if some people here had a chance to play with base-GPT-4 (the access is given very selectively for research purposes) and would not mind sharing some of their impressions?

I know that some people have been playing with it, but I've never seen a discussion of impressions and lessons from that. And I know that it is quite nontrivial to get access to this model, but that some access is given.

I think it would be super-interesting for many people here to hear this kind of conversation...
Quantopian contest, but for food intake and weight
by Lucent — 2023-11-08T05:41:35.050Z

Twelve years ago, I lost 100 lbs. in a fairly boring manner by eating 1200 calories a day. Brutally unpleasant and catabolic, but mostly successful. It's creeping back. There are no low-hanging fruits like sugarwater, fast food, eating out. The core problem is my satiety point is ~1000 calories above my RMR, and my body is not fooled by "high satiety" foods.

Since then, I've recorded every bite eaten every day, whether 10 calories of broth on a weekend fast or a 4000 calorie binge of my favorite deep dish "lucent" pizza, using a food scale for everything, not once thoughtlessly pouring oil into a recipe or blindly applying peanut butter to a sandwich. This cultivated superhuman calorie estimation abilities. "You say this is 280 kcal? It tastes like 350. Let me see the recipe… I see, the food label for ghee is off by half." My goal in recording calories is to predict true fat loss by subtracting intake from RMR so daily scale variations aren't discouraging. This is why I believe my records to be so much more accurate than others'—my incentives reward precision over the default of "undercount to stay under budget." I weigh frequently, have impedance body composition data, periodic RMR data from an indirect calorimeter (which I now own), sleep-wake times, all gym sessions, and daily step counts.

Until last year, I believed CICO was cause rather than effect. For whatever reason, I thought, I'm hungrier or have less willpower, and so I must suffer to achieve a healthy weight. My beliefs have since turned toward a meandering set point controlled by a yet unknown homeostat. Whether contamination, omega ratios, a gravitostat, or some combination, anecdotal analysis suggests the set point is a hidden variable, confounding simple analysis. Weight loss when above it is trivial, and below it, fiendishly difficult. This is why I believe a control system is the most likely candidate for a successful model. Anything simpler would be a low-hanging fruit of simple correlation long ago ferreted out by nutrition science. It's unlikely I'd be this size a century ago—almost no one was—so what is it I'm getting too much or not enough of?

Ideally, the massive longitudinal dataset contains dozens or hundreds of overlapping micro experiments (months of keto, weekend fasts, low protein, high fiber, potatoes, waves of monotonous meals) that taken together exceed the statistical significa...