trentmkelly/LessWrong-43k huggingface dataset:
[Intuitive self-models] 4. Trance
by Steven Byrnes — 2024-10-08T13:30:41.446Z

4.1 Post summary / Table of contents

Part of the Intuitive Self-Models series.

“Trance” is an umbrella term for various states of consciousness in which “you lose yourself”, somehow. The first kind that I learned about was hypnotic trance, as depicted in the media:

Source: tvtropes

With examples like that, I quite naturally assumed that hypnotism was fictional.

Other types of trance, particularly “spirit possession” in traditional cultures (e.g. Haitian Vodou), and New Age “channeling”, initially struck me as equally fictional—especially the wild claim that people would “wake up” from their hypnotic or other trance with no memory of what just happened. But when I looked into it a bit more, I found myself believing that these are indeed real experiences, even if I couldn’t explain them.

(…Not veridical experiences! Obviously the New Age “channelers” are not literally

“receiving information from paranormal sources”

! But real experiences. See §

1.3.1

–

1.3.2

for the difference.)

Meanwhile, the term “trance” is also applied to the more down-to-earth experience of “losing oneself” in drugs, music, slot machines, or (of course) writing good code. Those seemed less weird to me. But I didn’t see how they had anything to do with the more exotic amnesia-inducing types of trance above.

Then finally I read the “Masks and Trance” chapter of

Keith Johnstone’s

Impro: Improvisation and the Theatre

(1979)

. Not only did he suggest themes and patterns connecting the various types of trance, but he also drew on his many years of personal experience figuring out how to induce trance states in students taking his acting classes. His own trance-induction technique centered around having the students wear masks, look at themselves in the mirror, and thus somehow “become” the mask. He elaborated on that core technique with various other supporting tricks, which I’ll reverse-engineer below. Anyway, that book chapter gave me lots of nuts-and-bolts details to chew on, enough that everything about trance started to click into place.

Thanks to everything we’ve learned in the previous three posts of this series under our belts, I claim that we’re in a great position to understand all the different types of trance. Exactly what are they? What do they have in common? Why do they happen? What makes them tick?

The rest of the post is organized as follows:

Section 4.2–4.3

...
Schelling game evaluations for AI control
by Olli Järviniemi — 2024-10-08T12:01:24.389Z

Playing Schelling games is a key dangerous capability for schemers: it’s much harder to control AIs that are very capable at Schelling coordination. In this post, I discuss the threat model and describe concrete projects for empirical work.

Suppose you wish to control models that adversarially try to cause a catastrophic outcome (“schemers”). A useful class of techniques is monitoring: before letting any action taken by the potentially-scheming policy be executed, you use another model as a monitor to verify the actions are not dangerous.

A key question about monitors is whether they are trustworthy or not, i.e. whether the monitor strategically tries to deceive humans and let dangerous actions be executed. Untrusted monitoring is a technique that uses possibly-untrustworthy models for monitoring. The benefit is that we have no restrictions on the models we may use for monitoring, and in particular we can use our most capable models.

The central drawback is, of course, that the monitor might lie and deliberately label dangerous actions as safe. A previous post by Buck gives an overview of how to prevent such collusion. A central dynamic is that humans can train and evaluate the monitor before deploying it, and thus the monitor needs to determine whether the dangerous actions it sees are produced by the humans or the policy - otherwise it would be selected against in training and/or be caught by humans in evaluations.

The policy can try to make this easier for the monitor by somehow conveying that this is indeed a real attack, not a fake one created by humans. Humans, in turn, try to make it difficult for the monitor to tell real and synthetic attacks apart (including by making it hard for the policy to communicate to the monitor).

The success of the AIs depends in particular on how good they are at Schelling-point coordination relative to humans’ counter-measures.

In more detail, there are two ways[1] the monitor can acquire information about whether the attack it sees is fake or real: via distributional shift in inputs and in actions.

Inputs: the input (which is seen by both the policy and the monitor) suggests that the models are not currently being evaluated. For example, the models are used as assistants in an external company, and the input has a lot of information that the AI company’s evaluators couldn’t hav

...
Thinking About a Pedalboard
by jefftk — 2024-10-08T11:50:02.054Z

As I've been playing more gigs with

Kingfisher I've been

thinking
about how to reduce the time I'm spending on setting up and
tearing down. It usually takes me about 35 minutes to get everything
plugged in, and 20 minutes to get it all packed away again.

Cecilia's
been pushing me to make a pedalboard; what would that look like?

I do have a lot of stuff by my feet:

The key things are the four velocity-sensitive pedals that I use for
drums and
bass. I'm
playing these nearly constantly, and they need to be positioned just
right.

The other things are all helpful, though I use them much less. The
pedals on the right are for modifying the sound of my mandolin, while
the talkbox on the left can apply to the mandolin or my keyboard
overlays. I need to be able to turn these on and off while
playing, but the ergonomics doesn't need to be as good.

Here's what it looks like in context:

There are also other components (usb hub, raspberry
pi) that I currently stick to the piano but would move to the
floor if I had a good place to put them.

I need to measure to figure out the minimum width I can fit these
into, but I think it's about 26". I'd really like this to be
something I can check for flying with, probably by packing my clothes
around it in a regular (<62 linear inches) suitcase.

My normal default would be to figure out the minimum area that would
work and use 3/4" plywood, but
that seems like it might be heavier than necessary. It seems like
it's common for people to use metal
ones but these would add height. Which isn't a total dealbreaker,
but then I'd need an even taller seat for playing on. I'm also not
sure whether the slight angle (usually ~1" over 1') would bother
me.

I think my ideal would be close to something like this but upside
down: metal construction for a good strength-to-weight ratio, but with
the stiffening edge coming up instead of going down so I don't gain a
lot of height.

Suggestions?

Comment via: facebook, mastodon
Overview of strong human intelligence amplification methods
by TsviBT — 2024-10-08T08:37:18.896Z

How can we make many humans who are very good at solving difficult problems?

Summary (table of made-up numbers)

I made up the made-up numbers in this table of made-up numbers; therefore, the numbers in this table of made-up numbers are made-up numbers.

Call to action

If you have a shitload of money, there are some projects you can give money to that would make supergenius humans on demand happen faster. If you have a fuckton of money, there are projects whose creation you could fund that would greatly accelerate this technology.

If you're young and smart, or are already an expert in either stem cell / reproductive biology, biotech, or anything related to brain-computer interfaces, there are some projects you could work on.

If neither, think hard, maybe I missed something.

You can DM me or gmail me at tsvibtcontact.

Context

The goal

What empowers humanity is the ability of humans to notice, recognize, remember, correlate, ideate, tinker, explain, test, judge, communicate, interrogate, and design. To increase human empowerment, improve those abilities by improving their source: human brains.

AGI is going to destroy the future's promise of massive humane value. To prevent that, create humans who can navigate the creation of AGI. Humans alive now can't figure out how to make AGI that leads to a humane universe.

These are desirable virtues: philosophical problem-solving ability, creativity, wisdom, taste, memory, speed, cleverness, understanding, judgement. These virtues depend on mental and social software, but can also be enhanced by enhancing human brains.

How much? To navigate the creation of AGI will likely require solving philosophical problems that are beyond the capabilities of the current population of humans, given the available time (some decades). Six standard deviations is 1 in 10^9, seven standard deviations is 1 in 10^12. So the goal is to create many people who are 7 SDs above the mean in cognitive capabilities. That's "strong human intelligence amplification". (Why not more SDs? There are many downside risks to changing the process that creates humans, so going further is an unnecessary risk.)

It is my conviction that this is the only way forward for humanity.

Constraint: Algernon's law

Algernon's law: If there's a change to human brains that human-evolution could have made, but didn't, then it is net-neutral or net-negative for inclusive relative genet...
Near-death experiences
by Declan Molony — 2024-10-08T06:34:04.107Z

On March 20th, 1995, five men boarded different trains in the Tokyo subway system during rush hour. They were each holding sealed plastic bags wrapped in newspaper and carrying umbrellas with sharpened tips. Placing the bags on the floor, they poked holes in them with the umbrellas. The men promptly departed the trains while the liquid content of the bags spilled onto the floors.

Metro riders began to experience blurred or missing vision, extreme nausea, and foaming at the mouth while having seizures. Unknowingly, they were living through the deadliest terrorist attack in Japanese history. The liquid spilling from the bags was a chemical weapon—sarin gas—which killed 14 people and injured thousands more.

Author Haruki Murakami interviewed some of the survivors. They spoke about their daily lives, how they felt during the incident, and their opinions about the religious cult who perpetrated the attack. Murakami published their responses in a book called Underground: The Tokyo Gas Attack and The Japanese Psyche. The story from one man, Mitsuteru Izutsu, stood out from all the rest:

The day after the gas attack, I asked my wife for a divorce. We weren’t on the best of terms at the time, and I’d done my fair share of thinking while I was [traveling for work] in South America. I had meant to come out and say my piece when I got home, then I walked straight into the gas attack. Still, even after all I’d been through, she would barely speak to me.

After being gassed I phoned home from the office to tell my wife what had happened and my symptoms and everything, but I got almost no reaction from her. Perhaps she couldn’t really grasp the situation, exactly what had occurred.

But even so, I knew then that we’d come to a turning point. Or else, the state I was in had gotten me all worked up. Maybe that’s why I came straight out with it and said I wanted a divorce. Perhaps if this sarin thing hadn’t happened, I wouldn’t have been talking about divorce so soon. I probably wouldn’t have said anything. It was a shock to the system and at the same time a kind of trigger.

My family had been in such a mess for so long, by then I didn’t consider myself very important. Not that the possibility of dying wasn’t real, but, had I died, I probably could have accepted it in my own way as just a kind of accident.

So, did the other survivors also have life-changing epiphanies

...
The unreasonable effectiveness of plasmid sequencing as a service
by Abhishaike Mahajan — 2024-10-08T02:02:55.352Z
Link: https://www.owlposting.com/p/the-unreasonable-effectiveness-of

Note: thank you

Mark Budde

, cofounder and CEO of Plasmidsaurus, and

Maria Konovalova

, a growth/marketing/talented-person at Plasmidsaurus, for talking to me for this article! Also thank you to

Eryney Marrogi

, who helped answer some of my dumb questions about plasmids.

Introduction

Here’s some important context for this essay: it really, really sucks to start a company in biology.

Despite billions in funding, the brightest minds the world has to offer, and clear market need, creating an enduring company here feels almost impossible. Some of this has to do with the difficulties of engaging with the world of atoms, some of it has to do with the modern state of enormously expensive clinical trials, and some of it still can be blamed on something else. To some degree, this is an unavoidable facet of this field; working in it means you’re here for the ‘love of the game’ than anything else.

But is it necessarily fair to equate all for-profit life-science endeavors with grueling, decade-long struggles to bypass scientific obstacles? Is there a world in which life-sciences startups can have a more traditional tech culture ethos in how they approach things? Unfortunately, probably not for a startup aiming to do the traditional therapeutics play.

But if we broaden our scope to companies to include in service provider biotechs, I can offer at least one example: Plasmidsaurus.

Plasmidsaurus was started in 2021 and is currently run by

Mark Budde

. Some historical context: Mark was the founder of a separate, but related company called Primordium Labs, which merged with another separate company called SNPsaurus. They both were largely doing the same thing, so, circa 2022, they agreed to combine underneath the Plasmidsaurus name.

People who currently work on the wet-lab side of biotech have likely not only heard of this company, but are also loyal customers. On the flip side, I would guess that not even computational folks at biotech companies have heard of them, much less anyone in other industries! This is a shame, and something I’ve been wanting to rectify since I first stumbled across this company.

At their start, they really only did one thing: sequence plasmids. We’ll get into what exactly that means later on in the post, but you can think of it as a sort of on-demand quality assurance for certain lab processes. And the way they do that plasmid sequencing isn’t particularl...
There is a globe in your LLM
by jacob_drori — 2024-10-08T00:43:40.300Z

Gurnee & Tegmark (2023) trained linear probes to take an LLM's internal activation on a landmark's name (e.g. "The London Eye"), and predict the landmark's longitude and latitude. The results look like this:[1]

Two angles of true world atlas, with predicted atlas hovering above. True locations are red points; predicted locations are blue, in a slightly raised plane, linked to the corresponding true location by a grey line.

So LLMs (or at least, Llama 2, which they used for this experiment) contain a pretty good linear representation of an atlas.

Sometimes, like when thinking about distances, a globe is more useful than an atlas. Do models use the globe representation? To find out, we can train probes to predict the (x,y,z) coordinates of landmarks, viewed as living in 3D space. Here are the results:

Left: Europe faces us. Middle: The Pacific faces us. Right: India faces us. Predicted points are scaled up by 1.5x for visualization purposes.

You can rotate the plot yourself (and see the code used to generate it) in this notebook.

The average Euclidean (i.e. "through-the-Earth") distance between the true and predicted locations is 535 miles: roughly the distance from Paris to Berlin, from LA to Portland, or from Mumbai to New Delhi.

So LLMs contain a pretty good linear representation of a globe!

I don't know if there's any useful takeaway from this. I just thought it was cool.

^

These plots are my reproduction. I used Llama-3-8B-Instruct, and Gurnee & Tegmark's dataset. For ~7,000 of the 40,000 landmarks, the model did not correctly answer the question "which country is {place} in?", so I removed these from the dataset. I fed landmark names into the model with no prefix, and cached final-token residual-stream activations 70% of the way through the model. All plots and R2 values in this post were generated using a held-out 20% test split.
MATS AI Safety Strategy Curriculum v2
by DanielFilan — 2024-10-07T22:44:06.396Z

As part of our Summer 2024 Program, MATS ran a series of discussion groups focused on questions and topics we believe are relevant to prioritizing research into AI safety. Each weekly session focused on one overarching question, and was accompanied by readings and suggested discussion questions. The purpose of running these discussions was to increase scholars’ knowledge about the AI safety ecosystem and models of how AI could cause a catastrophe, and hone scholars’ ability to think critically about threat models—ultimately, in service of helping scholars become excellent researchers.

The readings and questions were largely based on the curriculum from the Winter 2023-24 Program, with two changes:

We reduced the number of weeks, since in the previous cohort scholars found it harder to devote time to discussion groups later in the program.
For each week we selected a small number of “core readings”, since many scholars were unable to devote time to read everything in the curriculum, and we thought that some readings were more valuable than others.

In addition, the curriculum was supplemented in two ways:

For some weeks, a summary of each reading was compiled for the benefit of discussion group facilitators.
There were some readings that we felt were valuable, but did not fit nicely into any particular week. These supplements were shared with scholars after the discussion series concluded, and are included in this post. When summaries exist, they are shown underneath the reading they summarize, and the additional readings are at the end of the post. Some summaries have been edited for accuracy since being shown to discussion group facilitators and scholars, but they still may contain flaws.

As in the post about the previous cohort’s curriculum, we think that there is likely significant room to improve this curriculum, and welcome feedback in the comments.

Week 1: How powerful is intelligence?

Core readings

The Power of Intelligence

(Rational Animations, Yudkowsky - 7 min)

Superintelligence

Chapter 6: cognitive superpowers (up to "Power over nature and agents") (Bostrom - 14 min)
[In retrospect, it was a mistake to assign a reading that people could not read just by clicking on the link. Instead, it would be better to assign

this LessWrong post

containing a summary and discussion of the relevant section.]

AI could defeat all of us combined

(Karnofsky - 20 min)

Other rea

...
2025 Color Trends
by sarahconstantin — 2024-10-07T21:20:03.962Z

Once again, I’ve compiled some statistics on color trends in the spring/summer 2025 ready-to-wear fashion collections!

Background and Methodology

We just got done with “fashion month”, the flurry of activity in the fall when fashion designers release their spring/summer collections for the coming year and display them on runways in New York, London, Milan, and Paris.[1]

Vogue Magazine generously shares many images from the collections — by my count, 13,570 photographs in all, typically each of a different outfit.[2]

My question is a simple one: which colors are most common in the SS25[3] collections in aggregate? And how do they compare to previous years? Are there changing trends in color popularity?

There’s an obvious and boring answer: the most popular color in clothing is always black. Followed by white. But things get more interesting as we go further down the list.

I use two methods for doing color stats: manual and automated.

The manual color stats are literally me counting the frequency of colors by eye. Each “look” or outfit that a color appears in, counts for one, whether the color is an accent, accessory, or the main color of a garment. Because this is time-intensive, I only count non-neutral colors manually; no black, white, grays, browns, beiges, etc.

The automated color stats are done with a script requesting GPT-4o to make a list of the “specific colors present in this image” for all the images.

Both human error and machine error are real possibilities here, plus “matters of taste” where I disagree with the AI but might also disagree with another human.[4] So it seems like including both scoring methods is best, just to be sure.

Results

Here’s the full GPT-4o automated results:

Automated top colors, SS25

Unsurprisingly, black and white top the list, along with some other neutrals; red, a perennial favorite, is the top non-neutral color.

If we just look at non-neutrals, we see a bold palette of mostly basic brights in the lead: red, pink, blue, yellow, green, orange, etc.

Automated top non-neutral colors, SS25

And if we look at manual scoring, there’s a fairly similar effect: red in the lead (of course), with other bold basic colors (bright yellows, royal blue, hot pink) close behind, as well as some pastels.

Where this gets more interesting is when you compare to collections in previous years.

Last year, the top colors after black & white we...
Clarifying Alignment Fundamentals Through the Lens of Ontology
by Ben Ihrig — 2024-10-07T20:57:33.238Z

Meta: This post grapples with the fundamentals of alignment, which are simultaneously well-trodden ground and quite confusing. I've spent enough time with this post that I no longer have a sense for if the topics are trivial or insightful – I can only imagine that will depend on the reader.

Introduction

Ontology is a common term in alignment thought, but it’s not always crisply clear what is meant by the word. I believe that understanding it well can shed light on some of the more central and confusing aspects of alignment. This is my attempt to clarify the subject – it is an exploration into the dynamics of ontology, how beings navigate it, and what this says about alignment.

In Part 1, I present a formalization of ontology which centrally focuses on how the elements of an ontology, concepts, are grounded – how they are ultimately defined. Personal ontologies are grounded in the perceptions of one being, and universal ontologies are grounded in the full expanse of information that describes the universe. Personal and universal ontology are analogous to map and territory.

Part 2 of the post describes correspondences between concepts in personal and universal ontology, which form the backbone of how beings relate to the world, and whose failure gives rise to some of the deep challenges in alignment.

Part 3 describes those challenges directly and presents a definition of alignment that adds precision to Christiano's characterization of intent alignment.

Much of this post aims at building intuitions for the various topics around ontology and correspondence. The most concrete portion of this post is part 3 and its technical definition of alignment; some readers may benefit from skipping directly to it if the topics are familiar or to get a sense for where things are going.

Background on Ontology

There are multiple senses in which the word “ontology” is used, some of them rather distant from alignment thought. The original usage, ontology as a philosophical discipline, is one such. Literally “the study of being,” the field of ontology,

in the context of philosophy

, is concerned asks the question: what sorts of things exist? What sorts of things have beingness, whatever that may mean? Specific things, like tables and chairs and apples, obviously exist. But do properties, like “redness,” exist? Do numbers exist? Etcetera.

One defining fea...