trentmkelly/LessWrong-43k huggingface dataset:
Vipassana Meditation and Active Inference: A Framework for Understanding Suffering and its Cessation
by sturb — 2024-03-21T12:32:22.475Z

I want to thank Jan Kulveit, Tomáš Gavenčiak, and Jonathan Shock for their extensive feedback and ideas they contributed to this work and for Josh Burgener and Yusuf Heylen for their proofreading and comments. I would also like to acknowledge the Epistea Residency and its organisers where much of the thinking behind this work was done.

This post aims to build towards a theory of how meditation alters the mind based on the ideas of active inference (ActInf). ActInf has been growing in its promise as a theory of how brains process information and interact with the world and has become

increasingly validated

with a growing body of work in the scientific literature.

Why bring the idea of ActInf and meditation together? Meditation seems to have a profound effect on the experience of people who practise it extensively, and in many cases purports to help people to come to great insight about themselves, reality, and in many cases profoundly alters their relationship to their lived experience. ActInf seems to promise a legible framework for understanding some of the mechanisms that are at play at the root of our experience. Considering these ideas seem to both be pointing at something fundamental about how we experience the world it stands to reason they might be talking about some of the same things in different languages. The hope is that we can use these two to explore these two theories and start to bridge some of the gap in science in providing a theoretical explanation for how these meditative techniques work.

This post will be quite speculative in nature without me providing much in the way of experimental evidence. This is a weakness in the work that I may try to address later but for now I would like to stick to what the theories say and how we can fit them together.

I will focus on the technique of Vipassana meditation and in a future post I will examine Anapana and Metta meditation. I’ll be talking about these techniques because I have a reasonable body of personal experience with them and because I have found their practice leads to fairly predictable and replicable results in those who practise them. My personal experience is the source of much of the discussion below.

Anecdotally, I have found that thinking about suffering in the way described below has helped me to recognise and escape from painful thought cycles where I was able to realise I was gene...
AI #56: Blackwell That Ends Well
by Zvi — 2024-03-21T12:10:05.412Z

Hopefully, anyway. Nvidia has a new chip.

Also Altman has a new interview.

And most of Inflection has new offices inside Microsoft.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. Open the book.

Clauding Along. Claude continues to impress.

Language Models Don’t Offer Mundane Utility. What are you looking for?

Fun With Image Generation. Stable Diffusion 3 paper.

Deepfaketown and Botpocalypse Soon. Jesus Christ.

They Took Our Jobs. Noah Smith has his worst take amd commits to the bit.

Generative AI in Games. What are the important dangers?

Get Involved. EU AI office, IFP, Anthropic.

Introducing. WorldSim. The rabbit hole goes deep, if you want that.

Grok the Grok. Weights are out. Doesn’t seem like it matters much.

New Nivida Chip. Who dis?

Inflection Becomes Microsoft AI. Why buy companies when you don’t have to?

In Other AI News. Lots of other stuff as well.

Wait Till Next Year. OpenAI employees talk great expectations a year after GPT-4.

Quiet Speculations. Driving cars is hard. Is it this hard?

The Quest for Sane Regulation. Take back control.

The Week in Audio. Sam Altman on Lex Fridman. Will share notes in other post.

Rhetorical Innovation. If you want to warn of danger, also say what is safe.

Read the Roon. What does it all add up to?

Pick Up the Phone. More good international dialogue on AI safety.

Aligning a Smarter Than Human Intelligence is Difficult. Where does safety lie?

Polls Show People Are Worried About AI. This week’s is from AIPI.

Other People Are Not As Worried About AI Killing Everyone. Then there’s why.

The Lighter Side. Everyone, reaping.

Language Models Offer Mundane Utility

Ethan Mollick on how he uses AI to aid his writing. The central theme is ‘ask for suggestions in particular places where you are stuck’ and that seems right for most purposes.

Sully is predictably impressed by Claude Haiku, says it offers great value and speed, and is really good with images and long context, suggests using it over GPT-3.5. He claims Cohere Command-R is the new RAG king, crushing it with citations and hasn’t hallucinated once, while writing really well if it has context. And he thinks Hermes 2 Pro is ‘cracked for agentic function calling,’ better for recursive calling than GPT-4, but 4k token limit is an issue. I believ...
An Affordable CO2 Monitor
by Pretentious Penguin — 2024-03-21T03:06:53.255Z

I found a seemingly-high-quality CO2 monitor here for less than $50, or on Amazon or from the manufacturer's website for $70.

The most recent discussions of different CO2 monitors on the market I could find on LessWrong[1] were a few years old, and it seems that prices have come down somewhat since then. I found this positive review of a CO2 monitor manufactured by Qingping,[2] and then found it on AliExpress for about $40 (though it seems to have gone up by about $5 since I ordered it). The review found that its measurements agreed quite well with the more expensive Aranet4, and it uses a Sensirion SCD40 CO2 detector, which is apparently well-respected.

I'm not entirely sure what the think of the evidence around CO2 negatively impacting cognition, but it seems plausible enough to be worth spending <$50 on a monitor.

In the comments of this post and this post. ↩︎

The review refers to it as the SmartAir, but the versions currently being sold on both Amazon and AliExpress have "qingping" in the location where the "SmartAir" branding would otherwise go. I assume it's the exact same product seeing as the product listings appear otherwise identical. ↩︎
DeepMind: Evaluating Frontier Models for Dangerous Capabilities
by Zach Stein-Perlman — 2024-03-21T03:00:31.599Z
Link: https://arxiv.org/abs/2403.13793

To understand the risks posed by a new AI system, we must understand what it can and cannot do. Building on prior work, we introduce a programme of new “dangerous capability” evaluations and pilot them on Gemini 1.0 models. Our evaluations cover four areas: (1) persuasion and deception; (2) cyber-security; (3) self-proliferation; and (4) self-reasoning. [Evals for CBRN capabilities are under development.] We do not find evidence of strong dangerous capabilities in the models we evaluated, but we flag early warning signs. Our goal is to help advance a rigorous science of dangerous capability evaluation, in preparation for future models.

At last, DeepMind talks about its dangerous capability evals. With details! Yay!

(These evaluations were done in January 2024; the Gemini 1.0 models were released in December 2023. DeepMind hasn't yet made RSP-like commitments — that is, specific commitments about risk assessment (for extreme risks), safety and security practices as a function of risk assessment results, and training and deployment decisions as a function of risk assessment results. Demis recently suggested that DeepMind might make RSP-like commitments this year.)

Random interesting note: DeepMind hired 8 superforecasters to make relevant predictions, most notably about when some eval-thresholds will trigger.

See summary on Twitter by last author Toby Shevlane.
Where are the Contra Dances?
by jefftk — 2024-03-21T02:00:03.887Z

About ten years ago

I made a heatmap of
contra dances, which I've

kept up to date.
I recently had a request from someone who'd like to print out a copy
for a poster, and while the one I built on the Google Maps API doesn't
have a way to generate a high-resolution version it seemed like fun to
make one from scratch:

This one includes both special events and regular dances. The code is on
github. While I usually don't find AI tools very helpful in
coding and don't bother, iterating on charts is a place where I find
they do speed me up a lot.

I'm pretty happy with it, but let me know if you have tweaks to suggest!

Comment via: facebook, mastodon
Slim overview of work one could do to make AI go better (and a grab-bag of other career considerations)
by Chi Nguyen — 2024-03-20T23:17:52.964Z

Many kinds of work one could do to make AI go better and a grab-bag of other career considerations

I recently found myself confused about what I’d like to work on. So, I made an overview with the possible options for what to work on to make AI go well. I thought I’d share it in case it’s helpful for other people. Since I made this overview for my own career deliberations, it is tailored for myself and not necessarily complete. That said, I tried to be roughly comprehensive, so feel free to point out options I’m missing. I redacted some things but didn’t edit the doc in other ways to make it more comprehensible to others. In case you’re interested, I explain a lot of the areas in the “Humans in control” and the “Misalignment” worlds here and to some extent here.

What areas could one work on? What endpoints or intermediary points could one aim for?

Note that I redacted a bunch of names in “Who’s working on this” just because I didn’t want to bother asking them and I wasn’t sure they had publicly talked about it yet, not because of anything else.

“?” behind a name or org means I don’t know if they actually work on the thing (but you could probably find out with a quick google!)

World it helps	The area (Note that this doesn’t say anything about the type of work at the moment. For example, I probably should never do MechInterp myself because of personal fit. But I could still think it’s good to do something that overall supports MechInterp.)	Biggest uncertainty	Who’s working on this
Hu- mans in con- trol

ASI governance | human-control

Who is in control of AI, what’s the governance structure etc.
Digital sentience
[...]

Is this tractable and is success path-dependent?	Will MacAskill, [redacted]?, indirectly: cybersec. folk?, some AI governance work?

Acausal interactions | human-control

Metacognition
Decision theory
Values of future civilisation
SPIs

[redacted]
SPIs for causal interactions | human-control	CLR
Mis- align- ment	Prevent sign flip and other near misses	Is this a real concern?	Nobody?

Acausal interactions | misalignment

Decision theory
Value porosity

Is this tractable?	[redacted]? [redacted]?
Reducing conflict-conducive preferences for causal interactions & SPIs | misalignment	CLR

Main- stream AI safety best thing to work on

Reduction of malevolence in positions of influence through improving awareness (also goes into the “Humans in control” category)	[redacted]? Nobody

...
How does AI solve problems?
by Dom Polsinelli — 2024-03-20T22:29:36.129Z

Epistemic Status: This is mostly just something that has been bouncing around in my mind, this is not a detailed well research essay with many citations about modern interpretability. I go back and forth on this being so obvious everyone here already implicitly agrees and it being something that was handily debunked a while ago and I just missed the memo.

I am able to solve a Rubik’s cube. This isn’t actually an impressive feat but there is a significant fraction of the population that can’t do this despite exerting all their effort when given the puzzle. I’m not especially fast, I never got into speed cubing, but I can comfortably solve the puzzle in under 3 minutes whereas someone unfamiliar with the cube will struggle for much longer than that and probably give up before solving it. What at least some people (who can’t solve the Rubik’s cube) don’t understand, is that my approach to solving it is completely different from theirs. There are three very different kinds of things that the term solving can apply to. The first is the naive attempt I took when I was a kid. That is, solving it like any other puzzle. You see the blue side is not all blue and you try to make it that way. Then, when you get that you try to make the white side and so on. (If you’re more clever you may attempt to solve based on layers but I digress). You want to get a particular scrambled state to the one solved state by whatever means necessary, usually by thinking quite hard about each move. I, at least as a child, did not have very much success with this method. However, you might try something very different. You may look for sequences of moves that change the position of the pieces in predictable and useful ways. Once you build up a collection of these sequences that can manipulate different pieces you can apply them without thinking deeply about each move in particular, you just select the sequence of moves that gets you from A to B without actually thinking about the messy middle. It is very hard to think through how to rotate an edge price without messing up the rest of the cube, but once you do it you can use it over and over again and now you just need to see when you ought to rotate an edge. Now the third meaning of the word “solve” becomes clear. It involves none of the hard thinking about each move or even the hard thinking to find the useful sequence of moves, it on...
What I Learned (Conclusion To "The Sense Of Physical Necessity")
by LoganStrohl — 2024-03-20T21:24:37.464Z

This is the concluding post in a sequence that demonstrates a complete naturalist study, specifically a study of query hugging (sort of), as described in The Nuts and Bolts of Naturalism. For context on this sequence, see the intro post.

Here is where my conceptualization of query hugging stands currently. It is still under construction, but this is what happens when I try to describe it right now.

The framing of “Hug the Query” suggests that in order to stay close to the issue, most of what’s needed is to not get distracted by less relevant things. It assumes that the reader is aware of a sense of how close to the issue they are (or, as Eliezer might put it, that they are tracking their location in The Great Causal Web), and then cautions them to heed that sense as they evaluate arguments.

I affectionately imagine Eliezer tearing at his hair while composing his essay. “Why do they keep getting distracted?! If they would just stay on track! It’s a theorem of graph theory, for godsakes!”

(My model of Eliezer often struggles with accidentally overestimating his audience on some unusual axis.)

I now think that Past Eliezer wrote slightly the wrong essay. It was an excellent essay, but it presented a solution to a different problem than the one I suspect people actually tend to have.

(I talked with Present Eliezer about this, and he seemed to agree with me; but I’m only describing my own view here. He may see things differently.)

In my study, I found that the thing to hug is not really a “query” in the sense of a question, a symbolically expressed topic; rather, it’s 1) whatever physically necessitates the answer be what it is, and (simultaneously) 2) whatever value deep in my heart cares about the topic in the first place.

The central skill behind query hugging—behind remaining close to the issue at hand—is a certain kind of awareness. Specifically, it is awareness of my sense of the physical necessity of things: my sense of the physical parts clicking together, of inevitability, the sense that tightens my core as I watch my cat knock a water glass off of a shelf and I know it will spill on my keyboard because I can see that there is nothing in the way. The prerequisite for query hugging is the ability to sink my guts into the physical systems. From there, I just need to be aware of what I personally care about, without separating myself from those systems.

My con...
Stagewise Development in Neural Networks
by Jesse Hoogland — 2024-03-20T19:54:06.181Z

TLDR: This post accompanies The Developmental Landscape of In-Context Learning by Jesse Hoogland, George Wang, Matthew Farrugia-Roberts, Liam Carroll, Susan Wei and Daniel Murfet (2024), which shows that in-context learning emerges in discrete, interpretable developmental stages, and that these stages can be discovered in a model- and data-agnostic way by probing the local geometry of the loss landscape.

Four months ago, we shared a discussion here of a paper which studied stagewise development in the toy model of superposition of Elhage et al. using ideas from Singular Learning Theory (SLT). The purpose of this document is to accompany a follow-up paper by Jesse Hoogland, George Wang, Matthew Farrugia-Roberts, Liam Carroll, Susan Wei and Daniel Murfet, which has taken a closer look at stagewise development in transformers at significantly larger scale, including language models, using an evolved version of these techniques.

How does in-context learning emerge? In this paper, we looked at two different settings where in-context learning is known to emerge:

Small attention-only language transformers, modeled after Olsson et al. (3m parameters).
Transformers trained to perform linear regression in context, modeled after Raventos et al. (50k parameters).

Changing geometry reveals a hidden stagewise development. We use two different geometric probes to automatically discover different developmental stages:

The local learning coefficient (LLC) of SLT, which measures the "basin broadness" (volume scaling ratio) of the loss landscape across the training trajectory.
Essential dynamics (ED), which consists of applying principal component analysis to (a discrete proxy of) the model's functional output across the training trajectory and analyzing the geometry of the resulting low-dimensional trajectory.

In both settings, these probes reveal that training is separated into distinct developmental stages, many of which are "hidden" from the loss (Figures 1 & 2).

Figure 1: The development of 2-layer attention-only language transformers.
Neural networks undergo stagewise development. This is often not visible in the loss (top left) but can be discovered by the local learning coefficient (bottom left) and essential dynamics (right three columns).

Figure 2: The development of linear-regression transformers. Plateaus in the LLC and particular

...
On the Gladstone Report
by Zvi — 2024-03-20T19:50:05.186Z

Like the the government-commissioned Gladstone Report on AI itself, there are two sections here.

First I cover the Gladstone Report’s claims and arguments about the state of play, including what they learned talking to people inside the labs. I mostly agree with their picture and conclusions, both in terms of arguments and reported findings, however I already mostly agreed. If these arguments and this information is new to someone, and the form of a government-backed report helps them process it and take it seriously, this is good work. However, in terms of convincing an already informed skeptic, I believe this is a failure. They did not present their findings in a way that should be found convincing to the otherwise unconvinced.

Second I cover the Gladstone Report’s recommended courses of action. It is commendable that the report lays out a concrete, specific and highly detailed proposal. A lot of the details, and the broader outline, seem good. The compute thresholds seem too aggressive. I would suggest working to get agreement on the structure of intervention, while also talking price on the thresholds, and hopefully getting them to a better place.

Executive Summary of Their Findings: Oh No

According to the report, things are very much Not Great, Bob.

Here is their Twitter summary thread.

Edouard Harris: Here’s what we’ve been working on for over a year:

The first US government-commissioned assessment of catastrophic national security risks from AI — including systems on the path to AGI.

TLDR: Things are worse than we thought. And nobody’s in control.

We started this work with concerns, but no preconceptions. We knew there were solid technical reasons that AI could eventually pose catastrophic risks.

But we went in looking for reasons to change our minds.

We found the opposite.

Our overriding goal was to get to the truth. To do that, we had to do more than just speak to policy and leadership at the AI labs.

We also connected with individual technical researchers, many of whom are way more concerned than their labs let on in public.

Many of these folks came forward on condition of anonymity to share stories.

Let me tell you some of the most insane stuff we learned.

First off, inside one lab there’s apparently a running joke that their security is so bad that they’re doing more to accelerate the AI capabilities of US adversaries, than the adversaries thems

...
Natural Latents: The Concepts
by johnswentworth — 2024-03-20T18:21:19.878Z

Suppose our old friends Alice and Bob decide to undertake an art project. Alice will draw a bunch of random purple and green lines on a piece of paper. That will be Alice’s picture (A). She’ll then make a copy, erase all the purple lines, and send the result as a message (M) to Bob. Bob then generates his own random purple lines, and adds them to the green lines from Alice, to create Bob’s picture (B). The two then frame their two pictures and hang them side-by-side to symbolize something something similarities and differences between humans something. Y’know, artsy bullshit.

Now, suppose Carol knows the plan and is watching all this unfold. She wants to make predictions about Bob’s picture, and doesn’t want to remember irrelevant details about Alice’s picture. Then it seems intuitively “natural” for Carol to just remember where all the green lines are (i.e. the message M), since that’s “all and only” the information relevant to Bob’s picture.

In this example, the green lines constitute a “natural latent” between the two pictures: they summarize all and only the information about one relevant to the other.

A more physics-flavored example: in an isolated ideal-ish gas, average energy summarizes “all and only” the information about the low-level state (i.e. positions and momenta of the constituent particles) at one time which is relevant to the low-level state at a sufficiently later time. All the other information is quickly wiped out by chaos. Average energy, in this case, is a natural latent between the gas states at different times.

A more old-school-AI/philosophy example: insofar as I view dogs as a “kind of thing” in the world, I want to track the general properties of dogs separately from the details of any specific dog. Ideally, I’d like a mental pointer to “all and only” the information relevant to many dogs (though I don’t necessarily track all that information explicitly), separate from instance-specific details. Then that summary of general properties of dogs would be a natural latent between the individual dogs.

Just from those examples, you probably have a rough preliminary sense of what natural latents are. In the rest of this post, we’ll:

Walk through how to intuitively check whether a particular “thing” is a natural latent over some particular parts of the world (under your intuitive models).
Talk about some reasons why natural latents

...