trentmkelly/LessWrong-43k huggingface dataset:
A primer on the current state of longevity research
by Abhishaike Mahajan — 2024-08-22T17:14:57.990Z
Link: https://www.owlposting.com/p/some-questions-and-answers-i-had

Note: This post is co-authored with

Stacy Li

, a PhD student at Berkeley studying aging biology! Highly appreciate all her help in writing, editing, and fact-checking my understanding!

Introduction

The last time I read about aging research deeply was around 2021. The general impression I was getting was that aging research was increasingly more and more funded (good!). Unfortunately, none of the money led to actionable or useful insights (bad).

Over time, you get slightly burnt out by all the negative news.

After getting a job in biotech, I kept a hazy eye on the subject but mostly tuned out of it entirely. But, especially today, I am curious: how has the aging field progressed in the last few years? Since 2021, what has changed?

In this post, I’ll share a list of immediate questions about the state of affairs in aging research, and the answers I’ve found for them. For each question, I’ll offer some basic background knowledge required to understand the question. Feel free to skip that part if you already understand the question!

Did the therapeutic focus on sirtuins amount to much?

Background

Sirtuins are a family of signaling proteins, commonly referred to by their corresponding gene name, SIRT1, SIRT2, all the way up to SIRT7. Their primary role is deacetylation, which is just the removal of a chemical marker (acetyl) on proteins. It was noticed in the 1980s that some sirtuin classes were especially involved in three key activities: modifying histones, which are proteins that tune the accessibility of DNA in the nucleus, transcriptional modification, which determines how DNA is interpreted by the body, and DNA repair, which speaks for itself. And anything involved in modifying and maintaining DNA is something worth paying attention to!

Studies in the 2000s showed that the activity of specific sirtuin classes strongly correlated with age; the young had more sirtuin activity, and the old had less. This seemed to be causative in aging;

overexpressing certain sirtuin genes led to lifespan increase

and

downregulation of them led to lifespan decrease

. The results were a bit mixed, and the results were for yeast cells — always a red flag — but there was some promise in viewing sirtuins as an aging target.

It turns out that editing humans to safely overexpress sirtuin genes is somewhat hard to do (as is expressing any gene in humans). But there was an easy way around that: f...
Some reasons to start a project to stop harmful AI
by Remmelt — 2024-08-22T16:23:34.132Z

Hey, I’m a coordinator of AI Safety Camp. Our program has supported many projects in the past for finding technical solutions, and my two colleagues still do!

Below is my view on safety, and what made me want to support pause/stop AI projects.

With safety, I mean constraining a system’s potential for harm.

To prevent harms, we must ensure that future AI systems are safe:

Safety is

context-dependent

. Harms are the result of the system’s interactions with a more complex surrounding world.
Safety must be

comprehensive

. Safety engineering is about protecting users and, from there, our society and ecosystem at large. If one cannot even design an AI product to not harm current users, there is no sound basis to believe that scaling that design up to larger scales will not also deeply harm future generations.

Today, companies recklessly scale designs and uses of AI models. There is a disregard for human safety. To hide this, companies utilise researchers to give users the impression of safety rather than actual safety. Safety researchers chase after the companies – trying new methods to somehow safely contain the growing combinatorial complexity (and outside connectivity) of models already in use. Billionaires sympathetic to the cause even support the companies to start up ‘safely’. This is a losing game.

Sincere researchers strove to solve lethal risks. Instead they discovered deeper problems that they could at best solve partially, using fragile assumptions then questioned by other researchers. No-one found a method to control the machinery once it starts scaling itself (to not converge on deadly changes to our environment needed for its self-preservation, etc). Some researchers are in fact discovering sharp limits to controlling AI.

There is dignity in informing the public: ‘We did our best to solve safety for autonomous open-ended AI systems. Sadly, we discovered that this problem is intractable.’

Therefore, it is not on us to solve all the risks that accelerationist CEOs and their engineers introduce by releasing unscoped designs. It is on us to hold firm: ‘You shall not pass. No longer shall we allow your reckless behaviour to put our world in true peril.’

We are not alone. Many communities want to prevent companies from harmfully scaling AI. Creatives and privacy advocates aim to stop AI freely feeding on pers...
The economics of space tethers
by harsimony — 2024-08-22T16:15:22.699Z
Link: https://splittinginfinity.substack.com/p/the-economics-of-space-tethers

Some code for this post can be found here.

Space tethers take the old, defunct space elevator concept and shorten it. Rockets can fly up to a dangling hook in the sky and then climb to a higher orbit. If the tether rotates, it can act like a catapult, providing a significant boost in a location where providing thrust is expensive. Kurzgesagt has a nice explainer and ToughSF has a great piece explaining the mechanics and some applications.

Tethers make it cheaper to explore space, but how much cheaper? Let’s look at the benefits.

Tether materials and characteristic velocity

The key performance metric for the tether material is the characteristic velocity:

Vc=√2Tρ

Where T is the tensile strength and rho is the density.

The stronger and lighter the material is, the faster the tether can spin, boosting payloads to higher speeds and saving more fuel. This quickly leads to thinking about exotic materials. Hexagonal boron nitride! Carbon nanotubes! I’m not immune to this kind of speculation, so I’ve added an appendix on the topic. But as I argue in another part of the appendix, we already have good enough materials to make a space tether. The potential gain from studying exotic materials is actually pretty small.

For what it’s worth, I like glass fibers. They’re pretty easy to make, the material can be be sourced in space, they can handle large temperature ranges, and they’re resistant to atomic oxygen environments and UV [1]. They can also get pretty good performance, S-2 glass fibers have a characteristic velocity close to 2 km/s while the best currently available material clocks in at 2.7 km/s.

Now let’s look at why the speed of the tether matters.

Delta V and fuel savings

Rockets have to reach a certain speed in order to orbit any object. For low earth orbit, that’s roughly 7.9 km/s; that’s over Mach 20 here on Earth. The change in velocity, or delta V (dV), required to reach orbit is the currency of spaceflight. You can essentially map out the solar system based on the delta V needed to reach different places:

Source

It takes a lot of fuel and engineering to get a payload up to these speeds, making launches expensive [2][3]. Tethers are exciting because they can wait in orbit and offer a rocket some extra delta V. A tether spinning at 1.5 km/s in LEO can grab a rocket moving at 5.8 km/s and release it at 8.8 km/s:

Source

It takes a while to visualize how these work. Star...
AI #78: Some Welcome Calm
by Zvi — 2024-08-22T14:20:10.812Z

SB 1047 has been amended once more, with both strict improvements and big compromises. I cover the changes, and answer objections to the bill, in my extensive Guide to SB 1047. I follow that up here with reactions to the changes and some thoughts on where the debate goes from here. Ultimately, it is going to come down to one person: California Governor Gavin Newsom.

All of the debates we’re having matter to the extent they influence this one person. If he wants the bill to become law, it almost certainly will become law. If he does not want that, then it won’t become law, they never override a veto and if he makes that intention known then it likely wouldn’t even get to his desk. For now, he’s not telling.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. AI sort of runs for mayor.

Language Models Don’t Offer Mundane Utility. A go or no go decision.

Deepfaketown and Botpocalypse Soon. How hard is finding the desert of the real?

The Art of the Jailbreak. There is always a jailbreak. Should you prove it?

Get Involved. Also when not to get involved.

Introducing. New benchmark, longer PDFs, the hot new RealFakeGame.

In Other AI News. METR shares its conclusions on GPT-4o.

Quiet Speculations. Are we stuck at 4-level models due to Nvidia?

SB 1047: Nancy Pelosi. Local Nvidia investor expresses opinion.

SB 1047: Anthropic. You got most of what you wanted. Your move.

SB 1047: Reactions to the Changes. Reasonable people acted reasonably.

SB 1047: Big Picture. Things tend to ultimately be rather simple.

The Week in Audio. Joe Rogan talks to Peter Thiel.

Rhetorical Innovation. Matthew Yglesias offers improved taxonomy.

Aligning a Smarter Than Human Intelligence is Difficult. Proving things is hard.

The Lighter Side. The future, while coming, could be delayed a bit.

Language Models Offer Mundane Utility

Sully thinks the big models (Opus, 405B, GPT-4-0314) have that special something the medium-sized models don’t have, no matter what the evals say.

A source for Llama-3.1-405-base, at $2 per million tokens (both input and output).

Accelerate development of fusion energy, perhaps? Steven Cowley makes the case that this may be AI’s ‘killer app.’ This would be great, but if AI can accelerate fusion by decades as Cowley claims, then what else can it also do? So few people ...
How do we know dreams aren't real?
by Logan Zoellner — 2024-08-22T12:41:57.380Z

Suppose you believe the following:

the universe is infinite in the sense that every possible combination of atoms is repeated an infinite number of times (either because the negative curvature of the universe implies the universe is unbounded or because of MWI)
Consciousness is an atomic phenomena[1]. That is to say, the only special relationship between past-you and present you is that present you remembers being past you.

In this case, we seem to get something similar to "dust" in Greg Egan's Permutation City, where any sequence of events leading to the present you having your present memories could be considered the "real you".

However, the "conscious you" of your dreams does not have any special attachment or memory to the waking you. That is to say at least sometimes when I'm dreaming (that I am driving a car or falling off a cliff or whatever) I am not also thinking "but this is all a dream and I will wake up soon".

Together, this seems to imply that when I dream there is (somewhere in the universe) a real person who is having the exact same conscious experience as my dream (but is awake).

Now, most of my dreams are fairly ridiculous, so I expect the "probability" that what I am dreaming is "real" is quite small (but not zero).

Maybe this is the same as Boltzmann Brains (where the probability is so small we just ignore it). But some of my dreams aren't that unrealistic.

So, the question is: does this imply that when I'm dreaming there is some probability that I never wake up and the dream me becomes the "real me"? If not, why not?

^

Is there an official name for this theory? ChatGPT suggests "bundle theory" or "Momentariness", but both of those seem to have additional philosophical baggage attached
Measuring Structure Development in Algorithmic Transformers
by Micurie — 2024-08-22T08:38:02.140Z

tl;dr: We compute the evolution of the local learning coefficient (LLC), a proxy for model complexity, for an algorithmic transformer. The LLC decreases as the model learns more structured solutions, such as head specialization.

This post is structured in three main parts, (1) a summary, giving an overview of the main results, (2) the Fine Print, that delves into various cross-checks and details and (3) Discussion and Conclusions.

Structure Formation in Algorithmic Transformers

In this work we study the development of simple algorithmic transformers, which are transformers that learn to perform algorithmic tasks. A major advantage of this setup is that we can control several (hyper)parameters, such as the complexity of the training data and network architecture. This allows us to do targeted experiments studying the impacts of these parameters on the learning dynamics. The main tool we use to study the development is the Local Learning Coefficient (LLC) and we choose cases where we have a reverse-engineered solution.

Why use the LLC for this purpose? It is a theoretically well motivated measure of model complexity defined by Lau et.al. For an overview of Singular Learning Theory (which serves as the theoretical foundation for the LLC) see Liam Carol's Distilling SLT sequence. For a brief overview of the LLC see e.g. this post.

We use the same setup as CallumMcDougall's October Monthly Algorithmic Mech-Interp Challenge. The model is an attention only transformer, trained on sorting numbers with layer norm and weight decay on a cross-entropy loss function using the Adam optimizer. The residual stream size is 96 and the head dimension is 48. It is trained on sequences of the form

[14,47,15,6,27,41,8,16,4,48,SEP,4,6,8,14,15,16,27,41,47,48]

and to predict the next token starting at the separation token. The numbers in the list are sampled uniformly from 0 to 50, which together with the separation token produce a vocabulary of 52 tokens. Numbers do not repeat in the list. The images making up the gifs can be found here.

1-Head Model

Let's first look at the case of a 1-head transformer:

Structure formation for 1-head attention model sorting lists. The heat-maps show the full QK and OV circuits. The upper plot shows the LLC together with other measures such as the translational symmetry[1] and the Rank, which is defined as the sum of the matrix ranks of

...
Deception and Jailbreak Sequence: 1. Iterative Refinement Stages of Deception in LLMs
by Winnie Yang — 2024-08-22T07:32:07.600Z

Executive Overview

As models grow increasingly sophisticated, they will surpass human expertise. It is a fundamentally difficult challenge to make sure that those models are robustly aligned (Bowman et al., 2022). For example, we might hope to reliably know whether a model is being deceptive in order to achieve an instrumental goal. Importantly, deceptive alignment and robust alignment are behaviorally indistinguishable (Hubinger et al., 2024). Current predominant alignment methods only control for the output of the model while leaving the internals of the model unexamined (black-box access, Casper et al., 2024). However, recent literature has started to demonstrate the value of examining the internal of the models that provide additional predictive power that is not affordable by the output of the model alone (Zuo et al., 2023). For example, a recent post (MacDiarmid., 2024) from Anthropic found that a simple probe is able to detect sleeper agents during training.

This is part one of a series of blogs using model internal state to study sophisticated safety related behaviors in large language models (LLMs). As a brief overview of the structure of the blogs:

Blog post 1 and 2 focuses on characterizing the internal state under two different types of safety related behaviors:

Blog post 1

Internal States of Deception
Accompanying

Colab Notebook

Blog post 2

Internal States of Jailbreaks
Accompanying Colab Notebook will be made available soon.

Blog post 3, 4 will focus on comparing two major model steering methods in mechanistic interpretability on the same task (Deception) :

Blog post 3

Focus on focus on contrastive steering
Accompanying Colab Notebook will be made available soon

Blog post 4

Focus on SAE steering
This will be part of my 16-hour project for Neel Nanda & Arthur Conmy's MATS application MATS application

Blog post 5

Steering on Jailbreak-related SAE features

This blog will start to explore dishonesty, focusing on using internal states of the model to characterize the three universal stages of lying, which we found to be consistent across models of different sizes and model families. This internal state oversight can help us in two important aspects

It helps us to dissect the question of whether models have internal representation of belief and build towards a path to measure it.
Importantly, more interpretable internals

...
Just because an LLM said it doesn't mean it's true: an illustrative example
by dirk — 2024-08-21T21:05:59.691Z

This was originally posted in the comments of You don't know how bad most things are nor precisely how they're bad.; I've broken it out into a post because I think it might be a useful corrective more generally for people inclined to cite LLM remarks as fact.

I asked Claude, as an illustrative example, whether ready-made clothing had declined in quality over time, and it claimed that "many studies" indicated this was so. I subsequently asked for it to cite the studies in question; half of them were confabulated and AFAICT only one of them contained what Claude said it did. (Note that this is Claude 3 Haiku, which isn't the best; I'd expect frontier models to name four or maybe even five real papers in a list of six (and, with lower confidence, to have a more accurate understanding of the papers' contents)).

Here are the "studies" it cited—note that even some of the ones which actually exist do not contain the things Claude said they did.

"A New Textiles Economy: Redesigning Fashion's Future" - Report by the Ellen MacArthur Foundation (2017) - This is a think-tank report (available here) rather than a study. However, it does actually exist. Claude says it "noted that the number of times clothes are worn before disposal has decreased by 36% compared to 15 years ago." It really does say this! Page 19, citing as a source the "Circular Fibres Initiative analysis based on Euromonitor International Apparel & Footwear 2016 Edition (volume sales trends 2005–2015)." A-. (Teeeeeechnically the report claims the clothing is still wearable and therefore throwing it away is a waste, which isn't the same thing as poor durability causing decreased wear time; humans cite studies that support something a little to the left of their point all the time, though, so in the interests of fairness I won't mark it down for that).
"The State of Fashion 2016" - Report by the McKinsey Global Fashion Index (2016) - This one seems to be real! (Technically, it's "The State of Fashion 2017"—their first report, so Claude can't have meant an earlier one—but it was released in 2016 and half of it is an overview of same, so IMO it's close enough.) It is not a research study, but rather a think-tank report (which is actually even worse than it seems, IMO, because on the few occasions I've checked sources on think-tank reports I've sometimes found that the results cited didn't seem to actual

...
How do you finish your tasks faster?
by Cipolla — 2024-08-21T20:01:41.306Z

I have the following problem: start with a goal that I know how to reach. Everything is there in my mind. But between the start and the finish I stray away from achieving it. It happens like this: I focus on a sub-task. Then, I focus on sub-(sub-task)s. Till I loose my focus.

I would like my process to be: finish the goal. Then, can focus on the sub-tasks and so on.

As now, I tried:

making lists on my laptop. But I then end up creating new lists that do not feature the original goal.
making lists on paper. It works much better. But I loose again the original goal.

If you have some app or method I would really be grateful. I am looking into the GTD system now.
AI Safety Newsletter #40: California AI Legislation
Plus, NVIDIA Delays Chip Production, and Do AI Safety Benchmarks Actually Measure Safety?
by Corin Katzke — 2024-08-21T18:09:33.284Z
Link: https://newsletter.safe.ai/p/aisn-40-california-ai-legislation

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.

Listen to the AI Safety Newsletter for free on Spotify or Apple Podcasts.

SB 1047, the Most-Discussed California AI Legislation

California's Senate Bill 1047 has sparked discussion over AI regulation. While state bills often fly under the radar, SB 1047 has garnered attention due to California's unique position in the tech landscape. If passed, SB 1047 would apply to all companies performing business in the state, potentially setting a precedent for AI governance more broadly.

This newsletter examines the current state of the bill, which has had various amendments in response to feedback from various stakeholders. We'll cover recent debates surrounding the bill, support from AI experts, opposition from the tech industry, and public opinion based on polling.

The bill mandates safety protocols, testing procedures, and reporting requirements for covered AI models. The bill was introduced by State Senator Scott Wiener, and is cosponsored by CAIS Action Fund, and aims to establish safety guardrails for the most powerful AI models. Specifically, it would require companies developing AI systems that cost over $100 million to develop and are trained on a massive amount of compute to implement comprehensive safety measures, conduct rigorous testing, and mitigate potential severe risks. The bill also includes new whistleblower protections.

A group of renowned AI experts has thrown their weight behind the bill. Earlier this month, Yoshua Bengio, Geoffrey Hinton, Lawrence Lessig, and Stuart Russell penned a letter expressing their strong support for SB 1047. They argue that the next generation of AI systems pose "severe risks" if "developed without sufficient care and oversight." Bengio told TIME, "I worry that technology companies will not solve these significant risks on their own while locked in their race for market share and profit maximization."

However, SB 1047 faces opposition from some industry voices. Perhaps the most prominent critic of the bill has been venture capital firm Andreessen Horowitz (a16z). They argue that it would stifle innovation, shutter small businesses, and “let China take the lead on AI development.”

Sen. Wiener responds to critics. In a recent open letter to a16z and Y Combinator, Sen. Wiener responds to c...
Should LW suggest standard metaprompts?
by Dagon — 2024-08-21T16:41:07.757Z

Based on low-quality articles that seem to be coming up with more regularity, and as mentioned in a few recent posts, AI-generated posts are likely to be a permanent feature of LW (and most online forums, I expect). I wonder if we should focus on harm reduction (or actual value creation, in some cases) rather than trying to disallow something that clearly people want to do.

I wonder how feasible it would be to have a LessWrong-specific workflow for using any or all of the major platforms to assist with (and not fully write) a LW question, a LW summary-of-research post, or a LW rationalist-exploration-of-a-question post (and/or others). This could simply be a help page with sample prompts for "how to generate and use a summary paragraph", "how to generate and modify an outline/thesis sketch", and "how to use the summary and outline to flesh out your ideas on a subtopic".

I've played with these techniques, but I tend to do it all in my captive meatware LLM rather than using an external one, so I don't have a starter example. Do any of you?