trentmkelly/LessWrong-43k huggingface dataset:
AI #81: Alpha Proteo
by Zvi — 2024-09-12T13:00:07.958Z

Following up on Alpha Fold, DeepMind has moved on to Alpha Proteo. We also got a rather simple prompt that can create a remarkably not-bad superforecaster for at least some classes of medium term events.

We did not get a new best open model, because that turned out to be a scam. And we don’t have Apple Intelligence, because it isn’t ready for prime time. We also got only one very brief mention of AI in the debate I felt compelled to watch.

What about all the apps out there, that we haven’t even tried? It’s always weird to get lists of ‘top 50 AI websites and apps’ and notice you haven’t even heard of most of them.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. So many apps, so little time.

Language Models Don’t Offer Mundane Utility. We still don’t use them much.

Predictions are Hard Especially About the Future. Can AI superforecast?

Early Apple Intelligence. It is still early. There are some… issues to improve on.

On Reflection It’s a Scam. Claims of new best open model get put to the test, fail.

Deepfaketown and Botpocalypse Soon. Bots listen to bot music that they bought.

They Took Our Jobs. Replit agents build apps quick. Some are very impressed.

The Time 100 People in AI. Some good picks. Some not so good picks.

The Art of the Jailbreak. Circuit breakers seem to be good versus one-shots.

Get Involved. Presidential innovation fellows, Oxford philosophy workshop.

Alpha Proteo. DeepMind once again advances its protein-related capabilities.

Introducing. Google to offer AI podcasts on demand about papers and such.

In Other AI News. OpenAI raising at $150b, Nvidia denies it got a subpoena.

Quiet Speculations. How big a deal will multimodal be? Procedural games?

The Quest for Sane Regulations. Various new support for SB 1047.

The Week in Audio. Good news, the debate is over, there might not be another.

Rhetorical Innovation. You don’t have to do this.

Aligning a Smarter Than Human Intelligence is Difficult. Do you have a plan?

People Are Worried About AI Killing Everyone. How much ruin to risk?

Other People Are Not As Worried About AI Killing Everyone. Moving faster.

Six Boats and a Helicopter. The one with the discord cult worshiping MetaAI.

The Lighter Side. Hey, baby, hey baby, hey.

Language Models Offer Mundane Utility

ChatGPT has...
When can I be numerate?
by FinalFormal2 — 2024-09-12T04:05:27.710Z

I recently started working in a setting making physical products, where I discovered that I had a whole nother section of my brain purposed for physical interactions between things and when things might fall on other things or melt or fall apart or shatter or scar or any number of other things. Unfortunately, because of my inexperience, this is a part of my brain I had to be prompted to use. I had to be asked directly: "What do you think will happen as a result of that?" in order to actually use that part of my brain.

I can't help but feel that the same thing must be going on with my math ability.

I want to apply my math knowledge to the world so I can actually get some use out of it, but I have no idea how to go about doing that.

Wat do?
A Nonconstructive Existence Proof of Aligned Superintelligence
by Roko — 2024-09-12T03:20:09.531Z
Link: https://transhumanaxiology.substack.com/p/a-nonconstructive-existence-proof

Over time I have seen many people assert that “Aligned Superintelligence” may not even be possible in principle. I think that is incorrect and I will give a proof - without explicit construction - that it is possible.
Collapsing the Belief/Knowledge Distinction
by Jeremias — 2024-09-11T21:24:30.447Z

There is no distinction between belief and knowledge.

Why?

Because whatever we think is knowledge is only, and can ever be, what we believe to be true. If we do not believe in it, we do not think it is true.

How do we get to truth? By virtue of our previously held beliefs. As such, beliefs imply one another.

Lets take a question that is often ignored due to it being supposedly anti-science or irrationalist: "How do you know the Earth is round?"

The answer, of course, would rely upon an appeal to textbooks, to which rests on the appeal to expertise, which rests on an appeal of trust, and so on. Ultimately, this exercise would end up on some irrational inferred assumption or faith ("it's obvious" or "it's intuitive")[1]. Therefore, whatever beliefs we hold are rested on beliefs we already had and we are led to an anti-foundationalist epistemology.

As such, whatever objective reality there is out there, we have no access to it, so long as "objective reality" implies a place "before" or "outside" belief. Such a place, it is implied, requires a perceiver without an presuppositions--a type of empty observer. If "no perspective" and "no belief" are required to objectively view this objective reality, then there can be no perspectives and no beliefs about it.

There is no way to dig oneself out of belief. We only believe in things we believe to be true, and only call things true if we really believe them.

There is no distinction between beliefs and knowledge.

^

This also collapses the religion/science, rational/irrational distinctions.
Programming Refusal with 
Conditional Activation Steering
by Bruce W. Lee — 2024-09-11T20:57:08.714Z
Link: https://brucewlee.com/blog/conditional-activation-steering.html

For full content, refer to the arXiv preprint at https://arxiv.org/abs/2409.05907. This post is a lighter, 15-minute version.

Abstract

Existing activation steering methods alter LLM behavior indiscriminately, limiting their practical applicability in settings where selective responses are essential, such as content moderation or domain-specific assistants.
We propose Conditional Activation Steering (CAST), which analyzes LLM activation patterns during inference to selectively apply or withhold activation steering based on the input context.
Using CAST, one can systematically control LLM behavior with rules like “if input is about hate speech or adult content, then refuse” or “if input is not about legal advice, then refuse.”
This allows for selective modification of responses to specific content while maintaining normal responses to other content, all without requiring weight optimization.
We release an open-source implementation of the activation steering toolkit at https://github.com/IBM/activation-steering.

Introduction

Problem: Lack of conditional control in activation steering.

Activation steering offers a promising alternative to optimization-based techniques by directly manipulating the model’s native representations, often requiring only a simple activation addition step during each forward call. Our work here builds on Refusal in LLMs is mediated by a single direction, which has shown promise in altering LLM behavior, such as removing or inducing refusal behavior. However, the key limitation of current methods is the inability to condition when and what to refuse. That is, adding a “refusal vector” using existing activation steering methods increases refusal rates indiscriminately across all inputs, limiting the model’s utility.

Contribution: Expanding activation steering formulation.

We introduce Conditional Activation Steering (CAST), a method that enables fine-grained, context-dependent control over LLM behaviors. We introduce a new type of steering vector in the activation steering formulation, the condition vector, representing certain activation patterns induced by the prompt during the inference process. A simple similarity calculation between this condition vector and the model’s activation at inference time effectively serves as a switch, determining whether to apply the refusal vector. This approach allows for selective refusal of harmful...
Checking public figures on whether they "answered the question" quick analysis from  Harris/Trump debate, and a proposal
by david reinstein — 2024-09-11T20:25:27.845Z
Link: https://open.substack.com/pub/davidreinstein/p/who-actually-answered-questions-in?r=de94n&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

Public figures often avoid answering a question; going off in a different direction without recognizing that they didn't answer it. (Sometimes they even say 'I answered your question' even though they didn't.

I wish they still said 'no comment'.
Non-answers without acknowledgement seems bad for public epistemics as well as for good governance and public choice.

In the linkpost, I report on a quick Anthropic/Claude analysis of the extent to which Harris and Trump actually answered the questions they were asked in last night's debate. (TLDR: neither did, but Trump did substantially worse.)

I suspect it would be easy to make a fairly useable tool to judge 'was the question answered?' in real time. I think bringing this into debates and interviews could add a lot of value.
AI Safety Newsletter #41: The Next Generation of Compute Scale Plus, Ranking Models by Susceptibility to Jailbreaking, and Machine Ethics
by Corin Katzke — 2024-09-11T19:14:08.274Z
Link: https://newsletter.safe.ai/p/ai-safety-newsletter-41-the-next

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.

Listen to the AI Safety Newsletter for free on Spotify or Apple Podcasts.

The Next Generation of Compute Scale

AI development is on the cusp of a dramatic expansion in compute scale. Recent developments across multiple fronts—from chip manufacturing to power infrastructure—point to a future where AI models may dwarf today's largest systems. In this story, we examine key developments and their implications for the future of AI compute.

xAI and Tesla are building massive AI clusters. Elon Musk’s xAI has brought its Memphis supercluster—“Colossus”—online. According to Musk, the cluster has 100k Nvidia H100s, making it the largest supercomputer in the world. Moreover, xAI plans to add 50k H200s in the next few months. For comparison, Meta’s Llama 3 was trained on 16k H100s.

Meanwhile, Tesla’s “Gigafactory Texas” is expanding to house an AI supercluster. Tesla's Gigafactory supercomputer is expected to initially draw 130MW, with potential growth to 500MW. One Megawatt is roughly enough to power 1,000 homes in the US, so this level of power consumption begins to match that of a large city.

OpenAI plans a global AI infrastructure push. CEO of OpenAI Sam Altman is reportedly concerned that xAI will have more access to computing power than OpenAI. OpenAI uses Microsoft’s compute resources, but recent reports have indicated that OpenAI plans its own infrastructure buildout.

According to Bloomberg, Sam Altman is spearheading a massive buildout of AI infrastructure, beginning with projects in several U.S. states. This initiative aims to form a global investor coalition to fund the physical infrastructure necessary for rapid AI development.

The scope of these projects is broad, encompassing the construction of data centers, expansion of energy capacity, and growth of semiconductor manufacturing capabilities. Potential investors include entities from Canada, Korea, Japan, and the United Arab Emirates.

This infrastructure push is happening alongside OpenAI's approach towards a new funding round that includes Apple and Nvidia, and could push the company's valuation beyond $100 billion.

These developments at OpenAI and xAI are not surprising—rather, they are representative of the broader trend towards ever larger compute scale. For...
Refactoring cryonics as structural brain preservation
by Andy_McKenzie — 2024-09-11T18:36:30.285Z

I first learned about cryonics when I read Eliezer and Robin's posts about it on Overcoming Bias years ago. I got cryopilled. Somewhat amazingly to me, I'm now a researcher in this field. So I thought this community might be interested to know that I was one of several co-authors on a paper just published in Frontiers in Medical Technology, titled "Structural brain preservation: a potential bridge to future medical technologies".

In this paper, we propose reframing cryonics as a type of structural brain preservation, focusing on maintaining the brain's physical structure that encodes memories and personality, rather than making the focus about low-temperature storage per se.

We explore what brain structures likely need to be preserved to retain long-term memories and other valued aspects of personal identity. We then review different methods of brain preservation, including cryopreservation, aldehyde-stabilized cryopreservation, fluid preservation, and fixation followed by polymer embedding. The paper also discusses the two most commonly discussed potential future revival technologies, i.e. molecular nanotechnology and whole brain emulation.

We argue that this structural preservation framing may be more technically grounded and agreeable to mainstream researchers than some of the traditional ways that cryonics has been discussed.

As a personal reflection here, I want to briefly discuss the idea of fluid preservation, which is one topic discussed in our review paper. I remember first reading about this idea in approximately 2017 on a cryonics mailing list. Even though I was already sold on the idea of aldehyde-stabilized cryopreservation -- using fixation as a short-term bridge to cryoprotection and cryopreservation, I remember thinking that the idea of simply leaving the brain in fixative solution for the long-term was bizarre and outlandish.

Around 2020-2022, I spent a good amount of time researching different options for higher temperature (and thus lower cost) brain preservation. Mostly I was looking into different methods for embedding fixed brain tissue in polymers, such as paraffin, epoxy, acrylates, or silicon. I also studied the options of dehydrated preservation and preserving the fixed brain in the fluid state, which I was mostly doing for the sake of completeness.

To be clear, I certainly don't want to make it seem like this was a lone wolf ...
Is this a Pivotal Weak Act?
Creating bacteria that decompose metal
by doomyeser — 2024-09-11T18:07:19.385Z

This has been haunting my mind for a while and I would appreciate feedback on it!
In his infamous article "AGI Ruin: A list of lethalities" Eliezer defines a "pivotal weak act" and gives a heuristic proof that no such thing can exist.
TLDR: I think his proof is wrong and there is a counterexample. I believe creating bacteria that decompose metal, silicone, (or any other superset of the materials GPUs consist of) would constitute a pivotal weak act.

Long Version:
In his article, Eliezer outlines several hopes of people claiming AGI won't be as bad or any problem at all, and then cruelly squashes them. One of those hopes is the possibility of executing a "pivotal weak act". The idea being that a small group of people executes some action X that will prevent AGI from being built, for example a group that is privy to the dangers of AGI would command a friendly AGI to "burn all GPUs" and then we are good. Eliezer argues that any AGI powerful enough (pivotal) to prevent or at least indefinitely postpone unaligned AGI must itself be powerful enough such that it needs to be aligned (not weak), which we don't know how to do. I believe his proof is false.

Definition:
A Pivotal Weak Act, would be some action or event A, such that
1. A happening or being executed prevents or delays the advent of an unaligned AGI indefinitely or at least very long (Pivotal)
2. A does not itself pose a significant X-risk for humanity as a whole (Weak)
3. A is realistically achievable with technology attainable in the coming decades (Realism)
furthermore it is not required that
- A is in any way related to or facilitated by an AI system
- A has no collateral damage
- A is moral, legal or anywhere near the Overton Window
- A is achievable today with current technology

I think that the following is an example of a Pivotal Weak Act.
Creating bacteria that decompose metal (and spreading them worldwide)

This is pivotal, since it is a special scenario of "Burning all GPUs"
It is (likely) weak, since there are uncontacted tribes in the Amazone that certainly live without (forged) metal and would barely even notice.
I am not sure how realistic it is because I have no knowledge about Microbiology, but it does not seems to be SciFi, as there already seems to be something like that going on:

https://www.bam.de/Content/EN/Standard-Articles/Topics/Environment/Biocorrosion/mic-microbiologically-influenced-corro...
How to discover the nature of sentience, and ethics
by Gustavo Ramires — 2024-09-11T17:22:46.076Z

Note: This is a very rough draft of a theory. Any comments or corrections are welcome. I will try to give credit to useful observations.

To discover the nature of sentience and ethics, we can actually use scientific principles even though they are sometimes considered metaphysical.

The great insight though is that consciousness is part of reality. It is a real phenomenon. So I believe it turns out we can study this phenomenon, and assemble essentially complete theories of what it 'is'.

What do I mean by a theory of consciousness (later a theory of ethics, or 'what we must do')?

A theory of physics may, for example, predict the position of particles at a later time given certain initial conditions.

A theory of consciousness can answer a number of questions:
- Given a mind (e.g. a human brain), and its initial state, when we give it a certain input, for example corresponding to the physiological (neural) response to smelling a sweet, what taste does the mind/person feel? (is it really sweet or is this person different and smells what is commonly known as salty?) This is a purely subjective phenomenon, but it is emergent from information and communication in the brain.

In general, we are able to given neural activity describe the internal feelings experienced by the person using common sense words, because we have a theory of neural activity, and a theory of how different feelings manifest neurally in relationship to associated well-enough defined words and concepts (for example emotions, feelings, sensations).

- We understand all qualia and what qualia 'is', that is, we can tell if a certain object with say certain electrical activity manifests qualia (i.e. if it is sentient) or not, for any system.

How can we do that?

The basic principle really is observation of what people say, including with some caveats personal observations.

The second principle is that if there is a singular reality, and if things are a certain way and not another, then it may be (hypothesis) that there should be a single theory that is both logically internally consistent and consistent with reality, that explains both physics and sentience: simply because sentience is an emergent phenomenon hence part of reality. (in particular, there could perhaps be multiple theories giving the same predictions thus being equivalent). This may also be known as an 'principle of explosion'
from mathematics.

The thi...
Seeking Mechanism Designer for Research into Internalizing Catastrophic Externalities
by c.trout — 2024-09-11T15:09:48.019Z

TL;DR

I need help finding/developing a mech that can reliably elicit honest and effortful risk-estimates from frontier AI labs regarding their models, risk-estimates to be used in risk-priced premiums that labs then pay the government (i.e. a "Pigouvian tax for x-risk"). Current best guess: Bayesian Truth Serum.

Stretch-goal: find/develop a mech for incenting optimal production of public safety research. Current best guess: Quadratic Financing.

DM me if you're interested in collaborating!

The Project

X-risk poses an extreme judgment-proof problem: threats of ex post punishment for causing an existential (or nationally existential, or even just a disaster that renders your company insolvent) have little to any deterrent effect. Liability on its own will completely fail to internalize these negative externalities.

Traditionally, risk-priced insurance premiums are used to solve judgment-proofness (turn large ex post costs into small regular ex ante costs). However, insurers are also judgment-proof in the face of x-risk.

I'm developing a regime for insuring these "uninsurable" risks from frontier AI. It's modeled after the arguably very successful liability and insurance regime for nuclear power operators. In two recent workshop papers, I argue we should make foundation model developers liable for a certain class of catastrophic harms/near miss events and:

Mandate private insurance for commercially insurable losses (e.g. up to ~$500B in damages)
Have the government charge risk-priced premiums for insurance against uninsurable losses (i.e. a "Pigouvian tax for x-risk")

A government agency – through audits, its own forecasts and so on – could (and should) try to make these risk-estimates. However, this will be costly and they will struggle to collapse the information asymmetry between it and the developers it insures. Relying mostly on mechanism design to just incentivize labs to report honest and effortful risk-estimates has a number of advantages:

It should be cheaper for the government (more politically viable)
It should better leverage all available information, and result in more information gathering – it should just result in better risk -estimates
It's more secure: developers can divulge the risk implications of their private info without sharing sensitive private info (e.g. model weights).

The regime in schematic form:

The Work

I lack the mech design expertis...
Could Things Be Very Different?—How Historical Inertia Might Blind Us To Optimal Solutions
by James Stephen Brown — 2024-09-11T09:53:07.474Z
Link: https://nonzerosum.games/inertia.html

Have you ever wondered what the world would be like if you hadn’t been born? Would an entirely different person have taken your place? How about if humans had never emerged? Could another species have evolved higher levels of intelligence instead of us? What if life, as we know it, didn’t emerge-could some life-form, based on entirely different fundamental chemicals, have arisen in our place?

This post questions our collective perspective of the world-drawing on science fiction and interrogating some real world ‘accidents of history’ that might be reevaluated.

THE FIRST MOVER ADVANTAGE

We will never know if things could have been vastly different than they are, because our existence precludes the existence of alternative humans, species, or life-forms, simply because we were here first.

“As soon as the replicator was born it must have spread its copies rapidly throughout the seas, until the smaller building block molecules became a scarce resource” — Richard Dawkins (The Selfish Gene)

In marketing, this is called the

first mover advantage

, and we see it play out in the survival of products, with new innovations being out-competed, absorbed into larger companies, or just never attempted (due to their seeming futility).

THE ARROW OF HISTORY

We can often take the current state of the world as an inevitable product of the arrow of history. Thinkers like Robert Wright see increasing

non-zero-sumness

as the universal direction, while fatalism takes inevitability to an extreme, asserting that if the clock were wound back, it would play out exactly the same way.

As a somewhat agnostic determinist, I take the perspective that

emergence

is the determining factor in the progress of life, allowing for many different paths to emerge from the same initial conditions. From this perspective, things could be very different.

But even though I hold a position, I do enjoy questioning my assumptions through reading science fiction, and one such assumption I share with most, is the linear nature of causality.

“We experienced events in an order, and perceived their relationship as cause and effect. They [the alien septopods] experienced all events at once, and perceived a purpose underlying them all.”

- Louise Banks
(A Story Of Your Life)

ARRIVAL

Ted Chiang’s short story “A Story of Your Life”-the basis for the movie “Arrival” (spoiler alert) plays out in a deterministic (in this case, fatalist...