trentmkelly/LessWrong-43k huggingface dataset:
Memetic Judo #1: On Doomsday Prophets v.3
by Max TK — 2023-08-18T00:14:11.322Z

There is a popular tendency to dismiss people who are concerned about AI-safety as "doomsday prophets", carrying with it the suggestion that predicting an existential risk in the near future would automatically discredit them (because "you know; they have always been wrong in the past").

Example Argument Structure

Predictions of human extinction ("doomsday prophets") have never been correct in the past, therefore claims of x-risks are generally incorrect/dubious.

Discussion/Difficulties

This argument is persistent and kind of difficult to approach/deal with, in particular because it is technically a valid (yet, I argue, weak) point. It is an argument by induction based on a naive extrapolation of a historic trend. Therefore it cannot be completely dismissed by simple falsification through the use of an inconsistency or invalidation of one of its premises. Instead it becomes necessary to produce a convincing list of weaknesses - the more, the better. A list like the one that follows.

#1: Unreliable Heuristic

If you look at history, these kind of ad-hoc "things will stay the same" predictions are often incorrect.
An example of this that is also related to technological developments would be the horse and mule populations in the US (back to below 10 million at present).

#2: Survivorship Bias

Not only are they often incorrect, there is a class of predictions for which they, by design/definition, can only be correct ONCE, and for these they are an even weaker argument, because your sample is affected by things like survivorship bias. Existential risk arguments are in this category, because you can only go extinct once.

#3: Volatile Times

We live in an highly unstable and unpredictable age shaped by rampant technological and cultural developments. The world today from the perspective of your grandparents is barely recognizable. In such times, this kind of argument becomes even weaker.
This trend doesn't seem to slow down and there are strong arguments that even benign AI would flip the table on many of such inductive predictions.

#4: Blast Radius Induction (thanks Connor Leahy)

Leahy has introduced an analogy of "technological blast radius" that represents an abstract way of thinking about different technologies in terms of their potential power, including their potential for causing harm either intentionally or by human error. As we are progressing through the...
Looking for judges for critiques of Alignment Plans
by Iknownothing — 2023-08-17T22:35:40.666Z

Hello!
AI-Plans.com recently held a "critique-a-thon," where participants submitted and refined 40+ critiques for AI Alignment plans. Here are the finalized critiques from the event: https://docs.google.com/document/d/1mW4SAxFN_aI6KyYXpl9qz5B9nVdeV9Xyc69GTNme5cA/edit?usp=sharing

We are looking for anyone who might be interested in helping to judge these final 11 critiques.
So far, we have gratefully had the assistance of Dr Peter S Park (MIT postdoc at the Tegmark lab, Harvard PhD) and Aishwarya G (AI Existential Safety Community Member at Future of Life Institute and Governance Course Facilitator at BlueDot Impact AI Safety Fundamentals), as well as some independent alignment researchers.
I would love to hear your thoughts!
Kabir Kumar (Founder, AI-Plans.com)
How is ChatGPT's behavior changing over time?
by worse — 2023-08-17T20:54:46.505Z
Link: https://arxiv.org/abs/2307.09009

Surprised I couldn't find this anywhere on lesswrong so thought I'd add it. Seems like there would be some alignment implications of LLM behavior changing over time, at the least gaining a bit more context.
Someone else I spoke to about this immediately deflated it with regards to some sort of experimental error that makes the paper's conclusions pretty void but I don't really see this.
Progress links digest, 2023-08-17: Cloud seeding, robotic sculptors, and rogue planets
by jasoncrawford — 2023-08-17T20:29:28.546Z
Link: https://rootsofprogress.org/links-digest-2023-08-17

Opportunities

Academic biotech founders: apply for Nucleate’s Activator program (via @kulesatony)

News & announcements

Neuralink raises $280M Series D led by Founders Fund
Waymo and Cruise have been approved to operate robotaxis in San Francisco
Rainmaker launches to “end global water scarcity and terraform Earth”

Podcasts

Dwarkesh Patel interviews Dario Amodei, CEO of Anthropic (via @dwarkesh_sp). “Dario is hilarious & has fascinating takes on what these models are doing, why they scale so well, & what it will take to align them”
Tyler Cowen interviews Paul Graham (via @tylercowen)

Links

Monumental Labs is building “AI-enabled robotic stone carving factories” to “create cities with the splendor of Florence, Paris, or Beaux-Arts New York, at a fraction of the cost.” Here’s a demo (via @devonzuegel). They are hiring stone carvers (who are still required for fine details and finishing)
The Codon Guide to Synthetic Biology (by @NikoMcCarty)
The AI License Raj: AI is facing serious bureaucratic hurdles in getting adopted (by @WillRinehart)
I had no idea there were this many rogue planets
“My personal history as a metascience venture capitalist” (by @stuartbuck1)
A303 in Hampshire, England, passes through Stonehenge and part of the ancient Roman Fosse Way is included in it (via @Rainmaker1973)

Social media

In the current YC batch, “a large number of domain experts in all kinds of different fields have found ways to use AI to solve problems that people in their fields had long known about, but weren’t quite able to solve”
If you got over-excited about LK-99, worth recalibrating your priors. But if you correctly predicted no RTS, there’s really no need to gloat. Related: LK99 demonstrated that the scientific community is perfectly capable of doing a peer review via arXiv and social media tools
Some of the lowest hanging fruit in biosecurity policy: Invest more resources in early warning systems that monitor wastewater
In the early 19th century, New York, Philadelphia, and Boston had residential densities upwards of 75,000 people per square mile
“NuScale started working toward regulatory approval in 2008. In 2020, when it received a design approval for its reactor, the company said the regulatory process had cost half a billion dollars, and that it had provided about 2 million pages of supporting documents to the NRC.” Related, my book review on why nuclear power flopped
Lear

...
Model of psychosis, take 2
by Steven Byrnes — 2023-08-17T19:11:17.386Z

(I am not by any stretch of the imagination an expert on psychosis. This is more like “live-blogging my thinking as I go”. I’m hoping to spur discussion and get feedback and pointers.)

1. Introduction

I suggested a model of psychosis in my blog post

“Schizophrenia as a deficiency in long-range cortex-to-cortex communication”, Section 4.2

last February. But it had some problems. I finally got around to taking another look, and I think I found an easy way to fix those problems. So this post is the updated version.

For the tl;dr, you can skip the text and just look at the two diagrams below.

2. Background: My “Model of psychosis, take 1” from earlier

The following is what I was proposing in

“Schizophrenia as a deficiency in long-range cortex-to-cortex communication”, Section 4.2

:

The idea is that, in psychosis, the green arrow is active and effective, but the red arrow isn’t, and therefore neither is the purple arrow. The result might be a kind of feeling that my arm was moved by an external force. That’s just one example—for different types of cortical outputs on the left, the corresponding first-person experience of psychosis would be different. But I claim that all symptoms of psychosis broadly fit into this template (update: I elaborated on “all symptoms of psychosis” in this comment and also Section 7.5 of this later post).

3. Three areas where that first model seemed to be missing something important

As I understand it, psychosis can come and go in people with schizophrenia, whereas (I believe) the deficiency in cortex-to-cortex communication is characteristic of schizophrenic brains, and is structural and permanent (absent future advances in medical technology).
Antipsychotics reduce psychosis, but the diagram above offers no way to explain that.
Psychosis can occur in other conditions besides schizophrenia. I think the most common example is the manic phase of bipolar. The diagram above cannot explain that.

4. My “Model of psychosis, take 2”

(The only change to the top part of the diagram is the new green text on the left that says “Signal strength = B”.)

Think of a part of the cortex as having an adjustable “volume”, in terms of how strongly and clearly it is announcing what it’s up to right now. For example, if the thought crosses your mind that maybe you might move your finger, then you might find (if you look closely and/or use scientific equipmen...
[Linkpost] Robustified ANNs Reveal Wormholes Between Human Category Percepts
by Bogdan Ionut Cirstea — 2023-08-17T19:10:39.553Z

This is a linkpost for https://arxiv.org/abs/2308.06887.

The visual object category reports of artificial neural networks (ANNs) are notoriously sensitive to tiny, adversarial image perturbations. Because human category reports (aka human percepts) are thought to be insensitive to those same small-norm perturbations -- and locally stable in general -- this argues that ANNs are incomplete scientific models of human visual perception. Consistent with this, we show that when small-norm image perturbations are generated by standard ANN models, human object category percepts are indeed highly stable. However, in this very same "human-presumed-stable" regime, we find that robustified ANNs reliably discover low-norm image perturbations that strongly disrupt human percepts. These previously undetectable human perceptual disruptions are massive in amplitude, approaching the same level of sensitivity seen in robustified ANNs. Further, we show that robustified ANNs support precise perceptual state interventions: they guide the construction of low-norm image perturbations that strongly alter human category percepts toward specific prescribed percepts. These observations suggest that for arbitrary starting points in image space, there exists a set of nearby "wormholes", each leading the subject from their current category perceptual state into a semantically very different state. Moreover, contemporary ANN models of biological visual processing are now accurate enough to consistently guide us to those portals.
Against Almost Every Theory of Impact of Interpretability
by Charbel-Raphaël — 2023-08-17T18:44:41.099Z

Epistemic Status: I believe I am well-versed in this subject. I erred on the side of making claims that were too strong and allowing readers to disagree and start a discussion about precise points rather than trying to edge-case every statement. I also think that using memes is important because safety ideas are boring and

anti-memetic

. So let’s go!

Many thanks to @scasper, @Sid Black , @Neel Nanda , @Fabien Roger , @Bogdan Ionut Cirstea, @WCargo, @Alexandre Variengien, @Jonathan Claybrough, @Edoardo Pona, @Andrea_Miotti, Diego Dorn, Angélina Gentaz, Clement Dumas, and Enzo Marsot for useful feedback and discussions.

When I started this post, I began by critiquing the article

A Long List of Theories of Impact for Interpretability

, from Neel Nanda, but I later expanded the scope of my critique. Some ideas which are presented are not supported by anyone, but to explain the difficulties, I still need to 1. explain them and 2. criticize them. It gives an adversarial vibe to this post. I'm sorry about that, and I think that doing research into interpretability, even if it's no longer what I consider a priority, is still commendable.

How to read this document? Most of this document is not technical, except for the section "What does the end story of interpretability look like?" which can be mostly skipped at first. I expect this document to also be useful for people not doing interpretability research. The different sections are mostly independent, and I’ve added a lot of bookmarks to help modularize this post.

If you have very little time, just read (this is also the part where I’m most confident):

Auditing deception with Interp is out of reach

(4 min)

Enumerative safety

critique (2 min)

Technical Agendas with better Theories of Impact

(1 min)

Here is the list of claims that I will defend:

(bolded sections are the most important ones)

The overall Theory of Impact is quite poor

Interp is not a good predictor of future systems

Auditing deception with interp is out of reach

What does the end story of interpretability look like? That’s not clear at all.

Enumerative safety?

Reverse engineering?

Olah’s Interpretability dream?

Retargeting the search?

Relaxed adversarial training?

Microscope AI?

So far my best ToI for interp: Outreach

Preventive measures against Deception seem much more workable

Steering the world towards transparency

Cognitive Emulations - Expla

...
Goldilocks and the Three Optimisers
by dkl9 — 2023-08-17T18:15:39.456Z
Link: https://dkl9.net/essays/goldilocks.html

Once upon a time, there was a little girl, Goldilocks, so named, we suppose, for her goldy locks — er, blonde hair.

A small (n=3) nuclear family of anthropomorphic bears lived in a house in a forest. Goldilocks, travelling in the forest (unaccompanied, apparently?), entered this house when the family was away.

Upon a dinner table, she found three bowls of porridge.

Goldilocks tasted from a bowl identified as first, but stopped after a small bite, finding it to be too hot. "This porridge was optimised for caloric content at fixed price!"[1] she observed, and moved on to a second bowl.

The second porridge was bland (more so than your average porridge) and induced brain-freeze, leading her to remark that "This porridge was optimised for preservation against microbes at fixed composition!" and convenience-sample from the third bowl.

Goldilocks ended up extending her convenience sample to the entire sampling frame (i.e. the bowl's contents), enjoying the final porridge. She supposed that "This porridge was optimised for taste at fixed nutrition."

She saw three armchairs, and decided to sit in one, during which she could advance her philosophy.

A first chair was far too large — it pressed against the ceiling — leading Goldilocks to ignore it, as she noted "That chair was optimised for capacity at fixed portability."

She skipped over the second chair — really more of a stool — upon noticing that "That chair was optimised for the user's mobility at fixed capacity."

The third chair looked appealing and reasonable, if perhaps suspiciously flimsy. She sat in it, enabling her to realise that she must have either misinterpreted these optimisation criteria, or underestimated the agents' intelligence/creativity. The true optima of all those goals are achieved by designs radically different from porridge or chairs. (It turns out she happened to be correct anyway. Remember, the house is run by bears, not the superintelligence Omega.)

After her shifting around in that chair, it collapsed. "That chair was optimised for material cost at fixed comfort!" Goldilocks realised.

By now, 'twas evening, so she looked for a bedroom — somehow completely unhesitant in her intent to sleep in a stranger's house — and found one containing three beds.

She briefly tried lying in the first bed, finding that "This bed was optimised for resilience against heat at fixed mass!" For it seemed to be made of...
Announcing Foresight Institute's AI Safety Grants Program
by Allison Duettmann — 2023-08-17T17:34:59.625Z

Foresight Institute has received funding to support projects in three AI safety areas we think are potentially under-explored:
1. Neurotechnology, brain computer interface, whole brain emulation, and "lo-fi" uploading approaches to produce human-aligned software intelligence
2. Computer security, cryptography, and related techniques to help secure AI systems
3. Multi agent simulations, game theory and related techniques to create safe multipolar AI scenarios that avoid collusion and foster positive sum dynamics
The grant application process is now open and accepts rolling applications. We expect to grant between $1 - 1.2 million per year across projects and look forward to receiving your project proposals that could make a significant difference for AI safety within short timelines.
Please visit

https://foresight.org/ai-safety/

for full details and application instructions.

I would appreciate it if you would consider sharing the opportunity with others who may benefit from applying.

Feel free to comment here or email me at allison@foresight.org with any questions, feedback, or ideas for collaborations.
The Negentropy Cliff
by mephistopheles — 2023-08-17T17:08:20.962Z

Something that is not often discussed explicitly and factors into the different intuitions people have about P(Doom) is how close to optimal biology and humans are in terms of harnessing negative entropy. This consideration pertains to equally to nanobots, ASI and artifical life in general.

Let's consider grey goo first: The race to turn all resources into copies of yourself has been going for a few billion years and is quite competitive. In order to supplant organic life, nanobots would have to either surpass it in carnot efficiency or (more likely) utilise a source of negative entropy thus far untapped. Examples of this previously happening are:

Photosynthesis
Aerobic respiration
Control of fire by early humans
Agricultural revolution
Industrial revolution

If, in the designspace of replicators, we are in a local (metastable) optimum and the ability to consume negative entropy, falls off a cliff in a place that is reachable by the synthetic but not organic life, we will get outcompeted quickly. So, are we stumbling in the dark, next to a civilisation swallowing precipice? Would the ASI need to discover new physics or are there already examples of negentropy sources that it could use better than biology?
"AI Wellbeing" and the Ongoing Debate on Phenomenal Consciousness
by FlorianH — 2023-08-17T15:47:42.128Z

“AI Wellbeing” by Goldstein & Kirk-Giannini (linked in the AI Safety Newsletter #18) presents arguments for viewing “Wellbeing” as a moral concern separable from Consciousness. I believe these arguments lead to right conclusions for the wrong reasons. We don’t need a new concept of moral value. However, they highlight (at least indirectly) a real problem with the existing AI consciousness debate: We don’t know where consciousness starts, and our implicit optimism on being able to observe it in future is entirely unfounded.

Terminology: Throughout, I use consciousness to refer to phenomenal consciousness or sentience, the experience of “it-feeling-like-something”. I write cold to emphasize the absence of consciousness. Wellbeing encompasses mental states such as knowledge or desire. Wellbeing-ists like Goldstein & Kirk-Giannini argue for wellbeing to be morally relevant independently of whether it invokes consciousness. Keeping this terminology precisely in mind helps avoiding confusion here with the otherwise conflationary terms.

Wellbeing and Consciousness

I had initially been surprised by the claim in “AI Wellbeing”, that little attention had been paid to the question: What conditions lead to an AI with (presumably ethically relevant) wellbeing? Haven’t we spent the past decade or so discussing this on LessWrong, in Effective Altruism, and, as of late, ad nauseam in all media? Eventually I realized my mistake. We have been discussing consciousness, much less wellbeing, which are not synonyms.

Then again, isn’t the critical issue for “AI wellbeing”, still: What possesses phenomenal consciousness? Surely we need some hint about this first. Then only, may we fully devote ourselves to whether the wellbeing of that conscious agent is high or low. There is hope that this second question is simpler than the first. At a high level, we likely agree on how wellbeing may differ across diverse human experiences or between you, your cat, and a factory-farmed chicken – assuming they all possess consciousness.

Wellbeing as independent moral concern

How then is the critical relevance of (cold) Wellbeing justified? The answer in “AI Wellbeing” is simple: It rejects the Consciousness Requirement. Put simply, this rejection means: You may be mistreating your computer even if it doesn’t develop phenomenal consciousness. This rejection comp...
AI #25: Inflection Point
by Zvi — 2023-08-17T14:40:06.940Z

Inflection.ai is the latest AI lab whose CEO is advocating for regulation of AI. I discuss that under the Quest for Sane Regulation. Amazon and Apple are incrementally stepping up their AI game. Hotz and Yudkowsky debate whether AI is existentially risky, cover all the usual bases with mixed results but do so in good faith. We have more discussion about whether GPT-4 is creative, and whether it can reason. Mostly we get the exact opposite of the title, more of the same.

Note: My posts get made into audio form via AI, for now you can listen to them at this link. This post will likely be available there later in the day on Thursday, or perhaps Friday.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. Creativity is hard to pin down.

Language Models Don’t Offer Mundane Utility. It’s the other way.

GPT-4 Real This Time. An easy way to prove ability to do something is to do it.

Go Team Yeah. If you organize the events and award points, they will red team.

Fun With Image Generation. Some have fun, others have less fun.

Deepfaketown and Botpocalypse Soon. Doesn’t have to be this soon.

They Took Our Jobs. Low estimates of economic impact, strange metaphors.

Get Involved. Anthropic is hiring for comms positions.

Introducing. Amazon AI customer review summaries, private-GPT, AI town.

In Other AI News. Apple joins the AI chorus, questions on influence functions.

Quiet Speculations. Let’s play the straight line extrapolation game.

The Quest for Sane Regulation. Inflection.ai’s CEO steps into the arena.

The Week in Audio. Hotz and Yudkowsky debate.

People Are Worried About AI Killing Everyone. Quite a lot of people.

Other People Are Not As Worried About AI Killing Everyone. All wrong anyways.

The Lighter Side. A well-deserved break.

Language Models Offer Mundane Utility

Replace crowdsourcing your business ideas, get a lower variance, lower upside set of concepts with a similar average quality. Does not seem especially useful, but can get ideas flowing perhaps. The AIs can give you many ideas, but were not very creative.

The connection between semantic diversity and novelty is stronger in human solutions, suggesting differences in how novelty is created by humans and AI or detected by human evaluators.

Alice Maz lays out how they get mundane utility from GPT,...