trentmkelly/LessWrong-43k huggingface dataset:
Why I'm Skeptical of De-Extinction
by Niko_McCarty — 2023-02-23T19:42:52.618Z
Link: https://cell.substack.com/p/extinction

This piece was co-authored with @Metacelsus

“For a successful technology, reality must take precedence over public relations, for nature cannot be fooled.”
—

Richard Feynman

The black-faced honeycreeper — or

po’o-uli

, as the Hawaiians called it — was native to Maui. Two birds were spotted in 2000 and then never seen again. The species was declared extinct in 2019; it lives on only as cells in a cryogenic freezer.

Or, consider the Pyrenean Ibex, a large goat native to Andorra’s mountains. The

Ibex

is the only animal to go extinct twice: Once in January 2000 and again in July 2003, after scientists cloned an animal and watched it die, shortly after birth, from a lung defect.

Hundreds of other plants and animals are listed as “

possibly extinct

” by the International Union for Conservation of Nature, which means they haven’t been seen in the wild for years. The list includes 156 amphibians, 22 birds, 29 mammals, and 93 insects. Captain Cook’s Bean Snail, the Wyoming Toad, and the She Cabbage Tree are all extinct in the wild. Today, they live only in zoos.

Numbers alone can’t convey the

Sixth Extinction

, because numbers alone aren’t inherently visual. Each digit is an entire species, with thousands or millions of years of history, that is at risk of disappearing from the Earth. We mask extinction by visiting zoos or fawning over new technologies. “Humans made this mess,” we think, “and humans will fix it.”

De-extinction, unfortunately, will not be our savior.

Many academic scientists and companies aim to ‘resurrect’ long-extinct animals using gene-editing and advanced reproductive technologies. One of them, Dallas-based

Colossal Biosciences

, recently raised $150 million to bring back the woolly mammoth, dodo, and thylacine, a fox-like marsupial that once roamed Tasmania. The dodo bird, native to Mauritius, was hunted to extinction by Dutch sailors in 1662. Colossal launched less than a year-and-a-half ago, and yet is already valued at

over $1 billion

.

The company, then, is a unicorn building technologies to make unicorns.

Press releases, though, won’t make de-extinction come true. No papers have been published by the company, and resurrecting the Pyrenean Ibex, which ended in failure, was massively simpler in comparison; living cells were already available. True de-extinction is nowhere near possible.

“It’s impossible to bring something back that’s an ident...
What causes randomness?
by lotsofquestions — 2023-02-23T18:50:31.315Z

If I clone two earths right when the first unicellular organisms formed, with everything down to the atom similar, will the two earths (or maybe just take them as universes composed of matter/energy) be the same? If not, why? What causes that difference? Is it randomness? If so, what causes this randomness?

Thank you, this is my first question.
Somerville Roads Getting More Dangerous?
by jefftk — 2023-02-23T18:20:03.354Z

In a local discussion of whether our attempts to reduce traffic
fatalities were working someone posted:

I've lived in Somerville for over 40 years and have visited even
longer. The roads are more dangerous than ever.

With all the Vision
Zero work, and earlier traffic safety progress, this would be
surprising and disappointing if it were the case. Is it?

Somerville is an interesting city to look at because it was mostly
built out decades ago. The last large change was bulldozing I-93
through in the late 1960s:

This makes fatality rates over time comparable in a way they wouldn't
be if you were looking at a city that had expanded, contracted, or
been seriously rebuilt.

I pulled accident date from the national Fatality
Analysis Reporting System (FARS) which goes back to 1975. Here's
what I see for the number of fatal crashes annually (code):

blue bars are annual counts; red line is a 10y centered moving average

Over the last 40y it's gotten about 3x better, though 2019 was a
pretty bad year.

I'd also be interested in whether our roads are becoming more
dangerous for pedestrians specifically. Is the decrease just due to
airbags, crash testing, and cars becoming safer for their occupants?
While pedestrian data in FARS only goes back to 1991, we can still
graph what we have:

It does look like we're making progress specifically within pedestrian
safety as well, though 2019 now sticks out even more.

(I'm being lazy here and charting "traffic accidents with a fatality
and involving at least one pedestrian". The FARS data does make it
possible to see whether it was a pedestrian that died, but this
requires joining CSVs in a way that's a bit awkward. In the ones I've
spot checked the fatality was a pedestrian, which is what you'd
expect.)

Comment via: facebook, mastodon
EIS XII: Summary 
by scasper — 2023-02-23T17:45:55.973Z

Part 12 of 12 in the

Engineer’s Interpretability Sequence

.

TAISIC = “the AI safety interpretability community”

MI = “mechanistic interpretability”

There might be some addenda later, but for now, this is the final post in The Engineer’s Interpretability Sequence. I hope you have found it interesting and have gotten some useful ideas. I will always be happy to talk to people about the topics from this sequence in the comments or via email. For now, the last thing I will do is offer a summary of key points post by post :)

A Prequel: Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks (Räuker et al., 2022)

A survey of over 300 works on inner interpretability from an AI safety perspective.
All opinions in this sequence, however, are my own and not necessarily those of coauthors or other affiliates.

EIS I: Intro

Lots of interpretability research exists, and the field is still rapidly growing.
Most of it is not very productive, and there is a significant gap between the research and practice. Interpretability tools aren't used much by engineers working on real alignment problems.
If one of our main goals for interpretability research is to help us with aligning highly intelligent AI systems in high stakes settings, we should be working on tools that are more engineering-relevant.

EIS II: What is “Interpretability”?

This post introduced a parable about two researchers trying to understand and fix a classifier displaying racial biases.

The moral of the story is that we should not privilege difficult or interesting methods over easy and simple ones. It is key not to grade different tools on different curves.

From an engineer’s perspective, the term “interpretability” isn’t that useful.

Whatever we call “interpretability” tools are entirely fungible with other techniques related to describing, evaluating, debugging, etc. in models.

Mechanistic approaches to interpretability are not uniquely important for AI safety. MI tools have the potential to help identify and fix deceptive alignment failures, but...

There are many non-deceptive ways AI could go wrong.
MI is not uniquely useful for fixing deceptive alignment and especially not uniquely useful for fixing non-deceptive alignment failures.

EIS III Broad Critiques of Interpretability Research

There is a growing consensus that interpre

...
How to survive in an AGI cataclysm
by RomanS — 2023-02-23T14:34:53.998Z

TLDR:

As with any other global risk, trying to prevent the AGI cataclysm is not enough. If there is a high risk that the cataclysm will happen, we should also start thinking how to survive during the cataclysm. There are realistic ways to survive.

Compare: a wise person works on preventing the nuclear war, but also builds nuclear bunkers for the case the war still happens.

The survival mindset

This post will bring some hope. But let's start with some bad news:

If you’re a human, you’re going to die anyway. Because humans die from old age, cancer, wars, etc etc.

If you live long enough, you may see efficient anti-aging therapies, mind uploading etc. But currently, death is the default.

The imminent AGI cataclysm doesn’t change the situation much. It doesn’t matter what kills you: the AGI or cancer.

After realizing this simple truth, many humans give up. But some – fight back.

The ones who fight back have a chance to survive.

For example, there is a chance that cryonics will work. And thus, even if you’re doomed to die from cancer, you still have a chance to survive. But the dying patients who reject cryonics – have no chance.

Another example: a nuclear war. Instead of giving up, you build a bunker, or move to New Zealand, or find another way to improve your chances of survival, regardless of how small are the chances.

The same general approach is helpful in every “hopeless” situation:
even if death is inevitable, you never give up. Instead, you search for ways to trick death.

There is no such thing as “death with dignity”. You live forever or die trying.

AGI is not so different from a global nuclear war

Imagine that in X years there will be an AGI who will disassemble you into useful atoms.

Obviously, you should try to prevent that from happening. For example, by helping with alignment research.

But even if you’re certain that the unaligned AGI is absolutely inevitable, you should keep the same survival mindset.

If the nuclear war is inevitable, prepare yourself. If the AGI cataclysm is inevitable, prepare yourself.

Types of AGI hostility

The reasons why AGI may try to kill you depend on its hostility:

1. no hostility: you’re made of useful atoms which it can use for something else

2. hostility against selected humans: you-in-particular and your pals are a threat to its survival

3. hostility against a large group of humans: e.g. ...
Covid 2/23/23: Your Best Possible Situation
by Zvi — 2023-02-23T13:10:01.887Z

No news is (often) good news.

At old Magic: The Gathering tournaments, judge Tony Parodi would often tell us, ‘if your opponent did not show up, that is your best possible situation.’

Every week, when I set out to write the Covid update, I held out the hope that at some point, perhaps soon, you would never read one of these again. There would be nothing to report. That the reports would no longer help anyone. I could go think about and write about something else.

Today is that day. I had to go see about… well, to start off, the possibility of AI destroying all value in the universe. Not my desired first pick. Once again, I much prefer the worlds where over the next weeks, months and years I get to deep dive into other very different aspects of the world instead.

It is still a joyous day. After three years, the weekly Covid posts are over.

From this point forward, I am no longer going to actively seek out Covid information. I am not going to check my Covid Twitter list.

I will continue to compile what Covid and related information I come still across, although with a much higher bar for inclusion going forward. If it seems worth its own post from time to time, I’ll do that. If not, I won’t. Unless something changes a lot, that will be a lot less common than weekly.

We have normality. Cherish it.

You’ll miss it when it’s gone.

Executive Summary

This will be the last weekly Covid post unless things change radically.

We have normality. I repeat, we have normality.

Anything you still can’t cope with is therefore your own problem.

Let’s run the numbers.

The Numbers

Predictions

Predictions from Last Week: 210k cases (-6%) and 2,625 deaths (-7%).

Results: 210k cases (-6%) and 2,396 deaths (-15%)

Predictions for Next Week: No more formal predictions. Expect continued slow declines in underlying numbers for a while.

Arizona reported 23k cases, which has to be a backlog dump, so I cut them down to a high but plausible 4k. Colorado reported negative deaths, so I changed that to zero.

I entirely forgot about Presidents Day, which makes the case number here unexpectedly high, and largely accounts for the deaths result. Overall slightly disappointing given the holiday.

Deaths

Cases

Physical World Modeling

I was alerted to quite the case of modeling: Metaculus has been successfully working with the Virginia state government to h...
Full Transcript: Eliezer Yudkowsky on the Bankless podcast
by remember — 2023-02-23T12:34:19.523Z

This podcast has gotten a lot of traction, so we're posting a full transcript of it, lightly edited with ads removed, for those who prefer reading over audio.
Intro

Eliezer Yudkowsky: [clip] I think that we are hearing the last winds start to blow, the fabric of reality start to fray. This thing alone cannot end the world, but I think that probably some of the vast quantities of money being blindly and helplessly piled into here are going to end up actually accomplishing something.

Ryan Sean Adams: Welcome to Bankless, where we explore the frontier of internet money and internet finance. This is how to get started, how to get better, how to front run the opportunity. This is Ryan Sean Adams. I'm here with David Hoffman, and we're here to help you become more bankless.

Okay, guys, we wanted to do an episode on AI at Bankless, but I feel like David...

David: Got what we asked for.

Ryan: We accidentally waded into the deep end of the pool here. And I think before we get into this episode, it probably warrants a few comments. I'm going to say a few things I'd like to hear from you too. But one thing I want to tell the listener is, don't listen to this episode if you're not ready for an existential crisis. Okay? I'm kind of serious about this. I'm leaving this episode shaken. And I don't say that lightly. In fact, David, I think you and I will have some things to discuss in the debrief as far as how this impacted you. But this was an impactful one. It sort of hit me during the recording, and I didn't know fully how to react. I honestly am coming out of this episode wanting to refute some of the claims made in this episode by our guest, Eliezer Yudkowsky, who makes the claim that humanity is on the cusp of developing an AI that's going to destroy us, and that there's really not much we can do to stop it.

David: There's no way around it, yeah.

Ryan: I have a lot of respect for this guest. Let me say that. So it's not as if I have some sort of big-brained technical disagreement here. In fact, I don't even know enough to fully disagree with anything he's saying. But the conclusion is so dire and so existentially heavy that I'm worried about it impacting you, listener, if we don't give you this warning going in.

I also feel like, David, as interviewers, maybe we could have done a better job. I'll say this on behalf of myself. Sometimes I peppered him with ...
Automated Sandwiching & Quantifying Human-LLM Cooperation: ScaleOversight hackathon results
by Esben Kran — 2023-02-23T10:48:08.766Z

We ran a

hackathon on scalable oversight

with Gabriel Recchia as keynote speaker (watch the talk) and Ruiqi Zhong as co-judge. Here, we share the top projects and results. In summary:

We can automate the “sandwiching” paradigm from Cotra [

1

] by having a smaller model ask structured questions to elicit a true answer from a larger model and getting a response accuracy rate as output.
We can understand coordination abilities between humans and large language models quantitatively using asymmetric-information language games such as Codenames.
We can study scaling and prompt specificity phenomena in-depth using a simple framework. In this case, word reversal is investigated to evaluate the emergent abilities of language models.

Watch the project presentations on YouTube.

Thank you goes to the local organizers, Gabriel Recchia, Ruiqi Zhong and the wonderful hackathon participants from across the world.

Automatic Sandwiching: Efficient Self-Evaluations of Conversation-Based Scalable Oversight Techniques

By Sophia Pung and Gabriel Mukobi

Abstract: Our project furthers the progress of Scale Oversight through automation of the sandwiching paradigm. In the Bowman et al. (2022) paper, the question is presented of how humans can effectively prompt unreliable, superhuman AIs to answer questions via conversation to arrive at accurate answers. We want to explore and evaluate the methods that humans can use reliably to elicit honest responses, from a more intelligent AI. We present a novel method, called Automatic Sandwiching, for implementing this paradigm. We implement a simplified version of this, evaluate our system on 163 training examples from Multi-task Language Understanding (MMLU) with 2 different oversight techniques. We provide code to reproduce our results at

sophia-pung/ScaleOversight

.

They replicate a non-expert human overseer by taking a smaller model (text-curie-001) and providing it specific instructions to get non-deceptive answers in a question and answer conversation with the larger model (text-davinci-003). The dataset has been used before in similar work [

2

] and is useful for eliciting structured conversations for oversight.

Gabriel Recchia’s comment: This is exactly the type of project I was hoping would come out of this hackathon! Replacing humans with LMs as has been done here, if it can be made to work, potentially allows for much faster research it...
How to estimate a pre-aligned value for a common discussion ground?
by EL_File4138 — 2023-02-23T10:38:18.489Z

There are actually all sorts of false science in the wild, such as "the earth is flat" and "pandemic is government-driven lies", extending their influence even though the rational community struggled to promote the rational thinking paradigm, which I believe is a strong prerequisite for the Alignment question to be solved.

So the question is:

How to define "absolute truth" under the rational thinking paradigm?

How to forcefully educate this "absolute truth" to the general public, for a common consensus to be presumed before any meaningful discussion can exist?

Edit: I found "forcefully" particularly not suiting under the context, but I can't bring a more proper expression at this time. This question is mainly on the first point, not by educating/convincing but by estimating a pre-aligned value.

Second Edit: This question might not be suitable because it's some long-standing question merged together with a manipulative intention I was not intended to express. I've changed the title to a more suitable one. Thanks for replying.

*Background: I've recently been struggling to educate some fellow artists with no ML backgrounds on generative ML models, but with little luck. *
Interpersonal alignment intuitions 
by TekhneMakre — 2023-02-23T09:37:22.603Z

Let's try this again...

The problem of aligning superhuman AGI is very difficult. We don't have access to superhuman general intelligences. We have access to superhuman narrow intelligences, and human-level general intelligences.

There's an idea described here that says: (some of) the neocortex is a mostly-aligned tool-like AI with respect to the brain of some prior ancestor species. (Note that this is different from the claim that brains are AIs partially aligned with evolution.) So, maybe we can learn some lessons about alignment by looking at how older brain structures command and train newer brain structures.

Whether or not there's anything to learn about alignment from neuroanatomy specifically, there's the general idea: there are currently some partial alignment-like relationships between fairly generally intelligent systems. The most generally intelligent systems currently existing are humans. So we can look at some interpersonal relationships as instances of partially solved alignment.

In many cases people have a strong need to partially align other humans. That is, they have a need to interact with other people in a way that communicates and modifies intentions, until they are willing to risk their resources to coordinate on stag hunts. This has happened in evolutionary history. For example, people have had to figure out whether mates are trustworthy and worthwhile to invest in raising children together rather than bailing, and people have had to figure out whether potential allies in tribal politics will be loyal. This has also happened in memetic history. For example, people have developed skill in sussing out reliable business partners that won't scam them.

So by some combination of hardwired skill and learned skill, people with some success determine the fundamental intentions of other people. This determination has to be high precision. I.e., there can't be too many false positives, because a false positives means trying to invest in some expensive venture without sufficient devoted support. This determination also has to be pretty robust to the passage of time and surprising circumstances.

This is disanalogous to AGI alignment in that AGIs would be smarter than humans and very different from humans, lacking all the shared brain architecture and values, whereas people are pretty much the same as other people. But there is some analogy, in that people...
Big Mac Subsidy?
by jefftk — 2023-02-23T04:00:03.996Z

I was reading a

Faunalytics
report on people who stopped being vegan or vegetarian,

via
the EA Forum, and was surprised to see:

by some estimates, a Big Mac would cost $13 without subsidies and a
pound of ground meat would cost $30.

This seemed much too high to me: I know we subsidize meat heavily, but
that's a much bigger ratio than I remembered seeing. Clicking through
to the
source, and 2022 AEIR (angle: anti-subsidy) blog post I see:

Research from 2015 shows this subsidization reduces the price of Big
Macs from $13 to $5 and the price of a pound of hamburger meat from
$30 to the $5 we see today.

Clicking through to their
source, a 2015 Berkeley Entrepreneurship & Technology
Technical Report (angle: anti-climate change) I see:

The U.S government spends $38 billion each year to subsidize the meat
and dairy industries, but only 0.04 percent of that (i.e., $17
million) each year to subsidize fruits and vegetables. A $5 Big Mac
would cost $13 if the retail price included hidden expenses that meat
producers offload onto society. A pound of hamburger will cost $30
without any government subsidies.

This is uncited, and not at all plausible. In 2013 a Big Mac cost $4.56, so
they're claiming $8.44/burger. There are about 550M Big
Macs sold annually, so the total Big Mac subsidy would be $4.6B.
That's 12% of the entire $38B. And Big Macs are only 0.4% of US beef
consumption (25.5B
lb of beef, 1/5lb each, 550M/y) let alone all the other subsidized
foods.

I did a bit more looking and found David Simon claiming in his 2013
book Meatonomics that the retail price of a big mac would
be $12. This is close enough that it might be the source of the
claim. He breaks it down as:

$0.38 for cruelty. A total of $20.7 billion in cruelty costs is
imposed on Americans each year. (Extrapolated from a study in which
auction participants bid to end cruel farming practices.)

$0.67 in environmental losses. This is a small piece of the $37.2
billion in annual environmental costs related to U.S. animal food
production each year. The figure includes the costs of soil erosion,
climate change, damage from pesticides and fertilizers, devaluation of
real property, and manure remediation.

$0.70 in subsidies. Toss in a few coins from the $38.4 billion in
government subsidies that American taxpayers pay to fund the meat and
dairy

...