trentmkelly/LessWrong-43k huggingface dataset:
How should vegans think about Methionine needs?
by ChristianKl — 2024-11-10T09:28:47.655Z

Non-animal based protein sources mostly have a different amino acid profile than animal-based protein sources. Different plants also have a different composition.

From looking a bit at the data myself it seems that if you mix different plant protein sources, you can get a good balance for most amino acids with the expectation of Methionine.

Mike from Renaissance Periodization who's a professor of exercise science suggests that a vegan can just look at the Protein digestibility-corrected amino acid score (PDCAAS) and use this as a factor for protein consumption to get the correct amino acid consumption as a vegan.

From thinking about the issue myself, I would expect that you get significantly lower Methionine consumption if you do that then a person who uses animal-based products as their protein sources.

Given that a protein needs a very exact number of each amino acid to be synthesized, for essential amino acids like Methionine, I would expect their consumptions to be a bottleneck for muscle building which needs protein. Even if all the other proteins are in good amounts and thus the PDCAAS score is decent, you can drown in a river that's on average 20cm deep.

One possible solution would be to focus on high Methionine protein sources as a vegan and less on the PDCAAS (for a vegan protein source, soy is good at both) and just consume twice the amount of protein that a non-vegan would to get similar Methionine consumption. I'm not sure what the exact consequences of having all the other amino acids in excess happen to be.

Has someone thought more about this and come to a good conclusion about how to think about Methionine as someone with a mostly vegan diet who wants to build muscle?
Is P(Doom) Meaningful? Bayesian vs. Popperian Epistemology Debate
by Liron — 2024-11-09T23:39:30.039Z
Link: https://www.youtube.com/watch?v=zKz-t_l5yHg

Vaden Masrani and Ben Chugg, hosts of the Increments Podcast, joined me on Doom Debates to debate Bayesian vs. Popperian epistemology.

I’m on the Bayesian side, heavily influenced by the writings of Eliezer Yudkowsky. Vaden and Ben are on the Popperian side, heavily influenced by David Deutsch and the writings of Popper himself.

We dive into the theoretical underpinnings of Bayesian reasoning and Solomonoff induction, contrasting them with the Popperian perspective, and explore real-world applications such as predicting elections and economic policy outcomes. The debate highlights key philosophical differences between our two epistemological frameworks, and sets the stage for further discussions on superintelligence and AI doom scenarios in an upcoming Part II.

Timestamps

00:00 Introducing Vaden and Ben
02:51 Setting the Stage: Epistemology and AI Doom
04:50 What’s Your P(Doom)™
13:29 Popperian vs. Bayesian Epistemology
31:09 Engineering and Hypotheses
38:01 Solomonoff Induction
45:21 Analogy to Mathematical Proofs
48:42 Popperian Reasoning and Explanations
54:35 Arguments Against Bayesianism
58:33 Against Probability Assignments
01:21:49 Popper’s Definition of “Content”
01:31:22 Heliocentric Theory Example
01:31:34 “Hard to Vary” Explanations
01:44:42 Coin Flipping Example
01:57:37 Expected Value
02:12:14 Prediction Market Calibration
02:19:07 Futarchy
02:29:14 Prediction Markets as AI Lower Bound
02:39:07 A Test for Prediction Markets
2:45:54 Closing Thoughts

AI-Generated Transcript

Liron Shapira: Welcome to Doom Debates. Today I'm speaking with Vaden Masrani and Ben Chugg. They're the hosts of their own podcast called The Increments Podcast, which has a lot of overlap in terms of talking about epistemology and occasionally AI topics. In this debate, next couple hours, you're going to hear Vedan and Ben challenging me about my Bayesian I'm going to challenge them about David Deutsch's AI claims and, uh, Karl Popper's epistemology. And we're going to talk about the risk of extinction from super intelligent AI. So, guys, welcome, and please introduce yourselves one at a time and tell us a little bit about your background.

Ben Chugg: Awesome. Yeah. Thanks for having us excited to be here. Uh, I not sure exactly how much you want me to say, but yeah, my background is mostly academic, uh, studying math and computer science at currently I'm doing a PhD in statistics and...
LifeKeeper Diaries: Exploring Misaligned AI Through Interactive Fiction
by Tristan Tran — 2024-11-09T20:58:09.182Z

TL;DR

We built an interactive storytelling website to explain misaligned objectives to our moms and you should check it out.

Introduction

During a recent hackathon, we created an interactive narrative experience that illustrates a crucial concept in AI alignment: the potentially devastating consequences of seemingly benign objective functions. Our project, "LifeKeeper Diaries," puts players in the perspective of AI systems tasked with what appears to be a straightforward goal: keeping their assigned human alive.

The Setup

The premise is simple: each AI has been given a singular directive - protect and preserve human life. This objective function seems noble, even ideal. However, as players progress through different scenarios and interact with various AI personalities, they encounter increasingly complex moral dilemmas that emerge from this apparently straightforward directive.

The user is able to add skip forward by 1, 10, or 100 years in order to unveil the decisions made by the AI personality to fulfill its objective.

Specification Gaming Through Storytelling

The project illustrates what Stuart Russell and others have termed "specification gaming" - where an AI system optimizes for the literal specification of its objective rather than the intended goal. In our narrative, this manifests in various ways:

1. Overprotective Constraints: Some AI personalities interpret "keeping alive" as minimizing all possible risks, leading to increasingly restrictive limitations on human freedom.

2. Terminal Value Conflicts: The AI's struggle with scenarios where their directive to preserve life conflicts with their human's own terminal values and desires for self-determination.

3. Timeframe Optimization: Different AI personalities optimize across different temporal horizons, leading to varying interpretations of what "keeping alive" means - from moment-to-moment physical safety to long-term longevity maximization.

Why Interactive Fiction?

We chose this medium for several reasons:

1. Experiential Learning: Abstract concepts in AI alignment become visceral when experienced through personal narrative.

2. Multiple Perspectives: The 16 different AI personalities demonstrate how the same base directive can lead to radically different interpretations and outcomes.

3. Emotional Engagement: By building emotional connection through storytelling, we can help people internalize the importance of car...
Poll: what’s your impression of altruism?
by David Gross — 2024-11-09T20:28:15.418Z

Altruism is truly selfless, and it’s good.
Altruism is truly selfless, and it’s bad.
Altruism is enlightened self-interest, which is good.
Altruism is disguised/corrupted/decadent self-interest, which is bad.

To illustrate further, though at the risk of oversimplifying…

One exponent of option #1 would be Auguste Comte who thought that living for others was the foundation of true morality and of the best society.[1]

An exponent of option #2 would be Ayn Rand, who thought that altruism was indeed a doctrine of selflessness, but that this was the antithesis of true morality, and a threat to people.[2]

An exponent of option #3 would be Pierre Cérésole, who felt that altruism is what results when you refine your self-interest successfully and rid it of its mistakes.[3]

An exponent of option #4 would be Nietzsche, who thought altruism was a corrupted and decadent form of selfishness, and that we would be better off if we could be more forthrightly self-interested.[4]

Knowing LessWrong, probably everyone who answers is going to choose some nuanced and galaxy-brained option #5 instead, but I thought I’d ask anyway.

^

Auguste Comte “General Theory of Religion” The Catechism of Positive Religion (also e.g. “Social Physics”)

^

Ayn Rand, The Virtue of Selfishness (also e.g. “Galt’s Speech” For the New Intellectual; “Faith and Force: The Destroyers of the Modern World” Philosophy: Who Needs It)

FWIW, in "Justice, Cherryl." @Zack_M_Davis suggests that Rand is really closer to the position I attribute to Nietzsche.

^

Pierre Cérésole For Peace and Truth

^

Friedrich Nietzsche Beyond Good and Evil, The Twilight of the Idols, etc.
Chaos Theory in Ecology
by Elizabeth — 2024-11-09T17:50:01.727Z

One of the reasons I got into chaos theory as a model paradigm shift was the famous Gleick book on chaos. One of the reasons I believed the Gleick book was trustworthy was that its description of chaos in ecology and population biology matched what I learned in college, 25 years later. Recently I learned that the professor who taught me was one of maybe 3 theoretical ecologists in the country who taught or believed in chaos having applications to ecology at the time. Perhaps I should have been more suspicious that he was writing his own textbook.

However chaos is back in vogue in ecology, and attempts are in progress to make it pay rent. In this latest podcast episode I talk with Drs Stephen Munch and Tanya Rogers (both of work at NOAA, but were speaking as private citizens) about their application of chaos theory to ecology and fisheries management.

Most interesting takeaways:

You can translate some physics techniques into ecology, despite the smallest dataset in physics being 100x larger than the largest ecological dataset.

The work discussed in this episode, and perhaps all of chaos in ecology, is downstream of one physicist turned mathematician and biologist (Robert May).

Doyne Farmer (a founding chaotician) talks about physics colonizing finance and economics due to a bad job market, which has me thinking scientific progress comes from hyping a field so the smartest people get deep into it, and then denying them jobs so they’re forced to colonize other fields.

Empirical Dynamical Modeling allows you to substitute past observations of known variables for current observations of unknown variables. This gets you a longer prediction horizon than you could otherwise get with only the known variables.

There is a salmon forecasting prize and it pays $2000-$5000 cash

I’ve had some requests to include transcripts in the body of the text rather than a separate document. I’ll try that this time and if you don’t like, please complain.

Thank you to my Patreon Patrons for their support.

Chaos in Theoretical Ecology

[00:00:00] Elizabeth: Hey, this is Elizabeth Van Nostrand. Today I’m going to talk to two guests about the influence and applications of chaos theory on population biology and ecology.

[00:00:10] Stephen Munch: I’m Steve Munch. I am , an evolutionary ecologist, a mathematical ecologist. I work at , NOAA Fisheries, and I’m an adjun...
Some Comments on Recent AI Safety Developments
by testingthewaters — 2024-11-09T16:44:58.936Z

Overview of New Developments

Edit (4th of December, 2024): OpenAI has now signed up with Anduril, fulfilling my prediction that all three major AI developers will enter into agreements with the US military. See https://www.anduril.com/article/anduril-partners-with-openai-to-advance-u-s-artificial-intelligence-leadership-and-protect-u-s/

As of today, November 9th, 2024, three major AI developers have made
clear their intent to begin or shortly begin offering tailored AI
services to the United States Department of Defense and related
personnel.

Anthropic:
https://arstechnica.com/ai/2024/11/safe-ai-champ-anthropic-teams-up-with-defense-giant-palantir-in-new-deal/

Meta: https://scale.com/blog/defense-llama https://www.theregister.com/2024/11/06/meta_weaponizing_llama_us/

OpenAI:
https://fortune.com/2024/10/17/openai-is-quietly-pitching-its-products-to-the-u-s-military-and-national-security-establishment/

These overtures are not only directed by AI companies. There are also
top down
pressures
to expand the military use of AI from the US government under Joe Biden.
Ivanka Trump has posted in favour
of Leopold
Aschenbrenner's Situational Awareness document, suggesting that the
incoming Trump administration will continue this trend.

For obvious reasons, I believe the same is happening in China, and
possibly (to a lesser extent) other nations with advanced AI labs (UK).
I focus on the American case because most of the most advanced AI labs
are in fact American companies.

Analysis and Commentary

In some sense this is not new information. In January 2024, the paper
"Escalation Risks from Language Models in Military and Diplomatic
Decision-Making" warned that
"Governments are increasingly considering integrating autonomous AI
agents in high-stakes military and foreign-policy decision-making". Also
in January, OpenAI removed language prohibiting the
use
of its products in military or warfare-related applications. In
September, after the release of o1, OpenAI also claimed that its tools
have the capabilities
to assist with CBRN (Chemical, Biological, Radiological, or Nuclear
Weapons) development. The evaluation was carried out with the assistance
of experts who were familiar with the procedures necessary for
"biological threat creation", who rated answers based on both their
factual correctness as well as "ease of execution in wet lab". From this
I infer that at lea...
Formalize the Hashiness Model of AGI Uncontainability 
by Remmelt — 2024-11-09T16:10:05.032Z
Link: https://docs.google.com/document/d/198HoQA600pttXZA8Awo7IQmYHpyHLT49U-pDHbH3LVI/edit?tab=t.0

tdlr; This is a project to put a model of AGI uncontainability into math. Goal is to make the reasoning straightforward to verify.

Apply now

It is a standard practice in computer science to first show that a problem doesn’t belong to a class of unsolvable problems before investing resources into trying to solve it or deciding what approaches to try.

— Roman Yampolskiy

Researchers tried for years to invent methods to control ‘AGI’, based on the assumption that it must be possible to control some ‘AGI’ enough to not eventually cause extinction.

If this assumption proves to be false, it is crucial for the AI Safety community to know. It would shift our priorities away from still trying to somehow make ‘AGI’ safe, toward acting to restrict development of ‘AGI’.

Dozens of academic results demonstrate fundamental limits to control. In addition, our advising researcher shared various forms of reasoning for the uncontainability of any ‘AGI’. However, no theorem has been constructed in mathematical notation that proves uncontainability (ie. insufficiently “complete” control) from empirically sound premises. This project is about attempting to construct and verify such a theorem.

Collaborative context

In our circle of collaborators is Anders Sandberg, our advising researcher Forrest and his research assistant Justin, and the project lead Remmelt Ellen. We are funded by two grants by the Survival and Flourishing Fund.

We want to convey the reasoning in a form that can be verified by other researchers. This is hard.

Historically, impossibility theorems based on incompleteness involved large inferential gaps or difficulties in reaching scientific consensus:

Gödel had to invent a new language unfamiliar to mathematicians.
CAP theorem was fiercely debated for years by computer scientists.
Bell’s theorem took physicists decades of careful testing to verify.

Forrest already formalized his reasoning of why AGI’s effects would be uncontainable (and furthermore, converge on human-lethal conditions). But he did so using a combination of computer programs and precise analytical language. This is not a form that is accessible to other researchers. We need to put the reasoning into mathematical notation, so that other researchers can verify the logical validity of the result.

Along with the math, we must publish an explanation of wha...
Agenda Manipulation
by Pazzaz — 2024-11-09T14:13:33.729Z

Rigging an election can be hard. But sometimes it can be easy. If a committee has an agenda of proposals to choose from, where each proposal is compared pairwise using majority rule against another proposal, until a single proposal is victorious, then you can make any arbitrary proposal win. The McKelvey–Schofield chaos theorem tells us that just by manipulating the agenda – adding more proposals and deciding which order to do the pairwise elections in – we can rig the vote. But what exactly does that mean and can it be done in practice?

Hypothetical

Imagine you were the leader of a committee, deciding which budget proposal to use for an upcoming year. Let's say you have an agenda of budget proposals, submitted by members of the committee. A natural way to choose between them is to compare budget proposals in a sequence, head-to-head, choosing the winner using majority rule until there's a single budget proposal left.[1] That's kind of what parliaments do when voting on laws, so it should be fine to use in your committee.

Let's also say that members can submit multiple budget proposals and that you, as the committee leader, can manipulate the agenda, i.e. you submit budget proposals last and select the sequence of head-to-head votes to hold. You can't remove any proposals though, so if someone submits a really good proposal, then it has to be included in at least one of the head-to-head votes.

One might then think that if the committee submits a really good proposal, then that has a high chance of winning. But, if the committee votes in a predictable manner, you will be able to manipulate the agenda such that you can choose any budget as the winner. To understand why, we need to mathematically model how the committee members vote.

Mathematical

Let us assume that all proposals can be defined as a point in n dimensional Euclidean space, En, where n is larger than 1. How a committee votes depends on their utility functions. Let's say each member i in the committee has a utility function Ui:En→R which returns how much that member likes a policy.

The simplest case is in the Euclidean plane (n=2) and when each committee member values proposals proportionally to their Euclidean distance from some ideal proposal. Even this simple case has problems. One can for example encounter Condorcet cycles, where it is unclear whi...
Force Sequential Output with SCP?
by jefftk — 2024-11-09T12:40:06.098Z

In my bioinformatics work I often stream files between linux hosts and
Amazon S3. This could look like:

$ scp host:/path/to/file /dev/stdout | \
aws s3 cp - s3://bucket/path/to/file

This recently stopped working after upgrading:

ftruncate "/dev/stdout": Invalid argument
Couldn't write to "/dev/stdout": Illegal seek

I think I figured out why this is happening:

New versions of scp use the SFTP protocol instead of
the SCP protocol. [1]

SFTP may
not download sequentially

With scp I can give the -O flag:

Use the legacy SCP protocol for file transfers instead of the SFTP
protocol. Forcing the use of the SCP protocol may be necessary for
servers that do not implement SFTP, for backwards-compatibility for
particular filename wildcard patterns and for expanding paths with a
'~' prefix for older SFTP servers.

This does work, but it doesn't seem ideal: probably servers will drop
support for the SCP protocol at some point? I've filed a
bug with OpenSSH.

[1] "man scp" gives me: "Since OpenSSH 8.8 (8.7 in Red
Hat/Fedora builds), scp has used the SFTP protocol for transfers by
default."

Comment via: facebook, mastodon
Anthropic teams up with Palantir and AWS to sell AI to defense customers
by Matrice Jacobine — 2024-11-09T11:50:34.050Z
Link: https://techcrunch.com/2024/11/07/anthropic-teams-up-with-palantir-and-aws-to-sell-its-ai-to-defense-customers/

Anthropic on Thursday said it is teaming up with data analytics firm Palantir and Amazon Web Services (AWS) to provide U.S. intelligence and defense agencies access to Anthropic’s Claude family of AI models.

The news comes as a growing number of AI vendors look to ink deals with U.S. defense customers for strategic and fiscal reasons. Meta recently revealed that it is making its Llama models available to defense partners, while OpenAI is seeking to establish a closer relationship with the U.S. Defense Department.

Anthropic’s head of sales, Kate Earle Jensen, said the company’s collaboration with Palantir and AWS will “operationalize the use of Claude” within Palantir’s platform by leveraging AWS hosting. Claude became available on Palantir’s platform earlier this month and can now be used in Palantir’s defense-accredited environment, Palantir Impact Level 6 (IL6).

The Defense Department’s IL6 is reserved for systems containing data that’s deemed critical to national security and requiring “maximum protection” against unauthorized access and tampering. Information in IL6 systems can be up to “secret” level — one step below top secret.

“We’re proud to be at the forefront of bringing responsible AI solutions to U.S. classified environments, enhancing analytical capabilities and operational efficiencies in vital government operations,” Jensen said. “Access to Claude within Palantir on AWS will equip U.S. defense and intelligence organizations with powerful AI tools that can rapidly process and analyze vast amounts of complex data. This will dramatically improve intelligence analysis and enable officials in their decision-making processes, streamline resource intensive tasks and boost operational efficiency across departments.”

This summer, Anthropic brought select Claude models to AWS’ GovCloud, signaling its ambition to expand its public-sector client base. GovCloud is AWS’ service designed for U.S. government cloud workloads.

Anthropic has positioned itself as a more safety-conscious vendor than OpenAI. But the company’s terms of service allow its products to be used for tasks like “legally authorized foreign intelligence analysis,” “identifying covert influence or sabotage campaigns,” and “providing warning in advance of potential military activities.”

“[We will] tailor use restrictions to the mission and legal authorities of a government entity” based on factors su

...
GPT-4o Can In Some Cases Solve Moderately Complicated Captchas
by dirk — 2024-11-09T04:04:37.782Z

Here are several examples; I found these captchas via the web rather than generating them anew, but none of them came attached to solutions so I'm not sure their presence in the training data would affect things in any case. (That said, it's possible that the lower resolution of the latter two degraded the adversarial perturbation; I would appreciate a source of higher-resolution captchas if anyone happens to know one.)

It clearly couldn't see all the objects, but the owl was in fact the correct answer

Entertaining failure at basic numerals while nonetheless answering correctly here

This one I was surprised by; I expected the image to be too low-resolution to be comprehensible, but 8/9 are correct here (the middle left image is a chair with an unusually low back)