trentmkelly/LessWrong-43k huggingface dataset:
Loving a world you don’t trust
by Joe Carlsmith — 2024-06-18T19:31:36.581Z

(Cross-posted from my website. Audio version here, or search for "Joe Carlsmith Audio" on your podcast app.)

This is the final essay in a series that I'm calling "Otherness and
control in the age of AGI." I'm hoping that the individual essays can be
read fairly well on their own, but
see here for
a brief summary of the series as a whole. There's also a PDF of the whole series here.

Warning: spoilers for Angels in America; and moderate spoilers for
Harry Potter and the Methods of Rationality.)

"I come into the presence of still water..."

~Wendell Berry

A lot of this series has been about problems with yang—that is,
with the active element in the duality of activity vs. receptivity,
doing vs. not-doing, controlling vs. letting go.[1] In particular,
I've been interested in the ways that "deep
atheism"
(that is, a fundamental mistrust towards Nature, and towards bare
intelligence) can propel itself towards an ever-more yang-y,
controlling relationship to Otherness, and to the universe as a whole.
I've tried to point at various ways this sort of control-seeking can go
wrong in the context of AGI, and to highlight a variety of
less-controlling alternatives (e.g.
"gentleness,"
"liberalism/niceness/boundaries,"
and "green") that I
think have a role to play.[2]

This is the final essay in the series. And because I've spent so much
time on potential problems with yang, and with deep atheism, I want to
close with an effort to make sure I've given both of them their due, and
been clear about my overall take. To this end, the first part of the
essay praises certain types of yang directly, in an effort to avoid
over-correction towards yin. The second part praises something quite
nearby to deep atheism that I care about a lot—something I call
"humanism." And the third part tries to clarify the depth of atheism I
ultimately endorse. In particular, I distinguish between trust in the
Real, and various other attitudes towards it—attitudes like love,
reverence, loyalty, and forgiveness. And I talk about ways these latter
attitudes can still look the world's horrors in the eye.

In praise of yang

Let's start with some words in praise of yang.

In praise of black

Recall "black," from my essay on
green. Black, on my
construal of the colors, is the color for power, effectiveness,
instrumental rationality—and hence, perhaps, the color most
paradigmatically associated with ya...
Book review: the Iliad
by philh — 2024-06-18T18:50:21.941Z

Translated by Emily Wilson

1.

I didn't know what the Iliad was about. I thought it was the story of how Helen of Troy gets kidnapped, triggering the Trojan war, which lasts a long time and eventually gets settled with a wooden horse.

Instead it's just a few days, nine years into that war. The Greeks are camped on the shores near Troy. Agamemnon, King of the Greeks, refuses to return a kidnapped woman to her father for ransom. (Lots of women get kidnapped.) Apollo smites the Greeks with arrows which are plague, and after a while the other Greeks get annoyed enough to tell Agamemnon off. Achilles is most vocal, so Agamemnon returns that woman but takes one of Achilles' kidnapped women instead.

Achilles gets upset and decides to stop fighting for Agamemnon. He prays to his mother, a goddess, wanting the Greeks to suffer ruin without him. She talks to Zeus, who arranges the whole thing, though Hera (his wife and sister) and Athena aren't happy about it because they really want Troy sacked.

So Zeus sends a dream to Agamemnon, telling him to attack Troy. Agamemnon decides to test the Greek fighters, telling them it's time to give up and sail home. So they start running back to the ships, but Athena tells Odysseus to stop them, which he does mostly by telling them to obey orders.

There's a bunch of fighting between Greeks and Trojans, and bickering among the gods, and occasionally mortals even fight gods. In the middle of it there's a one-day truce, which the Greeks use to build a wall, complete with gates. Poseidon says it's such a good wall that people will forget the wall of Troy, which upsets him because they didn't get the gods' blessing to build it.1 Agamemnon tries to convince Achilles to fight again by offering massive rewards, including the woman he stole earlier, whom he swears he has not had sex with as is the normal way between men and women.

Eventually the Trojans fight past the wall and reach the Greek fleet. At this point Patroclus, Achilles' bff (not explicitly lover, but people have been shipping them for thousands of years), puts on Achilles' armor and goes to fight. He's skilled and scary enough that the Trojans flee. Achilles told him not to chase them, but he does so anyway, and he's killed by Hector. Hector takes the weapons and armor, and there's a big fight for his body, but the Greeks manage to drag it back to their camp.

When Achilles finds out he...
AI Safety Newsletter #37: US Launches Antitrust Investigations
Plus, recent criticisms of OpenAI and Anthropic, and a summary of Situational Awareness
by Corin Katzke — 2024-06-18T18:07:45.904Z
Link: https://newsletter.safe.ai/p/ai-safety-newsletter-37-us-launches

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.

Listen to the AI Safety Newsletter for free on Spotify or Apple Podcasts.

US Launches Antitrust Investigations

The U.S. Government has launched antitrust investigations into Nvidia, OpenAI, and Microsoft. The U.S. Department of Justice (DOJ) and Federal Trade Commission (FTC) have agreed to investigate potential antitrust violations by the three companies, the New York Times reported. The DOJ will lead the investigation into Nvidia while the FTC will focus on OpenAI and Microsoft.

Antitrust investigations are conducted by government agencies to determine whether companies are engaging in anticompetitive practices that may harm consumers and stifle competition.

Nvidia investigated for GPU dominance. The New York Times reports that concerns have been raised about Nvidia's dominance in the GPU market, “including how the company’s software locks customers into using its chips, as well as how Nvidia distributes those chips to customers.” In the last week, Nvidia has become one of the three largest companies in the world by market capitalization, joining the ranks of Apple and Microsoft, with all three companies having nearly equal valuations.

Microsoft investigated for close ties to AI startups. The investigation into Microsoft and OpenAI will likely focus on their close partnership and the potential for anticompetitive behavior. Microsoft's $13 billion investment in OpenAI and the integration of OpenAI's technology into Microsoft's products may raise concerns about the creation of barriers to entry for competitors in the AI market. In addition, the FTC is looking into Microsoft’s deal with Inflection AI, where Microsoft hired the company’s co-founder and almost all its employees, and Microsoft paid $650 million in licensing agreements.

In January, the FTC launched a broad inquiry into strategic partnerships between tech giants and AI startups, including Microsoft's investment in OpenAI and Google's and Amazon's investments in Anthropic. Additionally, in July, the FTC opened an investigation into whether OpenAI had harmed consumers through its collection of data.

These developments indicate escalating regulatory scrutiny of AI technology. This heightened attention from regulators is driven by the rapid advancement and wides...
Suffering Is Not Pain
by jbkjr — 2024-06-18T18:04:43.407Z
Link: https://jbkjr.me/posts/2024/06/suffering_is_not_pain/

“Pain is inevitable; suffering is optional.”

The motivation of this post is to address the persistent conflation between suffering and pain I have observed from members of the EA community, even amongst those who purport to be “suffering-focused” in their ethical motivations. In order to best address the problem of suffering, it is necessary to be clear about the difference between suffering and mere pain or ordinary displeasure.

The parable of the second arrow

In the Buddhist parable of the second arrow, the Buddha illustrates the distinction between suffering and pain with the tale of a man struck by two arrows. The first arrow represents the pain that life inevitably brings. The second arrow, however, represents the suffering that arises from his reaction to the pain. The Buddha teaches that while the first arrow (pain) is unavoidable, the second arrow (suffering) is optional, and that by letting go of the resistance to the pain (aversion), one will not suffer the sting of the second arrow.

Defining pain and suffering

Pain: An unpleasant physical sensation or emotional experience.[1]
Suffering: The unsatisfactoriness that arises from craving, aversion, and clinging/attachment to sensations and experiences; dukkha.

I feel it is important to clarify at this point that, while the above definition of suffering derives from historically-Buddhist teachings about dukkha and its cause, I am not endorsing this definition because it is Buddhist but rather because I believe it best identifies suffering as it can actually be observed in phenomenal experience. For those who are skeptical (possibly deeply so) about the claims and teachings of Buddhism, I ask that you consider the distinction I am advocating with reference to your own experience(s) of pain and suffering. While both pain and suffering are phenomena that “feel bad” experientially, I maintain that the sensations and experiences to which the terms/concepts “pain” and “suffering” respectively refer are actually distinct as differentiated by the above definitions. As a tradition, Buddhism is almost entirely concerned with suffering, its cause, its cessation, and the way to its cessation, so I do not consider it far-fetched to think that the way(s) in which it describes suffering are quite useful in distinguishing it as it is to be found in actual experience.

Additionally, a distinction between pain and suffering...
Lamini’s Targeted Hallucination Reduction May Be a Big Deal for Job Automation
by sweenesm — 2024-06-18T15:29:38.490Z

Lamini recently posted a paper explaining their “memory tuning” methodology of using a Mixture of Memory Experts to significantly reduce LLM hallucinations over a limited domain of knowledge. They describe using this technique with a Mistral 2 open-source LLM model to achieve 95% accuracy on a text-to-SQL query task for a business, using specific facts about that business’ database schema. They claim a 10X reduction in hallucinations from 50% to 5% using this technique.

They also have a high-level write up that mentions achieving 88% accuracy for product suggestions over a 50,000 product database. If these claims are true and businesses can use Lamini’s method to tune LLM’s for significantly fewer hallucinations over the business’ relevant specific knowledge domains, it seems like significant job automation in these domains might not be far behind.
On DeepMind’s Frontier Safety Framework
by Zvi — 2024-06-18T13:30:21.154Z

On DeepMind’s Frontier Safety Framework

Previously: On OpenAI’s Preparedness Framework, On RSPs.

The First Two Frameworks

To first update on Anthropic and OpenAI’s situation here:

Anthropic’s RSP continues to miss the definitions of the all-important later levels, in addition to other issues, although it is otherwise promising. It has now been a number of months, and it is starting to be concerning that nothing has changed. They are due for an update.

OpenAI also has not updated its framework.

I am less down on OpenAI’s framework choices than Zach Stein-Perlman was in the other review I have seen. I think that if OpenAI implemented the spirit of what it wrote down, that would be pretty good. The Critical-level thresholds listed are too high, but the Anthropic ASL-4 commitments are still unspecified. An update is needed, but I appreciate the concreteness.

The bigger issue with OpenAI is the two contexts around the framework.

First, there’s OpenAI. Exactly.

A safety framework you do not adhere to is worth nothing. A safety framework where you adhere to the letter but not the spirit is not worth much.

Given what we have learned about OpenAI, and their decision to break their very public commitments about committing compute to superalignment and driving out their top safety people and failure to have a means for reporting safety issues (including retaliating against Leopold when he went to the board about cybersecurity) and also all that other stuff, why should we have any expectation that what is written down in their framework is meaningful?

What about the other practical test? Zach points out that OpenAI did not share the risk-scorecard for GPT-4o. They also did not share much of anything else. This is somewhat forgivable given the model is arguably not actually at core stronger than GPT-4 aside from its multimodality. It remains bad precedent, and an indication of bad habits and poor policy.

Then there is Microsoft. OpenAI shares all their models with Microsoft, and the framework does not apply to Microsoft at all. Microsoft’s track record on safety is woeful. Their submission at the UK Summit was very weak. Their public statements around safety are dismissive, including their intention to ‘make Google dance.’ Microsoft Recall shows the opposite of a safety mindset, and they themselves have been famously compromised recently.&...
[Linkpost] Transcendence: Generative Models Can Outperform The Experts That Train Them
by Bogdan Ionut Cirstea — 2024-06-18T11:00:04.629Z
Link: https://arxiv.org/abs/2406.11741v1

Authors: Edwin Zhang, Vincent Zhu, Naomi Saphra, Anat Kleiman, Benjamin L. Edelman, Milind Tambe, Sham M. Kakade, Eran Malach.

Abstract:

Generative models are trained with the simple objective of imitating the conditional probability distribution induced by the data they are trained on. Therefore, when trained on data generated by humans, we may not expect the artificial model to outperform the humans on their original objectives. In this work, we study the phenomenon of transcendence: when a generative model achieves capabilities that surpass the abilities of the experts generating its data. We demonstrate transcendence by training an autoregressive transformer to play chess from game transcripts, and show that the trained model can sometimes achieve better performance than all players in the dataset. We theoretically prove that transcendence is enabled by low-temperature sampling, and rigorously assess this experimentally. Finally, we discuss other sources of transcendence, laying the groundwork for future investigation of this phenomenon in a broader setting.

Summary thread: https://x.com/nsaphra/status/1803114822445465824.

Some previous discussion in an alignment-relevant context: GPTs are Predictors, not Imitators.
I would have shit in that alley, too
by Declan Molony — 2024-06-18T04:41:06.545Z

After living in a suburb for most of my life, when I moved to a major U.S. city the first thing I noticed was the feces. At first I assumed it was dog poop, but my naivety didn’t last long.

One day I saw a homeless man waddling towards me at a fast speed while holding his ass cheeks. He turned into an alley and took a shit. As I passed him, there was a moment where our eyes met. He sheepishly averted his gaze.

The next day I walked to the same place. There are a number of businesses on both sides of the street that probably all have bathrooms. I walked into each of them to investigate.

In a coffee shop, I saw a homeless woman ask the barista if she could use the bathroom. “Sorry, that bathroom is for customers only.” I waited five minutes and then inquired from the barista if I could use the bathroom (even though I hadn’t ordered anything). “Sure! The bathroom code is 0528.”

The other businesses I entered also had policies for ‘customers only’. Nearly all of them allowed me to use the bathroom despite not purchasing anything.

If I was that homeless guy, I would have shit in that alley, too.

I receive more compliments from homeless people compared to the women I go on dates with

There’s this one homeless guy—a big fella who looks intimidating—I sometimes pass on my walk to the gym. The first time I saw him, he put on a big smile and said in a booming voice, “Hey there! I hope you’re having a blessed day!” Without making eye contact (because I didn’t want him to ask me for money), I mumbled “thanks” and quickly walked away.

I saw him again a few weeks later. With another beaming smile he exclaimed, “You must be going to the gym—you’re looking fit, my man!” I blushed and replied, “I appreciate it, have a good day.” He then added, “God bless you, sir!” Being non-religious, that made me a little uncomfortable.

With our next encounter, I found myself smiling as I approached him. This time I greeted him first, “Good afternoon!” His face lit up with glee. “Sir, that’s very kind of you. I appreciate that. God bless you!” Without hesitation I responded, “God bless you, too!” I’m not sure the last time I’ve uttered those words; I don’t even say ‘bless you’ after people sneeze.

We say hi to each other regularly now. His name is George.

Is that guy dead?

Coming home one day, I saw a disheveled man lying facedown on the sidewalk.

He’s not moving. I crouched to ...
The thing I don't understand about AGI
by Jeremy Kalfus — 2024-06-18T04:25:32.987Z

Recently I've been hearing a lot about AGI, specifically that it's 5-10 years out. As someone with an interest in neuroscience, I don't understand how any system so much less complex than the human brain would be able to achieve such a thing. To me, I feel that current models are incapable of actual logical reasoning (which I know is a horribly vague idea -- sorry about that) and that any apparent logical reasoning that they are capable of is just a result of the fact that they have been trained on every possible verbal test of logical capacity.

Like, for example, it makes sense that a future LLM would be able to explain a mathematical concept that has been documented and previously discussed but I just can't see it solving existing frontier problems in mathematical theory, as it's a completely different "skillset".
Is my understanding of how LLMs work flawed? Can they perform logical reasoning?
--
P.S. Apologies for the informalities as this is my first post.
Calling My Second Family Dance
by jefftk — 2024-06-18T02:20:01.492Z

Yesterday I called my second family dance, which went pretty well!
I've stopped calling regular contra dances because I don't enjoy it
very much anymore, but these two have been a lot of fun and I'm up for
doing more.

As usual we played with an open band, and this time Nora decided to play bass:

Despite the flyers, no one
from the kids classes showed up. They've had much more luck
specifically inviting one or two friends. On the other hand, it
probably didn't help that it was father's day.

Here's what I called, which borrowed a lot from last
time. Since that was almost two years ago, though, I think it's
fine!

La
Bastringue, variant where you keep your partner. Same dance I
started with last time, and as before I didn't do a walk through and
called while I danced. It went well!

The Low-Backed
Car. Same as last time. Another good dance!

Heel-Toe Polka. This is
the only dance I called where you don't keep your partner, but I told
people it was fine to dance tandem. This dance is half-length, so
it's important to make sure the band plays AB.

Water break. It was hot!

Bridge of Athelone. We
had slightly too many couples for the music at ~10, but it worked ok
and we didn't get off the music.

The Blob Dance. Would have
been slightly better with more dancers, but it was fine.

Sasha. This one Lily
called; her first time! She did great! Probably the most popular
dance we do, and the only one I have kids requesting by name.

Water break. We like to end Sasha unreasonably fast, so it was
welcome.

Jacob's Potato.
This dance has a whole-set circle in a lonyways dance, which is fun.

Galopede. I hadn't been
planning to do eight dances in an hour, but the timing worked out for
it. This was slightly too close to the other longways dances we'd
done; I should work up a dance that is a bit more distinct.

It was a good time!

Comment via: facebook
LLM-Secured Systems: A General-Purpose Tool For Structured Transparency
by ozziegooen — 2024-06-18T00:21:13.887Z

Researcher Notes:
This is a semi-polished ideation & formalization post. I spent about a week on it. It's a bit outside our main work at QURI, but I thought it was promising enough to be worth some investigation. I've done some reading into this area, and have included a few related links, but I'm not at all an expert in internet privacy or LLMs.

If you skim this, I suggest reading the beginning until you understand the main concept, then going to the Potential Uses section.

The main controversial point might be the name, "LLM-Secured Systems". I'm unsure about this, but wanted something to refer to for this piece. Other suggestions are welcome!

This piece works well with my previous one on on Superhuman Governance. Arguably, these systems could be a key component to better governance, and governance might be the place where these systems are most useful.

I used Claude and DallE-3. I also got advice from a handful of helpful anonymous people.

Summary:

This is a formalization for using Large Language Models (LLMs) as privacy gatekeepers, dubbed LLM-Secured Systems (LSSs). The core idea is to entrust continuously-updating confidential information to an AI-based oracle system that can manage and share private data securely. Compared to techniques like homomorphic encryption, LSSs emphasize flexibility and generality over speed and privacy guarantees.

The main concepts are straightforward, have existed for quite some time, and have been explored in other discussions. This work primarily aims to propose a formal structure and compile a list of applications. I think there are a lot of potentially great applications.

The post explores a hypothetical implementation of a basic LSS using existing tools and outlines potential use cases in government, corporate accountability, personal privacy, interpersonal interactions, and more. It also discusses the implications of an ecosystem with multiple interacting LSSs, drawing parallels to a world where all communication is mediated through lawyers.

While acknowledging the current limitations of LLMs in terms of reliability and the potential for misuse, this outlines how LSSs could lead to higher levels of trust, institutional alignment, and reduced data abuse in the long term. The impact of LSSs may vary depending on the political context, with democratic countries potentially benefiting from enhanced accou...