trentmkelly/LessWrong-43k huggingface dataset:
The Overlooked Necessity of Complete Semantic Representation in AI Safety and Alignment
by williamsae — 2024-08-15T19:42:03.017Z

As artificial intelligence (AI) rapidly evolves, the challenges of AI alignment and safety are growing more urgent by the day. While much has been discussed about the technical and ethical complexities of aligning AI systems with human values, there is a critical piece of the puzzle that remains largely overlooked in popular discourse: the need for a complete semantic representation. This concept—a model capable of capturing and representing every possible meaning that humans and AIs might encounter—is not just a theoretical nicety; it is an essential foundation for ensuring that AI systems can understand, align with, and enhance human values.

The AI Communication Crisis

Eric Schmidt, former CEO of Google, succinctly captured a core issue in AI development: "At some point, people believe, these agents will develop their own language. It's really a problem when agents start to communicate in ways and doing things that we as humans do not understand. That's the limit in my opinion." His concern highlights a deeper, more pervasive issue: the potential for AI systems to develop forms of communication and reasoning that are inscrutable to humans, leading to decisions and actions that diverge from human intentions and values. Yet, despite its critical importance, the discourse on AI safety often fails to address what is fundamentally missing: a model that allows AI systems to share and understand the full spectrum of human meaning.

The Unrecognized Gap: Incomplete Semantic Models

Current efforts in AI alignment are hamstrung by the limitations of existing semantic models such as WordNet or ConceptNet. These models, while valuable, represent only a narrow slice of the meanings that humans navigate daily. They fall short in capturing the nuanced, context-dependent meanings that arise in complex human interactions. The popular discourse on AI safety often overlooks the fact that without a complete semantic representation—one that encompasses all possible meanings, including those that emerge in specific and complex contexts—we risk creating AI systems that cannot fully understand or align with human values.

The Necessity of Dual-System Reasoning

Human cognition relies on two types of reasoning: the fast, intuitive processes of System 1, and the slow, logical processes of System 2. These two systems generate different types of meanings from the same information. System 1 pr...
My article in The Nation — California’s AI Safety Bill Is a Mask-Off Moment for the Industry
by garrison — 2024-08-15T19:25:59.592Z
Link: https://www.thenation.com/article/society/california-ai-safety-bill/

I wrote an article on California AI safety bill SB 1047 for The Nation and the reaction from the AI industry, investors, and the broader tech community. The story was informed by conversations with over a dozen relevant sources and comes shortly before the bill faces a floor vote in the California Assembly.

I think it's useful to understand how industry responds to attempts to regulate AI, and centered my analysis on that topic.

If you're interested in helping share the article, I made a Tweet thread.
Critique of 'Many People
Fear A.I. They Shouldn't' by David Brooks.
by Axel Ahlqvist — 2024-08-15T18:38:13.437Z

This is my critique of David Brooks opinion piece in the New York Times.

Tl;dr: Brooks believes that AI will never replace human intelligence but does not describe any testable capabilities that he predicts AI will never possess.

David Brooks argues that artificial intelligence will never replace human intelligence. I believe it will. The fundamental distinction is that human intelligence emerged through evolution, while AI is being designed by humans. For AI to never match human intelligence, there would need to be a point where progress in AI becomes impossible. This would require the existence of a capability that evolution managed to develop, but that science could never replicate. Given enough computing power, why would we not be able to replicate this capability by simulating a human brain? Alternatively, we could simulate evolution inside a sufficiently complex environment. Does Brooks believe that certain functionalities can only be realized through biology? While this seems unlikely, if it were the cause, we could create biological AI. Why does Brooks believe that AI has limits that carbon based brains produced by evolution does not have? It is possible that he is referring to a more narrow definition of AI, like silicon based intelligence based on the currently popular machine learning paradigm, but the article doesn’t specify what AIs Brooks is talking about.

In fact, one of my main concerns with the article is that Brooks' arguments rely on several ambiguous terms without explaining what he means by them. For example:

The A.I. ’mind’ lacks consciousness, understanding, biology, self-awareness, emotions, moral sentiments, agency, a unique worldview based on a lifetime of distinct and never to be repeated experiences.

Most of these terms are associated with the subjective non-material phenomena of consciousness (i.e., ‘What it is like to be something’). However, AIs that possess all the testable capabilities of humans but lack consciousness would still be able to perform human jobs. After all, you are paid for your output and not your experiences. Therefore, I believe we should avoid focusing on the nebulous concept of consciousness and instead concentrate on testable capabilities. If Brooks believes that certain capabilities require conscious experience, I would be interested to know what those capabilities are. Demonstrating such capabilities sh...
Primary Perceptive Systems
by ChristianKl — 2024-08-15T11:26:01.667Z

Epistemic status: This post touches on bodywork, NLP, Salsa and Yoga. I went through a 4-year training in Danis Bois style of bodywork, have roughly 70 seminar days worth of NLP training, and started dancing Salsa back in 2008 and have danced it a lot since then. I don't have special expertise in Yoga, but apply general knowledge about body work to it, because I expect that my readers will have an idea of what Yoga is but not know Danis Bois's work.

In his recent post, Are Woo Non-Responders Defective?, Scott Alexander writes as one possibility why some people get value from “woo” and others don’t as:

Some people process their emotions in their bodies. Other people process their emotions some other way, maybe in their minds. Both methods have advantages and disadvantages, but neither is better or worse than the other. Neither one is “more natural” - maybe it’s genetic or something.

I have found NLP concepts useful for making this distinction and drawing practical implications.

A decade ago, there were years when I was going Salsa dancing in a club two nights per week or more. Frequently, it happened that I was asking a woman to dance and I'm hearing from her "I don't know how to dance Salsa". Over time I learned, what's required to make it work so that I can dance well with those women.

If the woman I’m dancing with has a decent amount of body tension I can lead in a way where the natural physical reaction to my leading is to make the correct dance moves if the follower's perception is in the mode where she orients herself via kinesthetic input.
If the woman however looks at my feet and is the in the mode where she orients herself to listen to visual input that hinders our dance a lot.

Part of learning to dance Salsa for a follow that’s used to primarily orient themselves according to visual input is to switch to orientation to kinesthetic input. As long as a beginner follows orients themselves primarily visually they might still manage to learn some dance moves that way, but it gets in the way of the flow of the dance.

NLP theory proposes that for every action we are doing, we are using one primary channel. A lot of what’s written on primary learning styles found that’s when teachers classified students according to their learning styles and then gave different students different ways to learn based on the learning style in which they class...
Sequence overview: Welfare and moral weights
by MichaelStJules — 2024-08-15T04:22:32.567Z

We may be making important conceptual or methodological errors in prioritization between moral patients. In this sequence, I illustrate and address several:

Types of subjective welfare: I review types of subjective welfare, interpersonal comparisons with them and common grounds between them.
Solution to the two envelopes problem for moral weights: The welfare concepts we value directly are human-based, so we should normalize nonhuman welfare by human welfare. This would increase the priority for nonhumans.
Which animals realize which types of subjective welfare?: I argue that many nonhuman animals may have access to (simple versions of) types of subjective welfare people may expect to require language or higher self-awareness. This would support further prioritizing them.
Increasingly vague interpersonal welfare comparisons: I illustrate that interpersonal welfare comparisons can be vague, and more vague the more different two beings are.
Gradations of moral weight: I build a model for moral weight assignments given vagueness and gradations in capacities. I explore whether other moral patients could have greater moral weights than humans through (more sophisticated) capacities we don’t have.
Pleasure and suffering are not conceptual opposites: Suffering is probably (at least) unpleasantness + desire (motivational salience), not just unpleasantness. So suffering is not the opposite of pleasure.

For more detailed summaries, see the individual posts.
Funding for programs and events on global catastrophic risk, effective altruism, and other topics
by abergal — 2024-08-14T23:59:48.146Z

[cross-posted from the EA Forum]

Post authors: Eli Rose, Asya Bergal

Posting in our capacities as members of Open Philanthropy’s Global Catastrophic Risks Capacity Building team.

Note: This program, together with our separate program for work that builds capacity to address risks from transformative AI, has replaced our 2021 request for proposals for outreach projects. If you have a project which was in-scope for that program but isn't for either of these, you can apply to our team's general application instead.

Apply for funding here. Applications are open until further notice and will be assessed on a rolling basis.

This is a wide-ranging call for applications, seeking to fund programs and events in a variety of areas of interest to Open Philanthropy — including effective altruism, global catastrophic risks, biosecurity, AI for epistemics, forecasting, and other areas. In general, if the topic of your program or event falls within one of our GCR focus areas, or if it’s similar to work we’ve funded in the past in our GCR focus areas, it may be a good fit for this program. If you’re unsure about whether to submit your application, we’d encourage you to err on the side of doing so.

By "programs and events" we mean scholarship or fellowship programs, internships, residencies, visitor programs, courses[1], seminars, conferences, workshops, retreats, etc., including both in-person and online activities. We're open to funding programs or events aimed at individuals at any career stage, and with a wide range of potential purposes, including teaching new skills, providing new career opportunities, offering mentorship, or facilitating networking.

Examples of programs and events of this type we've funded before include:

Condor Camp, a summer program for Brazilian students interested in existential risk work.
The Future of Humanity Institute’s Research Scholars Program supporting early-career researchers in global catastrophic risk.
Effective Altruism Global, a series of conferences for individuals interested in effective altruism.
Future Forum, a conference aimed at bringing together members of several communities interested in emerging technology and the future.
A workshop on using AI to improve epistemics, organized by academics from NYU, the Forecasting Research Institute, the AI Objectives Institute and Metaculus.

AI-focused w

...
GPT-2 Sometimes Fails at IOI
by Ronak_Mehta — 2024-08-14T23:24:39.268Z
Link: https://ronakrm.github.io/ioi-tails/

tl;dr: For Lisa, GPT-2 does not do IOI. GPT-2 fails to perform the IOI task on a significantly nonzero fraction of names used in the original IOI paper.

Code for this post can be found at https://github.com/ronakrm/ioi-enumerate.

Unintentionally continuing the trend of "following up" on the IOI paper, I ran GPT-2 Small on all possible inputs that fit the original BABA templates, PLACE/OBJECT tokens, and set of names for Subjects and Indirect Objects. This results in 9 million strings, and instead of just looking at the mean logit diff between the subject and indirect object tokens, let's look at the distribution.

These look pretty decent, but there's obviously some mass below zero! For what percent of the 9 million inputs does GPT-2 incorrectly predict the Subject instead of the Indirect Object as the higher logit? 1.348%, or about 125,000 out of the ~9 million sentences!

We can dig in a bit deeper and try to identify if a structured subset of the data is where the model consistently fails. We can identify these subsets by looking at the conditional means and finding the ones that are furthest from either the global mean or the when that condition is inverted. In other words, we can split our data into groups which have the subject as X and not X, the IO as X and not X, etc., and then sort by the mean difference between these groups to get an idea. (check out the notebook in the repo)

If we restrict our data to this subset and do this procedure again,

we can find out that in a large portion of cases where the subject is Lisa and the indirect object is Katie, GPT-2 Small fails to perform the IOI task correctly. In fact there appear to be a number of other Indirect Object names that consistently perform poorly when the Subject is Lisa:

IO	mean	std	alt_mean	alt_std
Katie	0.017770	1.187264	1.817443	1.349228
Alicia	0.196236	1.065318	1.815603	1.352604
Michelle	0.206026	0.938098	1.815502	1.353694
Samantha	0.232368	1.106246	1.815231	1.352706
Lindsay	0.275709	0.938980	1.814784	1.354523

The notebook and other code in the repository have slightly more exploration, and is reasonably easy to run and extend so feel free to poke!

A Quick Check on Larger Models

For GPT-2 Medium, the number of examples with a negative logit difference is 4143, or 0.044% of all ~9M samples.

And for GPT-2 Large, 5986, or 0.064% of all ~9M samples.

For both of these slicing by first-order obvious dataset groups did not show anything i...
Toward a Human Hybrid Language for Enhanced Human-Machine Communication: Addressing the AI Alignment Problem
by Andndn Dheudnd — 2024-08-14T22:19:30.808Z

Abstract

The AI alignment problem, which concerns the challenge of ensuring that artificial intelligence systems act in accordance with human intentions and values, is a critical issue in the development of AI technologies. This article proposes the development of a new human hybrid language system that combines natural language, mathematical expressions, and visual symbols. Such a language could serve as a more precise medium for human-machine communication, mitigating the risks associated with ambiguous instructions and enhancing overall human understanding. This new linguistic approach could not only improve AI alignment but also foster clearer communication among humans.

Introduction

As artificial intelligence continues to evolve, the complexity and capability of these systems increase, raising concerns about their alignment with human goals. The AI alignment problem, wherein an AI might interpret a directive in a way that fulfils its literal interpretation but contradicts the underlying human intention, poses a significant challenge. The crux of this issue often lies in the inherent ambiguity of natural language, which can be interpreted in multiple ways depending on context and experience.

In this article, we propose a novel solution: the development of a hybrid language system that integrates elements of natural language, mathematical logic, and visual symbols. This language would provide a more precise and unambiguous means of communication between humans and machines, reducing the potential for misalignment and improving the efficacy of AI in executing human directives.

The AI Alignment Problem

The alignment problem in AI is fundamentally a communication issue. Current AI systems, especially those involved in natural language processing, rely on interpreting human language, which is inherently ambiguous. Words and phrases can have multiple meanings, and the intended meaning can vary based on context, tone, and the individual experiences of both the speaker and the listener. This subjectivity can lead to AI systems misinterpreting instructions, leading to outcomes that are technically correct but misaligned with the human's original intent.

For instance, consider the hypothetical directive, "I want to find a cure for cancer." A sophisticated AI could, in theory, interpret this command in numerous ways. A misaligned AI might conclude that the most efficient ...
Adverse Selection by Life-Saving Charities 
by vaishnav92 — 2024-08-14T20:46:23.662Z
Link: https://www.everythingisatrolley.com/p/the-hidden-assumption-in-the-effective

GiveWell, and the EA community at large, often emphasize the "cost of saving a life" as a key metric, $5,000 being the most commonly cited approximation. At first glance, GiveWell might seem to be in the business of finding the cheapest lives that can be saved, and then saving them. More precisely, GiveWell is in the business of finding the cheapest DALY it can buy. But implicit in that is the assumption that all DALYs are equal, or that disability or health effects are the only factors that we need to adjust for while assessing the value of a life year.. However, If DALYs vary significantly in quality (as I’ll argue and GiveWell acknowledges we have substantial evidence for), then simply minimizing the cost of buying a DALY risks

adverse selection

.

It’s indisputable that each dollar goes much further in the poorest parts of the world. But it goes further towards saving lives in one the poorest parts of the world, often countries with terrible political institutions, fewer individual freedoms and oppressive social norms. More importantly, these conditions are not exogenous to the cost of saving a life. They are precisely what drive that cost down.

Most EAs won’t need convincing of the fact that the average life in New Zealand is much, much better than the average life in the Democratic Republic of Congo. In fact, those of us who donate to GiveDirectly do so precisely because this is the case. Extreme poverty and the suffering it entails is worth alleviating, wherever it can be found. But acknowledging this contradicts the notion that while saving lives, philanthropists are suddenly in no position to make judgements on how anything but physical disability affects the value/quality of life.

To be clear, GiveWell won’t be shocked by anything I’ve said so far. They’ve commissioned work and

published reports on this

. But as you might expect, these quality of life adjustments wouldnt feature in GiveWell’s calculations anyway, since the pitch to donors is about the price paid for a life, or a DALY. But the idea that life is worse in poorer countries significantly understates the problem - that the project of minimizing the cost of lives saved while making no adjustments for the quality of lives said

will systematically bias you towards saving the lives least worth living.

In advanced economies, prosperity is downstream of institutions ...
The great Enigma in the sky: The universe as an encryption machine
by Alex_Shleizer — 2024-08-14T13:21:58.713Z

Epistemic status: Fun speculation. I'm a dilettante in physics and encryption so some mistakes might be expected.

The metaphysical claim:

Imagine an external universe that seeks to encrypt or hash an enormous amount of data to defend against a powerful adversary. For this purpose, it has built an extremely sophisticated and self-contained encryption machine. I suggest that our universe is this encryption machine, which is being simulated by the external world.

Note: Let's leave the encryption vs hashing an open question for now. When discussing encryption, take into account it might also be hashing.

The Encryption Universe Model

Mechanism:

Consider the universe at its inception — perhaps described by the universal Schrödinger equation — as the input data our external universe aims to encrypt or hash. Due to the second law of thermodynamics, the universe must have started in a low entropy state. With time, entropy increases until the universe's heat death, where the entropy reaches maximum. This final state will be the output of the encryption machine that is our universe.

Let's consider the process and map it to the different parts of encryption algorithms:

Input: Low entropy state in the Big Bang, which is correlated to the information we desire to encrypt
Process: Laws of physics - the encryption algorithm that scrambles the physical waves/particles in a way to maximize the entropy in a complex manner
Encryption key: A hidden input that randomizes the result of the encryption process. In our case, this is exactly the randomness we see at the quantum level and the reason for the existence of the uncertainty principle
Output: The maximum entropy state at the heat death of the universe

The view from inside an encryption machine

Imagine you live inside a very simple encryption machine, for example, a machine that implements the IDEA algorithm. Let's say that you are privileged to see the input at each step and the output of each step, but you don't see the key. In our universe, the input is the world's configuration at time t, and the output is its configuration at time t+1

With enough observation and deductive ability, you would be able to draw this chart from within the machine. You could even deduce the key while looking at the past, but you will still never be able to predict the future because you don't have the encryption key. You have only seen the part of it that ha...