trentmkelly/LessWrong-43k huggingface dataset:
mental number lines
by bhauth — 2023-07-19T21:01:22.702Z

This paper says:

it was shown that second-grade children unevenly placed numbers on a 0–1000 number line as their placement pattern was best fit by a logarithmic rather than by a linear function. The same age group placed numbers more linearly (i.e., resulting in a constant distance between consecutive numbers) on a 0–100 number line. In contrast, adults produced a linear distribution when placing numbers on a 0–1000 number line

Two developmental leaps occurring during elementary school were recognized: (1) the division of the number line into two segments and (2) consistent use of different anchor points on the number line—the left endpoint in first grade, the right endpoint in second grade, and finally the midpoint in third grade.

People mentally represent numbers as positions on a number line. I consider this a latent space representation. Human conceptual capabilities imply ability to at least average latent space representations, and possibly add them, which could then be used for fast approximation of sums. If only averaging is possible, then a 2-step (average -> double) process would be needed. The accuracy of that approach is insufficient for solving arithmetic problems, so people often instead use a much slower algorithm with exact accuracy.

Per the linked paper, 2nd-grade children have a tendency to place large numbers logarithmically, which implies to me an approximately logarithmic latent space representation. In that case, an accurate logarithmic representation should also be something humans are capable of with some training. In an accurate logarithmic representation, addition and subtraction are, of course, equivalent to multiplication and division.

Is it possible for people to do fast mental multiplications with the accuracy of a typical slide rule, by learning a logarithmic mental representation of numbers between 1 and 10? It seems plausible to me. Any big mental arithmetic fans around here?

But of course, the latent spaces humans typically use are high-dimensional, with limited range in any given dimension. It's possible that the "linear" number line in human mental representations is curved in its latent space to better utilize high dimensionality. In that case, averaging representations of distant numbers doesn't work so well.

Supposing that the mental number line is curved, how is it linearized when people write numbers on a line? My...
Any suggestions for an impactful master's thesis in Political Science?
by Klara Helene Nielsen — 2023-07-19T17:44:32.728Z

I have a particular interest in the interplay between political science and artificial intelligence. My focus is less on safety - more on implementation/use-case. I have especially focused on education and decision-making more broad and in the context of the Global South (ex Khan Academy use of AI for personal tutoring), but I am open for any inputs and ideas as I am in a very early phase (I begin in January).

The idea is not to be blinded by quick or conventional ideas, but to gain an understanding on WHY a certain focus could be impactful - and use this unique opportunity to dive-in to a topic as a first step towards an impactful "career" further on.

Any inputs are welcome :-)
Incident reporting for AI safety
by Zach Stein-Perlman — 2023-07-19T17:00:57.429Z

Zach Stein-Perlman, Sebastian Lodemann, Štěpán Los, and Moritz von Knebel

We made this (quickly) because a resource on incident reporting as a tool for AI safety should exist; we are not experts; suggestions are welcome.

Zach recommends engaging with this post by reading §1 and skimming §4 (focusing on sources near the top). Read §2 for recommendations, §3 for existing efforts on incident reporting in AI, or §6 for incident reporting outside of AI, according to your interests.

1. Introduction

Incident reporting is a widely-used and generally accepted practice across many domains as a way to support safety practices. Incident reporting

can help

"expose problematic systems, improve (or in some cases, shut down) those systems, and seek redress" for anyone harmed. When accidents are publicly disclosed, system designers are forced to address them and at least fix ongoing safety issues.

Incident reporting

can be

"vital for helping others avoid training risky systems, and for keeping AI developers accountable." Going forward, if other developers of AI systems can easily find past accidents and near misses, and see how they were addressed, they can better avoid similar incidents.[1] If there is an expectation that all incidents are reported, there is also an added motivation for model developers to avoid such incidents.

More broadly

, "open, non-punitive reporting of accidents and near-misses is typically part of safety culture both instrumentally, and because reporting builds a norm of admitting mistakes, noticing them, and sharing lessons learned."

What is an 'incident' for AI? Different approaches, different typologies of incidents, and different sets of goals for reporting lead to defining "incident" more or less broadly.[2]

Governments can help support incident reporting in many ways, including by helping actors coordinate, requiring incident reporting in some cases, and by protecting actors from liability for voluntarily-reported incidents in some cases.

Incidents should probably include issues reported via bug bounty or responsible disclosure systems. Labs' bug bounty and responsible disclosure systems should cover model outputs in addition to security. See

Recommendation: Bug Bounties and Responsible Disclosure for Advanced ML Systems

(Vaniver 2023).

Information about some incidents is

infohazardous

and should not be shared widely. In particula...
Alignment Grantmaking is Funding-Limited Right Now
by johnswentworth — 2023-07-19T16:49:08.811Z

For the past few years, I've generally mostly heard from alignment grantmakers that they're bottlenecked by projects/people they want to fund, not by amount of money. Grantmakers generally had no trouble funding the projects/people they found object-level promising, with money left over. In that environment, figuring out how to turn marginal dollars into new promising researchers/projects - e.g. by finding useful recruitment channels or designing useful training programs - was a major problem.

Within the past month or two, that situation has reversed. My understanding is that alignment grantmaking is now mostly funding-bottlenecked. This is mostly based on word-of-mouth, but for instance, I heard that the recent lightspeed grants round received far more applications than they could fund which passed the bar for basic promising-ness. I've also heard that the Long-Term Future Fund (which funded my current grant) now has insufficient money for all the grants they'd like to fund.

I don't know whether this is a temporary phenomenon, or longer-term. Alignment research has gone mainstream, so we should expect both more researchers interested and more funders interested. It may be that the researchers pivot a bit faster, but funders will catch up later. Or, it may be that the funding bottleneck becomes the new normal. Regardless, it seems like grantmaking is at least funding-bottlenecked right now.

Some takeaways:

If you have a big pile of money and would like to help, but haven't been donating much to alignment because the field wasn't money constrained, now is your time!
If this situation is the new normal, then earning-to-give for alignment may look like a more useful option again. That said, at this point committing to an earning-to-give path would be a bet on this situation being the new normal.
Grants for upskilling, training junior people, and recruitment make a lot less sense right now from grantmakers' perspective.
For those applying for grants, asking for less money might make you more likely to be funded. (Historically, grantmakers consistently tell me that most people ask for less money than they should; I don't know whether that will change going forward, but now is an unusually probable time for it to change.)

Note that I am not a grantmaker, I'm just passing on what I hear from grantmakers in casual conversation. If anyone with more knowledge wants to c...
Zener Science
by Screwtape — 2023-07-19T16:40:54.974Z

Summary: Parapsychology (the study of unorthodox mental phenomena such as hypnosis or telepathy) is widely considered to be the study of something that doesn’t exist. Many parapsychology experiments are uncommonly good ways to practice science, since the experiments can be performed without expensive apparatus and require some understanding of statistics. Let’s do a few experiments and see if these things exist!

Tags: medium

Purpose: To learn how to do science on a confusing phenomenon.

Materials: Zenner cards are useful but a 52 deck can do in a pinch. Some variations may suggest special decks or other tools entirely.

Announcement Text: “

The Control Group is Out of Control

is a popular Scott Alexander article. In it, he examines parapsychology, the field of science that investigates such phenomenon as hypnosis, telepathy, and astral projection. If you think that psychic powers aren't real, then this is a fascinating field: it's a good example of how attempts to do science can give you the wrong answer, returning false positives. My science classes mostly focused on easily replicable experiments like testing acceleration due to gravity or the resistance different materials had to electricity. When studying something like telepathy or antidepressants where not every person responds the same way, how do you sort actual effects from statistical noise?

Designed by Karl Zener in the 1930s, Zener Cards are a set of twenty-five cards with one of five different visually distinct symbols on one side. Karl Z. used them as part of a series of parapsychology experiments attempting to prove psychic powers existed; the typical experiment involved a test subject attempting to guess which card was being held by the test administrator. The experiments initially showed that some people are naturally psychic, guessing correctly more often than chance. Later experiments concluded that this was entirely down to poor experimental design. So my question to you: are any of us psychic? What do you think you know, and how do you think you know it?

I'm bringing a couple packs of Zener cards and a couple packs of regular 52 decks. Let's do some science!”

Image from Wikipedia. https://en.wikipedia.org/wiki/Zener_cards

Description: The basic zener card experiment works like this; one person (the Tester) takes a deck of 25 cards and shuffles them, then one by one draws a card fro...
Desiderata for an AI
by Nathan Helm-Burger — 2023-07-19T16:18:08.299Z

I think a main focus of alignment work should be on redesigning AI from the ground up. In doing so, I think we should keep in mind a set of desirable characteristics to aim for in a better AI.

Most of my ideas here aren't original to me, so I'll add some links to sources. For lots more background material, check out the posts associated with the tags on this post. If you're familiar with AInotkilleveryoneism background material, you'll probably already have come across the content of these links. I'm not familiar with all these ideas being placed together as a set of goals to be pursued by a research program, which is why I'm writing this post.

These are meant to be pointers towards more true and fundamental ideals, not complete descriptions without edge cases. Also, I feel sure that there are desirable qualities missing from this list. Hopefully readers will think of some and put them in the comments. Iteratively improving these descriptions would be an important aspect of the research.

What isn't covered: I don't think that even a model with all these characteristics would be safe for use by a malicious human or group. This post is presuming wise, careful, kind operators with good intentions towards all of humanity. That's a big fragile assumption, and something that I think needs more work.

Interpretable

Background: Theories of Impact of Interpretability, How Interpretability can be impactful, World-Model Interpretability

Interpretable meaning that the model is fundamentally easy to understand for human observers. This should minimize the dependency on special tools or techniques in order for observers to understand what's going on in the reasoning processes of the model. I don't think it would make sense to include this on the list if I didn't think it was feasible. I believe, from years of studying neuroscience, that we can design a different kind of model which is inherently more interpretable. I will make another post talking about these hypotheses.

Corrigible

Background: Paul Christiano on corrigibility

Corrigible is, to my mind, a sort of wise and genuine obedience. A corrigible assistant is one who seeks to understand the spirit of your requests, not just the letter, and obey both as far as able. Where there is confusion, asks for clarification. When it thinks it cannot achieve what has been asked of it, lets you know, and perhaps su...
Valuism - an approach to life for you to consider
by spencerg — 2023-07-19T15:23:19.218Z

I finally finished my series of 5 essays on my life philosophy (Valuism)! In a nutshell, Valuism suggests that you work to figure out what you intrinsically value, and then that you try to use effective methods to create more of what you intrinsically value.

While simple and straightforward at first glance, few people seem to approach life this way, and Valuism ends up with a number of surprising implications and, I think, provides a perspective that can help shed light on a number of different domains.

Interestingly, Effective Altruism is implied by Valuism plus a specific set of strong intrinsic values that most Effective Altruists have (reducing suffering + truth).

Here is the sequence of essays, if you feel like checking them out:

Part 1: Doing what you value as a life philosophy – an introduction to Valuism

Part 2: What to do when your values conflict?

Part 3: Should Effective Altruists be Valuists instead of utilitarians?

Part 4: What would a robot value? An analogy for human values

Part 5: Valuism and X: how Valuism sheds light on other domains

A big shoutout goes to Amber Dawn Ace who wrote these essays with me.
Hedonic Loops and Taming RL
by beren — 2023-07-19T15:12:42.327Z

Crossposted from my personal blog.

Everybody knows about the hedonic treadmill. Your hedonic state adjusts to your circumstances over time and quickly reverts to a mostly stable baseline. This is true of basic physiological needs -- you feel hungry; you seek out food; you eat; you feel sated, and you no longer seek food. This also applies to more subjective psychological states. You cannot, normally, feel extremely happy, or extremely sad, forever. If life is getting better, you feel some brief initial happiness, but this quickly reverts to baseline. This also happens to a surprising extent on the downward side as well. Badness can quickly become normalized and accepted. This cycle of valuation and devaluation makes human behaviour difficult [1] to interpret as being that of a utility maximizer with a fixed reward or utility function. This is also one of the reasons why human experience is, in many ways, very unlike that of a utility maximizer.

It is also a good fit to many biological problems, and indeed many other real-world problems which are satisficing rather than maximizing objectives. For example, usually you want to eat enough food to have sufficient energy to go about your other objectives, not maximize your total amount of food consumption. From an AI safety perspective, satisficing objectives, if we can figure out how to robustly encode them and make them stable, are likely to be much safer than maximization objectives[2], especially if the satisfiable region is broad. This is because the objective is bounded and, if relatively easily achievable, there is a much less of a strong incentive towards instrumental convergence to generic power-seeking behaviour. Satisficing objectives in the brain do not appear to be implemented by some simple method like quantilizing a reward function. Instead there is a more complex PID-like control loop, where the salience and importance of varying objectives are flexibly increased and decreased in line with physiological needs to maintain homeostasis, and I have written previously about the benefits of dynamic reward functions and homeostatic control for alignment and specifically preventing the many pathologies of pure maximization.

However, there is a more important angle. Beyond just satisficing, hedonic treadmills are an existence proof of dynamic, flexible, and corrigible value changes occurr...
What Caused the Puzzling Decline in Activism Against Policy Violence Towards Black People?
by ChristianKl — 2023-07-19T14:40:26.120Z

I find myself surprised by the noticeable decline in the fervor surrounding activism against policy violence towards Black people from its peak in the summer of 2020 to the subsequent years of 2021 through 2023. Initially, I had anticipated that demands for increased use of body cams would result in the surfacing of more videos documenting the maltreatment of Black individuals by law enforcement officers, inevitably spurring more protests.

To attempt to make sense of this, I have considered several hypotheses:

- FBI Infiltration: The FBI may have successfully infiltrated activist networks, consequently dismantling their internal structure and hindering their ability to organize.

- Improved Law Enforcement Conduct: Police officers might have made significant changes to their behavior, thus reducing the chances of any controversial incidents being captured on camera.

- Shift in Mainstream Media Focus: The mainstream media, which previously stoked the flames of protest during the Trump administration, might have lost interest under Biden's administration, leading to a reduction in protest activities.

- Democratic Influence: Democratic organizers may have found that the 'defund-the-police' activism was polling poorly, leading to concerted efforts to tone it down.

- Misallocation of BLM Funds: Perhaps there's been some level of corruption within the BLM movement, with funds being directed towards personal extravagances instead of being allocated for organizing protests.

- Lack of Organic Support: Unlike the sustained energy seen in movements like the Tea Party, the impetus behind these protests might have been primarily media-driven, evaporating once the media's interest waned.

- Successful Reforms: Maybe there have been enough successful reforms in the policy violence sphere, leading people to feel less need to advocate for the cause.

While these are speculative theories, I'm curious hear from those who were less surprised by this development. I'm interested in understanding what you perceive to be the primary factors behind this trend.
Lisa Feldman Barrett versus Paul Ekman on facial expressions & basic emotions
by Steven Byrnes — 2023-07-19T14:26:05.675Z

1. Summary / Table of Contents

This post is mostly a book review of

How Emotions Are Made: The Secret Life of the Brain

by Lisa Feldman Barrett.

Barrett is very interested in arguing against a particular view that she attributes to Paul Ekman, so I also read some of Ekman’s work, including his book

Emotions Revealed

, and his paper

“An argument for basic emotions”

.

My assessment is that Barrett is correct that the view she attributes to Ekman is wrong, and Ekman is equally correct that the view he attributes to his intellectual opponents is wrong. But they are directly disagreeing with each other much less than they seem to think they are, and I’ll try to paint a single coherent picture that captures the best parts of both perspectives at once.

(Note for my regular readers: This post has no mention of AI safety or alignment, but I consider it vaguely related to my long-term research project described at

this link

.)

Table of Contents (with section summaries)

Section 2 presents three positions:

The “anti-Barrett” position is Barrett’s punching-bag that she has worked tirelessly to refute. She calls it “the classical view of emotions”. It claims (to oversimplify / caricature a bit) that everyday emotion concepts like “anger” have a perfect 1-to-1 correspondence with discrete innate behavioral programs.
The “anti-Ekman” position is Ekman’s punching-bag that he has worked tirelessly to refute. It claims that facial expressions are 100% social conventions, with no cross-cultural correlation whatsoever.
My own position is kind of a compromise. I think we have a bunch of “innate behaviors” (like vomiting, laughing, disgust-reactions, and

Duchenne-smiles

), associated with cell groups in the hypothalamus and brainstem, and I think these are what Ekman is trying to talk about. And, I think we learn emotion concepts within our lifetimes, just like we learn every other kind of concept, and they wind up stored in our cortex, and these emotion concepts are what Barrett is talking about.

Section 3 argues that cross-cultural studies of facial expressions disprove both the anti-Barrett and the anti-Ekman positions, thus nicely explaining why both sides keep declaring victory.
Sections 4 & 5 muse on whether Barrett agrees or disagrees with the extreme “anti-Ekman” position, and then whether Ekman agrees or disagrees with the extreme “a

...
AISN#15: China and the US take action to regulate AI, results from a tournament forecasting AI risk, updates on xAI’s plan, and Meta releases its open-source and commercially available Llama 2
by Corin Katzke — 2023-07-19T13:01:00.939Z
Link: https://newsletter.safe.ai/p/ai-safety-newsletter-15

Welcome to the AI Safety Newsletter by the

Center for AI Safety

. We discuss developments in AI and AI safety. No technical background required.

Subscribe here to receive future versions.

Both China and the US take action to regulate AI

Last week, regulators in both China and the US took aim at generative AI services. These actions show that China and the US are both concerned with AI safety. Hopefully, this is a sign they can eventually coordinate.

China’s new generative AI rules

On Thursday, China’s government released new rules governing generative AI. China’s new

rules

, which are set to take effect on August 15th, regulate publicly-available generative AI services. The providers of such services will be criminally liable for the content their services generate.

The rules specify illegal content as:

“inciting subversion of national sovereignty or the overturn of the socialist system, endangering national security and interests or harming the nation’s image, inciting separatism or undermining national unity and social stability, advocating terrorism or extremism, promoting ethnic hatred and ethnic discrimination, violence and obscenity, as well as fake and harmful information [...]”

Providers are also expected to respect intellectual property rights, protect personal information, and avoid using discriminatory training data.

The rules are some of the strictest regulations of generative AI to date. In order to publicly release generative AI services, Chinese companies will need to be able to tightly control what their models generate.

According to

Helen Toner, the Director of Georgetown’s Center for Security and Emerging Technology (CSET), the rules represent “a pretty significant set of responsibilities, and will make it hard for smaller companies without an existing compliance and censorship apparatus to offer services.”

However, even large companies will face regulatory challenges. Baidu is

still waiting on government approval

to release its chatbot, Ernie. Baidu isn’t alone — currently no LLM is publicly available in China.

China is balancing regulation with promoting technological innovation. Critically, the rules only apply to publicly-available generative AI. They do not regulate research done by AI labs. This distinction represents a change from an

earlier draft

of the rules, which also regulated the “research and development” of generative AI. The earlie...