trentmkelly/LessWrong-43k huggingface dataset:
Do you believe in hundred dollar bills lying on the ground? Consider humming
by Elizabeth — 2024-05-16T00:00:05.257Z

Introduction

[Reminder: I am an internet weirdo with no medical credentials]

A few months ago, I published some crude estimates of the power of nitric oxide nasal spray to hasten recovery from illness, and speculated about what it could do prophylactically. While working on that piece a nice man on Twitter alerted me to the fact that humming produces lots of nasal nitric oxide. This post is my very crude model of what kind of anti-viral gains we could expect from humming.

ETA 6/6: I made a major error in this post and its numbers are incorrect. The new numbers show that matching Enovid’s nitric oxide content, or even getting close enough for a meaningful effect, takes way more humming than anyone is going to do.

I’ve encoded my model at Guesstimate. The results are pretty favorable (average estimated impact of 66% reduction in severity of illness), but extremely sensitive to my made-up numbers. Efficacy estimates go from ~0 to ~95%, depending on how you feel about publication bias, what percent of Enovid’s impact can be credited to nitric oxide, and humming’s relative effect. Given how made up speculative some of these numbers are, I strongly encourage you to make up speculate some numbers of your own and test them out in the guesstimate model.

If you want to know how nitric oxide reduces disease, check out my original post.

Math

Estimating the impact of Enovid

I originally estimated the (unadjusted) efficacy of nitric oxide nasal sprays after diagnosis at 90% overall reduction in illness, killing ~50% of viral particles per application. Enovid has three mechanisms of action. Of the papers I looked at in that post, one mentioned two of the three (including nitric oxide) a second mechanism but not the third, and the other only mentioned nitric oxide. So how much of theat estimated efficacy is due to nitric oxide alone? I don’t know, so I put a term in the guesstimate with a very wide range. I set the lower bound to ⅓ (one of three mechanisms) to 1 (if all effect was due to NO).

There’s also the question of how accurate the studies I read are. There are only two, they’re fairly small, and they’re both funded by Enovid’s manufacturer. One might reasonably guess that their numbers are an overestimate. I put another fudge factor in for publication bias, ranging from 0.01 (spray is useless) to 1 (published estimate is accurate).

How mu

...
Tackling Moloch: How YouCongress Offers a Novel Coordination Mechanism
by Hector Perez Arenas — 2024-05-15T23:13:48.501Z

Moloch, as articulated by Scott Alexander, represents the coordination problems that lead to outcomes that leave everyone worse off. While prediction markets explore what people think will happen, YouCongress aims to aggregate beliefs and desires regarding ideal outcomes. This open-source platform proposes a novel coordination mechanism, making use of public opinion polls with delegation and AI to enable large-scale participation and pave the way for more rational and impactful policy decisions.

Numerous proposals have been put forward to improve democracy through concepts like liquid democracy—a hybrid of direct and representative democracy—and augmented democracy. Yet, the goal of YouCongress is not to make binding referendums but to assist in decision-making, whether these decisions are made through democratic referendums, by parliaments, companies, or individuals. By aggregating beliefs and desires, YouCongress aims to provide a clearer picture of what people truly want, thereby facilitating more effective and aligned decision-making across various domains.

Delegation

YouCongress empowers users not only to cast votes directly on issues but also to delegate their votes to a chosen list of representatives. Similar to following users on social media, this delegation feature allows individuals to align their votes with the consensus of their selected delegates. For example, if a user is represented by five delegates who have voted on a specific issue, with three in favor and two against, the user's vote will automatically align with the majority and be counted in favor. Importantly, this delegated vote is overridden when a user chooses to vote personally; in such cases, the delegates' votes become irrelevant to that individual's vote.

An additional delegation mechanism to be implemented is topic-based delegation:

We will be able to integrate these mechanisms to have a list of delegates per topic:

While topic-based delegation is not yet implemented, the majority list remains effective, as many individuals choose not to vote or voice opinions on matters outside their expertise or areas of interest. For instance, a climate activist might frequently discuss environmental issues, but typically refrains from commenting on AI.

Other delegation mechanisms that we may explore are priority lists and consensus lists:

Another potential modification to scale votes is the in...
Introducing Statistical Utility Mechanics: A Framework for Utility Maximizers
by J Bostock — 2024-05-15T21:56:48.950Z

This is independent alignment research. A decent grasp on stat mech is needed to understand this post fully, but I have recently written a stat mech sequence (up to part 3) which should do the trick to catch someone up to speed rapidly.

TL;DR

Statistical Utility Mechanics is a framework which draws on stat mech to model the actions of utility-maximizing agents. I introduce this framework here, and use it to demonstrate why the most naive possible alignment plan (by which of course I mean one not too far off the one that most AI companies are currently using) is doomed by scaling laws, and we should expect to see sudden increases in AI's potential for harm around when AI reaches roughly human level.

The framework consists of just three core rules

The world is a probability distribution over states
Each agent assigns a value to each state
Agents can affect the probability distribution over states with limited power

I'll go into more detail with these in this toy model:

Toy Example: Why AI Suddenly Becomes Dangerous

The world is described by a multivariate normal distribution over states s∈Rd, centred on the origin. All variances are 1, so the distribution can be written as follows:

P0(S=s)=1(2π)d/2exp(−12s2)⟺S∼N(0,Id)

This has the nice property of being totally spherically symmetrical. I will also write P0(S=s)≡ps0 for brevity in some cases.

Let us consider an AI agent AX, which has the simplest possible utility function: xs=vX⋅s for some vector vX, and a "power level" OX∈R. Without loss of generality we will assume that |vX|=1 (the reason for this will be clear soon)

This AI will impact the world's probability distribution P0AX→P1 according to two rules:

DKL(P1∥P0)≤OX
Subject to rule 1, E(X)=∫P1(S=s)xsds is maximized

Normally stat mech tells us about entropy given expected values, but here we can flip it on its head to tell us about expected values given KL-divergence (see appendix for why we use KL-divergence here). This is ps1=ps0exp(bvX⋅s) for some constant b, and gives the distribution P1∼N(bvX,Id)

Using the formula for the KL-divergence of two multivariate gaussians, we get:

DKL(P1∥P0)=12b2(vX⋅vX)=12b2

OX=12b2∴b=√2OX

P1∼N(√2OXvX,Id)

E(X)=√2OX

This framework naturally captures the fact that rescaling or adding a constant to our utility function does not change the behaviour of the a...
Taming Infinity (Stat Mech Part 3)
by J Bostock — 2024-05-15T21:43:03.406Z

The canonical example of quantum mechanics in action is the harmonic oscillator, which is something like a mass on a spring. In classical mechanics, it wobbles back and forth periodically when it is given energy, if it's at a position x, wobbling about x0 and moving with velocty v we can say its energy contains a potential term V proportional to (x−x0)2, and a kinetic term T proportional to v2, with an overall form:

E=12k(x−x0)2+12mv2

We could try and find a distribution over x and v, but continuous distributions tend not to "play well" with entropy. They're dependent on a choice of characteristic unit. Instead we'll go to the quantum world.

One of the major results of quantum mechanics is that systems like this can only exist in certain energy levels. In the harmonic oscillator these levels are equally-spaced, with a spacing proportional to the frequency associated with the classical oscillator. Since the levels are equally-spaced, we can think about the energy coming in discrete units called "phonons".

Our beliefs about the number of phonons N in our system can be expressed as a probability distribution P(N=n) over n∈N:

This is progress: we've reduced an uncountably infinite set of states to a countable one, which is a factor of infinity! But if we do our normal trick and try to find the maximum entropy distribution, we'll still hit a problem: we get P(N=n)=0 for all n∈N.

The Trick: Distribution Families

Thinking back to our previous post, an answer presents itself: phonons are a form of energy, which is conserved. Since we're uncertain over N, we'll place a restriction on E(N) of our distribution. We can solve the specific case here, but it's actually more useful to solve the general case.

Maths, Lots of Maths, Skippable:

Consider a set of states of a system s∈S. To each of these we assign a real numeric value written as s→xs∈R. We also assign a probability s→ps∈R+ constrained by the usual condition∑s∈Sps=1.

Next, define E(X)=∑s∈Spsxs and H(S)=−∑s∈Spslnps.

Imagine we perform a transformation to our distribution, such that the distribution is still valid and E(X) remains the same. We will consider an arbitrary transformation over elements {1,2,3}:

p1→p1+dp1, p2→p2+dp2, p3→p3+dp3

dp1+dp2...
Let's Design A School, Part 2.3
School as Education - The Curriculum (Phase 2, Specific)
by Sable — 2024-05-15T20:58:50.981Z
Link: https://affablyevil.substack.com/p/lets-design-a-school-part-23

In the previous post, we outlined three phases that students would go through, where each student matriculated through them at their own speed.

Phase 1 was literacy and numeracy.

Phase 2 was core civilizational requirements and survey courses.

Phase 3 was core adulting requirements and self-study.

There are two specific curricula involved in these phases: core civilizational requirements and core adulting requirements.

In this post, we’ll go into more detail about the core civilizational requirements.

Core Civilizational Requirements

What does it mean to live in this day and age?

Where does the material abundance we take for granted come from?

What was life actually like for most of human history, and why is it so much better now?

These are questions that everyone ought to be able to answer, and the fact that most students - and most people, I suspect - can’t is an indictment of our educational system.

The context in which we live and the forces that shape it are core to our understanding of our place in history and the systems, tools, and structures that got us here.

This curriculum draws heavily from the field of progress studies.

We mentioned six classes in the previous post: basic economics, basic statistics, basic industrial history, basic civics/governance, basic scientific method, and basic information (previously media) literacy. We don’t expect this list to be exhaustive, but it’s a good starting point.

Basic Economics

Motivation

I am far from an expert on economics, but I can tell you what supply and demand are, and why they matter. I can talk about the elasticity of supply and demand, and how that generally determines where the burden of a tax falls. I can tell you what price floors and ceilings do to markets.

I can tell you why markets work, and why it matters.

This should not be some kind of elite knowledge. It shouldn’t be hidden behind a college course. It is literally the framework within which our entire economy functions.

This should be taught to every child in every school.

The curriculum will center around this diagram:

There are already plenty of basic economics courses, so it shouldn’t be too difficult to adapt these courses to a level suitable to those in phase 2 (around 10-13). Keep in mind that students are not matriculated by age as well - if they need to be older to grasp these concepts, that’s fine. There’s no need for a whole lot of math - in fact, a...
"A Paradigm for AI Consciousness" - Seeds of Science call for reviewers
by rogersbacon — 2024-05-15T20:55:32.898Z

Abstract

AI is the most rapidly transformative technology ever developed. Consciousness is what gives life meaning. How should we think about the intersection? A large part of humanity’s future may involve figuring this out. But there are three questions that are actually quite pressing, and we may want to push for answers on:

1. What is the default fate of the universe if the singularity happens and breakthroughs in consciousness research don’t?

2. What interesting qualia-related capacities does humanity have that synthetic superintelligences might not get by default?

3. What should CEOs of leading AI companies know about consciousness?

This article is a safari through various ideas and what they imply about these questions.

Seeds of Science is a scientific journal (funded through Scott Alexander's ACX grants program) publishing speculative or non-traditional research articles. Peer review is conducted through community-based voting and commenting by a diverse network of reviewers (or "gardeners" as we call them). Comments that critique or extend the article (the "seed of science") in a useful manner are published in the final document following the main text.

We have just sent out a manuscript for review, "A Paradigm for AI consciousness", that may be of interest to some in the LessWrong community so I wanted to see if anyone would be interested in joining us as a gardener and providing feedback on the article. As noted above, this is an opportunity to have your comment recorded in the scientific literature (comments can be made with real name or pseudonym).

It is free to join as a gardener and anyone is welcome (we currently have gardeners from all levels of academia and outside of it). Participation is entirely voluntary - we send you submitted articles and you can choose to vote/comment or abstain without notification (so no worries if you don't plan on reviewing very often but just want to take a look here and there at the articles people are submitting).

To register, you can fill out this google form. From there, it's pretty self-explanatory - I will add you to the mailing list and send you an email that includes the manuscript, our publication criteria, and a simple review form for recording votes/comments. If you would like to just take a look at this article without being added to the mailing list, then just reach o...
Why you should learn a musical instrument
by cata — 2024-05-15T20:36:16.034Z

I have liked music very much since I was a teenager. I spent many hours late at night in Soulseek chat rooms talking about and sharing music with my online friends. So, I tend to just have some music floating around in my head on any given day. But, I never learned to play any instrument, or use any digital audio software. It just didn't catch my interest.

My wife learned to play piano as a kid, so we happen to have a keyboard sitting around in our apartment. One day I was bored so I decided to just see whether I could figure out how to play some random song that I was thinking about right then. I found I was easily able to reconstitute a piano version of whatever melody I was thinking of, just by brute-forcing which notes were which, given a lot of patience. So that was satisfying enough that I wanted to keep doing it.

What I didn't know is how immediately thought-provoking it would be to learn even the most basic things about playing music. Maybe it's like learning to program, if you used a computer all the time but you never had one thought about how it might work.

Many of the things I learned immediately that surprised me were about my perception of the music I had listened to for all of my life. In my mind, my subjective experience of remembering music that I am very familiar with seems very vivid. I feel like I can imagine all the instruments and imagine all the sounds, just like they were in the song. But once I had to reconstruct the music myself, it quickly became clear that I was tricking myself in a variety of ways. For example, my memory of the main melody would be very clear. But my memory of any harmony or accompaniment was typically totally vague. I absolutely could not reconstruct something to play with my left hand on the piano, because I wasn't actually remembering it; I was just remembering something more abstract, I guess.

Sometimes I would be convinced I would remember a melody and reproduce it on the keyboard, but then I would listen to the real song and be surprised. The most common way I got surprised was that in my memory, I had adjusted it so that I could physically sing or hum it, even though I don't often sing. If there was a big jump up or down the scale, I would do something in my memory that sounded sort of OK instead, like replace it with a repeated note, or the same thing moved an octave, and then forget that it had ever been any...
Contra Caller Gender III
by jefftk — 2024-05-15T19:40:03.907Z

When I looked at the genders of dance callers at large contra dance
events

several years
ago there was an interesting pattern where events were more likely
to book a man and a woman than you'd expect by chance. With more
years worth of data to look at, I thought it was worth checking if
this was still the case.

To see the effect most clearly, I looked at events with two
binary callers. [1] Here's what I saw:

You can see that events with one female and one male caller are the
most common, but you'd also expect this even if bookers were
selecting both slots independently because there are two ways to get a
male-female pair and only one way each to get a male-male or
female-female pair. What we can do, however, is generate another set
of lines representing the expected number in each category if bookers
were selecting independently:

The idea is that if, say, 40% of slots went to men than you'd normally
expect to see two men 16% (40% * 40%) of the time, two women 36% (60%
* 60%) of the time, and one of each 48% (40% * 60% + 60% * 40%) of the
time. That we see one of each 63% of the time for 2026-2024 instead
of 48% tells us that about some bookers are gender-balancing callers.

What these statistics don't tell us, however, is the gender
distribution we would get if bookers didn't consider this as a
factor. You could think that this factor leads to fewer women being
booked (because bookers are avoiding booking two women and there are
more female callers) or that this leads to fewer men being booked
(because booking two male callers looks regressive). One way to try
to get at this would be to look at events with only one caller. While
they might try to balance over years, I would at least expect the
effect to be smaller? Here's what I see:

(Note that I don't have this data for before 2016)

It seems to me that at the beginning of this period the booking
preference for gender balance was leading to fewer female callers at
two-caller events, since single caller events were much more likely to
book a female caller. More recently, however, these have converged,
which I think means that, under the assumption that single-caller
events reflect the uninfluenced ratio of male to female callers, the
preference for gender balance is affecting which gigs men and women
get but isn't affecting how many?

I was curious whether this ph...
Instruction-following AGI is easier and more likely than value aligned AGI
by Seth Herd — 2024-05-15T19:38:03.185Z

Summary:

We think a lot about aligning AGI with human values. I think it’s more likely that we’ll try to make the first AGIs do something else. This might intuitively be described as trying to make instruction-following (IF) or do-what-I-mean-and-check (DWIMAC) be the central goal of the AGI we design. Adopting this goal target seems to improve the odds of success of any technical alignment approach. This goal target avoids the hard problem of specifying human values in an adequately precise and stable way, and substantially helps with goal misspecification and deception by allowing one to treat the AGI as a collaborator in keeping it aligned as it becomes smarter and takes on more complex tasks.

This is similar but distinct from the goal targets of prosaic alignment efforts. Instruction-following is a single goal target that is more likely to be reflexively stable in a full AGI with explicit goals and self-directed learning. It is counterintuitive and concerning to imagine superintelligent AGI that “wants” only to follow the instructions of a human; but on analysis, this approach seems both more appealing and more workable than the alternative of creating sovereign AGI with human values.

Instruction-following AGI could actually work, particularly in the short term. And it seems likely to be tried, even if it won’t work. So it probably deserves more thought.

Overview/Intuition

How to use instruction-following AGI as a collaborator in alignment

Instruct the AGI to tell you the truth

Investigate its understanding of itself and “the truth”;
use interpretability methods

Instruct it to check before doing anything consequential

Instruct it to us a variety of internal reviews to predict consequences

Ask it a bunch of questions about how it would interpret various commands
Repeat all of the above as it gets smarter
frequently ask it for advice and about how its alignment could go wrong

Now, this won’t work if the AGI won’t even try to fulfill your wishes. In that case you totally screwed up your technical alignment approach. But if it will even sort of do what you want, and it at least sort of understands what you mean by “tell the truth”, you’re in business. You can leverage partial alignment into full alignment—if you’re careful enough, and the AGI gets smarter slowly enough.

It's looking like the critical risk period is probably going to involv...