trentmkelly/LessWrong-43k huggingface dataset:
What evidence is there for (or against) theories about the extent to which effective altruist interests motivated the ouster of Sam Altman last year?
by Evan_Gaensbauer — 2024-01-18T05:14:54.002Z

There was of course initially much speculation that effective altruism was solely responsible. That was a common accusation from some among the effective accelerationism community (https://en.wikipedia.org/wiki/Effective_accelerationism), a.k.a. "e/acc." Many of them may remain forever convinced of that, in spite of whatever accurate conclusion may eventually be substantiated.

It's nonetheless important for the EA community to try understanding how some of those among its ranks other than former OpenAI board members may have impacted the course of events leading up to and after Sam Altman was fired as CEO of OpenAI. Obviously, the ethos of EA in relation to AI safety played a role. The choices made by Helen Toner and Tasha McCauley in their multiple professional roles related to AI safety/risk would obviously be influenced by their prior affinity for EA, and views on AI safety common among the community. The same is of course true for Holden Karnofsky, also a former OpenAI board member.

Yet since the infamous course of events has ended, there is still widespread doubt that the board directors who fired Sam Altman acted alone. Public speculation persists that they must have acted in part on the behest or direction of other influential figures in or around the EA community. Almost two months out from Sam Altman being fired, it's worth checking if more info, other than the usual speculation on social media, has surfaced as to the definite motives for that move being taken. That of course includes other evidence substantiating motives not directly to AI safety.

In the last 2 months, from within the EA and rationality communities themselves, there has been surprisingly little review of such potential evidence, or a re-evaluation of the still incomplete narrative that had taken shape by December. There appears to have been one relevant post on the subject on each of the EA Forum, and LessWrong, respectively

On November 22 of last year, effective altruist Garrison Lovely published a Substack post, largely in response to a report from Semafor, challenging the notion that EA as a philosophy and movement is squarely and singularly to blame (https://garrisonlovely.substack.com/p/is-effective-altruism-really-to-blame). Specifically, Garrison points out that:

OpenAI board member Adam D'Angelo, while being a close colleague of Dustin Moskowitz, doesn't publicly or personal

...
Does literacy remove your ability to be a bard as good as Homer?
by Adrià Garriga-alonso — 2024-01-18T03:43:14.994Z

Epistemic status: probably we did lose the ability to memorize long songs due to improper practice, but it may be possible to enjoy the benefits of literacy and epic memory simultaneously.

Thanks to Niels uit de Bos for better links and editing, and to Ryan Kidd for encouragement to post.

You probably know that Socrates thought writing was terrible and it would destroy people’s ability to memorize things, because now they’re written down and don’t need to be memorized. I always thought that was a little ridiculous, maybe the effect was there and memorization would be less good, but not to a crazy extent.

Well, Milman Parry and Albert Lord traveled to Yugoslavia in the 1930-1950s, and recorded performance of gusle-player (guslar) bards. The greatest of them was Avdo Međedović. From Wikipedia:

At Parry’s request, Avdo sang songs he already knew and some songs he heard in front of Parry, convincing him that someone Homer-like could produce a poem so long. Avdo dictated, over five days, a version of the well-known theme The Wedding of Meho Smailagić that was 12,323 lines long, saying on the fifth day to Nikola (Parry’s assistant on the journey) that he knew even longer songs. On another occasion, he sang over several days an epic of 13,331 lines. He said he had several others of similar length in his repertoire. In Parry’s first tour, over 80,000 lines were transcribed.

All of the bards, which recited incredibly long songs from memory and composed slightly new lyrics on the fly “at the rate of [10-20] ten-syllable lines a minute”, and they could not have been geniuses, because there were too many of them. Instead, they had a “special technique of composition”: they were illiterate. From The Singer of Tales:

[Albert] Lord sees the existence of literacy and written/printed texts as deadly-- not to the songs themselves, but to the method of composition by which they are realised [which in the end amounts to the same thing]--schools, cities, and literacy eventually put [an end] to it in urban areas

“We must remember that the oral poet has no idea of a fixed model text to serve as his guide. He has models enough but they are not fixed and he has no idea of memorizing them in a fixed form. Every time he hears a song sung, it is different”

Once the idea that there is a fixed text enters the bard’s minds, they stop being able to compose new versions on the fly. Also presumably ...
D&D.Sci Hypersphere Analysis Part 4: Fine-tuning and Wrapup
by aphyer — 2024-01-18T03:06:39.344Z

As before, this is extreme spoilers for this D&D.Sci scenario, if you want to solve on your own try that first!

Last time we got a solution that I'm pretty sure will work. Everything from here on is just showing off a bit, and seeing if we can find any stuff that abstractapplic didn't expect anyone to get :P.

Finetuning our Model

The good news about having a reasonably firm model after last time is that we now have much better data for looking into any datapoints we want: before our data was extremely noisy, but now that we've controlled out the large majority of variance we're able to zoom in much more precisely on the few remaining effects. We've gone from having graphs of the effect of Pi that looked like this to vaguely suggest that 'Pi around the true value is probably good':

to having graphs that look like this to highlight that 'we're putting a threshold at 3.14159 when it should be at 3.15':

So let's make a pass through all the continuous variables and see if we can get anything else out of them. We're plotting the value of each variable against the difference between the value of Penalty our model predicts vs. what we actually see: so a positive number means 'our model is predicting a lower penalty, and so higher performance, than actually occurs for sites like this' and a negative number means the reverse.

Latitude

Nothing stands out to fix here. The two-blobs shape is because latitudes near 0 are rarer. That single point near the top left is our highest-error site in the universe, hopefully it'll shrink a bit as we fix other things.

Shortitude

...wow. Okay, that certainly stands out very dramatically now that we've got a cleaned-up model. Our initial looks at Shortitude saw things like this:

I guess we just didn't see the effect because it wasn't very big? A Penalty Factor of...this looks like about 0.015? - corresponds to only a 3-4% drop in performance. And because of the multiplicative nature of these metrics, a 3-4% drop in average performances around 20-25% would be a <1 percentage point[1] diff, which we couldn't pick out on that graph but can pick out now. Let's rerun the model with an extra parameter for [Shortitude is >45]...

...alright, our largest diff has shrunk from 4.74% to 2.72%, that's quite a drastic jump in our confidence. This doesn't actually change our...
Some heuristics I use for deciding how much I trust scientific results
by  — 2024-01-18T02:48:54.968Z

I've done nothing to test these heuristics and have no empirical evidence for how well they work for forecasting replications or anything else. I’m going to write them anyway. The heuristics I’m listing are roughly in order of how important I think they are. My training is as an economist (although I have substantial exposure to political science) and lots of this is going to be written from an econometrics perspective.

How much does the result rely on experimental evidence vs causal inference from observational evidence?

I basically believe without question every result that mainstream chemists and condensed matter physicists say is true. I think a big part of this is that in these fields it’s really easy to experimentally test hypotheses, to build really precisely test differences in hypotheses experimentally. This seems great.

On the other hand, when relying on observational evidence to get reliable causal inference you have to control for

confounders

while not controlling for

collider

s. This is really hard! It generally requires finding a natural experiment that introduces randomisation or having very good reason to think that you’ve controlled for all confounders.

We also make quite big updates on which methods effectively do this. For instance until last year we thought that

two-way fixed effects

did a pretty good job of this before we realised that actually

heterogeneous treatment effects are a really big deal

for two-way fixed effects estimators.

What’s more, in areas that use primarily observational data there’s a really big gap between fields in how often papers even try to use causal inference methods and how hard they work to show that their

identifying assumptions

hold. I generally think that modern microeconomics papers are the best on this and nutrition science the worst.

I’m slightly oversimplifying by using a strict division between experimental and observational data. All data is observational and what matters is credibly you think you’ve observed

what would happen counterfactually without some change

. But in practice, this is much easier in settings where we think that we can change the thing we’re interested in without other things changing.

There are some difficult questions around scientific realism here that I’m going to ignore because I’m mostly interested in how much we can trust a result in typical use c...
In Strategic Time, Open-Source Games Are Loopy
by StrivingForLegibility — 2024-01-18T00:08:40.909Z

For many situations we'd like to analyze, information flow is more important than chronological order. When game theorists say that two players choose their actions simultaneously, they mean that they each make their choice without knowing the action chosen by the other player. This is usually contrasted with sequential decisions, where one player is able to observe the action chosen by another, and potentially choose differently based on the action observed.

Open-source games are ones in which each player delegates their choice to a computer program, and each program has access to the source code of all delegated programs at runtime. This introduces a new type of information relationship between decisions, where two agents can attempt to condition their output on each other's output. It's this strategic time loop that enables agents in the Robust Cooperation paper to Cooperate, if and only if they are logically certain that the other agent will also Cooperate. And amazingly, such agents actually do Cooperate with each other!

This is a more powerful form of prediction than agents in closed-source games can perform, and they're already classically assumed to be infinitely intelligent and logically omniscient. What makes the difference is that the information about each agent's output is available to each agent at the time they're making their decisions. Any Nash equilibrium of a game can be realized by a program equilibrium, but the joint policies chosen in some program equilibria are not Nash equilibria. The program equilibrium (FairBot, FairBot) leads to the joint policy (Cooperate, Cooperate), but this is not a Nash equilibrium of the underlying Prisoners' Dilemma.

In this post I'll describe how strategic time is modelled in compositional game theory, which is my current favorite model of non-loopy strategic information flow. I'll also go into more detail about strategic time loops, how they're different from ordinary consequentialism, and what properties we want from systems that enable these loops in the first place.

Strategic Time

Compositional game theory aims to model complex games as being composed of simpler games, put together according to simple rules. One major aim of the field is to make it easier to reason about complex games, in the same way that we reason about complex software systems that are composed of simpler modules. To this end, formal rule...
Four visions of Transformative AI success
by Steven Byrnes — 2024-01-17T20:45:46.976Z

Tl;dr

When people work towards making a good future in regards to Transformative AI (TAI), what’s the vision of the future that they have in mind and are working towards?

I’ll propose four (caricatured) answers that different people seem to give:

(Vision 1) “Helper AIs”,
(Vision 2) “Autonomous AIs”,
(Vision 3) “Supercharged biological human brains”,
(Vision 4) “Don’t build TAI”.

For each of these four, I will go through:

the typical assumptions and ideas that these people seem to typically have in mind;
potential causes for concern;
major people, institutions, and research directions associated with this vision.

I’ll interject a lot of my own opinions throughout, including a suggestion that, on the current margin, the community should be putting more direct effort into technical work towards contingency-planning for Vision 2.

Warning 1: Oversimplifications. This document is full of oversimplifications and caricatures. But hopefully it’s a useful starting point for certain purposes.

Warning 2: Jargon & Unexplained Assumptions. Lots of both; my target audience here is pretty familiar with the AGI safety and alignment literature, and buys into widely-shared assumptions within that literature. But DM me if something seems confusing or dubious, and I’ll try to fix it.

Vision 1: “Helper AIs”—AIs doing specifically what humans want them to do

1.1 Typical assumptions and ideas

By and large, people in this camp have an assumption that TAI will look, and act, and be trained, much like LLMs, but they’ll work better. They also typically have an assumption of slow takeoff, very high compute requirements for powerful AI, and relatively few big actors who are training and running AIs (but many more actors using AI through an API).

There are two common big-picture stories here:

(Less common story) Vision 1 is a vision for the long-term future (

example

).
(More common story) Vision 1 is a safe way to ultimately get to Vision 2 (or somewhere else)—i.e., future people with helper AIs can help solve technical problems related to AI alignment, set up better governance and institutions, or otherwise plan next steps.

1.2 Potential causes for concern

There’s a risk that somebody makes an autonomous (Vision 2 below) ruthlessly-power-seeking AGI, either accidentally or deliberately. We need to either prevent that (presumably through governance), or hope that humans-with-AI-helpers can d

...
AI Disclosure Ballot Initiative (and voting method)
by Aaron Hamlin — 2024-01-17T20:02:02.383Z

I shared that I'm helping to raise funds for a ballot initiative earlier (it also includes approval voting), but some of you may be interested that this initiative has a component requiring disclosure for AI generation in campaign ads. This could be the first time AI usage is required to be disclosed by law. This could also lead the way for other AI protection laws.
If you're interested in seeing this take place, feel free to message me. The campaign still has a shortfall of around $1M at this point. As a bonus, this ballot initiative would also bring approval voting to its first state. Our funding target for this is March of 2024.
Hatching the Cosmic Egg (Hymn to Dionysus)
by rogersbacon — 2024-01-17T18:34:30.251Z
Link: https://www.secretorum.life/p/hatching-the-cosmic-egg-hymn-to-dionysus

Invocation

Dionysus, deep-hearted one, granter of words,
words heard best by the drunken and the mad,
all ecstasies and inspirations are of your lips,
I praise and honor you, I thank you for your blessings.

Dionysus, rapturous one, god of the vine,
who is lord of the far reaches of the mind,
whose hand is ever open, ever within reach,
in the thick of the woods along forgotten paths,
in the shadows of dusk and of dawn you roam the world,
I praise and honor you, I thank you for your blessings.

Dionysus, god who runs in the night,
who sees with eyes shut tight,
who dances to the heart’s strong beat,
ever are you as you are, the changeless changeling,
ever are you in the forest, and the truth is a pathless land,
I praise and honor you, I thank you for your blessings.

(now read it out loud like you were supposed to the first time)

(dishonor Dionysus at your own peril)

Orphism

is the name given to a set of religious beliefs and practices originating in the ancient Greek and Hellenistic world, associated with literature ascribed to the mythical musician and healer Orpheus, who descended into the Greek underworld and returned. This type of journey is called a katabasis and is the basis of several hero worships and journeys. Orphics revered Dionysus (who once descended into the Underworld and returned) and Persephone (who annually descended into the Underworld for a season and then returned). Orphism has been described as a reform of the earlier Dionysian religion, involving a re-interpretation or re-reading of the myth of Dionysus and a re-ordering of Hesiod’s Theogony, based in part on pre-Socratic philosophy.

Roman

Orpheus mosaic

, a very common subject. He wears a

Phrygian cap

and is surrounded by the animals charmed by lyre-playing

The suffering and death of the god Dionysus at the hands of the Titans has been considered the central myth of Orphism. According to this myth, the infant Dionysus is killed, torn apart, and consumed by the Titans. In retribution, Zeus strikes the Titans with a thunderbolt, turning them to ash. From these ashes, humanity is born. In Orphic belief, this myth describes humanity as having a dual nature: body (Ancient Greek: σῶμα, romanized: sôma), inherited from the Titans, and a divine spark or soul (Ancient Greek: ψυχή, romanized: psukhḗ), inherited from Dionysus. In order to achieve salvation from the Titanic, material existence, one

...
What do people colloquially mean by deep breathing? Slow, large, or diaphragmatic?
by VipulNaik — 2024-01-17T18:01:06.721Z

Some version of the idea of "relax, take a deep breath" is common
across cultures. But what does the "deep" in "deep breath" mean to
people? I've come across three plausible interpretations, and I'm
curious what people colloquially mean (I suspect it's a mix or
conflation of the meanings).

Deep as in slow (low number of breaths per minute)

One interpretation of "deep" is "slow", from the idea that "going
deep" into something takes time. So, deep breathing just means slow
breathing. This is particularly relevant to somebody who's doing a lot
of breaths per minute: "relax, take a deep breath" means "slow down."

Deep as in large (high breath volume)

Another interpretation I've seen of "deep" is "large" -- take a lot of
good air in, and expel all the bad air out. Get a fresh start by
cleansing the bad air from inside you.

The Johns Hopkins page on
hyperventilation
uses the term "deep breathing" (in a critical fashion) equating it to
rapid, large breaths.

Deep as in diaphragmatic

Here,
the term "deep" seems to be referring to the location of the trigger
for breathing (the diaphragm is deeper down/inside in the body
compared to the upper chest?).

This is the most scientifically "correct" definition of deep
breathing; see for instance
Wikipedia or
Harvard Health
Publishing.

All three?

See this WebMD
article
that seems to combine all three meanings: slow, large, and
diaphragmatic. A lot of breathing instruction and direction I see in
popular culture does seem to combine all three elements when using the
term "deep breathing".

Are the meanings in conflict?

Generally speaking, diaphragmatic breathing is likely to be slower,
and slower breathing is likely to be diaphragmatic, so those meaning
are not in conflict, though they still mean different things.

However, the relationship between breath rate and breath volume is
very unclear, with any of the 2 X 2 combinations possible.

In particular, it's very much possible to take breaths that are
"shallow" in terms of breath volume, while being slow and
diaphragmatic, and in fact the whole idea of "reduced breathing" (as
seen in things like the Buteyko
method)
is to try to reduce total breathing volume per unit time by doing some
combination of slow breathing (reducing the rate of breathing) and
"shallow" breathing (in the sense of reduced breathing volume). In
this viewpoint, then, the "slow" part is good but the "large" part is...
AlphaGeometry: An Olympiad-level AI system for geometry
by alyssavance — 2024-01-17T17:17:30.913Z
Link: https://deepmind.google/discover/blog/alphageometry-an-olympiad-level-ai-system-for-geometry/

[Published today by DeepMind]

Our AI system surpasses the state-of-the-art approach for geometry problems, advancing AI reasoning in mathematics

Reflecting the Olympic spirit of ancient Greece,

the International Mathematical Olympiad

is a modern-day arena for the world's brightest high-school mathematicians. The competition not only showcases young talent, but has emerged as a testing ground for advanced AI systems in math and reasoning.

In a paper published today in

Nature

, we introduce AlphaGeometry, an AI system that solves complex geometry problems at a level approaching a human Olympiad gold-medalist - a breakthrough in AI performance. In a benchmarking test of 30 Olympiad geometry problems, AlphaGeometry solved 25 within the standard Olympiad time limit. For comparison, the previous state-of-the-art system solved 10 of these geometry problems, and the average human gold medalist solved 25.9 problems.

Links to the paper appear broken, but here is a link:

https://www.nature.com/articles/s41586-023-06747-5

Interesting that the transformer used is tiny. From the paper:

We use the Meliad library for transformer training with its base settings. The transformer has 12 layers, embedding dimension of 1,024, eight heads of attention and an inter-attention dense layer of dimension 4,096 with ReLU activation. Overall, the transformer has 151 million parameters, excluding embedding layers at its input and output heads. Our customized tokenizer is trained with ‘word’ mode using SentencePiece and has a vocabulary size of 757. We limit the maximum context length to 1,024 tokens and use T5-style relative position embedding. Sequence packing is also used because more than 90% of our sequences are under 200 in length.