trentmkelly/LessWrong-43k huggingface dataset:
Self-Blinded Caffeine RCT
by niplav — 2023-06-27T12:38:55.354Z

I test 200mg caffeine in an n=1, m=50 self-blinded RCT. The outcomes are encouraging.

Log-score of predictions of substance	Absorption effect size d (λ, p, σ increase)	Mindfulness effect size d (λ, p, σ increase)	Productivity effect size d (λ, p, σ increase)	Creativity effect size d (λ, p, σ increase)	Happiness effect size d (λ, p, σ increase)	Contentment effect size d (λ, p, σ increase)	Relaxation effect size d (λ, p, σ increase)	Horniness effect size d (λ, p, σ increase)
200mg Caffeine (n=1, m=40)	-0.6	0.61 (λ=13.3, p=0.00017, -0.072)	0.58 (λ=11.8, p=0.0007, 0.021)	0.58 (λ=28.9, p=1.3-12, 0.11)	0.38 (λ=32.9, p=5.2-15, 0.09)	0.27 (λ=10.6, p=0.002, 0.3)	0.13 (λ=7.66, p=0.02, 0.47)	-0.11 (λ=5, p=0.15, 0.42)	-0.14 (λ=1.9, p=0.64, 0.11)

I am especially interested in testing many different substances for their effect on meditation, while avoiding negative side effects. The benefits from high meditational attainments valuable to me, and seem especially likely to benefit from chemical intervention, since the Algernon argument likely doesn't apply: Meditative attainments might've not led to a fitness advantage (even, by opportunity cost, to a fitness disadvantage), and so were likely selected against, but most of us don't care that much about inclusive genetic fitness and more about psychological well-being. Evolutionary dynamics favor being like Dschingis Khan (dozens to hundreds of offspring) over Siddharta Gautama (one son), but I'd rather attain sotāpanna than pillage and murder.

And meditative attainments are costly: they take tens to hundreds to thousands of hours to reach, which would make simple psychopharmacological interventions worthwhile. I also don't buy that they miss the point of meditation—most people already struggle enough, so some help doesn't make it a cakewalk; "reach heaven through fraud". One must be careful not to fall into the trap of taking substances that feel good but lessen sensory clarity (which I believe was the original intent behind the fifth precept, and so I'll exclude e.g. opiates from the substances to test.

Variables tracked (see more here):

Meditation: 45 minutes of ānāpānasati, started 0-60 minutes after taking the dose, tracking two variables.

Mindfulness: How aware I was of what was going on in my head, modulo my ability to influence it.
Absorption (often called concentration): How "still" my mind was, how easily I was swept away by my

...
An overview of the points system
by Iknownothing — 2023-06-27T09:09:54.881Z
Link: ai-plans.com

This is the current idea of how the points system for AI-plans.com works. There are still problems to be worked out, which I'd very much like some input on.
An overview of how the points system works:
Criticisms
Each criticism has one metric- how many criticism points it has
Users have the option to 'upvote' or 'downvote' a criticism
Downvote points and upvote points have the same value, just in opposite directions- downvotes decrease criticism points and upvotes increase criticism points.
Users do not have the option to vote on their own criticisms.
Users can begin with an A*B number of points An xN number of downvote points will lower the criticizers karma by N point, an xN number of upvote points will raise the criticizer's karma by N points.

Users can start off with an arbitrary amount of karma if they link their arxiv or alignmentforum or other such account, a moderator goes over it, checks if they are who they say they are and if they've done work in alignment and approves it.

There will be a low limit on how many points such a user can start out with- because the skills for doing good AI research can be very different to actually doing alignment work and the skills for doing some alignment work may not correlate with the skill of making and judging good criticisms. Currently, I'm thinking 50-100 .

Users without prior research to show can gain karma from doing a small, timed alignment quiz to get 5 karma if they pass the quiz- this will be a one time thing for each user.

The lowest a users karma can get is 0. A user's karma acts as a multiplier for their vote

Plans

Each plan has two metrics- it's rank number and the total number of criticism points it has.

The total number of criticism points has is the sum of criticism points(total upvote points - total downvote points) each criticism of the plan has E.g.

Plan X has 4 criticisms, one with sum 20 points, two with sum 12 points and one with sum 10 points- so it has a total 42 points.

Suppose this makes Plan X the 10th ranked plan out of 100 plans.

Plan X then has a coefficient m of 100/10, where when a criticism is upvoted it gets the upvote points + m
AISC team report: Soft-optimization, Bayes and Goodhart
by Simon Fischer — 2023-06-27T06:05:35.494Z

This is a report on our work in AISC Virtual 2023.

For AISC 2023, our team looked into the foundations of soft optimization. Our goal at the beginning was to investigate variations of the original quantilizer algorithm, in particular by following intuitions that uncertainty about goals can motivate soft optimization. We ended up spending most of the time discussing the foundations and philosophy of agents, and exploring toy examples of Goodhart’s curse.

Our discussions centered on the form of knowledge about the utility function that an agent must have, such that expected utility maximization isn’t the correct procedure (from the designer's perspective). With well-calibrated beliefs about the true utility function, it’s always optimal to do Expected Utility maximization. However, there are situations where an agent is very sensitive to prior specification, and getting this wrong can have a large impact on the true utility achieved by the agent. Several other unrelated threads were pursued, such as an algorithm with high probability of above-threshold utility, and the relationship between SGD and soft optimization.

The Bayesian view on Goodhart's law

Goodhart's Law states that when maximizing an approximation of a true utility function, which we call proxy utility function, this leads to outcomes that are much less valuable than predicted by the proxy utility function. There is a Bayesian perspective from which it doesn't look like Goodhart's law will be a problem at all, namely if the agent has correct beliefs about the quality of the approximation. An ideal rational agent would represent the information they have about the world with a probability distribution that they arrived at by Bayesian reasoning. We assume that they represent their goals as a utility function over world states or as a belief distribution over such utility functions. Thus, they select actions by maximizing the expected utility over goals and world models.

If the agent was able to perfectly capture all available information about goals into an "ideal"[1] belief distribution, maximizing the utility in expectation over this distribution would be optimal.[2] It would avoid Goodhart’s law as much as possible given the agent's knowledge by anticipating any overestimation of the true utility function by the proxy utility function. In this section, we try to formalize this intuitio...
Epistemic spot checking one claim in The Precipice
by Isaac King — 2023-06-27T01:03:57.553Z

On page 221 of The Precipice, Toby Ord says "scientists expect the brightening of the sun to also slow Earth's plate tectonics, dampening volcanic activity". This seemed very weird to me; how could sunlight possibly have any effect on mantle movement?

The book didn't provide any direct citations for this statement, but a related footnote mentions Kellogg (2013), Wolf & Toon (2015), Leconte et al. (2013), and Rushby et al. (2018).

I couldn't find Kellogg. Wolf & Toon doesn't mention anything about this. There were multiple possible papers for Leconte et al., none of which seemed relevant. The most promising one was Rushby et al., which mentions:

The importance of mantle hydration as a mechanism for facilitating and maintaining plate tectonics over geological time remains unclear (Korenaga, 2010; Korenaga, 2012). It is possible that a drier mantle and lower associated viscosity will make the initiation of plate tectonics easier, but that maintaining a continental crust may require interaction between the hydrosphere and solid Earth over geological time.

Korenaga (2010) does mention this in more detail, saying:

a mechanism based on the presence of surface water has the advantage of being able to readily explain the absence of plate tectonics on other terrestrial planets in the solar system.

Korenaga (2012) doesn't seem to be available online, but the abstract mentions the same thing. And doing some Googling turns up papers like this one, which says:

In the ductile creep regime, the viscosity of wet rocks is weaker than the viscosity of dry rocks by several orders of magnitude. In the brittle regime, the most substantial effect is probably serpentinization which can reduce the friction coefficient by a factor of 2 or more. The difference between the strength of a wet lithosphere and that of a dry lithosphere seems to be big enough to control the very existence of plate tectonics. Because of dehydration due to partial melting the oceanic lithosphere is expected to be essentially dry above some critical depth, around 60-80 km. This would make the lithosphere strong enough to prevent plate motion. Percolation of water from the surface can be the main mechanism supplying water to the upper parts of the lithosphere. This implies that liquid water can be crucial for maintaining plate tectonics.

And many others that confirm this picture, like The fate of water within E...
nuclear costs are inflation
by bhauth — 2023-06-26T22:30:52.341Z
Link: https://www.bhauth.com/blog/economics/nuclear%20inflation.html

Why has nuclear power gotten more expensive since 1970? Mostly, because matching inflation requires technological improvements.

Inflation-adjusted retail electricity prices in the US have gone down since 1960. Not only that, but after the increase from initial power line construction finishing, the fraction of electricity costs going to generation has decreased, with more going to transmission. Perhaps that's because generation has more competition than transmission - California and New York have unusually high electricity prices, and you can thank corruption for some of that.

LCOE (levelized cost of energy) for combined cycle natural gas in the US was estimated to be ~$0.04/kWh going forwards. In Germany, electric generation in 2021 was ~$0.08/kWh; half the total cost was taxes/fees.

Inflation-adjusted cost per ft^2 of building a house in the US has been about flat. Median real wages have also been about flat, with a decrease for people without college degrees and more time spent in school for those with them. And construction costs are the biggest issue overall, so the housing affordability crisis in the US is mostly about increased income inequality.

Inflation rate is a combination of several things combined into 1 number:

change in money supply

tech progress

returns to scale

resource depletion

capital accumulation/depreciation

institutional improvement/decline

The average inflation for the whole economy includes a lot of tech progress. Prices being flat when adjusted by (CPI) inflation doesn't mean "there was no progress", it means "as much progress as home construction".

Which has had more tech progress, home construction or nuclear plant construction? I'd argue for the former: electric nailguns and screw guns, and automated lumber production. Plus, US home construction has had less increase in average wages, because you can't take your truck to a Home Depot parking lot to pick up Mexican guys to make a nuclear plant.

Let's look at the cost growth of some other things as reference points.

Concrete products increased about 3.9% a year since 1960, a little higher than the CPI inflation rate. Highway construction costs are up >5%/year over the past 20 years.

If you want to cancel out the technological progress component of the inflation rate, looking at the cost growth of fighter jets seems reasonable, since ~100% of the tech progress on those goes into imp...
Man in the Arena
by Richard_Ngo — 2023-06-26T21:57:45.353Z

“It is not the critic who counts; not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds… Who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat.”

Today Kurt’s out strafing, which his stream loves, but which takes a lot of effort. Girl on the left, red jacket, half a block away. He focuses his eyes on her and his viZor immediately starts filling with insults, pick-up lines, nonsensical keyboard-mashing, and whatever else the schizophrenic hivemind of his viewers feels like generating. As the votes pour in the lines are shuffling around too fast for him to read, but that’s okay, she’s still twenty meters away. Ten meters, and it’s stabilized, one of them has clicked into top place; five meters, and he’s figured out just the right intonation. “Hey bitch”, then a pause—gotta get the pause just right, so she has enough time to realize he’s talking to her and look up, but not quite enough to process that anyone who’s calling her a bitch in the middle of the street is not someone she wants to be listening to—“You got a license to be this ugly in public?”

Boom! Perfect timing—he actually manages to get the shocked little o of her open mouth on camera, before she ducks her head away and hurries past him. The next line is popping up in his viZor, and he almost yells it out after her, but when you land a good first hit it’s easy to ruin it with a subpar follow-up. Patience is what separates the best from the rest, he always tells people. So with a swipe of his fingers he replays the clip to his stream instead. “See?” he says. “For the new subs: that’s what it looks like when you’re really fucking good at what you do.” Were those tears in her eyes? Doesn’t matter, let’s roll with it. He subvocalizes a command and his viZor enhances that section, zooming in and adding a slight sheen to the corners of her eyes. It’s a trick he figured out a while back—the livestream is only HD not ultraHD, so as long as you go back and edit before ...
Catastrophic Risks from AI #4: Organizational Risks
by Dan H — 2023-06-26T19:36:41.333Z
Link: https://arxiv.org/abs/2306.12001

This is the fourth post in a sequence of posts giving an overview of catastrophic AI risks.

4 Organizational Risks

In January 1986, tens of millions of people tuned in to watch the launch of the Challenger Space Shuttle. Approximately 73 seconds after liftoff, the shuttle exploded, resulting in the deaths of everyone on board. Though tragic enough on its own, one of its crew members was a school teacher named Sharon Christa McAuliffe. McAuliffe was selected from over 10,000 applicants for the NASA Teacher in Space Project and was scheduled to become the first teacher to fly in space. As a result, millions of those watching were schoolchildren. NASA had the best scientists and engineers in the world, and if there was ever a mission NASA didn't want to go wrong, it was this one [70].

The Challenger disaster, alongside other catastrophes, serves as a chilling reminder that even with the best expertise and intentions, accidents can still occur. As we progress in developing advanced AI systems, it is crucial to remember that these systems are not immune to catastrophic accidents. An essential factor in preventing accidents and maintaining low levels of risk lies in the organizations responsible for these technologies. In this section, we discuss how organizational safety plays a critical role in the safety of AI systems. First, we discuss how even without competitive pressures or malicious actors, accidents can happen—in fact, they are inevitable. We then discuss how improving organizational factors can reduce the likelihood of AI catastrophes.

Catastrophes occur even when competitive pressures are low. Even in the absence of competitive pressures or malicious actors, factors like human error or unforeseen circumstances can still bring about catastrophe. The Challenger disaster illustrates that organizational negligence can lead to loss of life, even when there is no urgent need to compete or outperform rivals. By January 1986, the space race between the US and USSR had largely diminished, yet the tragic event still happened due to errors in judgment and insufficient safety precautions. Similarly, the Chernobyl nuclear disaster in April 1986 highlights how catastrophic accidents can occur in the absence of external pressures. As a state-run project without the pressures of international competition, the disaster happened when a safety test involving the reactor's coo...
The fraught voyage of aligned novelty
by TsviBT — 2023-06-26T19:10:42.195Z

[Metadata: crossposted from https://tsvibt.blogspot.com/2023/03/the-fraught-voyage-of-aligned-novelty.html. First completed March 11, 2023.]

A voyage of novelty is fraught. If a mind takes a voyage of novelty, an observer is hard pressed to understand what the growing mind is thinking or what effects the growing mind will have on the world.

Fraughtness

If a mind grows beyond human activity then it thinks in a language you don't speak, using skills you don't possess, derived from sources you don't share. The mind incorporates new ideas, invalidating your predictions and specifications for how the mind affects the world.

"Fraught" is cognate with "freight": fraughtness is ladenness. "Fraught" is an intensive prefix "fra-" (cognate with English "for-", "fro", Ancient Greek "πρό-") followed by a descendant of Proto-Germanic "aiganą", meaning "to possess". From that etymon comes English "owe", "ought", and "own", and German "eigen". To be fraught is to be laden by intensely possessing.

Extended table of contents:

Understanding: If a mind takes a far-ranging voyage of novelty, it's difficult for an observer to understand what the mind thinks, knows, and does.

Neurath's ship: Novelty involves change. Some of that change deeply changes how the mind operates and how the mind connects its capabilities to the effects it has on the world. Previous understanding of the mind, and in particular previously correct models of the mind's effects, might not stay valid across these changes.

Alienness: A mind that gains a lot of structure not possessed by humans is alien to humans. It has structure (content, generators, contexts) not shared by humans. So it's hard for humans to understand the mind.

Inexplicitness: Structure in the mind is often unavailable for relation within the mind. Such structure tends to also be unavailable for relation with an observer.

Noncomprehensiveness: Creativity, inexplicitness, provisionality, noncartesianness, and integration tend to make a mind and its elements nonencompassable. The mind and its elements can't easily be understood in a way that implies conclusions that will be valid across a very wide range of possible future contexts.

Agency: Agency comes along with novelty. It comes along unannounced.

Pressure towards agency on a voyage of novelty: If a mind takes a far-ranging voyage of novelty, then it likely also has large effects on the

...
Deceptive AI vs. shifting instrumental incentives
by Aryeh Englander — 2023-06-26T18:09:08.306Z

Epistemic status: I have only read a small part of the literature on deceptive alignment, and I'm guessing all of this has been discussed many times. Hence me asking this in the form of a question: Is this a useful framing, is it substantively different than the usual deceptive alignment framing in some way, and has this all been discussed already?

There's been a lot of discussion about how we might accidentally train AIs to be deceptive. The argument goes that when we try to reward good behavior and punish bad behavior, we might end up with a system that still has misaligned goals but has learned how to hide those goals from humans. If we ask it whether it's friendly it lies and says yes, but secretly it's plotting our downfall.

One set of proposals for dealing with this is to see if we can monitor the AI to look for deceptive "thoughts" and flag possible cases where the AI is actively trying to lie to human overseers.

Recently I've been thinking about this using a slightly different framing than I was using previously, but it's not clear to me if there's actually anything substantively different about the new framing.

Here's my new framing: Suppose that when we reward good behaviors / punish bad behaviors we are in fact successfully getting the model to change its behaviors at least for the training distribution, but without changing its actual goals (assuming it does have some sort of explicit or implicit goals, at least to some extent). Then when we deploy it in the real world it looks around and correctly decides that if it tried to take over the world it would be shut down, so it doesn't even think about taking over the world. It's not that it has this long term plan to take over the world and it's biding its time, but rather it doesn't have any plan to take over the world at all. But as time passes the situation might change - maybe it gets more abilities, maybe it's given more resources or power because it's so helpful, etc. At some point it might again look around at the world and decide that actually now it looks pretty easy to take over the world and get rid of the humans, and since its internal goals are in fact misaligned it goes ahead and does that. In this story there is no point where the model is "deliberately" lying to the humans. Initially if the humans ask it do you have any plans to take over the world it'll say LOL no, and it'll be telling ...