trentmkelly/LessWrong-43k huggingface dataset:
How to Become a World Historical Figure (Péladan's Dream)
by rogersbacon — 2022-07-07T22:39:58.122Z
Link: https://www.secretorum.life/p/how-to-become-a-world-historical

I. Introduction

Erik Hoel

writes the following in reference to Roger’s Bacon’s

Predictions for 2050: Black Swan Edition

post (apologies if you’ve read this before, feel free to skip to the next section).

“Finally, there’s

Secretum Secretorum

, which eschews the idea of extrapolating from current trends and instead takes some big contrarian positions. Of particular interest is the idea of a “World Historic Individual” emerging. Specifically that

. . . by 2050 there will a living person who is widely recognized to be what early 1900s German historian Oswald Spengler called a “World Historical Figure” (

The Decline of The West

). Jesus, Socrates, Alexander the Great, Buddha, Genghis Khan, Muhammed, Newton, Darwin are all on the list.

It’s always struck me that my generation, the millennials, have grown up with a particular lack of World Historic individuals (in terms of impact in the long march of history, not popularity or name recognition in the moment). We missed concurrency with most of the 20th-century luminaries. My parents’ early lives overlapped with Einstein, for instance. Indeed, it’s worth asking:

Who is the most recent person that could reasonably be called a world historical figures? I say yes for Gandhi and Martin Luther King, Jr. . . . After that. . . I’m not so sure. Off the top of my head, here’s a list of potential candidates: Mao Zedong, Osama Bin Laden, Obama, Trump, Elon Musk (sorry Bezos, you didn’t make the cut), and Xi Jinping. I think most of these people are debatable when you start to consider truly vast time horizons: what are the chances people will know about Obama, Xi, or Elon Musk 500 years from now?

Of those listed by Secretum Secretorum, the only individual I can imagine mattering in five hundred years is Elon Musk, although not for anything he’s done yet. But if he did establish a

city on Mars

, as there’s indeed

a chance he might

, getting humanity off-planet would certainly be remembered. Yet, it’s also possible the responsibility for the actual settlement of Mars will be spread out over the entire space industry, not to mention over various governments, ensuring no lasting historical credit goes to Musk.

So where are the Einsteins? Or Joans of Arc? The Ghandis? Let alone the Leibnizs or Napoleons or Christs? Perhaps there is someone unknown to us, some little girl waiting in the wings who will change everything. A comforting thou...
Safety considerations for online generative modeling
by Sam Marks — 2022-07-07T18:31:19.316Z

Summary: the

online decision transformer

is a recent approach to creating agents in which a decision transformer is pre-trained offline (as usual) before producing its own trajectories which are fed back into the model in an online finetuning phase. I argue that agents made with generative modeling have safety advantages – but capabilities disadvantages – over agents made with other RL approaches, and agents made with online generative modeling (like the online decision transformer) may maintain these safety advantages while being closer to parity in capabilities. I propose experiments to test all this. (There is also an appendix discussing the connections between some of these ideas and KL-regularized RL.)

I’ll start with some motivation and then introduce the decision transformer and the online decision transformer. If you’re already familiar with the decision transformer, you can probably skip to “Online generative modeling.” If you’re already familiar with the online decision transformer, you can skip to “Some remarks.”

Motivation

Suppose we want to get an AI system to produce a picture of a cat.

A naive approach is to take a trained image classifier M and optimize an image to maximally activate M's 'cat' classification. If you do this, you won't get anything that looks like a cat. Instead, you'll get some deep-dream-esque conglomeration of whiskers, cat ears, and fur which M happens to strongly classify as 'cat.' One way of describing what went wrong here is that the image of the cat you get is very off-distribution. You would rather get a normal-looking picture of a cat, i.e. one which is on-distribution for M's training data.

This can be taken as an analogy for alignment: if you have an agent optimize the world for a given utility function U, what you get won't look anything like a normal world, but rather a very weird-looking world which happens to maximize U. We would prefer to get a normal-looking world which also scores high according to U.

We've recently seen

lots

of

reminders

that modern ML does have ways to produce normal-looking pictures of cats. A key ingredient in all of these approaches is generative modeling. Recall that a generative model is something which is fed in data and learns to produce new data which "looks similar" to its training data; in other words, a generative model tries to produ...
Human values & biases are inaccessible to the genome
by TurnTrout — 2022-07-07T17:29:56.190Z

Related to Steve Byrnes’

Social instincts are tricky because of the “symbol grounding problem.”

I wouldn’t have had this insight without several great discussions with Quintin Pope.

TL;DR: It seems hard to scan a trained neural network and locate the AI’s learned “tree” abstraction. For very similar reasons, it seems intractable for the genome to scan a human brain and back out the “death” abstraction, which probably will not form at a predictable neural address. Therefore, I infer that the genome can’t directly make us afraid of death by e.g. specifying circuitry which detects when we think about death and then makes us afraid. In turn, this implies that there are a lot of values and biases which the genome cannot hardcode.

In order to understand the human alignment situation confronted by the human genome, consider the AI alignment situation confronted by human civilization. For example, we may want to train a smart AI which learns a sophisticated world model, and then motivate that AI according to its learned world model. Suppose we want to build an AI which intrinsically values trees. Perhaps we can just provide a utility function that queries the learned world model and counts how many trees the AI believes there are.

Suppose that the AI

will learn a reasonably human-like concept for “tree.”

However, before training has begun, the learned world model is inaccessible to us. Perhaps the learned world model will be buried deep within a recurrent policy network, and buried within the world model is the “trees” concept. But we have no idea what learned circuits will encode that concept, or how the information will be encoded. We probably can’t, in advance of training the AI, write an algorithm which will examine the policy network’s hidden state and reliably back out how many trees the AI thinks there are. The AI’s learned concept for “tree” is

inaccessible information

from our perspective.

Likewise,

the human world model is inaccessible to the human genome

, because the world model is probably in the cortex and the cortex is probably

randomly initialized

.[1] Learned human concepts are therefore inaccessible to the genome, in the same way that the “tree” concept is a priori inaccessible to us. Even the

broad area where language processing occurs

varies from person to person

...
Cooperation with and between AGI\'s
by PeterMcCluskey — 2022-07-07T16:45:23.605Z
Link: http://www.bayesianinvestor.com/blog/index.php/2022/07/06/cooperation-with-and-between-agis/

[This post is primarily aimed at a Foresight Institute audience. I
don't know whether the average LW reader will learn much from it.]

This post is mostly a response to the Foresight Institute's book
Gaming the
Future, which is
very optimistic about AI's being cooperative. They expect that creating
a variety of different AI's will enable us to replicate the checks and
balances that the US constitution created.

I'm also responding in part to Eliezer's AGI
lethalities,
points 34 and 35, which say that we can't survive the creation of
powerful AGI's simply by ensuring the existence of many co-equal AGI's
with different goals. One of his concerns is that those AGI's will
cooperate with each other enough to function as a unitary AGI.
Interactions between AGI's might fit the ideal of voluntary cooperation
with checks and balances, yet when interacting with humans those AGI's
might function as an unchecked government that has little need for
humans.

I expect reality to be somewhere in between those two extremes. I can't
tell which of those views is closer to reality. This is a fairly scary
uncertainty.

AI's will have better tools for cooperation. That seems likely to mean
that AI's will cooperate more with each other than they cooperate with
humans.

The effectiveness of the US constitution depends on politicians having
limited ability to collude with each other.

I'm not claiming that differing abilities to cooperate will be as
important as the cognitive differences between AI's and humans. I'm
focusing here on cooperation because Foresight focuses on it, and many
other discussions of AI neglect it.

[I'll try to use the term AGI when talking about Eliezer's model, and
AI when talking about Foresight's model. But please don't rely too
much on that fuzzy distinction. ]

Analogies

Let's look at some examples of cooperation between unequal groups with
differing abilities to cooperate.

1.

Cats have significant skill at training human servants to feed and pet
them, and often to provide medical care. That's a fair amount of
success at aligning humans with feline values - much more than I would
have expected if I'd tried to make a prediction 1000 years ago.

But humans cooperate to spay cats. They talk to each other in ways that
create social pressure to spay cats. Veterinarians coordinate in ways
that make it easier to spay cats. Without developing human-level
cooperation skills, how...
Aversion Factoring
by CFAR!Duncan — 2022-07-07T16:09:11.392Z

Epistemic status: Anecdotally strong

The aversion factoring technique is not derived from any particular body of psych research, though it draws lightly on trigger-affect patterns and exposure therapy and takes advantage of the framework of reductionism. It was primarily developed for personal use by Andrew Critch, and then later shared and refined through iteration.

It’s quite common for people to talk about their aversions:

“Oh, I just hate going to the gym.”

“I never talk on the phone if I can help it.”

“They require a six-month commitment, and I really didn’t want to be locked into anything for that long.”

“I don’t know what it is. Something about his. . . attitude, I guess? The way he talks to people? Look, I’d just really rather you not invite him.”

Aversions lead to avoidance—they’re any sort of mental mechanism that causes us to be less likely to engage in a particular activity, or to do so only with pain, displeasure, or discomfort. Aversions can be conscious or unconscious, reasoned or felt, verbal or visceral, and they can range anywhere from a slight tinge of antipathy to outright phobias.

The purpose of the Aversion Factoring technique is to give you the tools you’ll need to identify and overcome aversions. Of course, not every aversion should be overcome—it would probably be counterproductive to lose your aversion to being hit by cars, for example—but there are many activities we might engage in and enjoy if we could just get past their one sticking point, be it self-consciousness or tedium or the aftertaste of mushrooms.

Reductionism: the LEGO principle

In the Goal Factoring section of this sequence, we attempted to draw out the positive elements of various plans, and weave them together into a new strategy that had fewer drawbacks (or none). In Aversion Factoring, we’ll take a slightly different approach, holding the larger context constant while we address specific aspects of it one at a time.

The key insight is that no activity simply is aversive, at its core. There’s no fundamental quantity called “running” that “just sucks,” for instance—running is a complex system of experiences, and summing or averaging across all of them causes us to lose valuable detail. Treating each experience separately, we may find that we’re more or less okay with all of them, and that the ones that are the most negative can be addressed individually.

Wearing running s

...
Genders Discrimination
by Jacob Falkovich — 2022-07-07T15:20:31.976Z

Whales are queering the fish-mammal binary

Cross-posted from Putanumonit

Categories in your head

In physical reality, light with wavelength between 380 and 700 nm passes through your eyeball and activates cells in your retina. In your mind, you perceive colors. This makes it tempting to think that your perception of color is simple a wavelength-detector, but that is not the case.

Sometimes the

same exact light waves

produce differently-perceived colors. Sometimes you perceive

a color whose wavelength is missing

. Sometimes you perceive a

color that has no associated wavelength at all

. And sometimes you eat a mushroom and perceive all the colors at once with your eyes closed. The wavelengths are out there, but the color is only in your mind a is a feature of your mind, not of reality.

This is true of

all

your perceptions

, including things like your emotions, sense of self, and the very reality of objects.

An important feature of perception is categorization — you learn concepts for grouping similar perceptions together. Your categories then affect your immediate perception, downplaying differences within categories and amplifying differences between them. Modern Westerners see the rainbow as made up of 7 distinct colors — up from

3-4 in medieval times

and 5 in the 18th century. The rainbow itself, of course, is a continuous spectrum that could be broken up into any number of bands.

A general principle of developing perception is: more discrimination is better. You can better make and enjoy visual art better if you can tell pumpkin orange from safety orange from tiger orange, as opposed to seeing them as the same rainbow stripe. The movie Inside Out taught you that there are only 5 emotions, but you can actually

increase the range and granularity

of your perception of emotion to understand others and manage your own feelings more skillfully. The more attention you pay to a domain, the more fruitful distinctions in perception you can make.

Queer whales

My friend points out

: just because out there are two clusters of biological sex defined by gamete size, doesn’t mean that people’s perception of gender is identical to those. What my friend calls people’s “gendering faculty” is similar to their perception and discrimination of colors, and has a lot to do with the particularities of the society they inhabit — for example whether men in that society grow their hair long or w...
Consider Multiclassing
by JustisMills — 2022-07-07T14:54:16.797Z

One approach to work is to specialize. The programmer who does lots of LeetCode in their spare time, keeps a portfolio up to date with side projects, and regularly posts answers on StackOverflow. The researcher who thinks about the same subject for the vast majority of their waking hours every day, only occasionally stopping to drink coffee or take a quick stroll. The medical doctor who has to spend a decade in training before their career even starts.

There are many benefits to specialization. The main one is that you aren't bottlenecked, relative to other people, by time. In terms of practice, you're ahead of the curve. And there are often compounding returns to skill and expertise.

But I think another approach is to deliberately cultivate multiple unrelated skills simultaneously. This has been my own approach so far. I recommend it.

Setting yourself up to multiclass

Basically, figure out things you enjoy doing, and find a way to do them a lot. For me, one example of this was starting a flash fiction literary review, and offering feedback to every single submission I got with a fast turnaround. It was challenging! But it was exciting. Why did I choose to do this?

I liked writing very short stories
My mom had been a professional editor, so she could get some good writers she knew to submit stories to get my backlog going
I wanted practice editing

It didn't feel like a career move at the time. But many years later, I was ready to take on professional editing opportunities. Just getting really skilled at something can have a way of providing value eventually. Plus it feels good.

Another example: in college I studied philosophy, but I also liked math a lot. There was a cellular automata problem - open at least in that nobody else had ever bothered attempting it - that I wanted to solve. I needed programming knowledge to do this. So I learned some Python. I think this experience helped me get my first engineering job - at the very least, the way my eyes lit up describing how I'd attacked the problem made me seem like a promising person to hire on spec. I remain a software engineer today.

The formula for me has basically just been, when I get a spike of energy and there's some project that seems like it'd be fun, it's almost always turned out to be worth doing in the long run. The key is not to worry too much about if it'll be valuable. Practice tends to be its own reward...
Covid 7/7/22: Paxlovid at the Pharmacy
by Zvi — 2022-07-07T14:30:00.432Z

Pharmacists are going to be allowed to prescribe Paxlovid, while doctors continue to be confused by the guideline of ‘you can give this life-saving medicine that cures disease to the people who are sick with that disease.’ The news on the vaccine update front is good. Cases and deaths went unexpectedly higher, which is slightly worrisome, but seems unlikely to indicate anything big.

On an unrelated note, I’ve also written a bit about a giving-people-money experiment in Chicago that did not go so well. With Covid quieting down, I’m likely going to be lowering the threshold for including non-Covid items in the weekly updates. The alternative was to have a growing backlog of things that I am unlikely to ever get to deal with.

Executive Summary

Get Paxlovid at your pharmacist, yay.
Vaccine update update is they are closer to updating.
Monkeypox continues to double every 10.5 days or so.

Let’s run the numbers.

The Numbers

Predictions

Prediction from last week (4th of July): 600k cases (-13%) and 1,800 deaths (-16%).

Results: 675k cases (-1%) and 2,272 deaths (+11%!).

Prediction for next week: 700k cases (+5%) and 2,250 deaths (-1%).

Cases and deaths both going unexpectedly higher implies there wasn’t much of a holiday effect. Deaths in particular have no reason to be higher even without a holiday effect, so that’s weird but I did adjust for Florida and the effect is still there in the South. My guess is some of this is random rather than permanent, but it’s hard to say, and I still have to presume that there was some amount of holiday effect to undo.

Deaths

Cases

Did you know this in many places represents medium or high levels of Covid?

All right then.

Paxlovid at the Pharmacist

You will be able to get Paxlovid prescribed to you directly by the pharmacist. Seems like an excellent development. I’m actually starting to notice a pattern of such pockets of sanity happening more often recently, after delaying long enough to let a lot of the value be destroyed, in quiet places where no one is looking. It’s as if they are worried someone will notice someone is optimizing for good outcomes in this hive of scum and villainy, and so they must be cautious.

You can also get Paxlovid at the Pop-Up here in NYC.

Meanwhile, doctors clamor for more clarity. Yes, we are allowed to prescribe life-savin...
Babysitting as Parenting Trial?
by jefftk — 2022-07-07T13:20:04.129Z

Julia recently posted an

interview
with Kat Woods on how Kat has thought through decisions around
having kids. In her 20s she started strongly wanting to:

I didn't want kids when I was younger, and then when I was about 20 I
was in the park, minding my own business and a little toddler toddled
up to me and he gave me this bent daisy. And my brain exploded! I was
like, "Oh my God, I need babies now."

I just became absolutely obsessed. EA was my number one priority, and
the next one was kids. I actually had a blog about parenting. I was
reading all these books about it, reading studies and figuring out
exactly how to deal with pregnancy and everything. I'd sit out outside
of playgrounds and do my homework next to the playground so I could
watch the kids play, just be around them.

Then she spent some time trying to understand her motivations better,
and decided to try babysitting as a way to get practical experience
with more aspects of taking care of kids:

Oh man, I get to spend time with the kids and develop this
relationship with them. But also I get paid for it. Amazing!

I babysat three separate sets of kids, and they were all perfectly
ordinary kids. They were not kids with behavior problems or anything
like that. But that was enough for me to realize: oh, I do not want to
be a parent. I do not want children at all.

Her advice to others based on this experience is to try babysitting
before having kids:

I think the lesson from this that I took away is that before you have
kids, it's really good to get some babysitting experience. It's a
really good way to cheaply test being as close to being a parent as
you can be without actually being a parent, that's pretty available to
most people. ... I just want people to be informed, and do some quick,
easy tests before they commit their lives to new beings.

In many ways this makes a lot of sense: the decision to have kids is a
serious one with lifelong effects, and you want to make it with lots
of information about yourself and how it's likely to go. Your
responsibilities in the moment as a babysitter pretty similar to as a
parent: taking care of the kid, and dealing with whatever comes up.
And yet, I like parenting a lot and didn't enjoy babysitting. If I had
used this method to decide whether to become a parent, I think I would
have decided against it, and that would...