trentmkelly/LessWrong-43k huggingface dataset:
On Virtue 
by Midnight_Analyst — 2022-01-13T16:36:56.306Z

This post is comprised partly of my own ideas, and partly of the ideas of Scott Alexander. Part of what I am doing here is just explaining my interpretation of his post. It is my first post, and it’s essentially just an experiment. I’d be highly appreciative of feedback and constructive criticism. I’m not 100% sure I’ve completely expressed what I want to, but I think I’ve at least made a reasonable start. In the spirit of writing quickly, I decided not to my doubts hold me back. My point might seem quite obvious/self-evident, but I do feel emotionally like it’s a point I need to make.

A lot of us have people we look up to, and some people feel inadequate because they can’t replicate the amazing feats that their idols can. Perhaps this post can go some way towards helping people to feel better about themselves relative to those they admire, and to feel more self-worth in general.

Virtue almost certainly doesn’t actually exist. But if we imagine for a moment that we’re living in a world where it does, it is almost certainly directly proportional to effort, and nothing else. Any other model of virtue seems, to me, somewhat incoherent.

The first part of Scott Alexander’s ‘I Can Tolerate Anything Except the Outgroup’ really struck a chord with me, and it illustrates quite well the point I want to make. His point is essentially encapsulated in the following extract:

“There are a lot of people who say “I forgive you” when they mean “No harm done”, and a lot of people who say “That was unforgivable” when they mean “That was genuinely really bad”. Whether or not forgiveness is right is a complicated topic I do not want to get in here. But since forgiveness is generally considered a virtue, and one that many want credit for having, I think it’s fair to say you only earn the right to call yourself ‘forgiving’ if you forgive things that genuinely hurt you.

To borrow Chesterton’s example, if you think divorce is a-ok, then you don’t get to “forgive” people their divorces, you merely ignore them. Someone who thinks divorce is abhorrent can “forgive” divorce. You can forgive theft, or murder, or tax evasion, or something you find abhorrent.

I mean, from a utilitarian point of view, you are still doing the correct action of not giving people grief because they’re a divorcee. You can have all the Utility Points you want. All I’m saying is that if you “forgive” s...
Plan B in AI Safety approach
by avturchin — 2022-01-13T12:03:40.223Z

Central claim: Ambitious alignment of superintelligent AI will not be solved in time, therefore different variants of plan B should be explored and implemented.

Below is a short overview and links to my works on AI safety. The central theme of all them is “plan B”. Thus it could be called the “plan B agenda”.

Ambitious alignment of arbitrary superintelligent AI can’t be solved before AGI creation: a) proof is too complex and unreliable b) it needs at least superintelligent AI safety team to be created с) Dangerous AI may appear soon – 2030. Many different types of AI-related x-risks are possible, not only paperclipper.

The best available solution is limiting the power of AI: artificial stupidity, a limited number of use cases, internal surveillance, multilevel boxing.

Safer AI architecture is needed: data-driven AI and human-like functional architecture for interpretability. Human-like AI could be modelled around a real-world person: it is not an upload, but very similar to side-loading. E.g. trained on someone’s texts.

Not only local but global solution of AI safety is needed: AI-nanny based on global surveillance and empowered by agreement between nuclear powers. Human-like AI or a first upload could be a “virtual tzar" of this.

The nature of human values depends on who is “moral subject": society, brain, personality or subpersonality. Human values are not stable preferences. “Real choices” are more important than values, as “values” are not always useful philosophical construction. Also, meta-values are more important: life extension, personal universes, – as they give anyone freedom to have their own values.

An AI that is learning human values is dangerous as it doesn’t know that values yet.

Unfriendly AI will not be very much interested to kill humans for atoms, as atoms have very small instrumental value, and living humans have larger instrumental value on all stages of AI’s evolution. But we have to help to understand this for future AI, especially for "Young AI”. We could empower our position by making some precomitments.

My works on AI Safety:

AI safety defense levels, a roadmap

Message to Any Future AI: “There are several instrumental reasons why exterminating humanity is not in your interest”

Catching Treacherous Turn: A Model of the Multilevel AI Boxing

Catastrophically Dangerous AI is Possible Befor...
Understanding as hierarchical model building
by invertedpassion — 2022-01-13T09:28:17.477Z

Crossposted from a note in my knowledge garden.

As mentioned in a note that I wrote earlier on the philosophy of science, reality is nothing but a set of predictive models we've deemed useful for predictions.

I like to imagine true reality as some kind of a fuzzy blob that doesn't have clear boundaries and access to which is forever out of our reach. The neat categories of reality (such as atoms, molecules, people) and so on exist in our mind as useful models to help navigate in that fuzzy blob of true reality. To put it a bit harshly, the reality that we perceive is an evolved illusion created by the mind to help us survive.

Since true reality is forever out of our reach and yet we have the illusion of understanding the world, the question of what does understanding something even mean becomes important.

The genesis of "aha"

The "aha" moment that comes with understanding something is a bit of a mystery. When we find ourselves dealing with an unfamiliar situation, there's a fog of confusion. Then after putting in some effort, the fog lifts and we get a nice, warm feeling of understanding.

I think this positive feeling of "aha" is a similar motivator as the feeling of happiness upon eating a cookie or the feeling of orgasm after having sex. These positive feelings were evolved to motivate us to do something that's beneficial for our survival and/or reproduction.

In case of understanding, what's clearly beneficial to us is to be able to predict the dynamics of the unfamiliar situation or object so that we can exert control over it for our own benefit.

So, that is what understanding boils down to ultimately:

Understanding something is building a (physical, mathematical or mentally simulated) mechanistic model of it.

In fact, we get a deep understanding or aha moment, when we're able to play in our mind with the model of the object to be understood and find that our simulated model's outputs match with what we observe in the object out there in reality.

Ways of understanding things

There are at least 3 ways of building a model of the thing to be understood:

Mathematical (or code)
Physical
Mentally simulated/imagined

An imagined model is actually simulated physics in the head. For example, when we're trying to understand how the moon remains in orbit around the earth, we invoke in our mind a rudimentary physical model of reality where forces act on objects and they make the obje...
Question Gravity
by Isaac King — 2022-01-13T06:30:56.013Z

Earlier today, I was in the shower. I had just finished a bottle of shampoo, and I tried balancing it on the thin railing of the shower door. I expected it to fall down, but instead it stayed there. From the side, it certainly looked like it was sticking out more than halfway over the edge. (Picture) Yet it stubbornly stayed put.

On a whim, I filled it with water and put it back. It fell over.

Wait, what? Filling it up shouldn't change the location of its center of gravity, so why would that make it fall over? I started performing more experiments. I rotated the bottle and balanced it again several times to see if there was some asymmetry causing it to have an off-center center of gravity. I thought maybe the water sloshing around could cause its center of gravity to move over the edge, so I I tried holding it there for several seconds before letting go, in order to let the water settle. I noticed the door had some wiggle room, and the weight of the bottle could be causing the door to shift to a different angle, so I tried holding it at different angles. Nothing caused any different behavior. I put it on the ledge empty and slowly filled it with water. When it got to around 1/20 full, it fell over. I found a second bottle and performed all the same tests on it; same results.

At this point I was thoroughly confused. I figured some assumption of mine must have been wrong, so I tried inspecting them. Does the center of gravity of an object really have to be over the edge in order for it to fall? I couldn't think of any situation where that wasn't true. (Generalizing from a non-exhaustive list of arbitrary examples is obviously not a great approach, but I couldn't do much better while in the shower.) Does filling it with water really not change its center of gravity? Oh wait; it does! The center of gravity doesn't move it laterally, but it does move it vertically. That assumption was wrong. (I probably wouldn't have made this mistake if I had been thinking of it as the center of mass rather than the center of gravity. Framing is important.) Unfortunately, that epiphany didn't lead to anything useful; I couldn't see how moving the center of mass vertically could affect whether it was balanced or not.

If you haven't figured it out yet, take a moment to think about it. You have all the information you need. Also, let me know in a comment if and how easily you figured i...
Paper claims: "Rationality" flavored words rose since 1850, began declining ~1980
by Raemon — 2022-01-12T23:11:46.232Z
Link: https://phys.org/news/2022-01-rationality-declined-decades.html

Epistemic status: found the link, thought it was kinda interesting, mostly kicking off discussion.

This article felt somewhat related to "Have Epistemic Conditions Always Been This Bad?", and the question of whether humanity was (briefly) in a golden age of epistemics.

Analyzing language from millions of books, the researchers found that words associated with reasoning, such as "determine" and "conclusion," rose systematically beginning in 1850, while words related to human experience such as "feel" and "believe" declined. This pattern has reversed over the past 40 years, paralleled by a shift from a collectivistic to an individualistic focus as reflected by the ratio of singular to plural pronouns such as "I"/"we."

Original paper here: https://www.pnas.org/content/118/51/e2107848118
The Unreasonable Feasibility Of Playing Chess Under The Influence
by Jan — 2022-01-12T23:09:57.679Z
Link: https://universalprior.substack.com/p/the-unreasonable-feasibility-of-playing

TL;DR: The wonderful tradition of playing chess drunk, Marr's levels of analysis, AlphaZero, and Iterated Amplification and Distillation.

A proud history of drinking and playing chess

Please enjoy this clip of the reigning chess world champion Magnus Carlsen playing a game of

hyperbullet

(30 sec total per player) while inebriated.

As the commenters are quick to point out, Magnus winning this game is partially due to his opponent blundering their Queen a couple of moves after Magnus takes over. But still. Even after a full night of sleep and being highly caffeinated I would struggle to complete a game of chess in 30 seconds[1]. Winning while chanting party bangers, and complaining about the state of one's own body, is kind of impressive.

Magnus is standing on the shoulders of giants in this game. Drinking and chess have been going hand in hand since at least

Paul Morphy

, whose family

started

the

Murphy brewery

. Since Morphy, two former world champions stand out as particularly heavy drinkers,

Alexander Alekhine

and

Mikhail Tal

. The story of

Mikhail Tal

, the Magician from Riga, is interesting as he was drinking and smoking en masse while producing

more brilliancies

[2] than any other player.

The Magician from Riga in his element.

His play under the influence might be the source of the common

urban myth

that drinking might improve one’s play. Tal's heavy drinking was likely not a character flaw, but

presumably

represented a type of

self-medication

for his congenital chronic illnesses. The same can not be said for another grandmaster, who triggered one of many[3] episodes of chess drama by drinking heavily and "

losing it

" live on Twitch during a chess streaming session. On the other hand, he was

awarded

the infamous title "best drunk chess player in the world" by world championship challenger

Fabiano Caruana

. Beyond the players, there is "

alcoholic chess

", the "

bongcloud opening

", and of course, the recent Netflix hit show "

The Queen's Gambit

" about a drug-addicted, female

Bobby Fisher

. Magnus Carlsen winning a game of hyperbullet after having a drink or two might appear impressive to an outsider - the aficionado just calls it "Tuesday".

How do they do it?

What's my point? I don't think we should be surprised that chess players drink alcohol. They are, visibly, human, which means the prior probability of them

having had a drink in the last year

is (based on loc...
New year, new research agenda post 
by Charlie Steiner — 2022-01-12T17:58:15.833Z

Thanks to Steve Byrnes, Adam Shimi, John Wentworth, and Peter Barnett for feedback.

In a nutshell, my plan A is to understand what we want from superintelligent AI really, really well. So well that we can write down a way of modeling humans that illuminates human preferences including higher-order preferences about how we want to be modeled, and do this in a principled rather than ad-hoc way. Achieving this understanding is highly ambitious, in a way that is mostly but not entirely parallel to "

ambitious

value learning."

If we understand value learning before we build superintelligent AI, there's a straightforward path to achieving a good future without paying a costly alignment tax - by alignment tax I mean all those things that slow down aligned AI being developed and "ready for liftoff," that collectively create selection pressure against safety. This problem is more tractable than many people think.

I think of this plan as an instance of the more general plan "solve value learning first." Some other tenable plans are "try to put humans in control," "get the AI to do good things prosaically," and "tinker with the AI until it value learns" - plus intermediate points between these.

What do I think the future is like?

I expect superintelligent AI in the short to medium term, centralized around a small number of points of development.

By short to medium term, I mean I'd put my 50% confidence interval between 2031 and 2049. I don't think we need compute to be many orders of magnitude cheaper, and I don't think we need two or more paradigm shifts on the order of neural nets overtaking support vector machines. The timeline is urgent, but not to the point that we should start ditching things like blue-sky research or gradual coalition-building.

By centralized, I mean it's possible to make big changes by having good solutions implemented in a small number of systems. Coordination may be important, but it isn't an inherent part of solving the problem.

All that said, that's just what I think is going to happen, not what's required for "solve value learning first" to be a good idea. Value learning research is still valuable in decentralized scenarios, unless we go so far as to avoid powerful agential AI long-term. Because it's more on the blue-sky end of the spectrum, longer timelines actually favor solving value learning over more atheoretic approaches, while if...
Political Office for Beginners?
by Asgård — 2022-01-12T17:27:30.199Z

If an American citizen was looking to run for office (from local, state, all the way to federal), what would be the recommended steps to take?
Specific questions that come to mind:
* How would a Millennial or Gen Z'er deal with existing social media accounts?
* How would a Gen X'er or Boomer create and navigate social media accounts and advertising?
* Where and how would the first $10,000 do the greatest good? First $100,000? Etc.
* Are there political grants/party grants/etc. available in the United States for less-funded campaigns?
* When should a citizen start campaigning for an election?
* Where and how would the first 100 hours do the greatest good? First 1,000? Etc.
Value extrapolation partially resolves symbol grounding
by Stuart_Armstrong — 2022-01-12T16:30:19.003Z

Take the following AI, trained on videos of happy humans:

Since we know about AI wireheading, we know that there are at least two ways the AI could interpret its reward function[1]: either we want it to make more happy humans (or more humans happy); call this R1. Or we want it to make more videos of happy humans; call this R2.

We would want the AI to learn to maximise R1, of course. But even without that, if it generates R1 as a candidate and applies a suitable diminishing return to all its reward functions, then we will have a positive outcome - the AI may fill the universe with videos of happy humans, but it will also act to make us happy.

Thus solving value extrapolation will solve symbol grounding, at least in part.

This is a massive over-simplification of what would be needed to define "happy" or anything similar. ↩︎
(briefly) RaDVaC and SMTM, two things we should be doing
by Eliezer Yudkowsky — 2022-01-12T06:20:35.555Z

UPDATE: Be advised that I no longer consider SMTM to be among the things we should be doing, though I don't regret having given them an earlier try. I wish somebody else was running out and trying the sort of things that SMTM is trying, but they're bad enough at analysis to not qualify for further support until that gets fixed. See eg this Twitter thread.

Having not apparently the energy to write this longly, I write it shortly instead, that it be written at all.

People sometimes go about saying now, in this community, that there is collectively enough money that we could potentially go do more things with it, if there were things worth doing.

If that's true, I'd like to see us planting seed grains now for replacing the now-defunct state capacity of the USA and Earth, with respect to biodefense and ultra-high-leverage med R&D.

As a poster child of a previous intervention I backed here, Seasonal Affective Disorder affects 0.5%-3% of the population and higher in countries at extreme latitudes. Call it a hundred million people at a guess. Standard lightboxes don't work very well for treating it. The Sun works great for treating it. Reason suggests trying more light. Earth, however, previously lacked the state capacity to investigate this obvious question of massive scope. I put a private funder in touch with a group that did a preliminary investigation; they need to run a larger study but preliminary results suggest that they may, in fact, need to use more light and that doing so might be effective. Again, that's for a problem that probably over a hundred million people have. So it's not very surprising that investigation of it bottlenecks on there being a single interested medical researcher, who writes Eliezer Yudkowsky as the person who posed that question, and gets funding arranged by that route. It was less than $100,000.

This sort of thing seems obviously competitive to me with QALYs/$ or DALYs/$ on global poverty interventions.

Really obviously competitive, actually. I will not argue the point further because I expect that most readers who can be persuaded found a glance at the numbers sufficient.

EA's ability to do more things in this space, if it is not bottlenecked on money, is very likely bottlenecked on: good ideas to pursue; people who can pursue those ideas; and/or admin st...
Morality and constrained maximization, part 2
by Joe Carlsmith — 2022-01-12T01:42:45.097Z

(Cross-posted from Hands and Cities)

This is the second in a pair of posts examining whether morality falls out of instrumental rationality, if you do the game theory right. David Gauthier thinks the answer is yes. I focus on his view as an example of a broader tendency, to which I expect many of my comments to generalize.

In my last post, I summarized the basics of Gauthier’s view. In this post, I get into more detail about the morality Gauthier proposes, and I discuss four objections to it:

1. It isn’t actually instrumentally rational.

I’m not fussed about whether Gauthier’s constrained maximization can ever be instrumentally rational. But Gauthier wants us to abide by the constraints we would’ve agreed on in a particular (hypothetical) situation we were never in. I think this requires more justification than Gauthier has given.

2. It gives the wrong type of reasons for moral behavior.

I used to treat this objection as decisive. Now I’ve softened a bit — but not fully.

3. It incentivizes threats and exploitation.

Gauthier thinks his constrained maximizers will avoid this via refusals to cooperate with agents that attempt it. I’m open to stuff in this vicinity, but I’m skeptical of his particular story.

4. It ties the moral consideration X receives too closely to X’s power, and licenses arbitrarily bad behavior towards the sufficiently disempowered and unloved.

This one looks true to me, and bad.

Overall, I don’t think Gauthier’s approach re-creates conventional morality, or the type of care towards others most important to my own ethical outlook. But I think it points at something important regardless.

I. Contracts you would’ve signed

When we last left Gauthier, he was telling us to be constrained maximizers: agents willing, in certain situations, to give up on maximizing utility, and to act on the basis of a collective strategy close enough to what would be agreed on via a process of rational bargaining. Morality, roughly speaking, is this strategy.

The project here fits a classic “social contract” formula: it attempts to ground a normative standard in what some set of agents would agree to in some situation. But social contracts come in different flavors – and on a spectrum from “idealistic” to “realpolitik,” Gauthier’s is on the latter end.

Thus: Rawls famously imagines agents behind a “veil of ignorance,” which denies them knowledge of who they are, and h...