trentmkelly/LessWrong-43k huggingface dataset:
Book Review: The Righteous Mind
by Tornus — 2022-01-20T21:42:00.883Z

I was inspired to read Jonathan Haidt’s The Righteous Mind by Julia Galef’s excellent interview with him. It’s an outstanding book that greatly improved my understanding of how humans understand morality and how to make moral arguments that humans will actually listen to.

More generally, I think it sheds some light on the challenges we face as we try to build a culture that is rational and ethical but still resonates with real humans.

Epistemic status

“All models are wrong but some are useful.” - George Box

Righteous Mind is a layperson’s introduction to Moral Foundations Theory, which as far as I can tell is broadly consistent with mainstream moral psychology but is just one of a number of competing theories.

I consider Haidt’s model of morality to be interesting, plausible, and useful. Some parts—most notably the evolutionary origins of certain instincts—are best regarded as interesting speculation.

Overview

You’re probably familiar with the idea that our eating behaviors are driven largely by instincts that evolved for a world very different from the one we currently live in. Our ancestors lived in a world of chronic food scarcity, and they evolved a powerful drive to consume high calorie foods whenever they got the chance. Even though there’s a McDonald’s on every street corner, our instincts drive us to eat as though we never know where our next meal will come from.

Our brains aren’t terrible at eating—they’re exquisitely well tuned for eating in an ancestral environment that was completely different from the modern world.

Haidt posits that morality, like eating, is an instinctive behavior. Morality evolved to help our ancestors navigate the social structure of hunter gatherer society and only makes sense when examined in that context.

To understand why humans routinely make terrible food choices—and to change those choices—you need to understand the evolutionary basis of our eating instincts. By the same token, to understand why humans routinely make terrible moral choices—and to change those choices—you need to understand the evolutionary basis of our moral instincts.

Part 1: the rider and the elephant

A quick review of Type 1 and Type 2 thinking

You’re probably already familiar with Dual Process Theory, which identifies two types of thinking:

Type 1 thinking (aka System 1) is fast, cheap, and subconscious. When you look at a glass of water and instantly “

...
The Liar and the Scold
by Tomás B. — 2022-01-20T20:31:14.765Z

When I found my faith, I had an almost solipsistic view of other people.

They say that is how we all start off. But I think I am unusual in that I remember the moment that I realized others, too, were themselves. I remember not because I was precocious. Rather, because I was stunted in this aspect of development to the point that it did not happen until I was five years old.

I knew I was myself, but others were, in my mind, these vast, mindless automatons. These strange giants, simultaneously servants and masters.

Towards my father I felt fear, for it was his hand that applied negative reinforcement. He was not a cruel man, and his slaps were light and always earned, but at that age even something as mild as a bee’s sting feels like death itself.

For my mother there was love, but it was the sort of abstract, superior love one has for a can-opener that has served you well for many years.

And then, everything changed.

Sometimes an insight comes so slowly you are not even aware your model of the world has changed. And sometimes just the opposite.

My father had taken me for a walk down a busy sidewalk near our apartment. A lazy child, I was not fond of walking and trailed sulkily behind, his hand grasped around my wrist, joining us like a leash on the neck of some reluctant dog.

At some point, I had enough. I sat down and refused to move. Busy people routed around us like water does about a stone.

He tried to drag me, but I remained still. He tried to cajole me but still I did not react.

I felt fear rise in me, as I knew he would slap me soon, but my stubbornness that day was strong enough to overcome a fear which proved unjustified.

He did not slap me, nor even threaten to. And I could not understand why. Until I did.

I looked up at him nervously eyeing the people walking past. And I felt his nervousness, his worry that others would judge him were he to resort to his usual methods. I looked at the face of a concerned woman walking past, and felt some semblance of her concern; I looked at the face of a bored young man listening sullenly to his headphones and I felt some semblance of his apathy.

And then I began to laugh. I laughed for what felt like hours but surely could not have been more than a few minutes. I laughed so hard that my father, too, began to laugh, and seeing him do so I felt a reflection of his amusement and began to laugh...
What's Up With Confusingly Pervasive Goal Directedness?
by Raemon — 2022-01-20T19:22:37.515Z

Fictionalized/Paraphrased version of a real dialog between me and John Wentworth.

Fictionalized Me: So, in the Eliezer/Richard dialogs, Eliezer is trying to get across this idea that consequentialism deeply permeates optimization, and this is important, and that's one[1] reason why Alignment is Hard. But something about it is confusing and slippery, and he keeps trying to explain it and it keeps not-quite-landing.

I think I get it, but I'm not sure I could explain it. Or, I'm not sure who to explain it to. I don't think I could tell who was making a mistake, where "consequentialism is secretly everywhere" is a useful concept for realizing-the-mistake.

Fictionalized John: [stares at me]

Me: Okay, I guess I'm probably supposed to try and explain this and see what happens instead of hoping you'll save me.

...

Me: Okay, so the part that's confusing here is that this is supposed to be something that Eliezer thinks thoughtful, attentive people like Richard (and Paul?) aren't getting, despite them having read lots of relevant material and paying attention and being generally on board with "alignment is hard."

...so, what is a sort of mistake I could imagine a smart, thoughtful person who read the sequences making here?

My Eliezer-model imagines someone building what they think is an aligned ML system. They've trained it carefully to do things they reflectively approve of, they've put a lot of work into making it interpretable and honest. This Smart Thoughtful Researcher has read the sequences and believes that alignment is hard and whatnot. Nonetheless, they'll have failed to really grok this "consequentialism-is-more-pervasive-and-important-than-you-think" concept. And this will cause doom when they try to scale up their project to accomplish something actually hard.

I... guess what I think Eliezer thinks is that Thoughful Researcher isn't respecting inner optimizers enough. They'll have built their system to be carefully aligned, but to do anything hard, it'll end up generating inner-optimizers that aren't aligned, and the inner-optimizers will kill everyone.

...

John: Nod. But not quite. I think you're still missing something.

You're familiar with the arguments of convergent instrumental goals?

Me: i.e. most agents will end up wanting power/resources/self-preservation/etc?

John: Yeah.

But not only is "wanting power and self preservation" convergent...
Emotions = Reward Functions
by jpyykko — 2022-01-20T18:46:52.584Z

... Or more specifically, a post on how, and why, to encode emotions to find out more about goals, rationality and safe alignment in general.

If one tries to naively fit reinforcement learning’s reward functions back onto the human mind, the closest equivalent one may find is emotions. If one delves deeper into the topic though, they will find a mish-mash of other “reward signals” and auxiliary mechanisms in the human brain, (such as the face tracking reflex, which aids us in social development) and ends up hearing about affects, the official term when it comes to the study of emotions.

At least that is what approximately happened with me.

Affects and reward functions seem to have a common functional purpose in agents, in that they both direct the agent’s attention towards what is relevant.

Both:

Evaluate the ‘goodness’ (valence) of a situation.
Are required for the ‘agent’ to perform any actions.
Define what the agents learn, what they value, and what goals they can create based on these values.

This means that if we can map and write all of the human affects into reward functions, we can compare various constellations of affects and see which ones produce what human-like behaviours. This in turn may lead to solutions for not only how to induce human-like biases into AI, but also investigate our own values and rationality from a new perspective.

The purpose of this post is to introduce a few ways to proceed with the task of encoding affects. First, there will be a rudimentary definition of the components and some motivational points on what this all could mean. After that there will be an introduction to three distinct levels of representation for various use cases, from philosophical to ontological, and finally pseudocode.

Disclaimers:

This post is meant to act as a conversation starter, so many points might be alarmingly concise.

The formalities are meant to replicate the functionality of human behaviour, and are not claimed to be exact copies of the neurological mechanisms themselves. Tests should be performed to find out what emerges in the end. Some points might be controversial, so discussion is welcome.

Definitions

Alright, let's define the two a bit more in depth and see how they compare.

Reward functions are part of reinforcement learning, where a “computer program interacts with a dynamic en...
Risk and Safety in the age of COVID
by Mike Harris — 2022-01-20T18:40:21.926Z

Intro:

When analyzing the dangers of COVID or proposing policies to combat it, there is a utilitarian cost-benefit framing which is implicitly assumed. Perhaps the best example of this is the Micro Covid calculator. Every action has a cost and benefit associated with it and the goal of any policy is to choose the best set of actions.

This framework, which I will call “risk based”, is easy to express numerically. This makes it feel objective and rational. However, this framework does not align with how people think about the dangers of COVID and is not how the policy decisions actually get made. By understanding the more intuitive framework which I will call “safety based” it is possible to understand the underlying motivations behind the policies and arguments being made. Understanding the constraints imposed by the safety based framework makes many of the policy failures throughout the pandemic understandable and in fact inevitable.

Safety Based Framing:

When thinking about an activity which involves risk, people do not, even implicitly, do a cost benefit analysis to determine if it is worth it. Instead they ask if it is safe or not. If an action is safe then it can be done without worrying about any residual risks. If an action is unsafe then it can only be done if extraordinary precautions are taken. If you fail to take these precautions then you are considered morally blameworthy for any bad outcomes which arise.

Determining if an action is safe or unsafe cannot be objectively determined. Although it is influenced by the relative risks involved it is inherently socially constructed. To take a simple example consider the difference between driving without a seatbelt and riding a motorcycle.

Riding a motorcycle is a much higher risk activity than driving without a seatbelt. However we treat them very differently. If you are taking the proper precautions (wearing a helmet, proper gear) then riding a motorcycle is safe but risky. On the other hand driving without a seatbelt is inherently unsafe[1].

In March 2020, as COVID cases surged in Italy and New York, there was a shift and COVID became unsafe, which meant that measures had to be taken to eliminate the unsafety. However, three broad camps emerged as to how COVID was unsafe. The seemingly small differences between these perspectives have lead to more wide fissures in rhetoric and policy as the pandemic has c...
Anchor Weights for ML
by jsteinhardt — 2022-01-20T16:20:20.390Z

In the previous post, I talked about several "anchors" that we could use to think about future ML systems, including current ML systems, humans, ideal optimizers, and complex systems.

In fact, I think we should be using all of these anchors (and any others we can think of) to reason about future ML systems. This is based on ideas from forecasting, where successful forecasters usually average over many worldviews and reference classes rather than focusing on a single reference class. However, we should also be discerning and weight anchors more if they seem like a better match for what we want to predict.

Below, I'll say what I personally think about most of the anchors we discussed so far, by assigning a numerical "weight" to each one. While these weights aren't perfect (the actual weight I'd use depends on the particular question), they hopefully provide a clear overall picture that is easy to agree/disagree with.

Here are the rough weights I came up with:

Anchor
Weight

Current ML
4

Complex systems
3

Thought experiments
2

Evolution
0.5

The economy
0.4

Humans
0.3

Corporations
0.2

Biological systems
0.2

Non-human animals
0.1

I primarily rely on Current ML, Complex Systems, and Thought Experiments, in a 4:3:2 ratio. In particular, I assign about twice as much weight to Current ML as to Thought Experiments, but I think the opposite ratio is also defensible. However, many people seem to implicitly put almost all their weight on Current ML, or almost all their weight on Thought Experiments. They have something like a 5:1 or 1:5 ratio, or even greater. I think neither of these stances is defensible, and I would be interested in anyone who disagrees writing up the case for assigning extreme weights (in either direction).

Relatedly, my last two posts were essentially an argument against a 5:1 ratio in favor of Current ML--first by arguing that Current ML often misses important developments, and second by arguing that thought experiments can sometimes catch these.[1]

Aside from this, my biggest disagreement with others would be assigning significant weight to the "Complex Systems" anchor, which I think most people overlook.

Finally, all anchors that correspond to a broad reference class (Current ML, Complex Systems, Thought Experiments) get significantly more weight than any anchor that is a single example (e.g. humans).[2] I give serious conside...
Covid 1/20/22: Peak Omicron
by Zvi — 2022-01-20T16:20:01.331Z

The Omicron and Weekly posts are now combined, so this includes the last day’s Omicron developments, which will be how it works going forward. Next week’s will include the probability updates section as well. When there is urgent news, I’ll supplement as appropriate.

This week was the peak. From here, it would be highly surprising if Omicron cases didn’t start declining. The other news is mostly more of the same, with several developments worth analyzing: The Supreme Court throws out one of Biden’s mandates but upholds the other, Djokovic gets deported a second time, a great proposal to expand Manhattan (which got its own post), and other neat stuff like that.

Executive Summary

If you live in USA and haven’t done so yet, order your free Covid-19 tests.
We are at or past the peak in Omicron infections in the United States and UK, but that doesn’t mean it’s over yet, next few weeks still rough.
UK lifts all Covid-19 restrictions, midnight is perhaps temporary after all.

Let’s run the numbers.

The Numbers

Predictions

Prediction from last week: 5.6mm cases (+13%) and 15,000 deaths (+30%).

Results: 4.9mm cases (+0%) and 12,606 deaths (+7%).

Prediction for next week: 4.4mm cases (-10%) and 14,500 deaths (+15%).

The deaths number is confusingly low this week. Cases increased by a huge amount with the three week lag, yet the deaths number isn’t increasing much. There’s about to be a much bigger three-week-lagged increase in cases, but the pattern is clear so my estimated size increase needs to adjust. I am fully prepared to miss low reasonably often here, if I wasn’t the estimate would be wrong.

For cases, it seems clear that we are peaking, so we should expect things to begin to decline. The question is how fast, with some areas not yet peaked, and the Northeast already having a much smaller share of total cases. The decline could easily be much faster than this, but things could also stay mostly steady for one more week.

Deaths

Even with Omicron being milder, the lack of more deaths is very good news here. The next week is the time when deaths should by all rights go much higher, so if they stay under control this week, we’re fully in the clear.

Cases

The Northeast already peaked, as probably did a number of other large states like Illinois, Florida and California. The South might have one more week left in aggregate, it ...
Estimating training compute of Deep Learning models
by lennart — 2022-01-20T16:12:43.497Z

by Jaime Sevilla, Lennart Heim, Marius Hobbhahn, Tamay Besiroglu, and Anson Ho

You can find the complete article here. We provide a short summary below.

In short:
To estimate the compute used to train a Deep Learning model we can either: 1) directly count the number of operations needed or 2) estimate it from GPU time.

Method 1: Counting operations in the model

2×# of connectionsOperations per forward pass×3Backward-forward adjustment×# training examples×# epochsNumber of passes

Method 2: GPU time

training time×# cores×peak FLOP/s×utilization rate

We are uncertain about what utilization rate is best, but our recommendation is 30% for Large Language Models and 40% for other models.

You can read more about method 1 here and about method 2 here.

Other parts of interest of this article include:

We argue that the ratio of operations of backward and forward pass of neural networks is often close to 2:1. More.

We discuss how the formula of method 1 changes for recurrent models. More.

We argue that dropout does not affect the number of operations per forward and backward pass. More.

We have elaborated a table with parameter and operation counts for common neural network layers. More.

We give a detailed example of method 1. More.

We discuss commonly used number representation formats in ML. More.

We share an estimate of the average performance of GPU cards each year. More.

We share some reported GPU usages in real experiments. More.

We give a detailed example of method 2. More.

We compare both methods and conclude they result in similar estimates. More.

We discuss the use of profilers to measure compute. More.

Complete Article

You can find the article here.
Choosing battles (on the Internet)
by PatrickDFarley — 2022-01-20T15:38:17.953Z

A couple years ago, I noticed that I'd been holding onto a mindset that was counterproductive, and I've often seen this same mindset in some of the online communities I visit. I like talking with people who believe in the truth-seeking power of good-faith disagreements, but some of these people have a tendency to overestimate the value of argument, which causes them to engage in conversations they shouldn't. And I think I have good heuristics for avoiding that without giving up on the project of rational discourse entirely.

The warm fuzzy hope of good-faith conversations

There's this optimistic belief that careful, rational arguments, even between strongly opposed people, will always lead to truth. How could they not? There is only one system of logic, and only one reality from which to gather evidence.

And let me be clear that I've benefited tremendously from engaging with people who think like this. They're basically right—good-faith arguments help you weed out your false beliefs and sharpen your correct ones, and having correct beliefs is good and advantageous.

But often with that optimism comes a fear that if you ever ignore an argument, you're missing an opportunity to learn the truth. If I say reality is X, and you say it's Y, and I refuse to argue with you for whatever reason, then I almost feel like I'm daring reality to be Y, leaving me helplessly ignorant about it for the rest of my life—or worse, until I get into a situation where knowing X vs Y really matters, and I make the wrong decision. And because that notion scares me, I stay engaged in our argument no mater what.

Failure modes

This is why I sometimes see well-meaning writers/commenters repeatedly typing out carefully worded paragraphs to their opponents, who are obvious trolls, or who will ignore most points and twist what remains, or who don't stay on topic, or who can't seem to make their point more than once without changing it, or who just aren't demonstrating enough intelligence to warrant any hope of mutual understanding. I see this a lot online and it saddens me, because the person who's being more careful to offer value in the argument is getting a worse deal for it.

It also worries me because it can eventually lead those kinds of people to snap and give up on the value of argument altogether. More on that later.

The well-meaning commenters think, "Sure my efforts don't seem to be working,...
Land Ho!
by Zvi — 2022-01-20T13:30:01.262Z

I love this modest proposal so much I am making an exception to my no-New-York-Times rule, and split it off from what was going to be a bonus section in the weekly Covid post. Time to think big.

Both Mayors Bill de Blasio and Michael Bloomberg offered climate-change plans that included extending the shoreline along the East River in Lower Manhattan. But these proposals, while admirable, would be small steps and would hardly make a dent with problems of such big scale.

This new proposal offers significant protection against surges while also creating new housing. To do this, it extends Manhattan into New York Harbor by 1,760 acres. This landfill development, like many others in the city’s past, would reshape the southern Manhattan shoreline. We can call the created area New Mannahatta (drawn from the name the Lenape gave to Manhattan).

It quickly became fashionable to mock this proposal. This is a sign of civilizational decline and inadequacy, because we used to do this kind of thing all the time and (unless there’s some logistical problem I’m missing that on one is talking about) it’s obviously insanely great.

The objections seem to be:

Ha ha.
Get a load of the nerd.
Who thinks America can do things, like, ever.
New York, in particular, ha ha.
Also, how would more housing help with a shortage of housing?
And how dare you build things when climate change is coming?
Evil bunch of idiot nerds.

I volunteer to be the mayoral candidate of the New Amsterdam (which is the obviously correct neighborhood name if we’re not making a multi-billion-dollar endorsement deal) party. My other platforms will include free and expanded subway service paid for at least in part by a 3%/year property tax surcharge on vacant apartments and houses, ending rent control on all new or vacant apartments, approval of all new construction everywhere, permanent outdoor dining, and ending of all city-based occupational licensing requirements. Construction costs will be paid for temporarily by a bond issue, then repaid with interest by a 100% tax on the unimproved value of the newly created land.

This was a proposal in The New York Times, so of course it was necessary to lead with talk about protection against climate change. That did not of course stop stupid people from saying how stupid it would be to build things at sea level, no matter how much this was used as part of a proje...