trentmkelly/LessWrong-43k huggingface dataset:
A Retrospective Look at the Guild of Servants: Alpha Phase
by moridinamael — 2021-03-22T20:34:30.520Z

Executive Summary

The Guild of Servants is an organization founded in August 2020 aimed at providing structure and community to rationalists, and others with similar ambitions, in the midst of the COVID-19 pandemic. The Alpha Phase of the project recently concluded.

The Alpha Phase was an experimental rollout and proof-of-concept involving ~35 volunteer participants, consisting of an online community component and an online course component. We have collected feedback from stakeholders in the form of interviews, and consolidated that feedback in this document.

The courses, as conducted in the Alpha, were determined to be too time-consuming and demanding, both for the instructors and the participants. It is recommended that the duration and intensity of courses be scaled down in subsequent phases. Large and complex courses such as the Character Sheet ought to be broken down into smaller discrete pieces. In this document we make further, more detailed suggestions for changing the format of the courses.

The cohort community structure, in which participants were placed into fixed cohorts of six members, was deemed a success. However, due to a natural expectation of attrition, it is recommended that the starting cohort size be increased to eight members, and that initial cohort sorting be based on mutual timezone compatibility.

The Alpha Phase was a process of learning and experimentation, and each founding Council member played many different roles. For the Beta Phase, it is recommended that the Guild leadership structure be reorganized such that each Guild Council member is assigned a specific administrative role, and that participation and contribution be compensated either monetarily or with ownership equity. It is further recommended that Guild Council members and other administrators not be expected to do both administrative and course-teaching duties at the same time.

The Alpha Phase is considered a success, based on our own stated expectations and goals in the planning. We avoided the pitfalls that we anticipated in our pre-mortem analysis. Before moving into the Beta Phase, it behooves us to recalibrate expectations toward a more ambitious goal, and specify our goals and vision as precisely as practical.

Introduction

August 2020, a group of five rationalists founded an organization to provide a formal community and development structure for rationalists. D...
On future people, looking back at 21st century longtermism
by Joe Carlsmith — 2021-03-22T08:23:06.743Z

(Cross-posted from Hands and Cities)

“Who knows, for all the distance, but I am as good as looking at you now, for all you cannot see me?”

– Whitman, Crossing Brooklyn Ferry

Roughly stated, longtermism is the thesis that what happens in the long-term future is profoundly important; that we in the 21st century are in a position to have a foreseeably positive and long-lasting influence on this future (for example, by lowering the risk of human extinction and other comparable catastrophes); and that doing so should be among the key moral priorities of our time.

This post explores the possibility of considering this thesis — and in particular, a certain kind of “holy sh**” reaction to its basic empirical narrative — from the perspective of future people looking back on the present day. I find a certain way of doing this a helpful intuition pump.

I. Holy sh** the future

“I announce natural persons to arise,
I announce justice triumphant,
I announce uncompromising liberty and equality,
I announce the justification of candor and the justification of pride…

O thicker and faster—(So long!)
O crowding too close upon me,
I foresee too much, it means more than I thought…”

– Whitman, So Long!

I think of many precise, sober, and action-guiding forms of longtermism — especially forms focused on existential risk in particular — as driven in substantial part by a more basic kind of “holy sh**” reaction, which I’ll characterize as follows:

Holy sh** there could be a lot of sentient life and other important stuff happening in the future.
And it could be so amazing, and shaped by people so much wiser and more capable and more aware than we are.
Wow. That’s so crazy. That’s so much potential.
Wait, so if we mess up and go extinct, or something comparable, all that potential is destroyed? The whole thing is riding on us? On this single fragile planet, with our nukes and bioweapons and Donald Trumps and ~1.5 centuries of experience with serious technology?
Do other choices we make influence how that entire future goes?
This is wild. This is extremely important. This is a crazy time to be alive.

This sort of “holy sh**” reaction responds to an underlying empirical narrative — one in which the potential size and quality of humanity’s future is (a) staggering, and (b) foreseeably at stake in our actions today.

Conservative versions of this narrative appeal to the spans of time that we might live on earth,...
Generalizing POWER to multi-agent games
by midco — 2021-03-22T02:41:44.763Z

Acknowledgements:

This article is a writeup of a research project conducted through the SERI program under the mentorship of Alex Turner. I (Jacob Stavrianos) would like to thank Alex for turning a messy collection of ideas into legitimate research, as well as the wonderful researchers at SERI for guiding the project and putting me in touch with the broader X-risk community.

Motivation/Overview

In the single-agent setting, Seeking Power is Often Robustly Instrumental in MDPs showed that optimal policies tend to choose actions which pursue "power" (reasonably formalized). In the multi-agent setting, the Catastrophic Convergence Conjecture presented intuitions that "most agents" will "fight over resources" when they get "sufficiently advanced." However, it wasn't clear how to formalize that intuition.
This post synthesizes single-agent power dynamics (which we believe is now somewhat well-understood in the MDP setting) with the multi-agent setting. The multi-agent setting is important for AI alignment, since we want to reason clearly about when AI agents disempower humans. Assuming constant-sum games (i.e. maximal misalignment between agents), this post presents a result which echoes the intuitions in the Catastrophic Convergence Conjecture post: as agents become "more advanced", "power" becomes increasingly scarce & constant-sum.

An illustrative example

You're working on a project with a team of your peers. In particular, your actions affect the final deliverable, but so do those of your teammates. Say that each member of the team (including you) has some goal for the deliverable, which we can express as a reward function over the set of outcomes. How well (in terms of your reward function) can you expect to do?

It depends on your teammates' actions. Let's first ask "given my opponent's actions, what's the highest expected reward I can attain?"

Case 1: Everyone plays nice

We can start by imagining the case where everyone does exactly what you'd want them to do. Mathematically, this allows you to obtain the globally maximal reward; or "the best possible reward assuming you can choose everyone else's actions". Intuitively, this looks like your team sitting you down for a meeting, asking what you want them to do for the project, and carrying out orders without fail. As expected, this case is 'the best you can hope for" in a formal sense.

Case 2: Everyone pla

...
Moving Data Around is Slow
by SatvikBeri — 2021-03-22T02:36:08.480Z

Data locality is a key part of writing fast data science code.

The core idea is simple: your data starts out in RAM (or disk), and to actually do anything useful with it, you need to move it to the CPU. This is actually pretty slow, so you want to minimize this transfer.

Say you have pairs of heights and weights, and you want to get the sum of all the weights. Let's start by generating 40 million pairs:

num_rows = 40000000
struct HeightAndWeight
height::Int
weight::Int
end
heights_and_weights = [HeightAndWeight(row[1], row[2]) for row in eachrow(rand(50:150, num_rows, 2))];

This gives us an array of HeightAndWeights like so:

40000000-element Array{HeightAndWeight,1}:
HeightAndWeight(138, 84)
HeightAndWeight(140, 136)
HeightAndWeight(87, 137)
HeightAndWeight(109, 143)

One approach is to iterate through each row and add the weight to a counter.

function my_sum(data)
sum_weights = 0
for row in data
sum_weights += row.weight
end
return sum_weights
end

Benchmarking this, it takes about 52 ms

using BenchmarkTools
@btime(my_sum(heights_and_weights))
> 52.535 ms (1 allocation: 16 bytes)

What if our heights and weights were separate, and we just pass the weights?

just_heights = [row.height for row in heights_and_weights]
just_weights = [row.weight for row in heights_and_weights]
@btime(my_sum2(just_weights))
> 23.495 ms (1 allocation: 16 bytes)

This is about twice as fast, modulo noise! Despite the fact that we're doing exactly the same number of additions, we're transferring half the data, which takes the majority of the time.

When we have an array of HeightAndWeight structs, it's very difficult to avoid passing in extra data. If the struct had even more fields, it would be even slower, despite the fact that we're adding up the same number of weights. This is a common situation with object-oriented programming or row-oriented tables.

The pattern of creating a collection of many HeightAndWeight objects is known as Array-of-Structs, while the pattern of having several arrays is known as Struct-of-Arrays (https://en.wikipedia.org/wiki/AoS_and_SoA .) Array-of-Structs is typically slower because of the increased data transfer, and the difference is most prominent in areas of your code where you're iterating over large collections of wide objects with many fields. Consider refactoring those to Struct-of-Arrays.
Burmese Days, Mar 22; Rules of the Game and Preference Falsification in the Army
by Tim Liptrot — 2021-03-22T00:12:21.648Z

I have an exam coming up so I am a low on time. I am skipping the predictions this week and diving right into the revolutionary situation.

What is winning?

The two factions here each seek control over the state. Control over the state has several components. It means the ability to use violence in the territory. It means the ability to command bureaucrats to carry out tasks. It means the ability to coerce the payment of taxes. It means control over the means to legitimate force and taxation. Those means of legitimation include judges, traditional leaders, media organs and religious leaders. There are others, but these are the most important.

In most modern societies a single faction or organization controls all of those powers. Split control over these powers is rare. In 1917 the Russian treasury refused to serve the Bolsheviks. On the 9th of Thermidor the Robbespiere faction controlled the prisons but the National Convention controlled the military. Controlling violence specialists is a powerful card, but no trump; in 1966 Mao transformed control over only the legitimation apparatus to complete dominance of the state. In each case the division of control was brief because the control of one state components makes it much easier to take others. Division of these powers is therefore unstable.

Currently the Hlaing regime has tight and uncontested control over the violence specialists. However, their command of the bureaucracy is weak and the legitimation apparatus is weak, restricted to hollow state media channels. The protestors, in contrast, have some fragmented control over the bureaucracy to launch a nationwide strike and

The conflict will end in one faction controlling the state, barring a civil war. For the military, that means using targeted sanctions and benefits to pick apart the strikers until they achieve some control over the bureaucracy. Crushing the protests is an important signal of strength, but not itself state power. For the revolutionaries, winning means compelling the regime to give up its control over the violence specialists and return to the barracks, likely in a negotiated settlement. They may also win if the regime slowly loses capacity to work together. Each side must strategically use its shattered pieces of the state to capture the others.

Review of events

The military evicted several hundred rail workers from a government housing uni...
Six-Door Cars
by jefftk — 2021-03-21T19:30:04.647Z

While at this point

my family has decided what
we want to do about cars, I'm still thinking about why the

market for moderately larger
cars is so awkward.

One issue with three-row vehicles is that the third row is hard to get
into. If that row is rear-facing, like in a traditional station wagon,
you get in through the rear door. If it is front facing, you either
remove one of the seats from the middle row, or have a way to slide or
fold them out of the way. This is awkward for the same reason that
the rear seats in a two door car are awkward, and there we fix it by
adding a second row of doors. Why don't we see six door cars?

Car manufacturers have made some:

Mercedes-Benz w124
six-door, source

Checker Aerobus, also
produced with eight doors

You also see aftermarket conversions:

These are mostly very long cars, but it seems to me like if a
manufacturer wanted to make a six door with a more restrained length
they could. For example, a Honda Fit is 161" and a typical rear seat
pitch is 35", so a hypothetical six-door version would be something
like 196". This is long, but still well within the standard range:
it's 6" shorter than the Toyota Sienna and 8" shorter than the Honda
Odyssey or Chrysler Pacifica.

Overall, it looks like this is something that can be done, has been
done, and hasn't been popular. Maybe I'm weird in liking the six
door look?

Comment via: facebook
Demand offsetting
by paulfchristiano — 2021-03-21T18:20:05.090Z

For the last few years I’ve been avoiding factory farmed eggs because I think they involve a lot of unnecessary suffering. I’m hesitant to be part of that even if it’s not a big deal on utilitarian grounds. This is a pain since factory-farmed eggs are used all over the place (e.g. in ice cream, pastries, pasta…). I’d prefer just spend a bit of money and not think too much about what I eat.

In this post I’ll describe a possible offsetting strategy that I think is unusually robust and should be satisfying for many moral perspectives. The same proposal would also apply to many other animal products and potentially to the environmental impacts of consumption.

Proposal

I think it’s possible to produce humane eggs where hens have positive lives and nothing horrifying happens to anyone. So my ideal would be to buy and use humane eggs. But this is tough since most of the time I’m eating eggs that someone else used as an ingredient (and even when I’m using them myself acquiring really humane eggs is kind of a pain).

So here’s an alternative that seems easier and just as good:

Some people raise humane eggs.
They sell these on the wholesale market as if they were totally normal eggs.
An inspector verifies that hens are treated extremely well and that they have sold N eggs on the wholesale market.
The inspector issues N “humane egg” certificates to the producer.
The producer sells these certificates in an online marketplace in order to cover the extra costs of humane eggs.
Whenever I eat an egg, I buy a humane egg certificate to go with it.

Analysis

If I buy an egg and a humane egg certificate, what is the net effect on the world?

Buying the egg increased demand for eggs. If I hadn’t also bought a certificate, that would indirectly cause someone to make one more factory-farmed egg.

Buying the positive-welfare certificate means that someone sold a wholesale egg on my behalf and increased the supply of eggs. If I hadn’t also bought an egg, that would indirectly cause someone to make one less factory-farmed egg.

So my net effect on factory farmed eggs is zero. It’s as if I was making my own positive-welfare egg and eating it, with no effect on how many factory-farmed eggs other people make or eat.

(In reality both of these actions will have other effects, e.g. causing other people to eat more or fewer eggs, but I think they still cancel out perfectly.)

This is an unusu...
A parable of brightspots and blindspots
by Remmelt — 2021-03-21T18:18:51.531Z

Cross-posted from Effective Altruism Forum.
For a concise summary of rationality and EA blindspots, read this post instead

I take the liberty to dramatically sketch a metaphor for early pioneers in the landscape. Then, I suggest quick checks for current charity managers in effective altruism.

This post started as a script – a friend said my real-life conference talk had a way bigger impact than reading the story again in his own voice. Listen here to the narrated story.

Here, take this narrative device. Strap on this head-mounted interface. Peer into virtual reality lenses as I guide you on a quest.

Enter a vivid simulation – of you and of the people around you.

You were a pioneer. You surveyed dark, foggy frontiers with your small community.

Eventually, you settled in a field. You gathered with like-minded comrades close to you. You set up your group’s headquarters, and fenced off your scope.

You waved off outsiders: ‘You no longer need to map here, we’ve got this area covered!’

But you haven’t got it covered. Birds of a feather flock together. So do you.

Together, your ingroup has a style of surveying the territory:

You filter observations, as you build up a map of the uncertain and ambiguous environment you’re part of.
You frame your map around waypoints relevant to you, of an environment more complex than just you. You may spot what you can gain from this land, yet miss all sorts of dangers lurking in between.

There’s so much to map and only so much you can process.

You point your headlight into the fog.

Point its bright spot nearby to observe concrete details up close. Or point it far away to observe the general outlines ahead.

But you can’t do both.

For wherever you focus your brightspot is enveloped by darkness – your blindspot.

This is where it gets tricky. Your group spots a problem. You’re all pumped to solve it, and make a positive impact. Together, you set out a target for solving your problem.

But you are interacting through a vast, informationally-complex landscape.

What if before you reach your target... you hit a trap instead?
What if you miss some crucial consideration... that was lurking in your blindspot?

You will be shocked to realise you had a massive negative impact or — at least — are now nowhere near where you expected to be.

The lesson is clear: Find a group whose brightspot is in your blindspot.
Talk to the other group, ...
Process Orientation
by lincolnquirk — 2021-03-21T15:21:32.085Z

There's a certain type of workplace where the only thing that matters is that you, as an individual, produce results that are valuable to the company. Netflix is famously like this (or claims to be); here's a quote from their culture page:

Succeeding on a dream team is about being effective, not about working hard. Sustained “B” performance, despite an “A” for effort, gets a respectful severance package. Sustained “A” performance, even with a modest level of effort, gets rewarded. Of course, to be great, most of us have to put in considerable effort, but hard work and long hours is not how we measure or talk about a person’s contribution.

There are a lot of ways this can be implemented. The "piece work" version (pay the worker a fixed wage per item produced) is traditional for manufacturing jobs. But for intellectual work, it is far harder to measure results on an individual basis. Let's take software engineering as an example. Maybe your boss wants you to build some features -- those features being shipped would be the result. But there are a lot of different ways you could build the features -- you could cut corners and introduce a lot of tech debt, or you could polish them for a long time. How is the boss going to evaluate your results? They would need to specify ahead of time the expected quality of the work, but specifying it precisely enough is about equivalent to doing the job -- so that won't work.

Isaac Lyman at the Stack Overflow blog has a piece about measuring developer productivity. He argues that "interdependencies and nuances of individual work are too complex to be measured by an outside observer," concluding that the only good way to do it is to measure it at the team level -- "does this team consistently produce useful software on a timescale of weeks to months?"

I suspect that of all their talk about measuring results, Netflix is probably (at the individual level) doing something more like what Lyman suggests -- using fuzzier methods to evaluate individual performance to decide whether to fire someone. (If anyone reading this works at Netflix, I would like to know whether I am right about this.)

Let's take a look at individual-evaluation methods. How should we decide if an individual is doing their part on a software engineering team? We can't measure something like hours worked, or lines of code committed; these measurements create bad in...
Gossip
by Yoav Ravid — 2021-03-21T13:36:34.693Z

Epistemic status: an attempt to model gossip, not based on research. confidant that this describes real dynamics, though there's room for refinement.

Gossip requires at least 3 parties - the Gossiper, the Target (who's being gossiped about), and the Recipient (who's being gossiped to).

The two basic questions we can ask about each party are:

How does gossip benefit them?
How does gossip harm them?

Suppose Alice gossips to Bob about Carol.

There are a few common ways of looking at this interaction.

Alice and Bob gossip because it's fun and interesting
Alice is gossiping to increase her own status or decrease Carol's status
Alice is gossiping to inform and benefit Bob
Whatever reason Alice is gossiping, she's actually hurting her status in the eyes of Bob and he will not trust her to protect his privacy.

The first one is a curiosity stopper; though true, it doesn't tell us much.

The other options also have truth to them, but none tell the whole story.

Harms and Benefits From Gossip

Here I'll explore a bunch of different gossip situations that either harm or benefit one of the parties, after that I'll explore what information can be gleaned from gossip, and what might be some good gossip norms.

The Gossiped

Harms

Gossip inherently breaches the privacy of the gossiped. When being gossiped about you have no control on what's being shared. Even if the gossiper has good intentions, they might share something which they didn't know they should have kept secret.

The typical case of gossip is when Carol did something bad, and Alice reveals that information to Bob. Now Bob might stay away from Carol or actively hurt her in some way (such as by further spreading the gossip).

Though I can think of some atypical cases.

If Carol just had something really good happen to her, like being accepted to a job she wanted, she might want to tell Bob herself, and would be upset if Alice already told him.

This can also happen with a negative thing Carol meant to tell Bob about. Suppose Carol broke a promise to Bob. She can make it up to him in part just by being honest and telling him. But if Alice tells Bob before Carol gets to, Bob might accuse Carol of not telling him. Now Carol can only claim that she meant to tell him, but she lost the chance to show it.

Benefits

The less discussed case is gossip about positive traits. When Alice tells Bob how generous and happy to help Carol is, ...