trentmkelly/LessWrong-43k huggingface dataset:
Guidelines for cold messaging people
by Severin T. Seehrich — 2022-01-17T12:07:48.214Z

I just received a random email from a stranger in which they ask me 5-10 hard-to-answer questions on effective altruism, emotional health, and moral cluelessness that bother them.

Reflecting on this, I came up with a list of eight rules of thumb you might want to consider when you cold message someone. You might not want to address all of them in every message (because that contradicts no. 8), but they may be a good pointer towards what would make people keener to respond:

1. Add a one-sentence introduction so they know how to interact with you.

2. Make clear how you got hold of their email address.

3. Make clear why they may want to talk to you.

4. Make clear why you think they are the right person to solve your problem.

5. Courteously emphasize that you don't mind if they are too busy to respond.

6. Start or end with a note of appreciation or gratitude.

7. Ask for their permission *before* you dump a long list of questions or any other kind of work on them. To minimize friction and time investment for them, add your list of questions to the first email right away. However, make clear that it is okay and encouraged for them to say *no*. For example, lead with a sentence like "Here are some questions I'm trying to answer for myself. Of course, no worries if you are too busy to help me with that."

8. Keep the message as short and clear as possible.

Let me know if there's anything you find valuable that I overlooked.
Being the Hero is hard with the void
by Johannes C. Mayer — 2022-01-17T11:27:31.020Z

Here is some prose that gets more and more metaphorical. Don't take it too literally.

What is a hero? I don't know. Maybe it is a matter of just behaving optimally towards some objective. Maybe, to call someone a hero you must share their objective. Maybe it is simply about trying your very best. Even in the face of great difficulty.

Me

I know that I have not been doing any of these. Maybe I have been getting closer than most people, but not nearly close enough. And I do think that I got better at doing this over time. So that means that there is room for me to improve.

Many factors seem to be outside of my control. Like when I just feel bad, and there is no apparent reason why this is so. In that situation or before, I need to figure out why I feel bad and then circumvent it, such that I can focus on increasing productivity towards my actual objective.

There is something wrong inside me. Why am I not excited to work on math and AIA as much as possible. It makes no sense on a logical level. But of course, my brain is far from trying to do the optimal thing. All my feelings seem to be governed by some heuristics that developed in another environment than the one I am in now. And they often conflict with what is best to do.

I should try to eradicate them. But it seems likely that I will not be able to surmount them all directly (though it might be possible using meditation). But for all of them, there is bound to be a way to work around them. I just need to find the approximately best way.

What is it like

Being a hero is not easy. Or at least there is one way to define "hero", which is all about doing what is right but difficult. What is courage without fear? What is being a hero without achieving difficult things? What is being a hero without achieving things that are difficult for you?

It certainly does not seem to be required to do what is best. Maybe all the easy things are the most important thing to do. But it seems that the main obstacle in your way to do the best thing will very often be difficult. That is just the curse of distributional shift.

And the hardest and most important thing you can do is to make the important things as easy to do for you as you can.

But you need to recognize that there will never come a time when doing the right thing will just always be easy. Even if right now you feel like you can do what you think is best, without any effort. ...
A decision tree for vaccinating children against Covid-19, or how to wisely make a monumental decision
by methree — 2022-01-17T11:23:45.785Z

So many things in your life
That you're bound to regret
Why didn't I do that?
Why didn't I do this?
So many chances you lost
That you'll never forget

—Meatloaf

(En español 🇲🇽: https://tinyurl.com/decisiones-monumentales)

I'm going to help you make difficult choices in the face of strong emotions and uncertainty. Decisions like whether or not you should vaccinate your child against Covid-19. I'm not going to make the choice for you but what I'll teach you will even come in handy when you're deciding which place to live in or whether to buy that used car. I call these decisions monumental because (a) undoing them is either impossible or very costly, and (b) you don't make them often. The potential for regret is enormous; they are decisions you don't want to get wrong. A look at the divorce rate, however, shows that we are often not great at making monumental decisions. Today, you are advised to vaccinate your child against Covid-19 (or get vaccinated yourself) and you think, this is a monumental decision, it has a lot of potential for regret! And then you make your first mistake.

Daniel Kahneman, who has been given a Nobel prize, studied decision-making for many, many years, in a controlled environment, and noticed everyone has two very different ways of making choices: a very fast, gut-feeling, autonomous way that is very good at, say, helping you drive a scooter through Mumbai; and a slow, reason-based way that is very good at helping you solve "computationally difficult" problems, like math.[1]

Here's a really cool experiment to see the latter in action: take the person nearest to you and ask them to look you directly in the eyes. Then, tell them to multiply 213 by 4 without looking away. Their pupils will immediately dilate. This is a sign that they have engaged the decision-making mechanism that is best suited for complex problem solving. Yes, it's kinda weird that it is physically observable.

Both ways of making decisions, which Kahneman calls "system 1" and "system 2", are essential for us. If an airplane loses control during takeoff and the pilots sit there, discussing whether it was a mechanical problem, the software had a bug or a goose got sucked into the engine, because knowing would help them best tackle the problem, it could cost lives. They need to react instantly, which is why they spend years training their "syst...
Challenges with Breaking into MIRI-Style Research
by Chris_Leong — 2022-01-17T09:23:34.468Z

Trying to break into MIRI-style[1] research seems to be much, much harder than trying to break into ML-style safety research. This is worrying if you believe this research to be important[2]. I'll examine two kinds of causes: those which come from MIRI-style research being a niche area and those which go beyond this:

Challenges beyond MIRI-style research being a niche area:

MIRI doesn’t seem to be running internships[3] or running their AI safety for computer scientists workshops
If you try to break into ML-style Safety and fail, you can always be reuse at least part of what you've learned to obtain a highly-compensated role in industry. Agent foundations knowledge is highly niche and unlikely to be used elsewhere.
You can park in a standard industry job for a while in order to earn career capital for ML-style safety. Not so for MIRI-style research.
MIRI publishes a lot less material these days. I support this decision I support as infohazards deserve to be taken seriously, but it also makes it harder to contribute.
There are well-crafted materials for learning a lot of the prerequisites for ML-style safety.
There seems to be a natural pathway of studying a masters then pursuing a PhD to break into ML-style safety. There are a large number of scholarships available and many countries offer loans or income support.
The above opportunities mean that there are more ways to gauge fit for ML-style safety research.
There's no equivalent to submitting a paper[4]. If a paper passes review, then it gains a certain level of credibility. There are upvotes, but this signaling mechanism is more distorted by popularity or accessibility. Further, unlike writing an academic paper, writing alignment forum posts won't provide credibility outside of the field.

Challenges that come from being a niche area

I think this probably should be a niche area. It would be a bit strange if foundations work were the majority of the research. Nonetheless, it's worth highlighting some of the implications:

General AI safety programs and support - ie. AI Safety Fundamentals Course, AI Safety Support, AI Safety Camp, Alignment Newsletter, ect. are naturally going to strongly focus on ML-style research and might not even have the capability to vet MIRI-style research.
It is much harder to find people with similar interests to collaborate with or mentor you. Compare to how easy it is to meet a bunch

...
A connectomic study of a petascale fragment of human cerebral cortex
by PointlessOne — 2022-01-17T08:49:10.165Z
Link: https://vcg.seas.harvard.edu/publications/a-connectomic-study-of-a-petascale-fragment-of-human-cerebral-cortex

Abstract

We acquired a rapidly preserved human surgical sample from the temporal lobe of the cerebral cortex. We stained a 1 mm3 volume with heavy metals, embedded it in resin, cut more than 5000 slices at ~30 nm and imaged these sections using a high-speed multibeam scanning electron microscope. We used computational methods to render the three-dimensional structure of 50,000 cells, hundreds of millions of neurites and 130 million synaptic connections. The 1.4 petabyte electron microscopy volume, the segmented cells, cell parts, blood vessels, myelin, inhibitory and excitatory synapses, and 100 manually proofread cells are available to peruse online. Despite the incompleteness of the automated segmentation caused by split and merge errors, many interesting features were evident. Glia outnumbered neurons 2:1 and oligodendrocytes were the most common cell type in the volume. The E:I balance of neurons was 69:31%, as was the ratio of excitatory versus inhibitory synapses in the volume. The E:I ratio of synapses was significantly higher on pyramidal neurons than inhibitory interneurons. We found that deep layer excitatory cell types can be classified into subsets based on structural and connectivity differences, that chandelier interneurons not only innervate excitatory neuron initial segments as previously described, but also each other’s initial segments, and that among the thousands of weak connections established on each neuron, there exist rarer highly powerful axonal inputs that establish multi-synaptic contacts (up to ~20 synapses) with target neurons. Our analysis indicates that these strong inputs are specific, and allow small numbers of axons to have an outsized role in the activity of some of their postsynaptic partners.
Entropy isn't sufficient to measure password strength
by benwr — 2022-01-17T06:41:18.073Z
Link: https://www.benwr.net/2022/01/16/password_entropy.html

If you know a lot about passwords, you probably know that it’s good to choose a password from a distribution Wwith a lot of “entropy”. The (Shannon) entropy of a distribution is defined as:

H(W)=∑wP(w)log1P(w)

This definition is an answer to the question “how much information is encoded in a chosen password, on average?” This is also the approximate number of bits we need to specify which password we picked. An adversary would need to try at least 2H(W)−2+1 passwords in order to guess yours, on average.[1]

This is typically how people think about choosing strong passwords, and it works well in the case where we’re choosing among equally likely passwords. But it breaks horribly when we allow our distribution to be non-uniform. Because choosing good passwords is about memorableness as well as sheer strength, we might sometimes want to use complicated password selection schemes that don’t necessarily give uniformly distributed selections, and Shannon entropy can really screw up here.

A comically bad password selection scheme

Consider the following password selection algorithm:

Roll a 20-sided die.
If the die comes up 20, choose a random password from n possibilities.
Otherwise, choose the password password.

Of course, this is a terrible password selection strategy. 19 times out of 20, your password will be the very first one the attacker tries, if they know your strategy. But the entropy here doesn’t reflect that - for example, if we pick a random 1000-character lowercase password when we get a 20, this has an overall entropy of about 235 bits. Entropy tells us that we have an extremely strong password, since on average it will take a long time to crack the password. But of course in the typical case, it will take no time at all.

Clearly something is wrong here. How can we fix it?

Min-entropy

The first thing to notice is that, as we’ve already implied, we’re engaged in a game of strategy with any would-be password crackers. We pick some strategy for choosing passwords, and then our opponent picks a strategy for guessing them. Our opponent’s best response is going to be to try the most likely passwords first.

So how should we really measure our password selection strategies? One way would be to use the information content of the most likely password, instead of the average information content:

Hmin(W)=minw∈Wlog1P(w)

This is called the min-entropy, and ...
Teaching Street Crossing
by jefftk — 2022-01-16T20:20:05.547Z

When Lily was about four we started practicing crossing streets. There
were two small streets on the way to the

farther park, and Lily was very
excited about the idea of crossing them by herself. After talking
through the process (look both ways, then if you don't see any cars
coming across slowly and steadily while continuing to look both ways)
I would stand with her at the corner and say "cross when you think it
safe", watching for cars myself so I could stop her if she stepped out
when she shouldn't.

Unfortunately, she took looking both ways to be about pointing her
head, not about actually looking. She would wiggle her head as fast
as possible, trying to maximize how many times she could look left and
right, much faster than she could actually tell what she was seeing.
(Lily: "when I first started learning how to cross streets I would
turn my head like a maniac.") This was not useful practice, and after
trying and failing to show her how to look I decided to wait until she
was older. After a while I stopped bringing up the idea, she stopped
bringing it up, and this was on hold for a couple years.

This summer, when Lily was seven and Anna was five, we tried again.
This time it went much better: they would both actually look for cars,
and would see them when they were coming. Pretty quickly we got to
where we would do this with most streets: I would ask whether they
wanted to practice, and they would cross on their own with me checking
that they were doing it right. On streets with parked cars, I taught
them to slow down and look again after they got to the "car line".
This is the point about 6 feet from the curb where sightlines are no
longer blocked by parked cars.

Over the next few weeks with lots of walks together they were both,
but especially Lily, reliably doing it right. I started giving them
permission when they had gone ahead of me, to where I could see them
and tell whether they were looking properly, but couldn't see
everything they could see. By September (7y5m) Lily was almost ready to do it on her own, and in October she was ready for a set
of small streets near our house. We used walkie-talkies, and she would check in
with us over the radio before crossing and after to let me know it
went ok. Lily could walk places herself, and could bring Anna if they
both wanted that.

The biggest street they wanted to cross...
Notes on Rationality
by David Gross — 2022-01-16T19:05:28.964Z

“The knowledge of the theory of logic has no tendency whatever to make men good reasoners.” —Thomas Macaulay[1]

This post examines the virtue of rationality. I’ve been dreading this one.

I have been writing a sequence of posts about virtues, and strategies for strengthening them in ourselves. My notes on notes on virtues post explains in more detail what I’m trying to accomplish and why.

I’m following the lead of virtue-oriented traditions and philosophers of the past as I choose which virtues to highlight. Many of these traditions included rationality (or some closely-related virtue like reason or love-of-truth). I agree that it’s a virtue (or at least that it can be useful to consider rationality in the form of a virtue) and a valuable one. But what can I add to the rich discussion about rationality that already exists here on LessWrong?

I decided that the best way I could advance the conversation would be by, instead of discussing rationality as a virtue, discussing rationality as a virtue.

What is a virtue?

“He who knows the truth is not equal to him who loves it, and he who loves it is not equal to him who delights in it.” ―Confucius[2]

I’m using the word “virtue” in the way I’m most familiar with it from the virtue ethics tradition.

“Virtue” can have implications that I don’t intend: In popular use, virtue is morally correct behavior, something urged on by the angel on your shoulder, in opposition to a devil’s temptingly delicious vice. You practice this sort of virtue because you’re s’posed to, or because God is watching, or because it is your unfortunate duty. Popularly, “virtue” can be prim, naïve, old-fashioned, maybe a little ostentatiously holier-than-thou. It also is more often singular than plural (do you exhibit virtue?) or is conceived of as something of a continuum (a person might be more-or-less virtuous).

By contrast, in the virtue ethics tradition I’m most familiar with, “virtues” are a variety of character traits. Those character traits that tend to help you to succeed at living an excellent human life (or that are themselves ways of living excellently) are virtues; those that interfere with this are vices; any others are just part of life’s rich variety. To consider rationality as a virtue is to consider it as one of the human excellences that individuals can strive to practice characteristically.

Other things you can probably say about virtues:

T

...
New Year Review Resources
by lynettebye — 2022-01-16T18:21:18.205Z

I’ve gathered together a handful of resources here on yearly reviews and setting new years goals. …Just in time for everyone to have wrapped up their new years reflections. Oh well, better late than never, right?

These aren’t particularly cohesive or polished. I recommend you skim through and take the most relevant ones for your situation.

Theory of change

If you’re not familiar with the concept, a theory of change is your best guess at the concrete steps required to cause the change you want to see in the world.

I have a pet theory that the key to becoming a high-impact EA is to develop a theory of change, use that to guide your efforts to build a better model of the topic, and then update your theory of change with your new information. Repeat, throw in some exploration, take care with epistemics, and you’re moving toward high-impact.

I don’t feel like restarting from scratch, so here are some previous resources on the topic:

Spreadsheet

template for making your own theory of change.

Post

on holding a theory of change loosely as a hypothesis that you test and iterate on.

Post

on why you should build models if you want to have a theory of change that actually works.

Post

with a good example of a theory of change from someone who already has a deep model of the topic.

Post

with a section of examples of theories of change.

One big caveat is that I don’t think my previous writing has emphasized enough how much iteration is necessary for good theories of change. Yes, you should be making theories of change right now. But your theories are going to be bad if you’re early in your career or you don’t know an area super well. Get more info and iterate. If this feels hard, reading about lean methodology might help. I liked The Lean Startup best so far. I recently heard good recommendations for The Phoenix Project, Working Backwards, and Sprint (I haven’t had a chance to check these out yet, but might update this later after I do.)

Check your trajectory

A yearly review is a good time to check that you’re on a good trajectory. (Think, “Where do I want to be in five years? Where am I going to be in five years based on my current trajectory?)

Note, I don’t think you necessarily need to spend five years. A lot of the time, you can aim for a much faster trajectory by cutting out all of the bullshit and using better learning methods. Perhaps ask...
Some thoughts on "The Nature of Counterfactuals"
by tailcalled — 2022-01-16T18:12:46.865Z

This is a response to the post The Nature of Counterfactuals by Chris_Leong. My post might not make much sense without having read Chris_Leong's post first.

I have been thinking a lot about causality. Since there is now a potential bounty on talking about causality, this seems like a good time to say some things.

TL;DR:

Counterfactuals are something we've made up, but causality isn't
Evolution "sniffs out" something resembling true counterfactuals by rolling out many organisms with slightly different DNA, thereby getting directly influenced by the causality of reality
If you select agents in a causal universe according to their ability to achieve certain consequences, as evolution does, then the optimal agents will use causal reasoning like Causal Decision Theory; presumably this is why humans do causal reasoning
We, for similar reasons, expect to have causal AI
Causality isn't just about decisions; it can cover many other things too

Causality is real, counterfactuals are not

Our universe appears to be a causal universe. By this, I mean that it appears to be characterized by having some particular 'starting state', and evolving forwards as a dynamical process according to some causal laws. However, actually showing this takes some steps.

Correlation is not causation, but before we go to the causal aspect, let's just consider the correlational aspect. Scientists have found that the universe to a good approximation follows physical laws that appear to be local[1]/continuous and translation-invariant[2] with respect to time and space. It also appears that the laws are deterministic, allowing the future to be predicted from the past.[3]

But just because we have descriptive laws does not mean that these laws are actually causal generators; after all, they're just correlations, so couldn't there just as well have been a common cause, or something else like that which could have produced these results?

No; this is where entropy comes in. The laws of physics are reversible, such that if you rewind the universe, the same laws will still apply.[4] Counterintuitively,[5] they are also measure-preserving, so that whenever we have some information about some specific time, such as how the state of the world is now, then we have equally much information about the past and future. However, in practice, the state of a system that is not continually replenished becomes le...