trentmkelly/LessWrong-43k huggingface dataset:
Self and No-Self
by Vaniver — 2019-12-29T06:15:50.192Z

Epistemic Status: sometimes you hear a platitude and it seems like nonsense, and then it clicks and you get it, but don’t have the ability to express it in a way that would help anyone who doesn’t already get it. Other times, it clicks and you have a mechanistic model that you think can explain it better than the platitude. This post feels something like 80% the first, 20% the second. It’s an analogy that crystallized for me recently, but ‘just an analogy.’

I view agency as integration of / balancing act between two ‘forces’. The first, which I’ll call ‘self’, is outward-flowing, preferences, morality, and particularism. The second, which I’ll call ‘no-self’, is inward-flowing, observations, system dynamics, and universalism. The labels come from meditative practices, and hopefully won’t cause too much of a namespace collision, but seem like the best labels of the thing to me at the moment.

The extremes, of course, are deficient. The person who is totally given over to their preferences hallucinates everything is already how they want it to be, and accomplishes nothing; the person who is totally given over to observing the unfolding of the universe also accomplishes nothing. Accomplishing something relies on observing how things are and what changes will make things different, and having a ‘something different’ to change towards.

The balancing act is complicated by the fact that the two have different types. It isn’t a battle between ‘good’ and ‘evil’, but instead a battle between ‘morality’ (a sense that it is better for things to go one way than another) and ‘amorality’ (a sense that there is one way that things will go, and ‘better’ has nothing to do with it). The fundamental element of

nihil supernum

, I think, is the visceral sense of the universe as no-self instead of self (in the sense of a loving God, or historical materialism, or so on); this is described in

Beyond the Reach of God

.

Well, not a ‘battle’. Those generally involve things of the same type; one army against another, one contestant against another, and generally they have a sole winner. This is closer to ‘design specifications’ meeting ‘engineering constraints’, where success looks like both being satisfied. One of the interesting features about balancing is that you can remain balanced while making both sides heavier; deepening in self often requires deepening in no-self, and vice versa. (Gai

...
Good goals for leveling up?
by wolverdude — 2019-12-29T05:25:09.661Z

I'm new to LessWrong and am trying to level up and internalize techniques to improve my life. I also want to increase my connection to the community, but that's less important for this question. I'm setting some goals for next year and was thinking about how to encode these meta-goals into SMARTer goals.

My question: what are some good goals for leveling up in a wholistic way? I don't just want to read things. I want to become conversant in them, and I want to put them into practice. I've written down some ideas below (I don't intend to do all of them; they're just ideas at this point). Feel free to critique them and/or ignore them and write your own.

Spend X hours per week interacting with people's Rationality posts on LessWrong and Facebook.

Spend X hours per week reading Rationality blogs, posts, and sequences.

Write X blog / LW post per month.

Learn X new Rationality technique per week and apply it to a problem I have (and post about it).

Read through X Sequence / Book

Again: As a newbie, what is a good set of goals to help me level up quickly and sustainably?

Edit: Based on the comments, I should specify that these would not be my only goals for the year. They would just be the ones that have to do with learning and applying Rationality techniques.
Programmers Should Plan For Lower Pay
by jefftk — 2019-12-29T03:20:01.343Z

Summary: we don't understand why programmers are paid so well. If
you're a programmer, there's enough of a chance that this is temporary
that it's worth explicitly planning for a future in which you're laid
off and unable to find similarly high-paying work.

Programmers are paid surprisingly well given how much work it is to
become one. Here's Dan Luu comparing it
to other high-paid careers:

If you look at law, you have to win the prestige lottery
and get into a top school, which will cost hundreds of thousands of
dollars. Then you have to win the grades lottery and get good enough
grades to get into a top firm. And then you have to continue winning
tournaments to avoid getting kicked out, which requires sacrificing
any semblance of a personal life. Consulting, investment banking,
etc., are similar. Compensation appears to be proportional to the
level of sacrifice (e.g., investment bankers are paid better, but work
even longer hours than lawyers).

Medicine seems to be a bit better from the sacrifice standpoint
because there's a cartel which limits entry into the field, but the
combination of medical school and residency is still incredibly brutal
compared to most jobs at places like Facebook and Google.

My sister is currently a second-year medical resident, and "incredibly
brutal compared..." feels like a understatement to me. She works 80hr
weeks, often nights, helping people with deeply personal and painful
issues that are hard to leave behind when you go home. This is after
four years in medical school, with still at least a year to go before
starting to earn doctor-level money. When I compare it to how I
started programming right out of college, making more
money for 40hr weeks and no on-call, I feel embarrassed.

What makes me nervous, though, is that we don't really understand why
programmers are paid this well, and especially why this has persisted.
People have a bunch of guesses:

Demand: as software eats the world there are far more profitable
things for programmers to do than people to do them.

Supply: it's hard to train people to be programmers, fewer
people are suited for it than expected, and bootcamps haven't worked
out as well as we'd hoped.

Startups: big companies need to compete with programmers
choosing to go off and start companies, which is harder to do in many
fields.

Novelty: the field is relatively n

...
What are the most exciting developments from non-Europe and/or non-Northern-Hemisphere?
by Raemon — 2019-12-29T01:30:05.246Z

My vague sense of "macrohistory of the world" is fairly Euro-centric, and Northern Hemisphere Centric. And I'm not sure how much of that is to with "well, for good or for ill Europe kinda took over the world and everything is entangled with that" and how much of it is "I grew up in a European-descended country which wrote European-descended history books".

At some point I'd be interested in figuring out how to build a proper sense of how history all fits together, but for now, just wanna get bombarded with cool stuff.

What are the most important bits of history, inventions, and developments from non-Europe and non-Northern history? Ideally, stuff that ended up playing a longterm role in humanity's capabilities, but random stuff like "The Incas independently invented writing" would be good too.
Safe exploration and corrigibility
by evhub — 2019-12-28T23:12:16.585Z

EDIT: I now think this post is somewhat confusing and would recommend starting with my more recent post “Exploring safe exploration.”

Balancing exploration and exploitation is a classic problem in reinforcement learning. Historically—with approaches such as deep Q learning, for example—exploration is done explicitly via a rule such as ϵ-greedy exploration or Boltzmann exploration. With more modern approaches, however—especially policy gradient approaches like PPO that aren't amenable to something like Boltzmann exploration—the exploration is instead entirely learned, encouraged by some sort of extra term in the loss to implicitly encourage exploratory behavior. This is usually an entropy term, though other more advanced approaches have also been proposed, such as random network distillation in which the agent learns to explore states for which it would have a hard time predicting the output of a random neural network, an approach which was able to set a state of the art on Montezuma's Revenge, a notoriously difficult Atari environment because of how much exploration it requires.

This move to learned exploration has a very interesting and important consequence, however, which is that the safe exploration problem for learned exploration becomes very different. Making ϵ-greedy exploration safe is in some sense quite easy, since the way it explores is totally random. If you assume that the policy without exploration is safe, then for ϵ-greedy exploration to be safe on average, it just needs to be the case that the environment is safe on average, which is just a standard engineering question. With learned exploration, however, this becomes much more complicated—there's no longer a nice “if the non-exploratory policy is safe” assumption that can be used to cleanly subdivide the overall problem of off-distribution safety, since it's just a single, learned policy doing both exploration and exploitation.

First, though, an aside: why is learned exploration so much better? I think the answer lies primarily in the following observation: for most problems, exploration is an instrumental goal, not a terminal one, which means that to do exploration “right” you have to do it in a way which is cognizant of the objective you're trying to optimize for. Boltzmann exploration is better than ϵ-greedy exploration because its exploration is guided by its exploitation—but it's still es

...
What is your recommended statistics textbook for a beginner?
by Rudi C — 2019-12-28T21:19:38.200Z

I have done a basic undergrad probability course (which used Ghahramani’s book.). My own university uses Wonnacott for statistics. (Which seems a tad dated.)
What’s your recommendation? I want to use the statistics eventually for ML.
Please state your reasons for the recommendation also.
Stupidity and Dishonesty Explain Each Other Away
by Zack_M_Davis — 2019-12-28T19:21:52.198Z

The explaining-away effect (or, collider bias; or, Berkson's paradox) is a statistical phenomenon in which statistically independent causes with a common effect become anticorrelated when conditioning on the effect.

In the language of d-separation, if you have a causal graph X → Z ← Y, then conditioning on Z unblocks the path between X and Y.

Daphne Koller and Nir Friedman give an example of reasoning about disease etiology: if you have a sore throat and cough, and aren't sure whether you have the flu or mono, you should be relieved to find out it's "just" a flu, because that decreases the probability that you have mono. You could be inflected with both the influenza and mononucleosis viruses, but if the flu is completely sufficient to explain your symptoms, there's no additional reason to expect mono.[1]

Judea Pearl gives an example of reasoning about a burglar alarm: if your neighbor calls you at your dayjob to tell you that your burglar alarm went off, it could be because of a burglary, or it could have been a false-positive due to a small earthquake. There could have been both an earthquake and a burglary, but if you get news of an earthquake, you'll stop worrying so much that your stuff got stolen, because the earthquake alone was sufficient to explain the alarm.[2]

Here's another example: if someone you're arguing with is wrong, it could be either because they're just too stupid to get the right answer, or it could be because they're being dishonest—or some combintation of the two, but more of one means that less of the other is required to explain the observation of the person being wrong. As a causal graph—[3]

Notably, the decomposition still works whether you count subconscious motivated reasoning as "stupidity" or "dishonesty". (Needless to say, it's also symmetrical across persons—if you're wrong, it could be because you're stupid or are being dishonest.)

Daphne Koller and Nier Friedman, Probabilistic Graphical Models: Principles and Techniques, §3.2.1.2 "Reasoning Patterns" ↩︎

Judea Pearl, Probabilistic Reasoning in Intelligent Systems, §2.2.4 "Multiple Causes and 'Explaining Away'" ↩︎

Thanks to Daniel Kumor for example LATEX code for causal graphs. ↩︎
Measuring Epistemic Rationality
by edoarad — 2019-12-28T18:46:24.041Z

Collecting some previous discussions on the matter, trying to reach a conclusion that would benefit me in my path to be more rational. Written with haste to serve as a reference for myself, possibly useful for you, and to be more accountable to be methodological in my rationality self-improvement.

1. The Rationality Checklist from CFAR has many rationality habits listed. I can go through it once in a while to feel progress, or focus on one that I feel I need to improve more and incorporate to my daily routine. (say- " I notice my mind flinching away from a thought; and when I notice, I flag that area as requiring more deliberate exploration. "). I'm not sure that I trust myself to reflect accurately. This is also "Not meant to be a way to get a ‘how rational are you?’ score, but, rather, to help you notice specific habits you might want to develop. "

2. Aliveness in Training - Gives the analogy of a Martial Artist, that improves by working against an opponent. I have three thoughts -

i. Work with a mentor, seriously consider criticism from friends, write down more of my beliefs publicly.

ii. Measure my research success. (Correlated with lots of other things, and unclear, but that's actually what I care about)

iii. Make lot's of predictions

3. In Swimming Upstream, TurnTrout gives his personal experience with instrumental rationality. Parts of this story makes me think that I can measure how seriously I take my beliefs by collecting my own decisions and counting how many of them were due to newly generated beliefs. Not sure about that, because I may be biased in the wrong way here. I think that a fine idea here is to track all decisions made, and their reasons, and afterwards I'll understand how to analyze them.

4. It turns out that there is a Rationality Quotient, by Keith Stanovich. I did not find the test, but I feel aversion toward it as a method to improve rationality. I think it's because I visualize myself Goodharting to it. It might be very useful as a one-time diagnostic tool.

In total, I think that I should

be systematic in analyzing my shortcomings
work on rationality habits like any other skill/habit
use my intuition to guide me to the most important skills I need to develop
be more clear about my reasoning, and seek criticism and help when communicating
track predictions and world view be

...
An Emergency Fund for Effective Altruists
by bice — 2019-12-28T18:02:38.449Z

One of the things stopping people from donating a substantial part of their income to charity is the risk of a future financial downfall, which can be anything from unemployment to illness. Even in a rich country with a solid social safety net, the idea of having no financial buffer at all seems daunting. However, if effective altruists precommitted to helping other effective altruists in need, they could all donate more money, as the risk would be spread.

This sounds a lot like insurance, and effective altruists could indeed insure themselves against a myriad of risks. Unfortunately, insurers will reject claims whenever possible, some risks cannot be insured, and insurance requires monthly payment, which requires a stable income. In other words, insurance is no replacement for a personal financial buffer. Insurance is useful to mitigate specific high-cost risks, but that's it.

(Additionally, insurance introduces the overhead of an insurance company. When I asked for a ballpark estimate at the insurer I interned at, I was told that at most one-third of all premium money is put back into payment of legitimate claims.)

I propose an intermediary charity that acts as an emergency fund. If a donor spends $100 through this charity, this intermediary charity directly forwards $80 of incoming donations to charities of the donor's choice. The remaining $20 is put in an emergency fund. In situations where the donor would otherwise make use of their financial buffer, they can take up to $95 from the emergency fund.

Recoupments would not need to be subjected to the same thorough vetting as insurance claims, since donors cannot recoup as much as they originally spent, so there is little opportunity for fraud.

The example above works under the assumption that, on average, donors need 20% of their donations back. It might make sense to create different funds for donors with different risk profiles: if a subset of donors think there's a 50% chance they'd need donated money back, they could donate to a fund which puts half of their donation in the emergency fund.

I would not recommend relying solely on the emergency fund as a replacement for a personal financial buffer, because it can be drained in times of national and global crisis. However, I believe that the mere existence of the fund will move people to donate more, not just by donating part of their emergency fund, but also ...
How was your decade?
by namespace — 2019-12-28T17:14:41.177Z
Link: https://www.youtube.com/watch?v=DBzuYNK95sM

Hi everybody,

With the New Year coming up and next year being 2020, I think it's worth reflecting on the last 10 years. As you probably remember the last Sequences post came out towards the end of the decade. How have things gone for you since then? If you're willing to share, here's some things to consider. You can answer as much or as little of this as you want, don't let it scare you off from posting:

(EDIT: This prompt seems to reliably cause people to write stuff that's uncomfortable to post as a comment. If you find this happens to you, go ahead and just excerpt a part which is comfortable to post as a comment and post that instead.)

Before & After

How were things for you at the start and end of the decade? Some categories to ponder:

Philosophy

Skills

Career

Lifestyle

Between

How did things change over time? What was your progression through the 2010's?

What made you say "oops" and "duh"? What did you update on?

How have your preferences changed over time? (e.g. Music, Food)

Did you break any promises? Did you keep some difficult ones?

Do you have any new habits? Did you break some?

Experiences

How did you experience the last 10 years? What proportion of the time was spent on good experiences vs. bad? Any notable highlights?

Worth Noting

When did you first read The Sequences, if ever?

Are you overall satisfied or dissatisfied with how the decade went?

Is there any advice you'd give to your past self? How do you think they'd react to it?
(Just so we don't have to read it 30 times, yes, you would buy bitcoin)

What do you think the zeitgeist of this decade was? What aspects and events do you think you'll remember the 2010's for? What do you think society at large will remember the 2010's for?

And of course if there's anything else you'd like to share, feel free. :)
Moloch Hasn’t Won
by Zvi — 2019-12-28T16:30:00.947Z

This post begins the Immoral Mazes sequence. See introduction for an overview of the plan. Before we get to the mazes, we need some background first.

Meditations on Moloch

Consider Scott Alexander’s Meditations on Moloch. I will summarize here.

Therein lie fourteen scenarios where participants can be caught in bad equilibria.

In an iterated prisoner’s dilemma, two players keep playing defect.
In a dollar auction, participants massively overpay.
A group of fisherman fail to coordinate on using filters that efficiently benefit the group, because they can’t punish those who don’t profi by not using the filters.
Rats are caught in a permanent Malthusian trap where only those who do nothing but compete and consume survive. All others are outcompeted.
Capitalists serve a perfectly competitive market, and cannot pay a living wage.
The tying of all good schools to ownership of land causes families to work two jobs whose incomes are then captured by the owners of land.
Farmers outcompeted foragers despite this perhaps making everyone’s life worse for the first few thousand years.
Si Vis Pacem, Para Bellum: If you want peace, prepare for war. So we do.
Cancer cells focus on replication, multiply and kill off the host.
Local governments compete to become more competitive and offer bigger bribes of money and easy regulation in order to lure businesses.
Our education system is a giant signaling competition for prestige.
Science doesn’t follow proper statistical and other research procedures, resulting in findings that mostly aren’t real.
Governments hand out massive corporate welfare.
Have you seen Congress?

Scott differentiates the first ten scenarios, where he says that perfect competition* wipes out all value, to the later four, where imperfect competition only wipes out most of the potential value.

He offers four potential ways out, which I believe to be an incomplete list:

Excess resources allow a temporary respite. We live in the dream time.
Physical limitations where the horrible thing isn’t actually efficient. He gives the example of slavery, where treating your slaves relatively well is the best way to get them to produce, and treating them horribly as in the antebellum South is so much worse that it needs to be enforced via government coordination or it will die out.
The things being maximized for in competitions are often nice things we care about, so at least we get th

...
Out in the Great Northwest
by jefftk — 2019-12-28T03:40:01.697Z

One of the songs my family would sing growing up was "Out in the Great
Northwest" (

mp3,

2011 recording). For me, the point of
the song was the verse:

A Scotsman went out there to live, he called his house a "hoos".

They showed him a great big animal, they said it was a moose.

Out in the great Northwest! Way out in the great Northwest!

The Scotsman said, "Now, what the deuce! You say you call that thing a moose!

I'd hate to see a rrrat get loose!" Out in the great Northwest.

My cousin Valerie wrote up a score (

pdf), and the full
lyrics of our version are:

I'm going way out West to where the buffalo used to roam.

I'll buy a big ten-gallon hat and build myself a home.

Out in the great Northwest! Way out in the great Northwest!

For men are men out there I swear, they wrestle with a grizzly bear.

Punch his nose and comb his hair, out in the great Northwest.

They have a brand of climate there, that puts hair on your chest.

You take a great big breath and bust the buttons off your vest.

Out in the great Northwest! Way out in the great Northwest!

A lady by the name of Weeks, was fond of swimming in the creeks

But she forgot the mountain peaks, out in the great Northwest

A Scotsman went out there to live, he called his house a "hoos".

They showed him a great big animal, they said it was a moose.

Out in the great Northwest! Way out in the great Northwest!

The Scotsman said, "Now, what the deuce! You say you call that thing a moose!

I'd hate to see a rrrat get loose!" Out in the great Northwest.

There are so many fishes there, in every mountain brook.

You have to hide behind a tree to get to bait your hook.

Out in the great Northwest! Way out in the great Northwest!

The rabbits there are very sly, they never go to school, but my,

How those things can multiply! Out in the great Northwest!

I was trying to figure out where this came from, and I found it in the

index
of the Intercollegiate Outing Club Association's Song Fest (1948).
Here are the lyrics as they appear in the seventeenth printing of the
New Song Fest:

My grandfather Phil brought it back from the CPS
Camps in Montana where he was smokejumping as a CO
in the early 1940s, though, so it wouldn't have been via this book.

Looking for earlier references, I found a short newspaper clip from
the Sunday
December 6th, 1931, Ardmore ...