trentmkelly/LessWrong-43k huggingface dataset:
[Letter] Advice for High School #2
by lsusr — 2021-04-30T08:42:34.511Z

This is amazing. Thank you, so, so much. I'll start coding in Hy with Vim. I'll start doing weekly blogs once my tennis season ends (three weeks from now), and I likely wouldn't have the guts to start if it wasn't for your advice. On the general high school advice; it's invaluable. I look forward to reading the books you've recommended. As a sidenote; I love books, and read as many as I can; any books that you'd recommend, on any subjects? I've started using Anki on a daily basis; the main bottleneck right now is that I'm making too few cards/day, but that can be addressed with more dedicated time, I think. Starting Strength is what my personal strength training needed. Meditation will be a useful intervention; I thank you for the guidance. I read How to... a year or so ago, and my social skills have improved significantly as a result; definitely a good recommendation. Luckily, I don't have any especially angry people in my life, but your Anger post serves as an excellent life guide. Do you know of any rationality/recommended programs near [redacted]? I've failed to find anyone near my intellect at my school/any other program I've done, and there aren't any rationality meetups nearby other than the [redacted] one, which I plan to go to, whenever possible. Again, thank you so much.

Yours truly,

[redacted]

Dear [redacted],

Don't worry about having too few cards in your Anki. Reviews pile up fast. Sustainability is more important than intensity. If you need more cards you can go to Ankiweb and download a frequency-based vocabulary deck for the foreign language of your choice.

Finding someone near your intellect…

Young people are sorted by geography. The average IQ at a typical high school is 100. Going to college bumps it up by a standard deviation to 115. An MD gets it a little higher to 120.

Enrolling in the hardest college major brings the IQ distribution a full two standard deviations above average to 130 which equals the average IQ of a science PhD. If you have an IQ of at least 145 (three standard deviations above the mean) then no undergraduate degree is sufficient.

College Undergraduate Major
IQ

Physics & Astronomy
133

Mathematical Sciences
130

Philosophy
129

Materials Engineering
129

Economics
128

Chemical Engineering
128

Other Engineering
128

Mechanical Engineering
126

Engineering
126

Electrical Engineering
126

Physical Sc

...
On silence
by Aaron Bergman — 2021-04-30T03:15:23.009Z
Link: https://aaronbergman.substack.com/p/on-silence

It has become common knowledge that some things—food, porn, outrage-share inducing articles—exploit a mismatch between our evolutionary heritage and the modern world, and do so at our expense. It seems, however, that we’ve collectively neglected one particular contemporary, omnipresent superstimulus: sound.

For a large proportion of my waking hours (70%?), sound is being directed straight into my eardrums. Not loud sound; I’m not worried about the health of my ears. Not “bad” sound; most of it is classical or electronic music, or ‘serious’ nonfiction audiobooks and podcasts. Yes, there’s a dash of delicious culture war content sprinkled among the podcasts and a healthy splash of music that might offend perhaps a 1980s Sunday school teacher, but I’m definitely not worried about the content.

What is starting to concern me, though, is the sheer lack of silence I experience in my daily life. Nothing specific is making me concerned, but it is gradually dawning on me that doing or consuming an excess of anything that was scarce in our ancestral environment, even the most benign music or informative nonfiction, can have a adverse effect on our mental and physical wellbeing.

Anyone listening?

While I am by no means the first to take notice, it isn’t like there’s an overwhelming amount of high-quality research and reporting on this. When I Google “Is silence important?” (the first phrase that came to mind, so not cherry-picking) plenty of listicles and a few other pop science articles come up: “

7 Benefits of Silence

,” “

An Ode to Silence,

” “

10 Reasons Why Silence is Really Golden

,” and “

The Hidden Benefits of Silence

,” to name a few. But the vast majority are some combination of poor quality and not actually about silence.

The

first link

, Google’s “featured snippet,” is a blog post by a soundproof enclosure company.

The

second

, Google’s first actual result, isn’t even about silence - it’s (mostly) about generalized hyperstimulation, stress reduction, and relaxation. Take, this paragraph, for instance:

“When we’re frazzled, our fight-or-flight response is on overload causing a host of problems,” says Dr. Sullivan. “We can use calm, quiet moments to tap into a different part of the

nervous system

that helps shut down our bodies’ physical response to

stress

.”

A lot of links cite the 2013 study

Is silence golden? Effects of auditory stimuli and their absence on adu

...
How do you solicit feedback effectively?
by Raj Thimmiah — 2021-04-30T00:16:07.211Z

It looks like I've been overly paranoid about the helpfulness of things I've done. A friend said that they could fill out feedback forms after conversations and while that wouldn't be perfect, I thought if I use it across multiple people it would likely be helpful.

I'm wondering if:

-anyone has already tried soliciting feedback from people about conversations already (and if so, to share your experience)

-has ideas about how I could do this not terribly
Low-stakes alignment
by paulfchristiano — 2021-04-30T00:10:06.163Z

Right now I’m working on finding a good objective to optimize with ML, rather than trying to make sure our models are robustly optimizing that objective. (This is roughly “outer alignment.”)

That’s pretty vague, and it’s not obvious whether “find a good objective” is a meaningful goal rather than being inherently confused or sweeping key distinctions under the rug.

So I like to focus on a more precise special case of alignment: solve alignment when decisions are “low stakes.” I think this case effectively isolates the problem of “find a good objective” from the problem of ensuring robustness and is precise enough to focus on productively.

In this post I’ll describe what I mean by the low-stakes setting, why I think it isolates this subproblem, why I want to isolate this subproblem, and why I think that it’s valuable to work on crisp subproblems.

1. What is the low-stakes setting?

A situation is low-stakes if we care very little about any small number of decisions. That is, we only care about the average behavior of the system over long periods of time (much longer than the amount of time it takes us to collect additional data and retrain the system).

For example, this requires that all of the AI systems in the world can’t corrupt the training process quickly or seize control of resources from humans. If they try, we can keep collecting data and fine-tuning them, and this will cause their behavior to change before anything irreversibly bad happens.

For a more formal definition see section 6.

2. Why do low stakes require only outer alignment?

If the stakes are low, we can train our model on the decisions that actually arise in practice rather than needing to anticipate tricky decisions in advance. Moreover, because the payoff from an individual action is always small, we can focus on average-case performance and achieve reasonable sample complexities without any additional tricks.

The main substantive claim is that we don’t need to worry about the “distributional shift” between past decisions and future decisions. When the distribution of inputs change, the system may behave poorly for a while, but if we keep retraining on the new data then it will eventually adapt. If individual decisions are low stakes, then the total cost of all of this adaptation is small. I give this argument in more detail in section 7.

Formally this resembles an online reg...
Where do LessWrong rationalists debate?
by Viliam — 2021-04-29T21:23:55.597Z

There are a few places where rationalists debate on internet outside of Less Wrong. Sometimes those places were created for debates that do not belong to the LW website, such as politics or investing. Sometimes they were created for a specific subgroup of rationalists, for example people who attend the same meetup. Some of them are on Slack, some of them are on Discord, some of them are on Facebook, and probably there are other places.

I propose the following:

Let's make a list of all the places where LessWrong rationalists debate currently.
Let's consider whether it would make sense to reduce the number of these places -- either by choosing among the existing ones, or creating a new one.
Afterwards, let's make the remaining places visible enough on the Less Wrong website, to avoid reinventing the wheel in the future.

Some clarifications:

This proposal only concerns the online forums of LessWrong rationalists, or a subset thereof (such as local meetups, or rationalists who are interested in some specific topic). It does not concern websites for wider audience, such as rationality-adjacent, or Slate Star Codex / Astral Codex Ten, or meta/post/dark/whatever rationalists, or effective altruists, or whatever. -- The idea is that the unified forum should still be populated by Less Wrong audience (rather than by general public), keeping the consensus on rationality, science, reductionism, and preserving the above-average quality of discourse.

On websites such as Slack or Discord, it is possible to create separate channels / chat rooms, and join a selected subset of them. I assume the unified forum would also have parts that are interesting only for a subset of members, such as specific topics, or maybe specific meetups. -- This would remove the need to create a separate forum for specific topics or meetups, as long as you want to debate these with (a subset of) fellow rationalists.

I would prefer not using Facebook, to keep our rationalist digital identities separated from our real names and jobs. (Also, Facebook is addictive, and its UI sucks.)

The thing I am specifically trying to avoid here, is the unnecessary fragmentation, where people duplicate things simply because they don't know these already exist at a different place. (Specifically, my trigger was getting an invitation to join a new discussion for "rationalists interested in investing", when I already know about...
Scott Alexander 2021 predictions: calibration and updating exercise
by Rob Bensinger — 2021-04-29T19:15:39.463Z

Scott Alexander makes yearly predictions on SSC/ACX. This year, I avoided looking at his blog post so I could make my own unanchored predictions about the same topics, using a copy of the post with Scott's probabilities removed. Then I looked at his and others' numbers, to see how much they update me.

For each question...

Bullet #1 is my initial Apr. 27 guess, based on about 20-120 seconds of googling and/or thinking.
Bullet #2 is my revised prediction after talking to my boyfriend and seeing their unanchored predictions.
Bullet #3 is my revised prediction after seeing Scott's probabilities (SA).
Bullet #4 is my revised Apr. 29 prediction after seeing Zvi Mowshowitz's arguments and probabilities (ZM) and market prices collected and interpreted by SimonM (MS).

My numbers were influenced a lot by the fact that I expect to round to the nearest 10% (or to 1% / 5% / 95% / 99% at the extremes) to check calibration.

US/WORLD

1. Biden approval rating (as per 538) is greater than 50%: [SA 80, ZM 80, MS 61]

Initial: 73%
Post-Boyfriend: 73%
Post-Scott: 75%
Final: 75%

2. Court packing is clearly going to happen (new justices don't have to be appointed by end of year): [SA 5, ZM 5, MS ~4-5+]

5%
7%
4%
4%

3. Yang is New York mayor: [SA 80, ZM 70, SM 69-77]

85%
85%
82%
74%

4. Newsom recalled as CA governor: [SA 5, ZM ?, MS ~7]

4%
6%
4%
10%

5. At least $250 million in damage from BLM protests this year: [SA 30, ZM ≤20]

45%
45%
30%
19%

6. Significant capital gains tax hike (above 30% for highest bracket): [SA 20, ZM 20]

40%
40%
20%
23%

7. Trump is allowed back on Twitter: [SA 20, ZM 10]

7%
8%
13%
12%

8. Tokyo Olympics happen on schedule: [SA 70, ZM 80, MS 75]

66%
66%
66%
74%

9. Major flare-up (significantly worse than anything in past 5 years) in Russia/Ukraine war: [SA 20, ZM ≤15, MS ~16]

6%
6%
20%
14%

10. Major flare-up (significantly worse than anything in past 10 years) in Israel/Palestine conflict: [SA 5, ZM 5-~15]

3%
3%
3%
11%

11. Major flare-up (significantly worse than anything in past 50 years) in China/Taiwan conflict: [SA 5, ZM 3]

2%
2%
2%
2%

12. Netanyahu is still Israeli PM: [SA 40, ZM 30, MS 22]

86%
86%
40%
35%

13. Prospera has at least 1000 residents: [SA 30, ZM 30]

13%
15%
24%
21%

ECON/TECH

14. Gamestop stock price still above $100: [SA 50, ZM 50, MS 60]

39%
39%
50%
59%

15. Bitcoin above 100K: [SA 40, ZM 25, MS 25]

20%
20%
36%
18%

16. Ethereum above 5K: [SA 50, ZM 30, MS 11]

20%
20%
40%
19%

17. Ethereum above 0.05 BTC: [SA 70, ZM 55, M...
We need a career path for invention
by jasoncrawford — 2021-04-29T18:11:23.058Z
Link: https://rootsofprogress.org/a-career-path-for-invention

If technological progress has slowed down, what is causing it? Here is a hypothesis.

Broadly speaking, there are three domains of activity important to technological progress: science, invention, and business. Science discovers new knowledge; invention creates useful machines, chemicals, processes, or other products; and business produces and distributes these products in a scalable, self-sustaining way. (Occasionally inventions are distributed by government: water sanitation is an example. But this oversimplified model will serve for our purposes.)

These domains do not form a simple linear pipeline, but they are distinct areas that attract different types of people, pose different challenges, and are judged by different standards. As such they create distinct communities and subcultures.

My hypothesis is that while science and business have functioning career paths, invention today does not.

Consider science. Suppose a high school or university student has a glimmer of desire to become a scientist. They will find that their road has already been paved. “Scientist” is a career. There’s an established path into the career: get a BS and then a PhD in a scientific field. There are research labs that hire scientists, organize them into teams, and give them space and equipment. There is funding for all of this, from government and philanthropy. There is an established deliverable: talks and papers, presented at conferences and published in journals. There are awards and honors that confer prestige within the discipline; some of these, such as the Nobel, are even well-known and respected among the general public.

All of this combines to create a career path for the scientist: anyone with even a modest level of commitment and effort can start down the path, and those who are exceptionally talented and ambitious can reach for inspiring goals. Importantly, there is a feedback loop in which progress down the career path opens opportunities. The more the scientist produces legible accomplishments, the more they are able to get grants, secure coveted positions, and attract talent to work with them. Money, prestige, and the opportunity to do meaningful work all (roughly) go together.

Entrepreneurship has different structures, but the career path is there nonetheless. “Startup founder” is not a job you get hired for; it is a job the founder must create for themselves. They must...
25 Min Talk on MetaEthical.AI with Questions from Stuart Armstrong
by June Ku — 2021-04-29T15:38:06.855Z

Previous Posts:
Formal Metaethics and Metasemantics for AI Alignment
New MetaEthical.AI Summary and Q&A at UC Berkeley

This time I tried to focus less on the technical details and more on providing the intuition behind the principles guiding the project. I'm grateful for questions and comments from Stuart Armstrong and the AI Safety Reading Group. I've posted the slides on Twitter.

Abstract: We construct a fully technical ethical goal function for AI by directly tackling the philosophical problems of metaethics and mental content. To simplify our reduction of these philosophical challenges into "merely" engineering ones, we suppose that unlimited computation and a complete low-level causal model of the world and the adult human brains in it are available.

Given such a model, the AI attributes beliefs and values to a brain in two stages. First, it identifies the syntax of a brain's mental content by selecting a decision algorithm which is i) isomorphic to the brain's causal processes and ii) best compresses its behavior while iii) maximizing charity. The semantics of that content then consists first in sense data that primitively refer to their own occurrence and then in logical and causal structural combinations of such content.

The resulting decision algorithm can capture how we decide what to do, but it can also identify the ethical factors that we seek to determine when we decide what to value or even how to decide. Unfolding the implications of those factors, we arrive at what we should do. All together, this allows us to imbue the AI with the necessary concepts to determine and do what we should program it to do.
On Chesterton's Fence
by Martin Sustrik — 2021-04-29T14:56:33.747Z

Let's think of the evolution as a conservative force for a bit.

The common narrative of evolution is that it is that great force of progress, optimizing the organisms, casting away inefficiencies, multiplying the species, and generally, driving living things to become more complex, starting from the simplest possible lifeforms, such as protozoans, up to that pinnacle of progress that is ourselves, the vertebrates.

But consider a plant that happens to have a mechanism to survive droughts, such as ability to store water in the roots. If there is no drought this year, that mechanism is a dead weight. It is costly to maintain and there's no immediate benefit form it. The plant would be better of without it. But the next year there's a drought and the plant survives. If the evolution was super progressive, the plant would optimize out its water-accumulating capacity in the first year and then die in the second year leaving no progeny behind.

A conservative may thus say: Evolution is a way to store knowledge of the past. It's a trove of mechanisms that have once proved useful for surviving. "Optimizing" it is just a nice way of saying that the instruments from that war chest are being thrown away.

And, of course, neither our strawman progressive or our strawman conservative are considering the full picture. Organisms benefit from having inherited the survival tricks from the past but also from being able to respond quickly and in novel ways to the new challenges. (But, to be fair, the conservative view is much easier to argue for. Arguing for the progressive view tends to lead to the theories on group selection, if not to some semi-mystical élan évolutif.)

In the end it's all probably just a race with time. The events that are frequent enough (the low temperature during the night, the cold in the winter) are worth optimizing for. Infrequent events (the ice ages) not much so. In the former case the tools for dealing with the event are worth keeping in the war chest. In the latter case they will be inevitably thrown away and we have to rely on the fact that at least some organisms will be, for random reasons, better suited to withstand cold and that those organisms will be the lucky winners of the evolutionary race when the next ice age comes.

To switch from evolutionary biology to political science, let's recall what Jean Monet once said on the topic of creating common E...
Covid 4/29: Vaccination Slowdown
by Zvi — 2021-04-29T13:50:01.010Z

Compared to expectations, excluding inevitable self-inflicted cratering of our vaccination rate, this was mostly a best realistic case scenario week. Johnson & Johnson was unpaused. India’s rate of case increases probably slowed down (they’re maxed out on testing capacity so it’s hard to know for sure). Biden seemed to come around to providing meaningful help to India and the world, at least to some extent.

Most of all, cases in America were down a lot, and it’s now clear that things are steadily improving. Unless something changes, we’re going to beat this thing at least in a large portion of the country, and do so on schedule.

The decline looks really bad though. Like the situation in India, it’s Malcom Reynolds-level worse than you know. When you look at first doses only, the lines are going straight down. If they go all the way to zero, many states and local areas won’t get to herd immunity from the vaccine.

Let’s run the numbers.

The Numbers

Predictions

Prediction from last week: Positivity rate of 5.1% (down 0.2%) and deaths decline by 4%.

Result:

Once again, Washington Post, your numbers don’t make sense together, but this is still clearly a big miss. 4.4% positivity rate is amazingly great in context. Johns Hopkins has the positivity rate declining from 4.7% to 3.9%. It’s worth noting that this ties the all-time low by their metrics.

As always, there’s the worry this ‘got ahead of ourselves’ somewhat, but I don’t think that’s the case. I think it’s more likely we’re making rapid progress, especially in the areas with higher rates of vaccinations.

Prediction for next week: Positivity rate of 3.9% (down 0.5%) and deaths decline by 6%.

Deaths

Date	WEST	MIDWEST	SOUTH	NORTHEAST	TOTAL
Mar 11-Mar 17	1492	1010	3217	1402	7121
Mar 18-Mar 24	1823	957	2895	1294	6969
Mar 25-Mar 31	1445	976	2564	1262	6247
Apr 1-Apr 7	1098	867	1789	1160	4914
Apr 8-Apr 14	1070	1037	1621	1145	4873
Apr 15-Apr 21	883	987	1747	1168	4785
Apr 22-Apr 28	752	1173	1609	1110	4644

I looked for an explanation for the jump in the Midwest, but there does not seem to be a data anomaly. It may be somewhat random, but it’s likely the delayed result of the local surge in cases in early April before things stabilized. While looking for a potential issue, I instead noticed that West Virginia reported -159 deaths one day, which I changed to 0, bringing their weekly total in line with expectations.

Overall the r...
Draft report on existential risk from power-seeking AI
by Joe Carlsmith — 2021-04-28T21:41:19.684Z

I’ve written a draft report evaluating a version of the overall case for existential risk from misaligned AI, and taking an initial stab at quantifying the risk from this version of the threat. I’ve made the draft viewable as a public google doc here (Edit: arXiv version here, video presentation here, human-narrated audio version here). Feedback would be welcome.

This work is part of Open Philanthropy’s “

Worldview Investigations

” project. However, the draft reflects my personal (rough, unstable) views, not the “institutional views” of Open Philanthropy.