trentmkelly/LessWrong-43k huggingface dataset:
Was a terminal degree necessary for inventing Boyle's desiderata?
by DirectedEvolution — 2020-07-10T04:47:15.902Z

My composition teacher in college told me that in some pottery schools, the teacher holds up your pot, examines it, comments on it, and then smashes it on the floor. They do this for your first 100 pots.

In that spirit, this post's epistemic status is SMASH THIS POT.

Is an MD/PhD practically ~necessary for groundbreaking scientific work of the 20th century?

I went into this project believing that an MD/PhD is ~necessary on a practical level to do groundbreaking STEM work. My model was this:

"These degrees come with credibility; access to expensive equipment, funding, and data; access to mentors and collaborators. A smart person who sets out to do groundbreaking STEM work will have a much lower chance of success if they don't acquire an MD/PhD. Massive, sustained social coordination is ~necessary to do groundbreaking research, and the MD/PhD pipeline is a core feature of how we do that. Without that degree, grant writers won't make grants. Collaborators won't want to invest in the relationship. It will be extremely difficult to convince anybody to let someone without a terminal degree run a research program."

Ideally, we'd answer this question with an RCT. Perhaps we'd take a pool of 100,000 successful biology PhD applicants, deny half of them the opportunity to enroll in a PhD or MD for the rest of their lives, and find some way to compare the two groups for their scientific accomplishments.

Nobody's going to let me run that experiment. So I tried looking at the proportion of recent STEM Nobel Prize winners who hold terminal degrees.

Of the 2018-2019 winners of the Nobel Prize in biology, chemistry, medicine, physics, and economics, all of them have a PhD. I also peeked at the 2018 Fields Medal winners. 3/4 have a PhD and the last one completed at least several years of PhD work. The most recent 3 individuals listed for an OpenPhil grant (7/9/2020) in Human Health and Wellbeing (a category I chose arbitrarily) all have PhDs.

Do credentials determine who gets the awards for groundbreaking work?

Zvi asks whether these awards might be mainly just reflections of the credentials of the applicants. Are these awards cherry-picking from among the set of all possible groundbreaking discoveries or projects the ones that are invented or headed by terminal degree-holders? If so, then these awards and grants might be severely under-representing groundbreaking STEM discoveries m...
As Few As Possible
by Decius — 2020-07-10T04:37:08.873Z

All of economics, every last bit of it, is about scarcity. About what is scarce and what is not, and about who does and who doesn’t get their needs (and sometimes wants) satisfied.

Much of the debate about healthcare is in fact a scarcity problem. There aren’t enough practitioners in practice to handle every patient, so some people don’t get doctors. It’s a combination of self-selection where people who can’t afford to take time off to have an ingrown toenail treated professionally hack at it with a pocketknife instead, and insurance selection where granny’s insurance will pay for hospice but not a new hip, and actual medical discretion now and then.

But this is about what medical care does right. Triage.

In times of very acute scarcity when many people are injured and need medical attention to survive, but few doctors are available to treat them, medicine does triage. The details can be complex, but one of the features of triage is that if doctors are too scarce to save everyone, they prioritize saving *as many people as possible*, even though that often means giving specific people no treatment at all. The scarcity is distributed so that *as few people as possible* suffer from scarcity.

This is not about mass trauma triage.

Imagine for a moment how utterly absurd it would be if there was an earthquake, but there were enough doctors to simultaneously treat everyone injured in it, but they insisted on following triage protocols and the very worst injured were given black tags indicating “do not treat”, even as surgeons who could save them sat idle, because everyone else was being treated. No civilized person would defend that kind of thing.

But this is not about hypothetical earthquakes with plenty of doctors.

The world produces significantly more food than needed to feed everyone. Quite a few people actually are angry that much of it is destroyed even though many people starve for lack of food. Many agree that is unacceptable that, even though there is enough to go around, not enough goes around.

This is not about hunger or food waste, or stores throwing bleach in the dumpsters to keep people from eating stale crackers.

This is about scarcity, and who gets it.

By it’s very nature as a lack of enough, if scarcity exists then someone must get it; there’s no accounting trick to pay Tuesday if there is...
Kelly Bet on Everything
by Jacob Falkovich — 2020-07-10T02:48:12.868Z

Cross-posted, as always, from Putanumonit.

It’s a

core staple

of Putanumonit to

apply ideas

from

math and finance

out of context to your everyday life. Finance is about making bets, but so is everything else. And one of the most useful concepts related to making bets is the Kelly criterion.

It states that when facing a series of profitable bets, your wagers should grow proportionally with your bankroll and with your edge on each bet. Specifically, that you should bet a percentage of your bankroll equivalent to your expected edge — if a bet has a 55% chance to go your way your edge is 55%-45%=10% and you should risk 10% of your bankroll on it (assuming equal amounts wagered and won). There could be

reasons to avoid betting the full Kelly

in practice: you’re not sure what your edge is, your bet size is limited, etc. But it’s a good guide nevertheless.

People’s intuition is usually that Kelly bets are too aggressive, that betting half of everything you have a on 75%-25% bet is too wild. But the Kelly criterion is actually quite conservative in that it maximizes not the expected size of your bankroll but it’s expected logarithm. “Exponential” means “fast and crazy”; logarithm is the inverse of that. It’s slow and cautious. If you have $1,000 and you’d risk no more than $750 for an equal chance to win $3,000, you’re logarithmic in dollars and should “bet the Kelly”.

Log scales apply to the difficulty and value you get for most things.

Life satisfaction grows with log(money)

. Making a new friend is probably one tenth as valuable to someone who has 10 friends than to someone who has one, so your social life depends on log(friends). It’s equally hard to double one’s number of blog readers, sexual partners, job offers etc regardless of how many you have, as opposed to incrementing each by a fixed amount. It’s equally valuable too.

And so, for most things, it makes sense to bet the Kelly. You’ll need to find out what bets are available, where your edge is, and what your bankroll is.

Money

Let’s start with the obvious one. What kind of Kelly bets can you make with money?

Investments are the obvious one

, and standard investment advice is to switch to high-risk-high-return assets when you have some money to spare.

You can also make bets on your ability to make money: take on a side project, look for a new job, start your own business, ask for a raise. Each one entail...
How far along are you on the Lesswrong Path?
by AiresJL — 2020-07-09T21:41:23.096Z

How far along are you on the Path?

Some say the Path has always existed and there have always been those who have walked it. The Path has many names and there are many notions of what the Path is.

To some the path is called philosophy, to others it is the art of rationality. To my people, it was called science and the pursuit of truth. Physics, psychology, neuroscience, machine learning, optimal control theory. All of these are aspects to the path. It has no true name.

With certainty, the only way to know whether you walk the path is by talking to those who don't walk the path.

One man, Scott Alexander describes the path as,

"Some people are further down the path than I am, and report there are actual places to get to that sound very exciting. And other people are around the same place I am, and still other people are lagging behind me. But when I look back at where we were five years ago, it’s so far back that none of us can even see it anymore, so far back that it’s not until I trawl the archives that realize how many things there used to be that we didn’t know. "

--------------------------------------------------------------------------------------------

How to tell where you are on the path?

One day, a man comes up to a woman and says:

"I am further down the path than you are."

In response, the woman says:

"No, I am further down the path than you are."

Instantly, both people recognize the solution as they have both at least progressed some degree through the path. The man speaks:

"Only one of us can be more right than the other. Let us make a prediction about all things in this world for all time and whoever is more correct about the state of the world for all time is closer to the end of the path."

Realizing the recursive time constraint of determining who is correct with infinite precision, the woman instead points to a random child.

"Whoever can guess correctly how this child behaves for the next 1 minute has walked further along the path and we will default to their position as truth when we are in conflict and have no time to discuss the details."

The man proceeds to describe in detail the neurobiological system of the baby and all the deterministic forces that would lead the baby to breathe, think, move in the manner in which he predicted it would. He then goes on to describe all ...
Why is the impact penalty time-inconsistent?
by Stuart_Armstrong — 2020-07-09T17:26:06.893Z

I showed in a previous post that impact penalties were time-inconsistent. But why is this? There are two obvious possibilities:

The impact penalty is inconsistent because it includes an optimisation process over the possible polices of the agent (eg when defining the Q-values in the attainable utility preservation).

The impact penalty is inconsistent because of how it's defined at each step (eg because the stepwise inaction baseline is reset every turn).

It turns out the first answer is the correct one. And indeed, we get:

If the impact penalty is not defined in terms of optimising over the agent's actions or policies, then it is kinda time-consistent.

What is the "kinda" doing there? Well, as we'll see, there is a subtle semantics vs syntax issue going on.

Time-consistent rewards

In attainable utility amplification, and other impact penalties, the reward is ultimately a function of the current state st and a counterfactual state s′t.

For the initial state and the initial state inaction baselines, the state s′t is determined independently of anything the agent has actually done. So these baselines are given by a function f:

f(μ,A,st,s′t).

Here, μ is the environment and A is the set of actions available to the agent. Since s′t is fixed, we can re-write this as:

fs′t(μ,A,st).

Now, if the impact measure is a function of st and μ only, then it is... a reward function, with R(st)=fs′t(μ,st). Thus, since this is just a reward function, the agent is time-consistent.

Now let's look at the stepwise inaction baseline. In this case, s′t is determined by an inaction rollout from the prior state st−1. So the impact measure is actually a function of:

f(μ,A,st,st−1).

Again, if f is in fact independent of A, the set of the agent's actions (including for the rollouts from st−1, then this is a reward function - one that is a function of the previous state and the current state, but that's quite common for reward functions.

So again, the agent has no interest in constraining its own future actions.

Semantics vs syntax

Back to "kinda". The problem is that we've been assuming that actions and states are very distinct objects. Suppose that, as in the previous post an agent at time t−1 wants to prevent itself from taking action S (go south) at time t. Let A be the agent's full set of actions, and A−S the same set without S.

So now the agent might be time-inconsistent, since

...
Featured Tags: July 2020
by Ruby — 2020-07-09T17:00:27.271Z

The LessWrong team is continuing to develop and validate the Tagging feature we introduced a few months ago. And we just opened up tag creation to everyone! (at least for now)

As inspiration and because they're great, I wanted to share some of the best tags we've got so far.

I've selected these for already having a pretty good collection of content on their respective topics.

Scholarship & Learning
Postmortems & Retrospectives
Decision Theory
Financial Investing
Progress Studies
Conversation
Book Reviews
Fiction

Note that you can change the ordering of posts with the sorting selector.

If you want to get involved with tagging (please do):

You can: 1) add tags to a post from the posts page OR 2) add posts to a tag page at the bottom right of the tag page.
You can create tags anywhere you see an "add tag" button, from www.lesswrong/tags/all or www.lesswrong.com/tag/create.
Any feedback is very welcome! We're doing a bunch of development know, so the more we hear from users the more we can incorporate that.

See the announcement for more details: The New Frontpage Design & Opening Tag Creation!
Arguments against myopic training
by Richard_Ngo — 2020-07-09T16:07:27.681Z

Note that this post has been edited to clarify the difference between explicitly assigning a reward to an action based on its later consequences, versus implicitly reinforcing an action by assigning high reward during later timesteps when its consequences are observed. I'd previously conflated these in a confusing way; thanks to Rohin for highlighting this issue.

A number of people seem quite excited about training myopic reinforcement learning agents as an approach to AI safety (for instance

this post on approval-directed agents

,

proposals 2, 3, 4, 10 and 11 here

, and

this paper

and

presentation

), but I’m not. I’ve had a few detailed conversations about this recently, and although I now understand the arguments for using myopia better, I’m not much more optimistic about it than I was before. In short, it seems that evaluating agents’ actions by our predictions of their consequences, rather than our evaluations of the actual consequences, will make reinforcement learning a lot harder; yet I haven’t been able to identify clear safety benefits from doing so. I elaborate on these points below; thanks to Jon Uesato, Evan Hubinger, Ramana Kumar and Stephan Wäldchen for discussion and comments.

I’ll define a myopic reinforcement learner as a reinforcement learning agent trained to maximise the reward received in the next timestep, i.e. with a discount rate of 0. Because it doesn’t assign credit backwards over time, in order to train it to do anything useful, that reward function will need to contain an estimate of how valuable each (state, action, next state) transition will be for outcomes many steps later. Since that evaluation will need to extrapolate a long way forward anyway, knowing the next state doesn’t add much, and so we can limit our focus to myopic agents trained on reward functions R which ignore the resulting state: that is, where R(s,a,s′)=M(s,a) for some M. I'll call M the approval function; we can think of such agents as being trained to take actions that their supervisor approves of at the time the action is taken, without reference to how the rest of the trajectory actually plays out. This definition can also include imitation learners, for which the approval function is calculated based on the agent’s divergence from the supervisor’s policy.

Although it&...
Covid 7/9: Lies, Damn Lies and Death Rates
by Zvi — 2020-07-09T13:40:01.359Z

Previously: Covid 7/2: It Could Be Worse

When I posted the link on Twitter, I joked that we already knew this week’s headline.

It turns out we didn’t. Not quite yet. I’m waiting on that one until next week.

Here’s what I said last week about death rates not picking up:

June 18 starts the surge in positive tests that represents the full second wave. June 23 represents when it accelerates. My default assumption has been one week to test positive, and about two weeks after that to see the average death.

That would mark the surge in deaths to start around July 2. In other words, today, with things picking up speed on July 7.

So no, this isn’t weird. Not yet. But if there is no spike in the next seven days, then that’s pretty weird. If that actually happened, I’d look more carefully at hospitalization data, which I usually disregard as not worth the trouble. But mostly I’d be terribly confused. The infection fatality rate seems to clearly have fallen, but why would it have fallen so much so quickly now that a surge in infections doesn’t kill more people? Quite the tall order.

Today is July 9. There was no rise in death rates starting on July 2. The holiday weekend shifted a bunch of reporting forward a few days, so tracking changes this week has been wonky. Death rate only picked up on July 7-8, and much of that was delayed reporting. Death rate this week is only slightly higher than last week’s.

It needs to be said up front. This is really weird. It’s not as weird as it looked before the last two days, but it’s still weird. I’m not going to back away and pretend it isn’t weird. Time to further investigate and break down potential causes, along with other news.

First, let’s run the numbers.

Positive Test Counts

Date
WEST
MIDWEST
SOUTH
NORTHEAST

May 7-May 13
22419
43256
37591
56892

May 14-May 20
22725
42762
40343
52982

May 21-May 27
23979
39418
42977
37029

May 28-June 3
32200
31504
50039
33370

June 4-June 10
35487
24674
55731
22693

June 11-June 17
41976
22510
75787
17891

June 18-June 24
66292
26792
107,221
15446

June 25-July 1
85761
34974
163,472
16303

July 2-July 8
103,879
40139
202,863
18226

Deaths by Region

Date
WEST
MIDWEST
SOUTH
NORTHEAST

May 7-May 13
1082
2288
1597
5327

Apr 23-29
1090
2060
1442
4541

Apr 30-May 6
775
1723
1290
3008

May 28-June 3
875
1666
1387
2557

June 4-June 10
743
1297
1230
1936

June 11-June 17

...
[Resource Request] What are good resources for best practice for creating bureaucracy?
by ChristianKl — 2020-07-09T12:05:54.262Z

Especially, after just having read The Utopia of Rules: On Technology, Stupidity, and the Secret Joys of Bureaucracy by David Graeber, I feel that most bureaucracy we have is very bad and might create more harm then benefits.

At the same time there are needs for bureaucracy. At Wikidata I care for it's functioning and it seems that some bureaucracy is needed to make norms legible for new users and also do reduce the necessity to waste time by having certain conflicts not resolved and refought over and over.

While Wikimedia projects are in some sense different then other organizations, it's likely that studies of other bureaucratic structures might provide valuable knowledge.

Is there any reading you can recommend?
Why does "deep abstraction" lose it's usefulness in the far past and future?
by Garrett Baker — 2020-07-09T07:12:44.523Z

Yesterday this post mentioned the "unreasonable effectiveness of abstraction" particularly noting

in the far past or the very far future, the state of the universe may be such that deep abstraction would be unlikely to remain useful (and thus "effective").

This makes no sense to me, and I was wondering if anyone could point me towards an explanation.
The New Frontpage Design & Opening Tag Creation!
by habryka — 2020-07-09T04:37:01.137Z

We just pushed a new frontpage design that we've been sitting and iterating on for a while, and we also just enabled the ability for everyone to create tags and edit the descriptions of existing tags.

New Frontpage design

A week ago we asked for your thoughts on the new frontpage design, and generally received quite positive feedback on the new design (and also almost the full LessWrong team has switched towards using the new design because we like it so much better).

I do think that pretty radical visual changes to a site can be distracting and break habits, and think that in this case it's worth that disruption, but generally want to be quite careful with doing redesigns like this to often, and don't want to force you all to reorient to a new design and visual language every time you visit the site.

The goal of this redesign was to make the frontpage a lot less visually cluttered, and make it easier to parse the structure of the site by adding more clear boundaries between foreground and background. This meant combining a bunch of elements in the Recommendations section, getting rid of a bunch of vertical dividers that were cluttering up the space, and using negative space more effective to communicate information without adding visual complexity.

The biggest change that comes with the new design is a lot more emphasis on our core tags and our frontpage selective filtering system that should allow you to personalize your frontpage, hide posts you don't want to see, and increase the frequency of topics that you are interested in. As a quick reminder, here is how to use the system:

Increase the karma of posts that are tagged with certain tags by hovering over the core tags and selecting the appropriate modifier

When you hover over one of the core tags on the frontpage, or a tag you added yourself to the filter list (by clicking the "+" icon on the right), you can add a karma modifier to all posts that are tagged that way, hide them completely, or mark a tag as required, hiding all posts that don't have that tag. I personally have a karma modifier of +10 on "Rationality" content and on "World Modeling" content, and a -10 to "Community" content.

I expect that over the next few weeks we will make changes to some more pages (most notable the frontpage) to bring it in line with the new visual direction of the site, and are also going to be running some experi

...
AI Research Considerations for Human Existential Safety (ARCHES)
by habryka — 2020-07-09T02:49:27.267Z
Link: https://arxiv.org/pdf/2006.04948.pdf

Andrew Critch's (Academian) and David Krueger's review of 29 AI (existential) safety research directions, each with an illustrative analogy, examples of current work and potential synergies between research directions, and discussion of ways the research approach might lower (or raise) existential risk.