trentmkelly/LessWrong-43k huggingface dataset:
Staying Grounded
by johnswentworth — 2021-08-14T17:43:53.003Z

What does it mean to “stay grounded”? Why does it matter? How does one stay grounded?

Examples

Let’s start with a bunch of examples of grounding failures - i.e. people failing to stay grounded.

The person who Volunteers to End Hunger, or End Poverty, or what have you, yet they do not actually make any significant difference to any actual people in hunger/poverty/etc (despite possibly believing that they have).
The cancer researcher who does lots of Experiments in the lab, and runs Statistical Calculations, and Publishes Papers, yet does not actually come any closer to understanding or curing cancer.
The person who does lots of Things Which Happy People Do in commercials, but is deeply unhappy anyway (despite possibly even convincing themselves that they are happy).
The person who makes themselves miserable to Eat Healthy and Exercise, yet is still far over their own preferred weight.
The person who gets Good Grades in high school, goes to a Good College, gets a Good Job, is generally Successful, but realizes sometime in middle age that they’re deeply unsatisfied with their life.
Cargo cultists: the pacific island tribes on islands which hosted airstrips during WWII who, after the war, would sometimes Talk into elaborate wooden “Radios” and Wave Sticks on the abandoned airstrip in hopes that planes would land with supplies.

Also, the various areas of academic study for which Feynman used the cargo cults as a metaphor.

The person who eats Good Food (possibly Healthy Food, possibly Expensive Food, possibly Ethical Food, depending on their social circles) but never really notices how much they enjoy the actual taste of different foods.
The person who buys Nice Clothes which are neither comfortable nor flattering for them in particular.
The startup founder who Writes Code, and Iterates, and Gets Funding, but never stops to think about how many people actually want their product or how much those people would pay for it.
The political activists who Organize The Movement and Raise Awareness to help X, get lots of media coverage and some laws passed, but in the end X doesn’t actually change much.
A regulatory agency puts in place lots of Rules and Regulations and Processes in order to Make People Safe, yet

doesn’t end up actually making people safer

.
The military (or guerillas) who Shoot Enemies and Destroy Their Stuff, but never manage to institute lasting regime change (or whate

...
Perhaps vastly more people should be on FDA-approved weight loss medication
by 25Hour — 2021-08-14T17:22:30.651Z

[Epistemic Status: I feel pretty good about most of this, but the life-years-saved-via-medication part is problematic on a number of levels, as pointed out by a few commenters. I include it since back-of-the-envelope calculations serve a purpose in ensuring we're comparing effects of approximately the appropriate magnitudes in doing risk/benefit analyses, but I wouldn't take it too seriously.]

[UPDATE 4/20/2024: Tirzepatide/Mounjaro is probably just a better version of Semaglutide, below. It's very effective, very safe, and approved by the FDA.]

Note that I’m not a doctor. Please speak to a doctor before doing any of this stuff Or You Will Die.

Introduction

Judging by posts in

r/loseit

, the existence of effective anti-obesity medications is not particularly well-known (and to the degree it is well-known, it's disapproved of.) Even posts on LessWrong, which tend to be very well-researched and exhaustive, simply ignore the topic of medication when weight loss methods or obesity are brought up; I suspect this is not because their authors had explicitly considered and discarded the various anti-obesity drugs currently available, but rather, because the existence of these drugs is very poorly-known. Which I’m attempting to remedy here! At least for the LessWrong crowd.

Quantifying Life-Years Saved by Losing A Certain Amount Of Weight

[Note: as pointed out by comments below, extrapolation to life-years saved is very speculative, since all the studies on this in humans are going to be confounded all to hell by healthy user bias and socioeconomic correlations and the like. That said, it feels like a fairly reasonable extrapolation given the comorbidity of obesity to various extremely problematic medical conditions. Be warned!]

According to

Genome-wide meta-analysis associates HLA-DQA1/DRB1 and LPA and lifestyle factors with human longevity | Nature Communications

, losing a single unit of BMI roughly corresponds to a 7-month gain in life expectancy in the overweight and obese. This seems basically in line with what I hear from popular sources, such as: “[L]ife expectancy for obese men and women was 4.2 and 3.5 years shorter” than people in the healthy BMI range.

This won’t count as a revelation. Obesity is unhealthy, news at eleven. My goal here is just to quantify what you’re getting relative to the risks involved in doing so...
Approaches to gradient hacking
by adamShimi — 2021-08-14T15:16:55.798Z

Gradient hacking is a fascinating type of deception, because of its anchoring in current ML. Even if it applies to more general settings, gradient hacking is considered foremost in the context of gradient descent, the most common training method at the moment.

Despite this, thinking about gradient hacking can lead one to very abstract and confusing questions, like how a deterministic neural net could decide to change its thinking for such reasons.

I propose instead in this post to discuss concrete ways a model could gradient hack, and the corresponding difficulties.

Thanks to Evan Hubinger for coining the term and discussing it many times with me, Kyle and Laria for discussions around gradient hacking and one proposal, Richard Ngo for pointing out the need for continuous failures, and Connor Leahy for feedback on my ideas.

Gradient hacking schemes

Exact gradient control

The most obvious way to gradient hack is to compute one's gradient on a given batch for a given loss, and then compute what is needed to make the gradient go in the wanted direction.

This requires:

Having incentives to change the gradient

Inner Misalignment, probably deceptive alignment

Knowing how to compute the loss, not only the loss function used but also the expected answer (correct label, reward, next word...)

Enough ML knowledge to understand how training and loss functions works
Educated guess (based on ML knowledge) of the loss function used
Competence at the task to know the correct answer
Incentive to learn how to compute the loss

Knowing how to compute the exact gradient from the loss

Compute the contribution of every neuron to the loss!

Knowing the model space enough to know where to go

Have a far superior than current ML knowledge of what neural nets

Being able to find the output which results in the right loss and gradient

Do all of the above while searching and testing output

Most of these sound doable to me for a neural net training on language or some intricate real-world task, especially since competence is required. But there is one massive issue: even if one could compute the expected loss, computing the gradient from there requires computing the gradient of the part of the network computing the gradient! In essence, the gradient is a transformation on the computation, but computing it makes the computation longer, and thus requires a recomputation of the gradient!

The problem is even more sev...
NJ Still Has Silly Rotary Rules
by jefftk — 2021-08-14T02:00:02.320Z

New Jersey has very strange rules determining who has the right-of-way
at rotaries / roundabouts when there aren't signs to clarify:

There are no set rules for driving into, around and out of a traffic
circle in New Jersey. Common sense and caution must prevail at all
times. In most cases, the circle's historically established traffic
flow pattern dictates who has the right-of-way. If a major highway
flows into and through the circle, it usually dominates the traffic
flow pattern and commands the right-of-way. Traffic control signs,
such as stop or yield signs, at the entrances to the circle also
govern which motorist has the right-of-way. Never enter a traffic
circle without checking all signs and determining the intentions of
the motorists already moving within the circle.

Whenever a motorist is in doubt concerning who has the right-of-way in a
circle, he/she should exercise extreme caution and remember the basic
rules governing any uncontrolled intersection: The vehicle to the left
yields the right-of-way to the vehicle approaching from the right.

—The
New Jersey Driver Manual, 2020

Having right of way depend on the "historically established traffic
flow pattern" means that anyone who approaches an unfamiliar
one doesn't know what the rules are. Which is definitely not how you
should set up your traffic system!

Additionally, the second paragraph says drivers already in the
rotary should yield to drivers entering, which is the opposite of the
system everyone else has standardized on.

I encountered this several years ago, but at the time the only source
I found was a dead link from Wikipedia. NJ
has revised their manual, however, and post of the 2020 version, so
I've updated Wikipedia.

On top of all that, NJ is apparently so embarrassed about this
that they won't allow you to copy text from the manual:

If I were in charge of this for NJ, I would announce that we were
going to switch to the standard "entering traffic yields" system in
some number of months, and ask people to submit any locations that
should be considered for alternative signage ahead of that change.
Absent-Minded Driver and Self-Locating Probabilities
by dadadarren — 2021-08-14T00:09:20.347Z
Link: https://www.sleepingbeautyproblem.com/about-the-absent-minded-driver/

The absent-minded driver problem is this:

Fig 1. (Taken from The Absent-Minded Driver by Robert J. Aumann, Sergiu Hart, and Motty Perry)

“An absent-minded driver starts driving at START in Figure 1. At X he can either EXIT and get to A (for a payoff of 0) or CONTINUE to Y. At Y he can either EXIT and get to B (payoff 4), or CONTINUE to C (payoff 1). The essential assumption is that he cannot distinguish between intersections X and Y, and cannot remember whether he has already gone through one of them.”

It can be easily shown the best strategy is to CONTINUE with a probability of 2/3 at an intersection. i.e. let p be the probability of CONTINUE at an intersection, the payoff would be 4p(1-p)+p2 which is maximized when p=2/3.

Yet, there is a paradox. Imagine you are the driver. When arrived at an intersection, you could assign a certain probability for “here is X”. Let this probability be α. Now the payoff function would be α[4(1-p)p+p2] +(1-α)[4(1-p)+p]. Maximizing it would cause p to be different from 2/3, no matter how one chooses to determine the value of α. Except of course if α=1. i.e. the probability of “here at X” is 100%, which is impossible to justify.

For example, a common approach is to let α=1/(1+p). The reason being the driver will always pass X with a probability of 1 and pass Y with a probability of p, ergo the probability of “here is X” should be the relative weight of the two. Using this value for α, the payoff is function is maximized when p=4/9. Different from 2/3.

Aumman’s Answer

Aumann, Hart, and Perry disagree with the above reasoning and deny the problem presents any paradox. First, they make a distinction between planning optimal and action optimal. The original p=2/3 is regarded as the planning optimal. The rational decision for the driver when arrived at an intersection is dubbed the action optimal. The action optimal is derived with the following observations:

The decision the driver makes only determines his action at that particular intersection. It does not affect the action at the other intersection.
The driver is aware he will make (has made) an identical decision at the other intersection too.

I will skip the details (Eliezer Yudkowsky made a post about it here). Their method of determining the action optimal is essentially treating it as a game of coordination between “me at this intersection” and “me at the other intersect...
D&D.Sci August 2021: The Oracle and the Monk
by abstractapplic — 2021-08-13T22:36:38.572Z

You are Oeis the Wise, professional Oracle. Your uncanny ability to predict how lists of whole numbers will continue – and only how lists of whole numbers will continue – creates infrequent but intense demand for your counsel: you usually leave your mountain home about once a month, whenever circumstances place a local noble or merchant lord in need of your services.

Today’s customer is Morgan, a monk. (Morgan has no money, but his temple has frequently provided you with food and shelter while travelling to more lucrative clients, so you’re fine doing this one for free.)

Morgan explains that he uses transcendental meditation to link himself to the substructure of the universe, observing the pulse and flicker of those lights which cast all we know as shadows. In his notes, he records flows of mana as they rise and ebb, forming the rhythms that undergird reality.

You tell him that sounds cool and you hope it works out for him.

However, Morgan continues, a recent supernova of unusual intensity has disrupted the leylines of the world, creating the possibility of flux in realms which have previously known only stasis. In ten days – day 384 in his record – he will have a once-in-several-lifetimes chance to not merely channel these flows, but change them: he plans to invoke two mana types simultaneously, wielding them in a ritual that subtly but profoundly reshapes the world for the better. If he is to do this, he must begin the endeavour now; to determine which sources of power he should prepare to conjure on that fateful day, he zealously entreats your assistance.

You tell him that sounds cool and you hope it works out for him. (Living alone on a mountain has not done wonders for your conversational repertoire.)

Morgan concludes by saying that his priorities for day 384 are as follows:

Firstly, the strongest of the two mana types he channels cannot be Void, Doom, or Spite. These may be used if you are certain they will be the weaker of the pair, but if they dominate, darkness will possess him and he will transform into an immortal demon, wreaking horror and bloodshed upon the world he sought to improve.
Secondly, the ritual must work. For this to occur, the two mana types together must have a combined power of at least 70.
Finally, as a strictly tertiary objective, increasing the total amount of mana channelled would allow him to increase the amount of good done; however,

...
Fudging Work and Rationalization
by J Bostock — 2021-08-13T19:51:44.531Z

If you want to do extraordinary things, you will need to do your best.

In practical sciences, this can come as a form of precision, typical analytical chemistry is done to some degree by hand (!) and involves literally weighing out milligram-precise quantities of substances, and putting them into sub-millilitre-precise quantities of solvents. It is easy to go wrong, or to be lazy. The most accurate flasks look like this:

Lucasbosch, CC BY-SA 3.0, via Wikimedia Commons

Note the single line. If you overshoot the volume, you have to start again. This is a real pain, and can sometimes take tens of minutes. It's also a little embarrassing. But what is overshooting? There's only a certain precision a human can go to, and after all, if you're only half a percent over it can't make much of a difference, can it? These mental patterns are easy to fall into, and they look like bargaining with the universe.

The accuracy of your results doesn't depend on how hard it was to measure out the right volume. They depend on how accurately you measured that volume. The universe cannot be bargained with.

It's a sort of rationalization, at least that's how it feels to me, when I fall into it. The feeling of rationalization is one of the most important ones to be able to notice as rationalists. I think there are a few reasons for this beyond the standard ones:

I expect to be punished for bad work on some level. When this happens, we bargain with the person doing the punishing, which is reasonable. If someone says "you measured that volume wrong", things like "well it's pretty much close" and "but measuring volumes is really hard!" may well be reasons for them to forgive you. But they are not reasons why the volume is right. When you're doing work like this, it's absurd to feel guilty at an occasional mistake, but unfortunately the guilt-making part of the brain doesn't know this.

I want to protect my opinion of myself as skilled. This has ruined many scientists' careers, when their results are disproven, they turn to fraud.

I want to keep doing things without having to redo the work. Schools and universities teach us that failed work is not something to redo. For the most important work (exams) redoing them is forbidden! In the real world, the more important the work, the more important it is to redo it if it's wrong. Also, redoing the work is lots of effort, and I want to move on to the n...
A deeper look at doxepin and the FDA
by DirectedEvolution — 2021-08-13T18:59:48.022Z

Doxepin is an old drug that functions as a sedative and, at high doses, as an antidepressant. It goes by many names, including Silenor and Sinequan. On GoodRx, you can buy 30 doses of Sinequan for about $10, but Silenor costs at least $100. But these are two names for exactly the same chemical. Physicians have been able to write off-label prescriptions for doxepin since the drug was first approved in 1969. It's a second-rate and unpopular drug relative to other antidepressants and sedatives, but the very expensive Silenor is still generating revenue for its parent company.

Scott Alexander interprets Silenor as an example of the FDA's role in perpetuating the high costs of our healthcare system. The company that brought it to market, Somaxon, spent $170 million on 3-4 phase III trials and other costs of winning FDA approval for its brand name of doxepin as an insomnia drug. Why don't physicians just prescribe the much cheaper Sinequan?

...there are various regulations that make off label prescriptions less convenient than the usual type, so most people stick with Silenor...

So the idea is that FDA approval creates trivial inconveniences for physicians writing off-label prescriptions. This creates an opportunity for drug companies to needlessly spend the money to get FDA approval for their brand so that physicians can prescribe it on-label. This alleviates a trivial inconvenience created by the FDA, but at tremendous cost to the public.

We should interrogate this hypothesis. Is it really inconvenient enough to write off-label prescriptions that it creates a market for companies to profit?

I spoke with a psychiatrist who's been practicing for decades, who had this to say about writing off-label prescriptions:

Once a drug has been approved by the FDA for any indication, a physician can prescribe it for anything without hassle or limitation. On occasion – but certainly not in a case like Silenor – there may be insurance hassles, refusal to cover medication for which the indication is suspect.

So the idea that off-label prescription writing is inconvenient enough to make physicians prefer expensive Silenor over cheap generic doxepin is controversial. But none of these claims are precise. We could be dealing with any number of physician concerns. It might take more time to prescribe off-label, patients may request a particular brand name, there might be concerns abo...
You are invited to apply to the Guild of the ROSE
by moridinamael — 2021-08-13T18:25:35.568Z

The Guild of the R.O.S.E. is an organization dedicated to improving the world, created by rationalists, for rationalists. We provide a support structure of a small Cohort for you to join, and a wider Guild to learn from.

As a member you will attend weekly classes using a flipped classroom model, in which readings and lectures are attended to asynchronously, while the class is hands-on activities and discussion. There will also be competitions with other Cohorts, and other extracurricular activities such as raising money for charity and Fermi Estimation challenges.

Our first Beta session will take place on September 1st. Our courses will begin with a series of classes on ‘Quantified Self-Assessment’, ‘Practical Decision Theory’, ‘Context and Communication’, and ‘Practical Social Networking’. Most future courses will broadly fall under the skills of Communication, Critical Thinking, and Meta-Skills (how to most effectively acquire skills).

Unlike the Alpha phase of the “Guild of Servants” (our previous moniker), the Beta phase has no set end date. We will continue the classes until we stress test the limits of our resources to our satisfaction. In addition to the application process on our website, accepted attendees will be contacted regarding a modest monthly fee for Guild Dues on Patreon.

If you are ready to join a community of people striving to help each other bloom, join the ROSE today at

guildoftherose.org

. We are also holding a public online mixer on August 28th.
In Most Markets, Lower Risk Means Higher Reward
by dawangy — 2021-08-13T17:32:16.585Z

The age old adage that high risk is associated with high expected return has not been true in the U.S. stock market, at least when using the academic standard of measuring risk with the either beta or standard deviation of returns. This claim is one of plain numbers, so by itself, it is not disputed. However, the reasons for the existence of this phenomenon, as well as its practical exploitability are. Here we will explore different hypotheses about why this is the case, including some hypotheses that are consistent with the EMH and some that are not.

Returns of Low vs. High Risk Stocks in the U.S. Since 1929

In my view, the most comprehensive empirical study of a similar result was in Frazzini and Pedersen. In this paper the authors examined the low beta phenomenon by constructing hypothetical portfolios that are long low beta stocks and short high beta stocks in market-neutral ratios. Using this approach, the authors determined that low beta was associated with excess returns in 19 out of 20 countries' stocks, and was also true in many alternative markets such as those for corporate bonds, treasuries, foreign exchange, and commodities. The in-sample effect was strongest in the Canadian stock market, and the Austrian stock market was the only studied market for which the effect was (very slightly) negative. The out of sample construction based on the methodology created in sample was also positive in all nations studied except Sweden. Note that this is slightly different from my headline, as it suggests that the "risk-adjusted" returns of low beta stocks is higher than those of high beta stocks. Not necessarily that the returns of low beta stocks are higher in general.

The hypothesis of Frazzini and Pedersen is that some investors are leverage constrained, meaning that they are less capable of borrowing funds in order to increase exposure to equity. This leads those less capable of leverage to seek the higher expected returns dictated by their preferences through exposure to riskier stocks, which reduces the risk adjusted expected returns of riskier stocks because investors will purchase them at discounted expected returns to achieve artificial market leverage. The paper illustrates that under the framework of modern portfolio theory, it is mathematically true that certain kinds of investors that they describe will yield the types of results they are des...