trentmkelly/LessWrong-43k huggingface dataset:
On the Cost of Thriving Index
by Zvi — 2023-06-26T15:30:05.160Z

Scott Winship argued recently that the ‘cost of thriving’ has fallen, pushing back once again against Oren Cass and his rather arbitrarily calculated ‘Cost of Thriving Index (COTI).’

Alex Tabarrok posted today on Marginal Revolution that Winship and coauthor Horpedahl were right, but that they face an uphill battle because people feel they are wrong, and suggested that our newly high time-value warps our perceptions. He points to Linder’s Theorem, which I hadn’t seen before, which states (correctly): “rising productivity decreases the demand for commodities whose consumption is expensive in time.”

Who is right, and why? If thriving is easier now, why does everyone think it isn’t?

Overview of the Disagreement

My analysis of the details shows that both perspectives are flawed.

Cass uses sloppy calculations, especially double counting employer-provided health insurance payments, and his calculations importantly exclude changes in taxation.

Winship and Horpedahl offer useful correctives to the sloppiness of the original Cass calculations in some places, and point out the rather glaring omission of taxation.

But they misunderstand the purpose of creating a Cost of Thriving Index. In several places, they therefore do the wrong calculation, confusing improved quality of goods with reduced cost of thriving.

COTI attempts to capture an important thing: Even if the quality of the standard ‘basket of goods’ is improved, that doesn’t change that there are huge pressures forcing people to buy whatever is the standard goods basket, and ‘improved quality’ is nice but fails to cancel this effect out. Winship does not seem to appreciate why that is an important thing to be measuring.

Excluding women and younger workers here is not arbitrary either. The whole idea is to ask what it takes for a typical man to support a typical family – you can’t simply decide to measure something else.

Let’s look at the proposed modifications, as argued in the Twitter thread.

Scott Winship: New from me and @jmhorp: “The Cost of Thriving Has Fallen: Correcting and Rejecting the American Compass Cost-of-Thriving Index”. Our critique of the @AmerCompass COTI developed by @oren_cass. Working paper version is up here.

Cass argues that there are not enough weeks in the year for the typical male worker to afford five basic “needs”, and that this “COTI” is up 22 weeks since 1985. We

...
"Safety Culture for AI" is important, but isn't going to be easy
by Davidmanheim — 2023-06-26T12:52:47.368Z
Link: https://forum.effectivealtruism.org/posts/7tDuaRLS7FXYB4f2P/safety-culture-for-ai-is-important-but-isn-t-going-to-be

This is a linkpost (to the EA forum version of this post, which is) for a new preprint, entitled "Building a Culture of Safety for AI: Perspectives and Challenges," and a brief explanation of the central points. Comments on the ideas in the post are welcome, but much of the content which clarifies the below is in the full manuscript.
Safety culture in AI is going to be critical for many of the other promising initiatives for AI safety.

If people don't care about safety, most safety measures turn into box-ticking. Companies that don't care avoid regulation, or render it useless. That's what happens when fraudulent companies are audited, or when car companies cheat on emissions tests.
If people do care about safety, then audits, standards, and various risk-analysis tools can help get them there.
Culture can transform industries, and norms about trying to be safe can be really powerful as a way to notice and discourage bad actors.

However, there are lots of challenges to making such a culture.

Safety culture usually requires agreement about the risks. We don't have that in AI generally.
Culture depends on the operational environment.

When people have risks reinforced by always being exposed to them, or personally being affected by failures, they pay more attention. In AI, most risks are rare, occur in the future, and/or affect others more than the people responsible.
Most safety cultures are built around routines such as checklists and exercises that deal with current risks. Most AI risks aren't directly amenable to these approaches, so we can't reinforce culture with routines.

Cultures are hard to change once they get started.

AI gets cultural norms from academia, where few consider risks from their work, and there are norms of openness, and from the startup world, where companies generally want to "move fast and break things."
AI companies aren't prioritizing safety over profits - unlike airlines, nuclear power operators, or hospitals, where there is a clear understanding that safety is a critical need, and everything will stop if there is a safety problem.
Companies aren't hiring people who care about safety culture. But people build culture, and even if management wants to prioritize safety, lots of people who don't care won't add up to organizations that do care.
We need something other than routinization to reinforce safety culture.

Thankfully, there are so...
Direct Preference Optimization in One Minute
by lukemarks — 2023-06-26T11:52:16.994Z

The

Direct Preference Optimization (DPO) paper

promises a more simple and efficient alternative to proximal policy optimization that is able to void the reward modeling phase, and thus optimize directly for the preferences expressed in preference data. This is achieved through the loss function:

LDPO(πθ;πref)=−E(x,yw,yl)∼D[logσ(βlogπθ(yw|x)πref(yw|x)−βlogπθ(yl|x)πref(yl|x))]

Where:

x is some prompt
πθ(yw|x) and πθ(yl|x) are the probabilities of the preferred and dispreferred completions under the current model.
β is controls the deviation from the reference policy πref.

In essence, DPO computes the log probabilities of preferred and dispreferred completions under the current model and optimizes parameters to increase the likelihood of the preferred completions and decrease the likelihood of the dispreferred completions.

The authors share the following results:

"Figure 2: Left. The frontier of expected reward vs KL to the reference policy. DPO provides the highest expected reward for all KL values, demonstrating the quality of the optimization. Right. TL;DR summarization win rates vs. human-written summaries, using GPT-4 as evaluator. DPO exceeds PPO’s best-case performance on summarization, while being more robust to changes in the sampling temperature."

"Figure 3: Left. Win rates computed by GPT-4 for Anthropic-HH one-step dialogue; DPO is the only method that improves over chosen summaries in the Anthropic-HH test set. Right. Win rates for different sampling temperatures over the course of training. DPO’s improvement over the dataset labels is fairly stable over the course of training for different sampling temperatures."

We evaluate different methods by sampling completions on the test split of TL;DR summarization dataset, and computing the average win rate against reference completions in the test set. The completions for all methods are sampled at temperatures varying from 0.0 to 1.0, and the win rates are shown in Figure 2 (right). DPO, PPO and Preferred-FT all fine-tune the same GPT-J SFT model. We find that DPO has a win rate of approximately 61% at a temperature of 0.0, exceeding the performance of PPO at 57% at its optimal sampling temperature of 0.0. DPO also achieves a higher maximum win rate compared to the best of N baseline. We note that we did not meaningfully tune DPO’s β hyperparameter, so these results may underest

...
Self-experiment: A supraphysiological dosage of testosterone.
by shapeshifter — 2023-06-26T10:26:50.951Z

Unfortunately, I haven't found a good way to conduct a long-term self-experiment in which I observe the effect a supraphysiological weekly dosage of testosterone has on me. However, I think I found a decent alternative.

Firstly, some stats:

21y/o male, sub-saharan descent. 173cm at 61kg of bodyweight. I have used steroids before, in a bodybuilding context but that was quite some time ago. I don't bodybuild as of now.

Experimental preparations are as follows: I have sort-of double blinded myself. I have prefilled one syringes with testosterone enanthate at a concentration of 200mg/ml. 300 mg/ml. Another syringe I prefilled with ordinary bacteriostatic water. I have - 20 minutes ago - injected my left buttcheek with the content (1ml) of one of the two syringes, I don't know which one though. I will wait at least 3 weeks (time it takes for this version of testosterone until it has cleared my body) before stabbing myself with the other syringe. In both cases, I will record any subjective effects here.

I will measure my cognitive abilities every three days using:

Zetamac (mental math website for processing speed)
Dual n-back (for working memory)
No. of uni slides I can work through in an arbitrary hour (for measuring productivity, I study urban planning).

How does that sound? Can you suggest any improvements (preferrably improvements which I can implement without letting others know of my plans)?

Edit: Corrected the concentration (300mg/ml) and added the amount of testosterone I injected (1ml).
Confused Attractiveness
by Vlad Loweren — 2023-06-26T09:33:27.406Z

Crossposted from Optimized Dating.

I.

Imagine that you are a human xeno-sociologist studying the society of insectoid aliens on another planet. To you, they all look alike the same way two ants look alike. Yet when talking about each other, they often use words that translate closely to “beautiful”, “attractive”, “plain” and “ugly”, indicating that they see an aesthetic quality in each other that you don’t see. How do you learn which specific insectoid individuals are attractive and which ones are not?

The Ant and the Grasshopper, J. J. Grandville, 1838 (!!!)

First thing you could try is to take photos of many aliens – let’s say a hundred of them. Then you make a survey which asks a simple binary question under each photo: “Is this individual attractive? Y/N”. You distribute the survey to a large group of aliens, collect the responses and arrange the photos from the one with least Y’s to the one with the most. You designate the first 10 aliens as “10th percentile rank”, next 10 as “20th”, and so on until the most attractive alien in your sample, who gets “99th”. Now you can compare the ranks of any two aliens in your study and see which one is more attractive than the other.

You proceed to gather data on attractiveness in more elaborate ways, not just with surveys but looking for revealed preference as well - for example, measuring

exoskeleton conductance response

when presented with different alien faces, or measuring

tips that the alien waiters get

. Crucially though, two facts hold constant throughout your studies:

Attractiveness is determined by conscious or unconscious reactions of other individuals (binary choice/conductance/tipping).
Attractiveness of an individual can be expressed as a percentile rank among all the other individuals in a study.

The data looks nice, but what if you need to check the attractiveness of an alien that’s not in your ranked sample? It’s expensive and cumbersome to run a full study with a large population each time you want to assess whether some specific alien you meet is attractive or not. It would be great to learn how to tell apart beautiful aliens simply by looking at them. Is there a way to do that?

You try to extract a set of features associated with each attractiveness rank. You discover that individuals in top attractiveness ranks have longer antennae and bulging thorax, while lowest-ranked aliens have small claws and l...
60+ Possible Futures
by Bart Bussmann — 2023-06-26T09:16:41.274Z

Introduction

I have compiled a list of possible future scenarios. I hope this list is useful in two ways:

As a way to make your own thinking about the future more explicit; how much probability mass do you put on each possible future?
As a menu of options to choose from; which of these futures do we want to make more likely?

This list is just a brainstorm, and I encourage readers to write any missing but probable futures in the comments. I will add any scenarios that do not substantially overlap with existing items and which I subjectively estimate as having at least a 0.01% probability of happening to the list (with attribution).

I have divided the possible futures into the following categories:

Futures without AGI, because we prevent building it
Futures without AGI, because we go extinct in another way
Futures without AGI, because we take a different path
Futures without AGI, because of strange factors
Futures with AGI, in which we die
Futures with AGI, in which we survive, and things are somewhat normal
Futures with AGI, in which we survive, but we're very different humans
Futures with AGI, in which we survive, and the universe gets optimized

Futures without AGI

Because we prevent building it

Successful Treaty: Humanity figures out that building AGI would be super-dangerous. After a long negotiation between world leaders, they succeed in agreeing on a FLOP quotum, which is far under the limit for potentially dangerous AGI. This policy is strongly enforced and prevents any individual or organization from developing AGI.
Surveillance: A world government is instantiated that recognizes that AGI would be dangerous. They ban AGI research and install an Orwellian surveillance machine that records and analyses every keystroke, voice command, and research meeting. This successfully prevents AGI from being created.
Regulation: Humanity enforces strong regulations on AI, mostly in order to combat non-existential risks such as discrimination, fairness, and loss of jobs. This makes R&D in AI unprofitable, as the resulting models cannot be deployed for any real-world use.
Humanity grows up: Humanity makes epistemic, technological, political and moral progress and learns how to defeat Moloch and cooperate at a planetary scale. We decide collectively that building AGI would be bad and is something we just don't do. Consequently, no one works on developing AGI.
Catastr

...
Bounded surprise exam paradox
by cousin_it — 2023-06-26T08:37:47.582Z

A small fun thing I came up with this morning.

The surprise exam paradox goes as follows: "you'll be given a surprise exam on one of the days next week, and you won't figure out the date of the exam on the previous day". Then you figure that the exam can't be on Sunday as it's the last day and you'd know it on Saturday, so it can't be on Saturday and so on, ruling out all days. Then the exam happens on Wednesday and you're surprised.

The standard solution, which I agree with, is to interpret it as a self-referential sentence S = "there is a natural number X from 1 to 7, such that from knowing any lower bound on X and also knowing S, it's impossible to prove the value of X". This isn't a liar kind of paradox, because the self-reference in S only talks about provability, not truth. So we can take any formal theory that's strong enough to talk about provability within itself (same conditions as Gödel's theorems), convert S into a non-self-referential sentence by using the diagonal lemma, and observe that S is false. And from a falsehood anything follows, so for the student's conclusions are only the student's problem.

It's a satisfying solution, but there's a wrinkle. What if we interpret the teacher's statement as being not about provability, but as a prediction of the future? Not "there's no proof", but "you specifically, as a deterministic algorithm running in finite time, will not find a proof"?

We can encode that by using two self-references instead of one. The student will be a program P, which on the evening of each day will enumerate all proofs in Peano arithmetic up to N symbols long, for some large N. The teacher's statement S will be: "The program P, quoted here in full, won't on any day find a proof that {S + knowledge of that day} implies the exam is the next day." All self- and mutual references can again be resolved by using the diagonal lemma. So P will either find a proof or won't. Which is it?

Well, if N is large enough, P can prove that the exam can't happen on the last day, then on the day before and so on, by the same method as in the original paradox. So on the very first day P will prove that S is false, and also that S implies the exam will be on Monday, and S implies the exam will be on Tuesday, all of that at the same time. In other words, even if the student is a bounded deterministic program without logical omniscience, the teacher is sti...
Model, Care, Execution
by Ricki Heicklen — 2023-06-26T04:05:50.065Z
Link: https://bayesshammai.substack.com/p/model-care-execution

When Alice harms Bob, it is likely that one of the following three things went wrong:

Model: Alice’s mental picture of what Bob would want, or of what is going on in the situation (which we’ll call her model), was incorrect and led her to behave in ways that caused Bob harm. If she’d had a better model, she would have acted differently, resulting in a better outcome for Bob.
Care prioritization: Alice prioritized something else—herself, another person, some abstract value—over Bob. Even if she cares a lot about Bob’s happiness, other factors outweighed her care for him in this situation.
Execution: Alice knew what Bob would want and cared enough about him, but failed to execute on bringing about the best Bob-outcome. Her execution failure could be the result of physical limitations, impulsivity, depression, structural barriers, or any of a number of other constraints.

Often, the problem is a combination of two or even all three of these: if I had taken the time to think a little about what you would want (a care problem), I would not have made the mistake I made (a model problem) or would have made a plan that actually worked (an execution problem).

We have found the framework of Model, Care, and Execution (MCE) to be useful when postmorteming cases of interpersonal harm. What went wrong, and how can we do better next time? More generally, what failure modes are most common for each of us, and what do we need to do to help each other improve in those areas?

Postmortem: Your Roommate Sleeps Through His Interview

It’s 1pm and your roommate Choni has fallen asleep on the couch. You see him sleeping and leave him alone. When he wakes up at 4:30pm, he has missed his 2pm job interview and is annoyed. What went wrong? Here are three plausible stories:

There was a modeling error: You falsely believed that Choni wanted to nap all afternoon—you didn’t know he had an important interview, or thought he had rescheduled it—so you decided to let him sleep, acting reasonably based on your bad model.
There was a care mismatch: You knew Choni would want to be awakened, but decided against doing so, perhaps because Choni has a tendency to punch people in the face when they wake him up, or because you were helping a friend in crisis and decided that was more urgent, or because you thought there was some chance he actually wouldn’t want to be woken up and y

...
The Fall of Rationality - The Senate of Admins
by Ace Delgado — 2023-06-26T01:49:00.457Z

Borrowing liberally from https://youtu.be/ecMfU3910WY. In fact, this won't make much sense without having seen that first. The following is definitely very accurate to the attitudes of the people and groups mentioned and should totally be thought of as representative.

Nick Bostrom, having successfully convinced EA to focus on long term risk of AI superintelligence over saving children from malaria and starvation, has marched to the gates of LessWrong at Mount Rationality with the conquered EAs in tow. The admins are in session, presided over by Emperor Yud.

Roko: Hail Nick Bostrom, conqueror of the wordcel EAs.

Admins: Hail Nick Bostrom!

Roko: Honorable Shape Rotators! And admins of LessWrong. Have you heard what is being proposed? Bostrom has asked that we, the admins of LessWrong, should give these EAs, these wordcels, LessWrong accounts and host their posts on our servers. To treat these inexact fools as brothers, equals. And what of our other adjacent communities? What of the Polyamorists? So loyal to us. What of the redditors, and the neoreactionaries which sends us so many recruits? If we make Rationalists of these wordcels, can we withhold LessWrong accounts from them? No! Then what becomes of the precious prize a LessWrong account once was? It becomes a cheap, common thing. It will be given away like Reddit Karma. I say no! We are Rationalists! Shape Rotators! Let us rid our minds of this poisonous idea. Ban their IP addresses. Send the rest to Reddit. Teach them once and for all what it is to make war on Rationality. That... is the Rationalist way.

Bostrom: Honorable Shape Rotators of LessWrong! You have greeted me as an author, who came in small triumph from his book sales. And I thank you for it. But you will have a hard decision to make here today! And it must not be I, as an author, who swayed your vote. I ask, therefore, of Yud, that a man who is neither poster nor admin, but a philosopher, a man of reason, be permitted to speak for me.

William MacAskill: Fellow Rationalists!

The assembled admins are outraged.

Admins: Wordcel! Wordcel!

MacAskill: Born a Wordcel, I became a Rationalist by choice.

Admins: EA! EA!

MacAskill: Bound to EA... I won my freedom. Fellow Rationalists. I am a teacher, and as a teacher I know that when I have tried to teach the same lesson for a hundred times and still the pupil does not ...
Another medical miracle
by Dentin — 2023-06-25T20:43:45.493Z

Or, "How I became even more disappointed in the medical practice in the USA than I already was".

I'm getting older - half a century now - and have recently been having health problems. It started with soreness in my right hip about four years ago, gradually getting worse and worse over time. At one point I damaged it by sitting in a funny position for too long; later on, I did what I thought was a very normal set of lunges in the gym that made me almost unable to walk for months afterward.

Then, about a year ago, I started having issues with my feet. At first, they were a little sore. They got worse and worse, eventually getting to the point where I again couldn't walk comfortably. They just would not heal, no matter how much time off I gave them.

In the past three years, I've seen half a dozen doctors. Hypotheses ranged from early onset arthritis, to joint damage, to tendon damage, to disc compression in my back. Physical therapy helped a small amount, but the gains tapered off quickly and did not last. I was making no progress, and had simply restricted activity in my life to the point where things were tolerable. None of the doctors had any idea what was going on, and every visit was hundreds of dollars to be told things I already knew. One doctor prescribed exercises for the elderly to build muscle tone, without even bothering to ask about my current exercise regime or current state of fitness. (At the time I could do sets of 40 push-ups and one-legged squats to the floor and back.) To say that I was frustrated is an understatement.

A bit over six months ago, I began to wonder if perhaps this was a side effect of the low dose statin I had started taking. The problems started prior to my taking the statin, but I found reports of statin side effects that looked vaguely similar to my issues. Perhaps I had fouled my cholesterol metabolism?

I concocted an experiment to stop taking the statin for a few weeks, and eat a ton of eggs to try to get more cholesterol in my system. I didn't know at the time that dietary cholesterol makes very little contribution to total cholesterol levels, but that turned out to be a good thing: I saw what seemed to be noticeable improvement. The next couple of months I learned about cholesterol metabolism and ran more experiments, eventually narrowing things down to the point that I knew statin use was not responsible. ...