trentmkelly/LessWrong-43k huggingface dataset:
For me the Christianity deal-breaker was meekness
by tcelferact — 2021-08-26T14:25:41.372Z

I was raised Catholic, became agnostic around 13, stopped thinking an afterlife made sense at 15, and noticed I was no longer religious at 16.

But I still spent the next ~12 years in pretty close proximity to Christianity. I did religious studies at school, I studied philosophy and theology at university, and most importantly I sung in church choirs, which meant I was regularly attending services.

I also wasn't crazy about the label 'atheist', as I didn't think my beliefs had much in common with famous or 'new' atheists (Richard Dawkins, Christopher Hitchens, AC Grayling). I often found their objections uninteresting, as they dealt with claims concerning things like 'metaphysics' and 'theodicy' that most Christians I knew didn't really care about. For those Christians, religion was more of a spiritual commitment or a decision to live a certain way, and, for example, their conservative approach to romance seemed like it could be valid even if the Church couldn't explain why God lets evil exist.

But if what you're doing is following a set of practices that are at most loosely inferred from a set of community texts and a history of community tradition, why take the extra step of identifying as Christian? Why not just say 'hey there's some good stuff in here, and I'll join in with the good and leave the bad'? What makes you want to take that extra step?

During the last 12 years I've wondered if I wanted to take that extra step again, until this year, when I realized any desire I had to do so was based in meekness.

Meek: Quiet, gentle, and easily imposed on; submissive.
- https://www.lexico.com/definition/meek

Meekness is praised in the Gospel:

Blessed are the meek,
for they will inherit the earth.
- Matthew 5:5

Not to mention that in the most important story in the whole Bible, the Son of God gives himself up *for death* without putting up a fight. He knows he will be betrayed but he waits for his captors to come and arrest him.

Meekness is a big deal for Christianity. And it shouldn't be. Meekness is not a virtue.

In the dictionary definition of meekness above, 'quiet, gentle' sounds fine; 'easily imposed on' and 'submissive', not so much. Meekness is when you don't stand up and ask for what you need, or it is when you allow what you need to be taken from you. This is a disaster for your personal wellbeing, and perhaps even worse when it comes to looking out for your...
Covid 8/26: Full Vaccine Approval
by Zvi — 2021-08-26T13:00:01.491Z

Great news, everyone. The Pfizer vaccine has been approved. Woo-hoo!

It will be marketed under the name Comirnaty. Doh!

(Do we all come together to form one big cominraty? Or should you be worried about the cominraties of getting vaccinated, although you should really be orders of magnitude more worried about the cominraties of not getting vaccinated? Did things cominraty or was there a problem? Nobody knows. Particle man.)

My understanding is that if a doctor were to prescribe the vaccine ‘off label,’ say to give to an 11 year old or to get someone an early booster shot, then they could potentially be sued for anything that went wrong, so in practice your doctor isn’t going to do this.

A reasonable request was made that my posts contain Executive Summaries given their length. Let’s do it!

Executive Summary of Top News You Can Use

Pfizer vaccine approved under the name Cominraty.
Vaccines still work. If you have a choice, Moderna > Pfizer but both are fine.
Boosters are still a good idea if you want even better protection.
Cases approaching peak.

Also, assuming you’re vaccinated, Krispy Kreme is offering two free donuts per day from August 30 until September 5.

Now that that’s out of the way, let’s run the numbers.

The Numbers

Predictions

Prediction from last week: 1,000,000 cases (+14%) and 8,040 deaths (+45%).

Results: 935k cases (+7%) and 7,526 deaths (+35%).

Prediction for next week: 950k cases (+2%) and 9,400 deaths (+25%).

I was confused how there could be such sharp peaks in other countries. It looks like we won’t get one of those. The trend lines seem clear, and it looks like we are approaching the peak. It would be surprising if we were still seeing increases week over week by mid-September, with the obvious danger that things could pick up again once winter hits.

Deaths

Date	WEST	MIDWEST	SOUTH	NORTHEAST	TOTAL
Jul 1-Jul 7	459	329	612	128	1528
Jul 8-Jul 14	532	398	689	145	1764
Jul 15-Jul 21	434	341	732	170	1677
Jul 22-Jul 28	491	385	1009	157	2042
Jul 29-Aug 4	693	477	1415	304	2889
Aug 5-Aug 11	705	629	2181	234	3749
Aug 12-Aug 18	912	851	3394	388	5545
Aug 19-Aug 25	1281	1045	4692	508	7526

Deaths continue to lag cases. News was slightly good, so adjusting expectations slightly in response. Peak should still be a month out or so.

Cases

Date	WEST	MIDWEST	SOUTH	NORTHEAST	TOTAL
Jul 1-Jul 7	27,413	17,460	40,031	7,065	91,969
Jul 8-Jul 14	45,338	27,544	68,129	11,368	152,379
Jul 15-Jul

...
MIRI/OP exchange about decision theory
by Rob Bensinger — 2021-08-25T22:44:10.389Z

Open Philanthropy's Joe Carlsmith and Nick Beckstead had a short conversation about

decision theory

a few weeks ago with MIRI's Abram Demski and Scott Garrabrant (and me) and LW's Ben Pace. I'm copying it here because I thought others might find it useful.

Terminology notes:

CDT is

causal decision theory

, the dominant theory among working decision theorists. CDT says to choose the action with the best causal consequences.
EDT is evidential decision theory, CDT's traditional rival. EDT says to choose the action such that things go best conditional on your choosing that action.
TDT is

timeless decision theory

, a theory proposed by Eliezer Yudkowsky in 2010. TDT was superseded by FDT/UDT because TDT fails on dilemmas like

counterfactual mugging

, refusing to pay the mugger.
UDT is

updateless decision theory

, a theory proposed by Wei Dai in 2009. UDT in effect asks what action "you would have pre-committed to without the benefit of any observations you have made about the universe", and chooses that action.
FDT is

functional decision theory

, an umbrella term introduced by Yudkowsky and Nate Soares in 2017 to refer to UDT-ish approaches to decision theory.

Carlsmith: Anyone have an example of a case where FDT and updateless EDT give different verdicts?

Beckstead: Is smoking lesion an example?

I haven't thought about how updateless EDT handles that differently from EDT.

Demski: FDT is supposed to be an overarching framework for decision theories "in the MIRI style", whereas updateless EDT is a specific decision theory.

In particular, FDT may or may not be updateless.

Updateful FDT is basically TDT.

Now, I generally claim it's harder to find examples where EDT differs from causal counterfactuals than people realize; eg, EDT and CDT do the same thing on smoking lesion. So be aware that you're not going to get the "standard view" from me.

However, TDT gets some problems wrong which UDT gets right, eg, counterfactual mugging.

Updateless FDT would not get this wrong, though; it appears to be all about the updatelessness.

To get EDT-type and CDT-type DTs to really differ, we have to go to Troll Bridge. EDT fails, FDT variants will often succeed.

Garrabrant: I feel like it is not hard to come up with examples where updateful EDT and CDT differ (XOR blackmail), and for the updateless question, I think the field is small enough that whatever Abram says is the “standa...
Learning can be deciding
by Jaime Ruiz — 2021-08-25T16:22:54.903Z

Consider the following questions:

A: "Will it rain in Paris at midnight tonight?"

B: "Will I blink in the next 5 seconds?"

If I wish to find the answer to question A, I might need to look at the conditions of the atmosphere over Paris and then use knowledge of how climate evolves. I might need readings of temperature, wind speed, humidity, and pressure, and I might also need complex mathematical models of the weather.

If I wish to find the right answer to question B, I can just decide to blink now.

There is an activity that I would roughly describe as "figuring out how the world is like." This activity is to be understood as a collective endeavour, not a personal one. Simple acts of information gathering, like opening a box to see what is inside, are examples of this activity. More impressive examples are Science and History, systematic disciplines that evolve across generations and produce large bodies of knowledge. The end goal of this activity might be a theory of everything.

I often hear people (particularly academics) speak as if figuring out how the world is like must always look like the process described above for answering Question A: science-y, focused on gathering data, distilling models or regularities, and applying them to particular cases.

However, Question B illustrates that in some cases, figuring how the world is like simply consists in deciding, or at least it includes a decision. This occurs when we pose questions about ourselves or about changes in the environment that occur as a consequence of our actions. For example: what will the temperature of this room be in the next 10 minutes? That depends on whether I will open the windows, turn on the AC, or set the furniture on fire.
How to turn money into AI safety?
by Charlie Steiner — 2021-08-25T10:49:01.507Z

Related: Suppose $1 billion is given to AI Safety. How should it be spent? , EA is vetting-constrained, What to do with people?

I

I have heard through the grapevine that we seem to be constrained - there's money that donors and organizations might be happy to spend on AI safety work, but aren't because of certain bottlenecks - perhaps talent, training, vetting, research programs, or research groups are in short supply. What would the world look like if we'd widened some of those bottlenecks, and what are local actions that people can do to move in that direction? I'm not an expert either from the funding or organizational side, but hopefully I can leverage Cunningham's law and get some people more in the know to reply in the comments.

Of the bottlenecks I listed above, I am going to mostly ignore talent. IMO, talented people aren't the bottleneck right now, and the other problems we have are more interesting. We need to be able to train people in the details of an area of cutting-edge research. We need a larger number of research groups that can employ those people to work on specific agendas. And perhaps trickiest, we need to do this within a network of reputation and vetting that makes it possible to selectively spend money on good research without warping or stifling the very research it's trying to select for.

In short, if we want to spend money, we can't just hope that highly-credentialed, high-status researchers with obviously-fundable research will arise by spontaneous generation. We need to scale up the infrastructure. I'll start by taking the perspective of individuals trying to work on AI safety - how can we make it easier for them to do good work and get paid?

There are a series of bottlenecks in the pipeline from interested amateur to salaried professional. From the the individual entrant's perspective, they have to start with learning and credentialing. The "obvious path" of training to do AI safety research looks like getting a bachelor's or PhD in public policy, philosophy, computer science, or math, (for which there are now fellowships, which is great) trying to focus your work towards AI safety, and doing a lot of self-study on the side. These programs are often an imprecise fit for the training we want - we'd like there to be graduate-level classes that students can take that cover important material in AI policy, technical alignment research, t...
The Validity of Self-Locating Probabilities (Pt. 2)
by dadadarren — 2021-08-25T01:53:17.616Z

In a previous post, I argued self-locating probabilities are not valid concepts. Many comments want me to use examples with solid numbers and bets. So here is the example that is essentially the Sleeping Beauty Problem with a small twist to highlight the problems of self-locating probability.

Cloning with Memory (with a coin toss)

The experiment is almost the same as before. Tonight during your sleep some mad scientist will scan your body at a molecular level to create a highly accurate clone. The process is highly advanced so the created person will accurately retain the original's memory to a degree not discernible by human cognition. So after waking up, there is no way to tell whether you are the Original or the Clone. However, the mad scientist will only perform the cloning if a fair coin toss lands on Tails (He will scan you regardless). Now, after waking up ask yourself this: "what is the probability that I am the Original?" Also, "What is the probability that the coin landed on Heads?"

Possible Answers

Let me present my answer first so it is out of the way: The probability of Heads is 1/2 since it is a fair coin toss. And the "probability I am the Orignal" is not a valid concept. "I" is an identification not based on anything but my first-person perspective. Whether "I" am the Orignal or the Clone is something primitive, not analyzable. Any attempt to justify this probability requires additional postulates such as equating "I" to a random sample of some sort.

Perhaps the more popular answer would be to say the probability that I am the Orignal is 2/3. Whereas the probability for Heads is 1/3. This would correspond to Thrider's camp in the Sleeping Beauty Problem. The rationale for it may not be the same for all Thirders. But it typically follows the Self-Indication Assumption, and finding myself exists eliminates the possibility that I am the Clone and the coin landed Heads.

Another camp would be saying the probability of Heads is 1/2, and the probability that I am the Orignal is 3/4. This corresponds to the Halfer camp in the Sleeping Beauty problem. This camp endorses the "no new information" argument. But have different reasons regarding how to update given self-locating information.

If You Say P(Heads)=1/3

Say the Mad scientist wants to encourage people to participate in his experiment. So he decides to give 2 gold...
Ordering the Special Rolls
by Alex Hollow — 2021-08-24T23:34:16.721Z
Link: https://alexhollow.wordpress.com/2021/08/24/ordering-the-special-rolls/

It took me three years to realize I should be ordering the special rolls when I got all-you-can-eat sushi.

At first, I reasoned that the other sushi was free, and paying infinity times as much for a slightly better piece of sushi couldn’t be worth it. In a sense, this was correct – for any non-zero number, you would want to avoid multiplying by infinity. The first few times I went, I thought through the decision. I had proof that I was willing to pay $0 for the normal rolls. Given that, was it worth it to spend money to get slightly better sushi? I felt like it wasn’t. Free was a compelling price point, and spending money to get a special roll felt like a waste.

But the special rolls at my local sushi joint cost $1. And they were definitely much better than most of the other rolls. One had mango, one used jalapeno to great success, and one was baked and covered in scallops and delicious flaky bits. (Though they weren’t all hits. One had a very smelly mushroom topping that caused my tablemates to literally gag.)

One day, looking at the menu, I realized the correct question to ask. Not “do I want to spend money right now?”, but “would I be willing to spend $1 more to get the special rolls at other restaurants?”. Once I framed the question like that, the answer was clear: at most other restaurants, I would rather pay $10 for a special roll than $9 for a regular roll. The marginal cost, not the percentage difference, is what matters.

Once I made this decision, I split my choices among the regular and special rolls. Not only did those rolls taste better, but having a wider variety made eating all I could much more enjoyable.
Welcome & FAQ!
by Ruby — 2021-08-24T20:14:21.161Z

The AI Alignment Forum was launched in 2018. Since then, several hundred researchers have contributed approximately two thousand posts and nine thousand comments. Nearing the third birthday of the Forum, we are publishing this updated and clarified FAQ.

Minimalist, watercolor sketch of humanity spreading across the stars by VQGAN

I have a practical question concerning a site feature.

Almost all of the Alignment Forum site features are shared with LessWrong.com; have a look at the

LessWrong FAQ

for questions concerning the

Editor

,

Voting,

Questions

,

Notifications & Subscriptions

,

Moderation

, and more.

If you can’t easily find the answer there, ping us on Intercom (bottom right of screen) or email us at team@lesswrong.com

What is the AI Alignment Forum?

The Alignment Forum is a single online hub for researchers to discuss all ideas related to ensuring that transformatively powerful AIs are aligned with human values. Discussion ranges from technical models of agency to the strategic landscape, and everything in between.

Top voted posts include

What failure looks like

,

Are we in an AI overhang?

, and

Embedded Agents

. A list of the top posts of all time can be

viewed here

.

While direct participation in the Forum is limited to deeply established researchers in the field, we have designed it also as a place where up-and-coming researchers can get up to speed on the research paradigms and have pathways to participation too. See How can non-members participate in the Forum? below.

We hope that by being the foremost discussion platform and publication destination for AI Alignment discussion, the Forum will serve as the archive and library of the field. To find posts by sub-topic, view the

AI section of the Concepts page

.

Why was the Alignment Forum created?

Foremost, because misaligned powerful AIs may pose the greatest risk to our civilization that has ever arisen. The problem is of unknown (or at least unagreed upon) difficulty, and allowing the researchers in the field to better communicate and share their thoughts seems like one of the best things we could do to help the pre-paradigmatic field.

In the past, journals or conferences might have been the best methods for increasing discussion and collaboration, but in the current age we believe that a well-designed online forum with things like immediate publication, distributed rating of quality (i.e. “peer review...
Extraction of human preferences 
👨→🤖
by arunraja-hub — 2021-08-24T16:34:14.408Z

Introduction

Developing safe and beneficial reinforcement learning (RL) agents requires making them aligned with human preferences. An RL agent trained to fulfil any objective in the real world will probably have to learn human preferences in order to do well. This is because humans live in the real world, so the RL agent will have to take human preferences into account as it optimizes its objective. We propose to first train an RL agent on an objective in the real world so that it learns human preferences as it is being trained on that real world objective and then use the agent’s understanding of human preferences to build a better reward function.

We build upon the work of

Christiano et al. (2017)

where they trained a human preference predictor as the reward signal. The preference predictor was trained on environment observations to give a high reward for states where the human preferences were satisfied and a low reward for states where the human preferences were not satisfied:

In our experiments, the reward predictor takes the activations (hidden states) of the RL agent as the input and is trained to predict a binary label depending on whether human preferences are satisfied or not.

We first train an RL agent in an environment with some reward function that’s not aligned with human preferences. After training the RL agent, we try different transfer learning techniques to transfer the agent’s knowledge of human preferences to the human preferences predictor. Our goal is to train the human preferences predictor to get a high accuracy with a small amount of labeled training examples.

The idea of training a human preference predictor off of the RL agent’s hidden (internal) states was already validated by

Wichers (2020)

. We wanted to validate it further by trying other techniques to train a human preference predictor, as well as to validate it in more environments.

Research question

The main research question we wanted to answer is: “Are human preferences present in the hidden states of an RL agent that was trained in an environment where a human is present?”

In order to answer this question, we conjectured that if human preferences are indeed present in the hidden states, we could:

leverage the hidden states of the RL agent in order to train an equally accurate preference predictor with a smaller amount of data
bootstrap a reward function that would

...
Framing Practicum: Turnover Time
by johnswentworth — 2021-08-24T16:29:04.701Z

This is a

framing practicum

post. We’ll talk about what turnover time is, how to recognize when turnover time is relevant in the wild, and what questions to ask when you find it. Then, we’ll have a challenge to apply the idea.

Today’s challenge: come up with 3 examples of approximately-independent turnover which do not resemble any you’ve seen before, and estimate their turnover times. They don’t need to be good, they don’t need to be useful, they just need to be novel (to you).

Expected time: ~15-30 minutes at most, including the Bonus Exercise.

What’s Turnover Time?

The proteins in biological cells are often damaged by radiation or uncontrolled chemical reactions. To avoid damaged proteins building up, organisms turn over their proteins regularly: they’re constantly broken down and replaced with fresh proteins. As long as the stressors which damage proteins are at a constant level, the proportion of damaged proteins reaches an equilibrium, at which turnover balances damage rate.

The typical time it takes a protein to turn over is its turnover time. (This might be an average time, median time, half-life, etc… it’s a Fermi estimate, so it doesn’t need to be exact.)

Another example, with a more abstract (but more general) notion of “turnover”: air molecules in a box. We can draw an imaginary divider down the middle, and then think about the typical time it takes a molecule on one side to bounce over to the other side. This is a turnover time.

What To Look For

In general, turnover time should come to mind whenever we have many parts which change, are created and destroyed, or which move between states roughly-independently over time.

Useful Questions To Ask

Imagine a biological cell is hit with a pulse of radiation or a splash of harsh chemical, so that a bunch of its proteins are damaged. Assuming the cell is still functional, how long will it take the proportion-of-damaged-proteins to return to equilibrium? The turnover time is a good Fermi estimate: we’re asking how long it will take for the damaged proteins to be replaced by new proteins, and the turnover time is the typical time it takes proteins to be replaced by new proteins.

Note that this is just a rough estimate - the cell might detect the extra damage and upregulate protein turnover, or a lot of the protein-turnover-machinery may itself be damaged; either of these would throw off the estimate, but probably not...