trentmkelly/LessWrong-43k huggingface dataset:
Lying to Save Humanity
by cebsuvx — 2022-11-14T23:04:16.848Z

You are an astronaut on an expedition to Mars when you encounter an Alien. Luckily, it speaks English, and you begin an exchange of cultures: history, art, technology, and food, whereupon the Alien interrupts:

“Our species has strong feelings about ice cream flavors. Tell me, what is your favorite flavor? If you answer chocolate, all is well, and our races can coexist. If you answer vanilla, we will annihilate every human.”

This is a dilemma, for you strongly prefer vanilla ice cream, but you have preferences against both lying and having humanity annihilated. You consider your moral framework.

Path 1

“I like chocolate ice cream,” you lie. One tiny moral lapse is worth saving all humanity.

“Aha! Liar!” screams the Alien. “The question was a test, for we have mind-reading technology. How do you think I learned English so quickly? In retribution for your treachery, we shall destroy all humans, starting with you.”

Outcome: Humanity annihilated.

Path 2

“I like vanilla ice cream,” you answer truthfully. You would never compromise your moral principles. Also, you aren’t convinced these Aliens could defeat your nukes.

“Very brave of you,” says the Alien. “What our species truly values is integrity. As a reward, I gift you this machine, which provides limitless energy, faster-than-light travel, immortality, and a few other things…”

Outcome: Humanity enters a trillion-year golden age.

Path 3

“I like…” you begin, then draw your phaser pistol and shoot the Alien in the face.

The Alien collapses, dead. You stand over its smoking corpse, not feeling any remorse, for you don’t think non-human species carry moral weight, and you are pleased at finding a way out of your ethical dilemma.

You wait to see if other Aliens might spring from behind the rocks, or if you are atomized by an orbital laser. Nothing happens.

Eventually, you decide there’s no point in wasting good protein, and you’re not vegetarian. You haul the Alien back to base, announce “Looks like meat’s back on the menu, boys!” to your fellow astronauts, and cook the Alien for dinner.

Outcome: your dinner has similar nutritional content to your customary astronaut nutrient slurry, but it tastes better. Zero lessons have been learned.
Moral contagion heuristic
by Mvolz — 2022-11-14T21:17:53.107Z

"Remember that time you went roller-skating with pink eyebrows? I was embarrassed even though it wasn't me, but because you're my mom." - My 7 year old

A prevalent and arguably universal human cognitive bias is the contagion heuristic - the idea that mere contact or association with a bad thing or person reflects badly on you. At a relatively young age, my daughter had picked up that others might judge her for her association with me.

"A contagion heuristic is psychological heuristic which follows the law of contagion and the law of similarity."

But would these others be rational in doing so?

In fact, the reason my eyebrows were pink were because my daughter had coloured them that way herself, the day before. We are also connected socially; they might consider her culpable for not noticing earlier and preventing me from leaving the house in such a state. (She ushered me to the bathroom on arriving at roller skating, but it was difficult to get it off without make-up remover.)

But there are clearly times in which the contagion heuristic is wrong. The clearest examples come from inaccurate assessment of the contagiousness of a disease. Historically some people believed diseases like cancer were contagious. Cancer patients were treated badly or shunned accordingly. The same was true for HIV patients, where ignorance about transmission risk caused people to be afraid to shake their hands. (Princess Diana gained moral status for shaking hands with HIV patients; Boris Johnson lost some for doing the same with COVID-19 patients).

With diseases, science helps us get this right. But what about moral contagion? Is morality... contagious? Or at least, does associating with someone with poor morals signal you might also have poor morals?

"Moral contagion is a phenomenon in which individuals or objects take on the moral essence of the people who are associated with them." [1]

First, we can say what little evidence we have suggests that morality is at least in part contagious. This makes sense; morality is at least in part socially constructed, and moral messages do spread contagiously[2] and can have long term impact.[3]

But even if morality weren't contagious at all, the law of similarity still applies here. In the case of Jeffery Epstein, it was less that people associated with him were thought to have become sexual predators because of him, bu...
Will we run out of ML data? Evidence from projecting dataset size trends
by Pablo Villalobos — 2022-11-14T16:42:27.135Z
Link: https://epochai.org/blog/will-we-run-out-of-ml-data-evidence-from-projecting-dataset

Summary: Based on our previous analysis of trends in dataset size, we project the growth of dataset size in the language and vision domains. We explore the limits of this trend by estimating the total stock of available unlabeled data over the next decades.

Read the full paper in arXiv.

Our projections predict that we will have exhausted the stock of low-quality language data by 2030 to 2050, high-quality language data before 2026, and vision data by 2030 to 2060. This might slow down ML progress.

All of our conclusions rely on the unrealistic assumptions that current trends in ML data usage and production will continue and that there will be no major innovations in data efficiency. Relaxing these and other assumptions would be promising future work.

Figure 1: ML data consumption and data production trends for low quality text, high quality text and images.

Historical projection	Compute projection
Low-quality language stock	2032.4
[2028.4 ; 2039.2]	2040.5
[2034.6 ; 2048.9]
High-quality language stock	2024.5
[2023.5 ; 2025.7]	2024.1
[2023.2 ; 2025.3]
Image stock	2046
[2037 ; 2062.8]	2038.8
[2032 ; 2049.8]
Table 1: Median and 90% CI exhaustion dates for each pair of projections.

Background

Chinchilla's wild implications argued that training data would soon become a bottleneck for scaling large language models. At Epoch we have been collecting data about trends in ML inputs, including training data. Using this dataset, we estimated the historical rate of growth in training dataset size for language and image models.

Projecting the historical trend into the future is likely to be misleading, because this trend is supported by an abnormally large increase in compute in the past decade. To account for this, we also employ our compute availability projections to estimate the dataset size that will be compute-optimal in future years using the Chinchilla scaling laws.

We estimate the total stock of English language and image data in future years using a series of probabilistic models. For language, in addition to the total stock of data, we estimate the stock of high-quality language data, which is the kind of data commonly used to train large language models.

We are less confident in our models of the stock of vision data because we spent less time on them. We think it is best to think of them as lower bounds rather than accurate estimates.

Results

Finally, we compare the...
I (with the help of a few more people) am planning to create an introduction to AI Safety that a smart teenager can understand. What am I missing?
by Tapatakt — 2022-11-14T16:12:22.760Z

Disclaimer: My English isn't very good, but do not dissuade me on this basis - the sequence itself will be translated by a professional translator.

I want to create a sequence that a fifteen or sixteen year old smart school student can read and that can encourage them to go into alignment. Right now I'm running an extracurricular course for several smart school students and one of my goals is "overcome long inferential distances so I will be able to create this sequence".

I deliberately did not include in the topics the most important modern trends in machine learning. I'm optimizing for the scenario "a person reads my sequence, then goes to university for another four years, and only then becomes a researcher." So (with the exception of the last part) I avoided topics that are likely to become obsolete by this time.

Here is my (draft) list of topics (the order is not final, it will be specified in the course of writing):

Introduction - what is AI, AGI, Alignment. What are we worried about. AI Safety as AI Notkilleveryoneism.
Why AGI is dangerous. Orthogonality Thesis, Goodhart's Law, Instrumental Convergency. Corrigibility and why it is unnatural.
Forecasting. AGI timelines. Takeoff Speeds. Arguments for slow and fast takeoff.
Why AI boxing is hard/near to impossible. Humans are not secure systems. Why even Oracle AGI can be dangerous.
Modern ML in a few words (without math!). Neural networks. Training. Supervised Learning. Reinforcement Learning. Reward is not the goal of RL-agent.
Interpretability. Why it is hard. Basic ideas on how to do it.
Inner and outer alignment. Mesa-optimization. Internal, corrigible and deceptive alignment. Why deceptive alignment seems very likely. What can influence its probability.
Decision theory. Prisoner's Dilemma, Newcomb's problem, Smoking lesion. CDT, EDT and FDT.
What exactly are optimization and agency? Attempts to define this concepts. Optimization as attractors. Embedded agency problems.
Eliezer Yudkowsky's point of view. Pivotal actions. Why it can be useful to have imaginary EY over your shoulder even if you disagree with him.
Capability externalities. Avoid them.
Conclusion. What can be done. Important organisations. What are they working on now?

What else should be here? Maybe something should not be here? Are there reasons why the whole idea can be bad? Any other advices?
Two New Newcomb Variants
by eva_ — 2022-11-14T14:01:05.097Z

Two Newcomb variants to add to the list of examples where optimal choice and optimal policy are diammetrically opposed. I don't think problems these exist anywhere else yet.

4 Boxes Problem

In a game show there are 4 transparent boxes in a row, each of which starts off with $1 inside. Before the human player enters, the show's 4 superintelligent hosts have a competition: they each have a box assigned to them, and they win if the human chooses their box. To motivate the human they cannot alter their box at all, but they are able to put $100 into any or all of the other 3 boxes.

Our first human is an adherent of CDT. Since the boxes are transparent, and he wants to get money, he obviously chooses the one with the most. It's not like his decisions can change what the hosts have already done. Putting money into the other boxes would only have hurt the hosts' chances, so they didn't do that. Instead, all the boxes have only the original $1. CDT is disappointed, but picks one at random and goes home.

Our second human is an adherent of EDT. She knows if she sees how much is in the boxes, she'll pick the one with the most, which will end up being only $1. Because of this, she blindfolds herself before walking on stage. She weighs the two left boxes together, to see how much is in them total, and the two on the right as well. "If the left two are heavier I'll pick one of them at random," she says, "if the two right I'll pick one of those, and if the same I'll pick any at random". EDT was quite happy when she found out she was going to do this. She'd worried that she wouldn't get the blindfold on in time and only get $1 but it seemed to work out. The hosts predicted this, of course, so the two on the left put $100 in eachothers boxes, to increase their own odds, and the two on the right did the same, and EDT picks at random and leaves with $101. [I'm not completely sure EDT can't do better than this, so corrections with even more elaborate schemes encouraged]

Our third human is an adherent of FDT. She considers the policies she could implement, and decides that she will take one of the boxes with the least money. Not wanting to give their opponents an advantage, the hosts all put $100 in every other box, and FDT leaves with $301.

5 Boxes problem

This time the game show wants to see who is best at prediction. Each of the 4 superintelligent hosts must try to ...
Improving Emergency Vehicle Utilization
by jefftk — 2022-11-14T14:00:01.140Z

Every ambulance that goes out represents a massive waste of
potential: it doesn't have any passengers! Ambulances need space to
transport people on the return trip, and it really says something
about humanity that we've gone so many years with that capacity
completely idle on the outward journey. It would be one thing if we
were talking about vehicles that travel normally across the
city—though of course I'd love to see more carpooling—but
these are high-speed vehicles! Traffic yields to open a path, they
don't have to wait for stoplights, they're the fastest way across the
city.

The reason we haven't fixed this already is that it's only recently
that smartphones have become widely available. Before, it would have
been too hard to connect travelers with first responders going their
direction within the narrow window between an alert coming in and a
vehicle going out. After, bypassing taxi supply cap regulations was a
larger market. But now, the time is ripe: introducing Ambolyft!

While our initial fleet is currently entirely ambulances, we're
expanding to police as well. Great side hustle opportunity!

Comment via: facebook, mastodon
X-risk Mitigation Does Actually Require Longtermism 
by DragonGod — 2022-11-14T12:54:53.237Z

Introduction

Myself — and many others — have argued that longtermism is not needed to argue for x-risk mitigation. That all such actions can be adequately justified within a neartermist framework, and given the poor reception/large inferential distance/inaccessibility of longtermist arguments, we might be better served arguing for x-risk mitigation within a strictly near termist framework.

But this is not fully accurate.

Averting extinction makes sense in near termist ethical frameworks (8 billion people dying is very bad), but extinction is not the only category of existential risk, and it's the only one that can readily be justified within neartermist frameworks.

Longtermism and Existential Risks

Excluding extinction, all the other existential risks — the very concept of an "existential risk" itself — implicitly rely on longtermism.

Toby Ord defined an existential catastrophe as an event that permanently curtails the longterm potential of humanity/human civilisation.

A few classes of existential catastrophe other than extinction:

Value lockin

Irreversible technological regression

Any discrete event that prevents us from reaching technological maturity

Any discrete event that leads to Bostrom's "Astronomical Waste"

(I would also add "technological stagnation" to the list. It's not a discrete event [so Ord didn't consider it as a catastrophe], but it has the same effect of curtailing the long term potential of human civilisation.)

We cannot even conceive of an existential catastrophe without a framework of "longterm potential of human civilisation", concepts of "technological maturity", "astronomical waste", etc.

All of these are concepts that are defined only within a longtermist framework.

Thus, existential risk mitigation is inherently a longtermist prospect.

Caveats

While extinction risks aren't the only existential risks, they are the one that has attracted the supermajority of attention and funding.

Excluding extinction risk mitigation, other longtermist projects looks like:

Grand strategy for humanity

Promoting more adequate/resilient institutions

Better mechanisms for coordination and cooperation

Governance of advanced/speculative technologies

Space settlement and colonisation

Etc.

Some of these actions may not have that large an effect on near term extinction risks.

Maybe there's an argument that we should argue for taking actions to mitigate near term e...
Why don't we have self driving cars yet?
by Linda Linsefors — 2022-11-14T12:19:09.808Z

1) Is it because of regulations?

2) Is it because robustness in the real world (or just robustness in general) turns out to be very hard for current AI systems, and robustness is much more important for self driving cars than other areas where we have seen more rapid AI progress?

3) Is it because no one is trying very hard? I.e. AI companies are spending much less compute on self driving car AIs compared to language models and image generators. If this is the case, why? Do they not expect to see a lot of profit from self driving cars?

Some other reason or some combination of the above?

I'm mostly interested in learning to what extent 2 is the cause, since this has implications in AI forecast, both timelines and what trajectories to expect.
Eigenvalues for Distance from The Buddhist Precepts And The Ten Commandments
by benjamin.j.campbell — 2022-11-14T05:50:24.100Z

I have a little ethical morality deep dive I could use your help with. I'm trying to work out how universally accepted the Buddhist precepts and the Christian 10 Commandments are in the LW community, so that I can engineer more utopian futures. I'm also trying to work out how far short of our own goals we fall on each one of the precepts/commandments. Then we can collapse these distinct eigenvalues down into a root-mean-square (RMS) of how far off our moral compass we are, collectively.

I've made two Google Forms quizzes.

Please do each one until you get sick of answering questions.

Nothing is mandatory.

Everything is encouraged.

I'll share my research data with anyone who wants it.

I could also make the Actions quiz more concrete, to focus on individual scenarios. I found myself tempted to adjust my scores as I was doing this to be more in line with what I thought I "should" answer. The is/ought brainrot is real.

Quiz 1: Level of Agreement with The Precepts/Commandments
https://forms.gle/hcxdoQRYx7FZ8yCy5

Quiz 2: How Much My Actions Align with The Precepts/Commandments

https://forms.gle/twoymHm89eYERf2h9

Thanks for participating and engaging! I hope it's fun and not too morally depressing to compare ourselves to our utopias. The Good News is that if you can be bad, you can also be good. And there's very little reason not to be.
AI Safety Microgrant Round
by Chris_Leong — 2022-11-14T04:25:17.510Z

We are pleased to announce an AI Safety Microgrants Round, which will provide micro-grants to field-building projects and other initiatives that can be done with less.

We believe there are projects and individuals in the AI Safety space who lack funding but have high agency and potential. We think individuals helping to fund projects will be particularly important given recent changes in the availability of funding. For this reason, we decided to experiment with a small micro-grant round as a test and demonstration of this concept.

To keep the evaluation simple, we’re focusing on field-building projects rather than projects that would require complex evaluation (we expect that most technical projects would be much more difficult to evaluate).

We are offering microgrants up to $2,000 USD with the total size of this round being $6,000 USD (we know that this is a tiny round, but we are running this as a proof-of-concept). One possible way this could pan out would be two grants of $2000 and two of $1000, although we aren’t wedded and we are fine with request for less. We want to fund grant requests of this size where EA funds possibly has a bit too much overhead.

The process is as follows:

Fill out this form at

microgrant.ai

(<15 min).
Shortlisted applications receive a follow-up call within two weeks after the applications close
We may also send an email with follow-up questions if we need more information. We would expect a reply within a few days so that we could confirm the grant within a week

To inspire applicants, here are some examples of projects where a microgrant would have been helpful:

Chris recently received a grant through an Australian-based organisation to hire two facilitators at $1,000 each for running a local version of the AGI safety fundamentals course. Intro fellowships have proven to be a great way of engaging people and we would be excited about funding this if it would help kickstart a new local AI safety group rather than just being an isolated project.
We might have funded something like the

AI safety nudge competition

to help people overcome their procrastination, and are excited about the potential to motivate a large number of people to accelerate their AI safety journeys and about experimenting with a potentially scalable intervention.
The

Sydney AI Safety Fellowship

was originally going to be funded with three p

...
Estimating the probability that FTX Future Fund grant money gets clawed back
by spencerg — 2022-11-14T03:33:05.507Z
Link: https://manifold.markets/SpencerGreenberg/will-projects-which-received-grants

It would be great to have more people involved in estimating the probability that FTX Future Fund grants end up getting clawed back (i.e., that projects that received such grants are asked to give the money back). This is a very important question for grant recipients right now, with a lot of murkiness around it - your forecasting help is appreciated!

If you have an opinion on the topic, would like to help make the forecasts more accurate, or are just interested in the question, here are three forecasting tournament pages related to the question:
In Defence of Temporal Discounting in Longtermist Ethics
by DragonGod — 2022-11-13T21:54:38.706Z

Introduction

My thoughts on temporal discounting in longtermist ethics.

Two Senses of Normative Ethics

There are two basic senses of a normative moral system:
1. Criterion of judgment: "what is right/good; wrong/evil?"
2. Decision procedure: "how should I act? What actions should I take?"

We can — and I argue we should — distinguish between these two senses.

Consider consequentialism:

Criterion of Judgment

Using consequentialism as a criterion of judgment, we can evaluate the actual ex post consequences of actions (perhaps over a given timeframe: e.g. to date, or all of time [if you assume omniscience]) to decide whether an action was right/wrong.

Decision Procedure

However, when deciding what action to take, we cannot know what the actual consequences will be.

For a decision procedure to be useful at all — for it to even qualify as a decision procedure — it must be actionable. That is, it must be possible — not only in principle, but also in practice — to act in accordance with the decision procedure. That is, the decision procedure must be:

Directly evaluable or
Approximately evaluable or
Robustly estimable or
Etc.

We should have a way of determining what course of action the procedure actually recommends in a given scenario. As such, the procedure must be something we can evaluate/approximate/estimate ex ante, before we know the actual consequences of our actions.

Because of coherence arguments, I propose that a sensible decision procedure for consequentialists is the "ex ante expected consequences[1] of (policies over) actions"[2].

Against Discounting

When considering how we should value the longterm future, I feel like MacAskill/Ord conflate/elide between the two senses[3].

They make a compelling argument that we shouldn't discount the interests (wellbeing/preferences) of future people:

Assigning a fixed discount rate per year (e.g. 1%) to future people and extrapolating back in time, you get a conclusion like the interests of Ancient Egyptian royalty (e.g. Cleopatra) outweigh the interests of everyone alive today, and this seems obviously lacking.

Temporal Discounting and the Two Senses of Normative Ethics

I agree that from a perspective of "morality as criterion of judgment", we should not discount the interests of future people. Plausibly, in a consequentialist-criterion-of-judgment-framework almost all the value of any action is det...