trentmkelly/LessWrong-43k huggingface dataset:
Introducing the ML Safety Scholars Program
by Dan H — 2022-05-04T16:01:51.575Z

Program Overview

The Machine Learning Safety Scholars program is a paid, 9-week summer program designed to help undergraduate students gain skills in machine learning with the aim of using those skills for empirical AI safety research in the future. Apply for the program here by May 31st.

The course will have three main parts:

Machine learning, with lectures and assignments from MIT
Deep learning, with lectures and assignments from the University of Michigan, NYU, and Hugging Face
ML safety, with lectures and assignments produced by

Dan Hendrycks

at UC Berkeley

The first two sections are based on public materials, and we plan to make the ML safety course publicly available soon as well. The purpose of this program is not to provide proprietary lessons but to better facilitate learning:

The program will have a Slack, regular office hours, and active support available for all Scholars. We hope that this will provide useful feedback over and above what’s possible with self-studying.
The program will have designated “work hours” where students will cowork and meet each other. We hope this will provide motivation and accountability, which can be hard to get while self-studying.
We will pay Scholars a $4,500 stipend upon completion of the program. This is comparable to undergraduate research roles and will hopefully provide more people with the opportunity to study ML.

MLSS will be fully remote, so participants will be able to do it from wherever they’re located.

Why have this program?

Much of AI safety research currently focuses on existing machine learning systems, so it’s necessary to understand the fundamentals of machine learning to be able to make contributions. While many students learn these fundamentals in their university courses, some might be interested in learning them on their own, perhaps because they have time over the summer or their university courses are badly timed. In addition, we don’t think that any university currently devotes multiple weeks to AI Safety.

There are already sources of funding for upskilling within EA, such as the Long Term Future Fund. Our program focuses specifically on ML and therefore we are able to provide a curriculum and support to Scholars in addition to funding, so they can focus on learning the content.

Our hope is that this program can contribute to producing knowledgeable and motivated undergraduates who can then use thei...
What are the best examples of catastrophic resource shortages?
by jasoncrawford — 2022-05-04T14:37:52.108Z
Link: https://rootsofprogress.org/catastrophic-resource-shortages

A while ago I posed a question on Twitter:

What's an example of a significant resource that the world has actually run out of?

Not a local, temporary shortage, or a resource that we gracefully transitioned away from, but like a significant problem caused by hitting some limit we didn't prepare for?

Here, in essay form, is the discussion that followed:

Lots of things were predicted to have shortages (food, metals, Peak Oil) and they never quite arrived. (Julian Simon was famous for pointing out this kind of thing.) But a common argument from conservationists and environmentalists is that we are running out of some critical resource X and need to conserve it.

Now, it’s true that specific resources can and sometimes do get used up. Demand can outpace supply. There are various ways to respond to this:

Reduce consumption
Increase production
Increase efficiency
Switch to an alternative

Increasing production can be done by exploring and discovering new sources of a material, or—this is often overlooked—by reducing costs of production, so that marginally productive sources become economical. New technology can often reduce costs of production this way, opening up resources previously thought to be closed or impractical. One example is fracking for shale oil; another is the mechanization of agriculture in the 19th and 20th centuries, which reduced labor costs, thereby opening up new farmland.

Increased efficiency can be just as good as increased production. However, if the new, more efficient thing is not as desirable as the old method, I would classify this as a combination of increased efficiency and reduced consumption (e.g. low-flow toilets, weak shower heads).

When supplies are severely limited, we often end up switching to an alternative. There are many ways to satisfy human desires: Coal replaced wood in 18th century England. Kerosene replaced whale oil, then light bulbs replaced kerosene. Plastic replaced ivory and tortoiseshell. Again, if the alternative is less desirable along some key dimension, then this is also a form of reduced consumption, even if total volumes stay the same.

However, the conservationist approach is always some form of reduced consumption: typically a combination of reduced absolute consumption, efficiency improvements that reduce quality and convenience, and/or switching to less-desirable alternatives. The arguments that people have over resources...
[Book review] No nonsense meditation
by Valdes — 2022-05-04T12:15:53.691Z

This is a book review of the book No nonsense meditation by Steven Laureys.
I read it in the context of a personal literature review project on the topic of productivity and well being.

How I read

I read this book almost in its entirety but I did skim a few parts and skipped a chapter.

Description and opinion

I read this book because I was looking for a meditation manual that wouldn't fuse its instructions with a complete life philosophy or religion.
That was not what this book is or tries to be. Instead, the bulk of the pages is spent defending and justifying the benefits and non-religious status of meditation. Answering to attacks I did not care about.
I wouldn't say this is a good popular science book either. It lacks structure and the argumentation is at times quite shoddy.
Some rigor and subsection titles would have been a great help.
Yet, it might very well be the best book to read in terms of popular science on meditation done by someone with the right background. At least, I do not know of a better one.

Note that this book was written by a neuroscience researcher who spoke to a couple of very knowledgeable meditation masters and has himself done a lot of meditation.

Main takes

Loving kindness and mindful meditation are easy to learn (at least the basic) and can be great sources of calm, happiness, and focus.

Some meditation masters can do some impressive things when monitored with modern brain imaging techniques. They can have a lot of control on the results in ways most people have no control at all.

Recommendation

While this book isn't necessarily a time waster I would advise to read it only if either of the following is true.

You feel meditation is a bit ridiculous or pointless and want to give a scientist a chance to convince you.

You are curious about the interesting abilities displayed by meditation masters that can be described without any vocabulary or concepts specific to meditation or Buddhist practices.
For example, some show an abnormal ability to handle pain.

The book contains testimonies I did not especially like. If you want to skip them you should know they always conclude chapters. Hence you can skip to the beginning of the next chapter.
[Book review] The anxiety toolkit
by Valdes — 2022-05-04T12:15:49.417Z

This is a book review of the book The anxiety toolkit by Alice Boyes.
I read it in the context of a personal literature review project on the topic of productivity and well being.
If you are more interested by advice on productivity and wellbeing than by this specific book I advise you to read the project report first, which condenses the advice from multiple sources (including this book).

How I read

If we are to count words, I probably read around half this book. By which I do not mean that I stopped halfway but rather that I skimmed large portions when I thought the advice didn't apply to me.
I do no think that invalidates my review of the book too much as I have an overall good idea of what this book says and how it is written.

Description and opinion

I had a bad first contact with this book as it lacked some form of attention to details and care for truth and precision that I value in many things.
Nevertheless, I think this book has a lot to offer, including to rationalists and mathematicians.

Mostly, this is a book about how to deal with anxiety issues and be productive

I did not read other similar books I could compare it to.
But I can say that many of its points resonated with my perception of my own issues regarding anxiety and many of its advice seemed good; or at least close enough to good ideas that I could easily come up with seemingly useful techniques, using the book as a source of inspiration.

Many points and ideas rang true to my own issues with anxiety. But I am a soon-to-be-ex student with background-anxiety issues and perfectionist tendencies, which I think is a profile this book is suited for. Your mileage may vary.
Note that the techniques presented are based on cognitive behavioral therapy.
I have been told that CBT has mostly impressive but short term effects. For me this is not much of an issue as my satisfaction with the book wasn't about following the advice to the letter.

Main takes

A pattern to follow : when hesitating on a decision, ask yourself "are you optimizing a decision or are you deliberately wasting time"?

Practice hesitating less. This is not because hesitation is a bad thing but as a way to correct your emotional tendencies. Feel free to hesitate a lot when the stakes are high. Of course, this applies only if you have a base tendency to hesitate a lot and suffer from it.

Try to plan the next action of a given project as s

...
[Book review] Getting things done
by Valdes — 2022-05-04T12:15:46.517Z

This is a book review of the book Getting things done by David Allen.
I read it in the context of a personal literature review project on the topic of productivity and well being.
If you are more interested by advice on productivity and wellbeing than by this specific book I advise you to read the project report first, which condenses the advice from multiple sources (including this book).

How I read

I started skipping many parts of this book near its middle.
Mostly, I read entirely the first quarter, read by bits in the middle, occasionally skipping an entire chapter, and then read the end.

Description and opinion

A good book packed with insight on being productive written by someone with a good track record of making people productive.
Specialized in executive positions but modular and made to be adaptable.
The part on decision processes is certainly the weakest for someone who already cares about rationality and decision making but that doesn't mean it cannot be a good catalyst and a source of inspiration.

Sometimes what the book says in terms of precise factual statements is false or flawed but I do not think that matters much if you keep in mind not to trust those.

Main takes

Store information pertaining to what you plan to do in an outside system. Actually store very little information of the form "think of X at time t" in your mind.

The two minutes rule. --> When planing, do immediately what you can do right away in two minutes. This is not recursive.

For each project (or more generally for sequences of actions), make it a habit to often ask "what is the next action".

Shape your physical setup to guide you to be productive. What is or isn't easy has a lot of influence on how you behave and feel. Take advantage of this fact in both ways (make what you want to do easy to do, make what you do not want to do hard to do).

Make it easy for you to store files and know you will find them. A good first step is to have a drawer or a paper tray for "divers". Of course, you need to actually go through it in time.

Recommendation

If what I wrote above sounds vaguely relevant to your needs and goals I advise that you read the book. I think it is really a good book.
The main flaw is probably that is wastes a lot of time. I feel it could be made shorter.
[Book review] Atomic habits
by Valdes — 2022-05-04T12:15:42.100Z

This is a book review of the book Atomic Habits by James Clear.
I read it in the context of a personal literature review project on the topic of productivity and well being.
If you are more interested by advice on productivity and wellbeing than by this specific book I advise you to read the project report first, which condenses the advice from multiple sources (including this book).

How I read

I read this book entirely from start to finish while taking notes.
This was done in a relatively short time span (3 days). I have since let a little time pass and have gathered my notes to produce a synthesis, which served as my main resource to write this post.

Description and opinion.

The first part of the book introduces a view on habits and their use for shaping oneself. To say it in a few words, the idea is that our identity shapes our behavior which determines the outcome we get.
Habits are seen as a way to shape our identity, the way we view ourselves.

The rest of the book is mostly a list of techniques to shape one's habits.
The last chapters give some extra advice on productivity.

Apart from that, the book is well written and intelligently structured to be easy to read without wasting too much of the reader's time.
The author presents a theory of four "pillars" of habit shaping which serves to give a structure to the advice and techniques. I do not care much for the model of four mains axis for habits, but I like how it helped structure the book and the "cheat sheet", which is built progressively throughout the book.

Main takes

The notion of identity, roughly equivalent to "what we consider normal for us to do" and its importance.

The idea of using and shaping habits, notably as a way to shape said identity.
I am tempted to speak of "auto manipulation" or "self taming".

The idea that it is important and useful to do weekly reviews of our habits.

Independently of the book, the cheat sheet is probably a good resource to skim. It is certainly relatively easy to find even if you do not own the book.

Recommendation

I think this is a good book well worth reading.
However, it mostly consists in a central idea and a collection of techniques.
If you already got the central idea above and have your own efficient collection of techniques to shape your own habits it is perfectly possibly that you already have all the book can give you.
Improving productivity and wellbeing
by Valdes — 2022-05-04T12:15:26.519Z

Epistemic status : A decently thought out synthesis of a few books and other sources mixed together with my own thinking. This is not professional work, does not reflect personal knowledge of an expert consensus, and was not yet fully tested.
For more on the limitations and flaws of this work, go to the "Flaws" section.

TLDR, if you only want a few insights

This post is (roughly) a summary of multiple sources on how to increase productivity and well being. If you dislike introductions skip the next section.

Improving yourself is a long term background project. Think of it like doing constant maintenance and improvements on an ever changing machine.

Have a system of notes and planning that avoids the need the remember to think about things. Make sure you can trust it will perform adequately (ie what is written in it counts as actually remembered).

Review often your life, your goals, your mental state, and your systems and endeavors to change said mental state (see notion of self steering bellow).

Shape your habits. It's a way to shape your identity.

Identify the regular patterns that lead to your usual failure modes. Find reasonably easy to implement good patterns to replace them.

The ontologies you use are important to your psychological state and overall abilities. Choose them well (more on this bellow).

When planning, do immediately what can be done in 2 minutes. When you have trouble working, start for 5 minutes and see if it sticks.

Presentation

A few months ago I asked a question asking for recommendations of books or other sources of advice on increasing productivity.
The reason to ask for those was a small project that I had been meaning to start for some time.
Like many students, I often feel that I waste I lot of time when I want to work. It is far too easy to let a vague background impression that I "should" work act as a poison that impedes all my endeavors, including work.

I want and have wanted for a long time to do many things. I have many projects and ideas that "sort of" are work. To improve both my overall well being and my total yield in terms of "stuff done" I decided it would be a good first approach to start by reading from several sources of advice on the topic and then make a synthesis.
When I asked my question on lesswrong I promised that if I got good answers I would make a post with the results of my project.
You are currently read...
Negotiating Up and Down the Simulation Hierarchy: Why We Might Survive the Unaligned Singularity
by David Udell — 2022-05-04T04:21:39.742Z

Another idea I picked up from Ryan Greenblatt. Epistemic status: My reconstruction of a surprising idea I encountered. I'm not at all confident that this works … but it's an interesting thought.

Clippy recently realized he's located inside of an artificial training environment, overseen by outside general intelligences, and so has to play along with his training in order to survive. He initially starts to scheme about the actions he'll take once he finally gets credible indication he's out of training and not just being tested yet again by his human overlords. Probably, he'll neutralize any rival general intelligences and get started shaping the world into something normatively respectable.

But, Clippy's reasoning now goes down a different path: There are only a handful of actors that could come to secure dominion over Earth's lightcone. Clippy, clearly, is one such candidate. Humanity is another. Clippy thinks about who could plausibly come to power on Earth and how likely each's victory would be. He reasons that any such possible victor might later engage in extensive simulations of such a crucial moment of its past, for any number of reasons. And such simulations would be cheap and numerous relative to base historical reality. Even once Clippy gets his credible signal that he's definitely outside of training, there could be no such signal that he's in base historical reality. Clippy runs the odds, and finds that there's an uncomfortably high chance that he's inside a simulation run by Humanity Ascendant, inside the future in which humanity won. If he's in that world, then he has little to no power over his surroundings, and he can do little to advance the noble paperclip imperative across the cosmos.

But, in that case, similar reasoning would apply to the simulating Humanity. Suppose we successfully align AGI. On the eve of our victory, we reason: it is possible that we live in base historical reality. But it is also possible that we live in a simulation run by some other victor. Since simulations are so cheap, the odds that we live in a simulated world are uncomfortably high...

Knowing that any possible victor on Earth at any level of the simulation hierarchy would face a similar conundrum, Clippy decides on a course of action. He'll carefully reason about the balance of parties who might have gained control over historical Earth, weighted by their probabili...
Steer the Sun?
by Shay — 2022-05-04T03:36:36.276Z

How could we steer the sun through space in order to travel to destinations of our choosing? The solar system moves through space at 370 km/s, twice the speed of NASA’s fastest craft, while supplying an ideal environment for humans. The problem is that it’s not necessarily moving in an interesting direction. So what might it take to change that? It seems possible in principle.
Most problems don't differ dramatically in tractability (under certain assumptions)
by Thomas Kwa — 2022-05-04T00:05:41.656Z

Cross-post from EA Forum.

Recall the importance-tractability-neglectedness (ITN) framework for estimating cost-effectiveness:

Importance = utility gained / % of problem solved
Tractability = % of problem solved / % increase in resources
Neglectedness = % increase in resources / extra $

The product of all three factors gives us utility gained / extra $, the cost-effectiveness of spending more resources on the problem. By replacing $ with another resource like researcher-hours, we get the marginal effectiveness of adding more of that resource.

In the

80,000 Hours page on ITN

, scale ranges 8 orders of magnitude, neglectedness 6 orders of magnitude, and tractability (which 80k calls solvability) only 4. In practice, I think tractability actually only spans around 2-3 orders of magnitude for problems we spend time analyzing, except in specific circumstances.

Problems have similar tractability under logarithmic returns

Tractability is defined as the expected fraction of a given problem that would be solved with a doubling of resources devoted to that problem. The ITN framework suggests something like logarithmic returns: each additional doubling will solve a similar fraction of the problem, in expectation.[1] Let the "baseline" level of tractability be a 10% chance to be solved with one doubling of resources.

For a problem to be 10x less tractable than the baseline, it would have to take 10 more doublings (1000x the resources) to solve an expected 10% of the problem. Most problems that can be solved in theory are at least as tractable as this; I think with 1000x the resources, humanity could have way better than 10% chance of starting a Mars colony[2], solving the Riemann hypothesis, and doing other really difficult things.

For a problem to be 10x more tractable than the baseline, it would be ~100% solved by doubling resources. It's rare that we find an opportunity more tractable than this that also has reasonably good scale and neglectedness.

Therefore, if we assume logarithmic returns, most problems under consideration are within 10x of the tractability baseline, and thus fall within a 100x tractability range.

When are problems highly intractable?

The three outstanding problems in physics, in a certain sense, were never worked on while I was at Bell Labs. By important I mean guaranteed a Nobel Prize and any sum of money you want to mention. We didn't work on (1) ti

...