trentmkelly/LessWrong-43k huggingface dataset:
Six weeks doesn’t make a habit 
by lynettebye — 2022-08-06T08:54:26.584Z

I occasionally get asked some version of “How long do I need to practice my habit before it will stick?” The coachee has often heard some magic number – 21 days, 6 weeks, 9 weeks – that will make a new habit automatic and robust. (I hear surprising different numbers, giving how similar the rest of the statement is.)

It’s an alluring idea. Habits form the backbone of our default actions. We probably would be happier, healthier, and wealthier if we could just make doing the “right thing” automatic.

But I think this mindset is missing something important about habits really work.

Oh, it’s not entirely wrong. We seem to learn cue->response patterns that we execute by default when we’re not deliberately choosing how to act. Changing those responses does reduce the need to effortfully intervene.

Yet, most habits weren’t chosen over a search space of one option. You didn’t pick up an afternoon cookie habit because that was the only food you’ve ever eaten for an afternoon snack. You picked up the afternoon cookie habit because the cookie tastes good!

It seems so obvious when you think about it. Of course we form habits we find rewarding! By default, we gravitate towards actions that feel good in the moment (or at the very least, not bad). Even maladaptive habits usually provide immediate pleasure or relief, like procrastinating on dealing with an email for weeks. The overall feeling of dread and lateness might be building, but each moment you can avoid dealing with the issue provides fleeting relief from facing it.

I don’t have a precise answer to “How rewarding does it need to be?” Roughly, it seems like the more immediately rewarding, the easier it is to form a habit. Like, I had a really easy time developing an afternoon ice-cream habit. I had to hunt more for the reward of daily Pilates (which for me is learning to enjoy the activity a bit, pride at keeping my streak going, and “reward cuddles” from my supportive partner).

Chapter 2 of Effortless by Greg McKeown has some good tips/examples of finding immediately enjoyable aspects of tasks. He mostly focuses on pairing clearly enjoyable actions with the necessary tasks, but I liked one exception example: “I noticed an element of the [weekly financial] meeting I’d previously overlooked: the whole exercise involved tidying up a messy area, the family finances! Just noticing that creating order out of chaos was a part o...
Why I Am Skeptical of AI Regulation as an X-Risk Mitigation Strategy
by A Ray — 2022-08-06T05:46:48.416Z

Epistemic Signpost: I'm not an expert in policy research, but I think most of the points here are straightforward.

1. Regulations are ineffective at preventing bad behaviors

Here's a question/thought experiment: Can you think of any large, possibly technology related, company who has broken a regulation. What happened as a result? Was the conclusion the total cessation of that behavior?

I'm not bothering to look up examples (at the risk of losing a tiny bit of epistemic points) because these sorts of events are so freaking common. I'm not going into the more heinous cases, where it wasn't even plausible to anybody that the behaviors were defensible (and instead just expected to not get caught) or cases where the behavior was more profitable than the fine imposed. My point is that the regulations failed to prevent the behavior.

If you try to defend this with "Regulations are about incentives and not safety (prevention)" -- then I think you, too, should be pessimistic about AI Regulation as a strategy for X-Risk mitigation.

2. What does effective regulation look like

Lest I be accused of making a fully-general counterargument, here's some examples of regulations that seem to actually work:

2.1. Defense Export Controls

The United States has a long history of enforcing export controls on defense-related technologies, and I'll just focus on the USML (United States Munitions List) and CCL (Commerce Control List) here.

The USML is the more "extreme" of the two, and this is what people are talking about if they're talking about ITAR (International Traffic in Arms Regulations). It covers things like firearms, missiles, and nuclear weapons, and licenses/export of these are pretty strictly controlled. I don't think anyone doubts that putting AI technology on the USML would cause some sort of impact, but it seems pretty unlikely (and also undesirable for other strategic reasons).

Lesser known is the CCL, and how it regulates a much more lightweight (but still strongly enforced) export license requirement. It's designed to facilitate American technology companies selling their tech abroad, even if the technology is relevant to national security.

An example relevant to AI technology is when the US government prevented the export of computer chips to China for use in a supercomputer. Everyone building a (publicly known) supercomputer claims it will be used for publicly good sci...
My advice on finding your own path
by A Ray — 2022-08-06T04:57:47.009Z

tl;dr - 4 steps

Give yourself permission to take the future seriously
Be willing to imagine successful or ambitious outcomes
Give yourself the gift of focused time and space
Write draw sketch diagram or whatever you need to empty your mind

Intro

(Feel free to skip to the method.)

When I was transitioning into technical AI alignment work, I benefited immensely from the mentorship and guidance of experts in the field. They helped me to understand the relevant concepts, to develop my skills, and to find my place in the research community.

In my early years of giving career advice to other people, I tried to replicate this experience for them. I listened to their questions and concerns, and I tried to provide what I thought were the best possible answers. These are questions like "What do you think are the most important problems to work on?" or "What should I do to get the highest impact job in AI alignment?", and while I tried to listen to them and give personalized advice, I think the answers always failed to capture what was the best fit for the individual.

Over the years, however, my advice has evolved. I have answered direct questions less and less directly, and tried to do more of asking leading questions in response. "What do _you_ think the most important problems are?" "What kinds of work do _you_ think you'd be highest impact at?"

As I've done this, I've noticed four specific hang-ups (probably others, but I'm simplifying and reducing here in order to make my points simpler). My advice has mostly turned into just four things that I say to everyone, and I think are more useful than specific answers to specific questions.

Method: 4 Steps

1. Give yourself permission to take the future seriously

I think for many people it can be difficult to imagine details about the future. There is a lot of uncertainty, and this can lead to people thinking that certain kinds of thinking or planning or predictions are inappropriate. Another way this sometimes looks is too much deference to "Serious Senior People" and not trusting their own internal senses and intuitions about the future. So I think it's important for people to give themselves permission to take the future seriously. To think about what could happen, what they would like to happen, and what they can do to make it happen. To trust their own judgments and not feel like they have to wait for someone else to tell them wha...
PredictIt is closing due to CFTC changing its mind
by eigen — 2022-08-06T03:34:44.937Z

https://www.predictit.org/platform-announcements

The staff of the Commodity Futures Trading Commission (CFTC) has withdrawn the No-Action letter (NAL) issued to Victoria University of Wellington and under which PredictIt has operated since 2014.

What does the LW community think about this happening for PredictIt in particular, and prediction markets in general?
Metaculus and medians
by rossry — 2022-08-06T03:34:01.745Z

Cross-posted from my personal blog.

(1)

Should I expect monkeypox to be a big deal for the world? Well, fortunately, Metaculus has a pair of questions that ask users to predict how many infections and deaths there will be in 2022:

203 users(!) made 817 predictions of infections, and Metaculus helpfully aggregates those into a "community prediction" of ~248k infections. 77 users made 180 predictions of deaths, with a community prediction of 541.

The y-axis is on a log scale (as are the predictors' distributions). This is a good choice! Whatever you expect the most-likely case to be, there's definitely a chance with things like this that one a misestimation or shift in one factor can make it bigger or smaller by a multiple, not just an additive amount.

What's not a good choice is to report the median outcome of the aggregate position as the "community prediction". This causes a headline reported value that is way too low. Like, four to seven times too low (at least for my intended purposes).

Because the predictors gave (and are scored on) probability distributions, Metaculus will happily give you an aggregate distribution, of which the 248k "community prediction" is the median scenario (the middle of the three dashed lines):

However, on the same plot, the aggregate distribution predicts a 10% chance of at least 4,950k infections. If it's 10% likely to be 5 million infections, then that's already lot more concerning than the 250k in the community prediction! And when I say I'm interested in how much monkeypox to expect, I do mean that in the sense of expected value, and I care about the mean case, not the median case.

Backing the mean out of the distribution provided on the page takes a small bit of spreadsheet work, but in the end I get 1,729k -- more than 7x higher than the community prediction!

Similarly, monkeypox deaths have a community prediction of 541, but a mean of more like 2,316 (4.3x higher):

(2)

Now, there are good reasons for Metaculus to report medians instead of means if their goal is to be a respectable-looking site with non-crazy-seeming community predictions at the top of each question. If a Lizardman constant of users predict that there will be 100 times as much of a thing as the truthful users, should Metaculus report a mean that's 5x higher than the non-lizardman consensus? Seems bad!

On the other hand, if you're trying to (1) aggregate actio...
Announcing the Introduction to ML Safety course
by Dan H — 2022-08-06T02:46:00.295Z

TLDR

We're announcing a new course designed to introduce students with a background in machine learning to the most relevant concepts in empirical ML-based AI safety. The course is available publicly here.

Background

AI safety is a small but rapidly growing field, and both younger and more experienced researchers are interested in contributing. However, interest in the field is not enough: researchers are unlikely to make much progress until they understand existing work, which is very difficult if they are simply presented with a list of posts and papers to read. As such, there is a need for curated AI safety curricula that can get new potential researchers up to speed.

Richard Ngo’s

AGI Safety Fundamentals

filled a huge hole in AI safety education, giving hundreds of people a better understanding of the landscape. In our view, it is the best resource for anyone looking for a conceptual overview of AI safety.

However, until now there has been no course that aims to introduce students to empirical, machine learning-based AI safety research, which we believe is a crucial part of the field. There has also been no course that is designed as a university course usually is, complete with lectures, readings, and assignments; this makes it more likely that it could be taught at a university. Lastly, and perhaps most importantly, most existing resources assume that the reader has higher-than-average openness to AI x-risk. If we are to onboard more machine learning researchers, this should not to be taken for granted.

In this post, we present a new, publicly-available course that Dan Hendrycks has been working on for the last eight months:

Introduction to ML Safety

. The course is a project of the

Center for AI Safety

.

Introduction to ML Safety

Philosophy

The purpose of Introduction to ML Safety is to introduce people familiar with machine learning and deep learning to the latest directions in empirical ML safety research and explain existential risk considerations. Our hope is that the course can serve as the default for ML researchers interested in doing work relevant to AI safety, as well as undergraduates who are interested in beginning research in empirical ML safety. The course could also potentially be taught at universities by faculty interested in teaching it.

The course contains research areas that many reasonable people concerned about AI x-risk think ...
«Boundaries», Part 2: trends in EA's handling of boundaries
by Andrew_Critch — 2022-08-06T00:42:48.744Z

This is Part 2 of my «Boundaries» Sequence on LessWrong, and is also available on the EA Forum.

Summary: Here I attempt to constructively outline various helpful and harmful trends I see in the EA and rationality communities, which I think arise from a tendency to ignore the boundaries of social systems (rather than a generic tendency to violate norms). Taken too far, this tendency can manifest as a 'lack of respect' for boundaries, by which I mean a mixture of

not establishing boundaries where they'd be warranted,
crossing boundaries in clumsy/harmful ways, and
not following good procedures for deciding whether to cross a boundary.

I propose that if some version of «respecting boundaries» were a more basic tenet of EA — alongside or within the principles of neglect, importance, and tractability — then EA would have fewer problems and do more good for the world.

The trends

Below are the (good and bad) trends I'd like to analyze from the perspective of respecting boundaries. Each of these trends has also been remarked by at least a dozen self-identified EAs I know personally:

expansive thinking: lots of EAs are good at 'thinking outside the box' about how to do good for the world, e.g., expanding one's moral circle (example: Singer, 2011).
niche-finding: EA helps people find impactful careers that are a good fit for their own strengths and weaknesses (example: 80k, 2014).
work/life balance: EA has sometimes struggled with people working too hard for the cause areas, in ways that harm them as individuals in an unsustainable way.
romances at work: people dating their co-workers are an example of a professional boundary being crossed by personal affairs.
social abrasiveness: EA culture, and perhaps more so rationalist culture, is often experienced by newcomers or outsiders as abrasive or harsh. (Hypocrisy flag: I think I've been guilty of this, though hopefully less over the past few years as I've gotten older and reflected on these topics.)
pivotal acts: numerous EAs seriously consider pivotal acts — i.e., a potential unilateral acts by a powerful, usually AI-enabled actor, to make the world permanently safer — as the best way to do good for humanity (as opposed to pivotal processes carried out multilaterally).
resistance from AI labs: well-established AI labs are resistant to adopting EA culture as a primary guiding influence.

I'm going to ana...
"Just hiring people" is sometimes still actually possible
by lc — 2022-08-05T21:44:35.326Z

Graham's design paradox is a maxim about organizations that says that if key management isn't competent at ${X}, it's often impossible for an organization to hire people who are, regardless of how well resourced that organization is. It was once described thusly:

Paul Graham’s Design Paradox is that people who have good taste in UIs can tell when other people are designing good UIs, but most CEOs of big companies lack the good taste to tell who else has good taste. And that’s why big companies can’t just hire other people as talented as Steve Jobs to build nice things for them, even though Steve Jobs certainly wasn’t the best possible designer on the planet. Apple existed because of a lucky history where Steve Jobs ended up in charge. There’s no way for Samsung to hire somebody else with equal talents, because Samsung would just end up with some guy in a suit who was good at pretending to be Steve Jobs in front of a CEO who couldn’t tell the difference.

The paradox has been invoked either directly or as an analogy to explain why the alignment problem is hard, why information security is hard, why identifying good programming talent is hard, and why American car companies don't design good looking cars.

In some cases, especially in cases where failure is irrecoverable (like in evaluating alignment solutions), overcoming the design paradox is in fact a pressing issue. However I'd argue those cases are probably rare. In practice, most appeals to it to explain organizational incompetence are misattributions of a litany of deeper problems.

One field that seems to most people like it'd be a perfect application of this maxim is information security. Major security breaches are sometimes described as black swan events, because they're relatively rare and when they happen they can be very disruptive. Your typical large technology company may not get real-world feedback on the performance of a subpar Chief Information Security Officer for years. It's also extremely difficult to evaluate the security of such systems in vitro if you yourself are not generally good at breaking them, and you generally want to protect them against the best computer hackers in the field and not just the average ones. Perfect example of a case where the design paradox applies, right?

It's true that companies tend to be bad at picking CISOs, and a very common tendency in the industry is for ...
The need for certainty
by Thomas McMurtry — 2022-08-05T20:18:17.339Z

I had a conversation with my one truly religious Aunt a few months back as we went for a walk in the East-end of Toronto for a mid-morning coffee. In our caffeine fueled ramblings I was trying to persuade her that humans love religion for its ability to provide certainty.

A man (a notable atheist I believe) once said something along the lines of, "religions will continue to exist so long as man is afraid of death". I believe the uncertainty of what comes after death is what makes a promise of an after-life so alluring to so many people, and has allowed religions to flourish for thousands of years. The comfort that can be conferred to someone who is grieving a lost family member, or on their death bed, that they will see all of their loved ones again together in a wonderful place. In other words, "the church" was able to provide certainty surrounding humans' preeminent fear. (and I believe it can be said most fear comes from uncertainty).

She did not agree with my hypothesis. I believe because it was providing a rather negative overall outlook on an institution which can admittedly provide much more than simply a perspective on death. Of course neither of us was entirely right or wrong.

Emerging "Scientism"

The context of this conversation was around the general public's new undying and unquestioning allegiance to government edicts. It was the deferral to the supposed "experts", and the need for absolute obedience that so alarmed me as well as my aunt.

"Follow the science" would be snarled at you, condescendingly, as you began to ask a simple question. The Science was set, and questioning it belied your ignorance or maybe immorality.

Regardless if the appropriate policies were followed, the inability to ask simple questions was frightening to the basic tenets of science, as well as citizenship in a free society.

Others before me have speculated on an emerging Scientism. As religiosity continues to decline in the Western world, so this institution of faith is replaced by (big S) Science as the new altar. I speculate on this as the need for certainty for humans to come from somewhere.

This has been the framework for totalitarian regimes of the past, in which religions are rooted out, and the leaders rule with absolute authority. Although I do not argue for a return to monotheism as the guiding force of society, I am weary of the direction our new purs...
Rant on Problem Factorization for Alignment
by johnswentworth — 2022-08-05T19:23:24.262Z

This post is the second in what is likely to become a series of uncharitable rants about alignment proposals (previously: Godzilla Strategies). In general, these posts are intended to convey my underlying intuitions. They are not intended to convey my all-things-considered, reflectively-endorsed opinions. In particular, my all-things-considered reflectively-endorsed opinions are usually more kind. But I think it is valuable to make the underlying, not-particularly-kind intuitions publicly-visible, so people can debate underlying generators directly. I apologize in advance to all the people I insult in the process.

With that in mind, let's talk about problem factorization (a.k.a. task decomposition).

HCH

It all started with HCH, a.k.a. The Infinite Bureaucracy.

The idea of The Infinite Bureaucracy is that a human (or, in practice, human-mimicking AI) is given a problem. They only have a small amount of time to think about it and research it, but they can delegate subproblems to their underlings. The underlings likewise each have only a small amount of time, but can further delegate to their underlings, and so on down the infinite tree. So long as the humans near the top of the tree can “factorize the problem” into small, manageable pieces, the underlings should be able to get it done. (In practice, this would be implemented by training a question-answerer AI which can pass subquestions to copies of itself.)

At this point the ghost of George Orwell chimes in, not to say anything in particular, but just to scream. The ghost has a point: how on earth does an infinite bureaucracy seem like anything besides a terrible idea?

“Well,” says a proponent of the Infinite Bureaucracy, “unlike in a real bureaucracy, all the humans in the infinite bureaucracy are actually just trying to help you, rather than e.g. engaging in departmental politics.” So, ok, apparently this person has not met a lot of real-life bureaucrats. The large majority are decent people who are honestly trying to help. It is true that departmental politics are a big issue in bureaucracies, but

those selection pressures apply regardless of the peoples’ intentions

. And also, man, it sure does seem like

Coordination is a Scarce Resource

and

Interfaces are a Scarce Resource

and scarcity of those sorts of things sure would make bureaucracies incompetent in basically the ways bureacracies ...