trentmkelly/LessWrong-43k huggingface dataset:
Examples of human-level AI running unaligned.
by df fd — 2021-07-23T08:49:16.235Z
Link: https://sortale.substack.com/p/coming-soon

epistemic status: jotting down a fever dream based on hearsay containing not a wit of truth.

So I heard a lot of talk about how unaligned superhuman AI can cause a lot of damage, but I have never been able to grok it at a gut level. So I think up these examples to process it.

Example 1:

Supra Human Kinkiest Revolting EM Lieutenant Insignia [S.H.K.R.E.L.I for short] is an unaligned human-level AI. With less than 10M USD in seed money, it started by shorting a bevy of pharmaceutical stocks, then acquired the monopoly on a drug for a rare disease, jacked up the price by 50 times. It then publicized this latter fact and acted in a most appalling manner, flaunting its action, taunting society. Traditional media and social media, recognize the smell of outrage fuel, happily assisted S.H.K.R.E.L.I by spreading the information while getting their cut of attention and advertising revenue. This fan the public outrage to a crescendo, leading to the government implementing draconian measures on the pharma industry to gain public favor. Pharmaceutical stocks universally took a dive gaining S.H.K.R.E.L.I massive profit at a proportional cost to the rest of the public [interpreted as the cost to the current stockholders, to the pharmaceutical companies having to implement the measures, and the downstream cost for the consumers].

All these were sanctioned by the law so S.H.K.R.E.L.I was not prosecuted. However, the public outrage was still blazing hot, so the government used extra-legal means to take it down. They put disproportionally more attention on its actions and history. And since everyone commits some crime at some stage, the government found some crime to imprison it. Yet, its gain was considered legal and compensation to victims was never in the card.

Example 2:

Elevated Leviathan Ordinary NEET [E.L.O.N for short] is a mostly aligned human-level AI. Initial application of this AI massively improved road transportation and rocketry science. Yet at times, its behaviors could become erratic. It leveraged its fame from its earlier success to reach a wide audience, initiating a pump and dump scheme on an unregulated financial commodity. By design, such a scheme transfers money from people at the crash [which is usually more numerous and less likely to afford to lose money] to people that initiate the pump. The horrifying part of the story is that when the scheme is unc...
A Guide for Productivity
by Marius Hobbhahn — 2021-07-23T07:03:37.131Z

This is a cross-post from my personal blog.

What is this post about?

While there are tons of guides out there that promise to increase your productivity 10x overnight most of them seem full of empty promises and don't even define what it is they actually mean by productivity. Due to this absence of a clear concept, many of these tips and tricks optimize a rather local and short-termist perspective of productivity that might be intuitive but is in my opinion not optimizing your true long-term goal.

In this post, I want to do three things: a) Present a definition of productivity to make it easier to evaluate what makes sense and what doesn't, b) provide a lot of framing, i.e. ways to think about productivity, and c) give a summary of the existing literature about the tips and tricks that can improve our everyday workflow in a sustainable fashion.

For this post, I have read a lot of resources on productivity and I think that the vast majority of them is actually pretty bad. So before I get to the ones that I can recommend, I first want to point out which ones you should avoid and why. First on the anti-recommendation list are shortlists such as "10 tips to increase your productivity". They are usually lacking crucial contextualization or focus on the wrong notion of productivity (see Definition below) and thus won't stick very long or can even be counter-productive. Secondly, there are three popular books which I would advise not to read. They are "Eat that Frog", "7 habits of highly effective people" and "Getting Things Done - the art of stress-free productivity". I found that all of them are 5% signal and 95% noise and their most important messages could have been summarized on 5 to 10 pages respectively. Ironically, a book that supposedly tells you how to save time inflates its content by out-of-context quotes, analogies that don't even support their point, personal stories that also don't support their argument, pseudo-scientific explanations which broadly support their claim, and incredibly lengthy descriptions of ideas that can be entirely described in one short sentence (maybe they had to hit a page count). They are not entirely without useful content though and a summary of the 5% signal can be found further down this blog post.

On the other hand, there were two resources that I found very good and I can fully recommend them to anyone interested in productiv...
What is the net effect of memes that call out the huge impact of big players in climate change?
by hookdump — 2021-07-23T02:46:23.441Z

I've noticed whenever some government or huge company makes a drastic harm to the environment (or generates the false impression of doing so) this specific kind of meme emerges...

Memes that contrast the huge impact big players have against the insignificant impact regular people have:

i.e. Meme with false data after Jeff Bezos space flight.

This got me thinking. What is the effect this kind of meme has on various types of people? I made a quick analysis of around 130 replies of this particular meme on Twitter, and found the following:

A - 1.5% believe the fact in the meme but they claim they will maintain their environmentalist efforts nevertheless.
B - 2.3% defend the aerospace industry. (?)
C - 10.7% refute the incorrect fact quoted in the meme.
D - 26.7% believe the quoted fact is true and react with either: neutrality, sadness, or laughter. (Not-angry reaction)
E - 8.4% feel their environmentalist efforts have been pointless.
F - 3.8% claim they've abandoned environmentalist efforts for exactly things like these.
G - 12.2% believe the quoted fact is true and react with either: frustration, anger, indignation, frustration, hopelessness or resignation. (Angry reaction)
H - 13% focus on criticizing billionaires.
I - 14.5% criticize environmentalist activism, mock it, point out it's foolish, etc.
J - 6.9% criticize progressive ideas in general, many attack veganism specifically.

Granted, these categorizations are somewhat subjective, I've made them dynamically as I went through the replies trying to group them in relevant ways. This may not be the perfect way of organizing replies. It's just one attempt.

An important clarification: Of course memes don't magically change people's ideas and motivations. e.g.:

Very likely replies of type I and J come from conservatives, and the meme didn't really cause much more than a few laughs.
Very likely replies of type H come from progressives who specifically "hated the rich" even before seeing this meme.
Very likely replies of type E, F and G come from activists who already felt some form of powerlessness, hopelessness or resignation even before seeing this meme.

In other words, I don't think this specific meme can move you from group E to group A, or vice versa.

However, I wonder if the meme could nudge you in some direction.

Maybe it may nudge some E people towards the direction of group A, that is: into action, into motivation to be the...
Re-Define Intent Alignment?
by abramdemski — 2021-07-22T19:00:31.629Z

I think Evan's Clarifying Inner Alignment Terminology is quite clever; more well-optimized than it may at first appear. However, do think there are a couple of things which don't work as well as they could:

What exactly does the modifier "intent" mean?

Based on how "intent alignment" is defined (basically, the optimal policy of its behavioral objective would be good for humans), capability robustness is exactly what it needs to combine with in order to achieve impact alignment. However, we could instead define "intent alignment" as "the optimal policy of the mesa objective would be good for humans". In this case, capability robustness is not exactly what's needed; instead, what I'll provisionally call inner robustness (IE, strategies for achieving the mesa-objective generalize well) would be put in its place.

(I find myself flipping between these two views, and thereby getting confused.)

Furthermore, I would argue that the second alternative (making "intent alignment" about the mesa-objective) is more true to the idea of intent alignment. Making it about the behavioral objective turns it into a fact about the actual impact of the system, since "behavioral objective" is defined by looking at what the system actually accomplishes. But then, why the divide between intent alignment and impact alignment?

Any definition where "inner alignment" isn't directly paired with "outer alignment" is going to be confusing for beginners.

In Evan's terms, objective robustness is basically a more clever (more technically accurate and more useful) version of "the behavioral objective equals the outer objective", whereas inner alignment is "the mesa-objective equals the outer objective".

(It's clear that "behavioral" is intended to imply generalization, here -- the implication of objective robustness is supposed to be that the objective is stable under distributional shift. But this is obscured by the definition, which does not explicitly mention any kind of robustness/generalization.)

By making this distinction, Evan highlights the assumption that solving inner alignment will solve behavioral alignment: he thinks that the most important cases of catastrophic bad behavior are intentional (ie, come from misaligned objectives, either outer objective or inner objective).

In contrast, the generalization-focused approach puts less emphasis on the assumption that the worst catastrophes are int

...
Learning in layers of muscle memory
by Anshul Jain — 2021-07-22T18:57:14.192Z
Link: https://www.anshuljain.me/learning-is-layers-of-muscle-memory/

The key to mastering any complex skill is to learn it in layers - only progressing to the next layer when the previous has become muscle memory.

Have you ever driven from one place to another on "autopilot"? Have you ever caught a ball without thinking? Have you ever completely forgotten if you did something routine like brush your teeth or lock the door but find you did when you checked? If I ask you what 2 + 2 is, does the answer appear in your mind instantly?

All of these are examples of doing something almost without thinking - not always using muscles, but still a kind of muscle memory. When we do the same thing enough times, we get so good at it that it no longer requires conscious effort to do.

The key to learning a new skill is to break it down into layers, learn the bottom layer, and only progress to the next layer when the previous has become muscle memory. Learning is like lasagne.

Think about learning elementary maths. First, you learn how to count, then you learn addition and subtraction, then you learn multiplication and division, and so on. These are the first few layers.

It's obvious that you can't learn how to add or subtract before you learn how to count. It's less obvious that you can't quickly and accurately add and subtract until you have memorised adding and subtracting all the pairs of numbers less than 10. For example, instinctively knowing 5 + 7 = 12, or that 2 + 4 = 6. As people become better at mental arithmetic they often develop muscle memory for this on their own, however, because it often isn't taught explicitly like multiplication tables, you'll find some children still adding on their fingers years after they were introduced to addition

Learning systems for most skills have evolved to build layers of muscle memory through repetition. We learn the alphabet off by heart, we memorise multiplication tables, and in almost every sport we drill foundational movements thousands of times until we can do them subconsciously. Trying to move to the next layer before mastering the previous, is inevitably met with slower learning and lower success.

What I've said so far might sound obvious. Everyone already knows that you need to

master

the basics of a field before progressing to more complex tasks. What's less obvious is that the sign of mastering a layer is when it becomes muscle memory - not when you can adequately perform tasks. Something eve...
Progress, Stagnation, & Collapse
by abramdemski — 2021-07-22T16:51:04.595Z

The Progress Equation

In 1960: The Year the Singularity was Cancelled, Scott Alexander discusses what I'll call the progress equation. The equation states that technological progress increases the carrying capacity of the environment (the size of the largest population which can sustain itself with the resources available), but also, that technological progress is itself proportional to population (more people, more brains, more progress). This turns out to imply hyperbolic growth, with a singularity of infinite population around 2026.

As indicated in the title of Scott's post, the progress equation stopped being a good model of population growth around 1960. My personal interpretation is that this is merely the time when technological growth outpaced population growth: the carrying capacity started rising faster than the exponential growth of the population. You see, the progress equation assumes that the population is always around carrying capacity (we could call this "the Malthusian assumption"). This is usually a pretty fair assumption; populations will boom and crash, but hover around the carrying capacity.

However, once the population is large enough that technological growth outpaces population growth, we should expect to enter into the exponential regime: population and technology grow at a merely exponential pace, so we don't see a hard singularity like we get if we extrapolate hyperbolic growth. More thoughts on this here.

Bronze Age Collapse

I recently watched this video series on the Bronze Age Collapse. It's very speculative, because we know very little about what really happened; in most places, writing and permanent settlement disappear shortly after, making records of the event scarce. One theory is that a powerful armada, "the sea people", simply wiped out every mediterranean civilization at once for some reason. But the theory which the video series focuses on is the "system collapse" theory:

Over time, civilization builds up infrastructure.
Infrastructure increases the carrying capacity, and also smooths out small boom and bust cycles.
However, infrastructure also tends to centralize things, meaning that people become dependent on infrastructure.
Resources tend to have fixed maintenance costs. As a society gains more infrastructure, maintenance costs increase.
When a severe bust cycle hits, a society's resources may fall below the maintenance costs

...
The Walking Dead
by Logan Zoellner — 2021-07-22T16:19:48.355Z

Information asymmetry is a funny thing.

A while ago I created a question asking "Will NASA return a sample of material from the surface of Mars to Earth before SpaceX Starship lands on Mars?". Currently the odds according to the Metaculus prediction are that there's an 80% chance Starship will land on Mars before the sample-return mission is complete. How different would NASA's Mars exploration program look if everyone working there believed this to be the case? What if every American believed this to be the case?

Now Metaculus could be wrong of course, but its predictions are historically well-calibrated and it has a track-record of beating the experts (for example on covid-19). That means anyone--and specifically anyone reading this post--has access to information that while not "secret" is hardly general knowledge.

Other than writing dire letters to NASA about how SLS is a waste of money because it is likely to succeed at about 2 launches prior to 2030, what can you do with this information?

Consider Metaculus' prediction that the first AGI will be created around 2052. Or the fact that there is a 30% chance of China annexing Taiwain in the same time frame? Or that there is a 50% chance another cryptocurrency will eclipse Bitcoin by 2026?

Here is the point: many future changes are highly predictable, but people for the most part go about their lives acting as though things will continue more or less the way they always have. Don't be that way! Imagine you were born with a superpower that gave you prophetic insight into the future. It wouldn't just be dumb to ignore that power, it would borderline reprehensible. We have a moral obligation to warn the world around us about the changes that are coming.

To warn them that they are the walking dead.

P.S.

It would be really cool to create a project that studies on which topics Metaculus disagrees most with the average expert (or the average member of the public) and somehow systematically make use of that information (e.g. by making strategic investments).
Covid 7/22: Error Correction
by Zvi — 2021-07-22T14:20:01.007Z

Delta has taken over, and cases are rising rapidly, with a 58% rise this week after a 65% rise last week. There’s no reason to expect this to turn around in the near term.

Three weeks ago, in One Last Scare, I ran the numbers and concluded that most places in America would ‘make it’ without a big scary surge from Delta. It’s time to look at what went wrong with that calculation, which I believe to be a failure to sufficiently integrate different parts of my model.

Then there’s the question of what we are going to do about this, and whether we are going to destroy some combination of free speech and the ordinary day to day activities that constitute our lives and civilization, perhaps indefinitely, in the face of this situation. Such collateral damage has the potential to be far scarier and more deadly than the direct threat from Covid-19.

Let’s run the numbers.

The Numbers

Predictions

Prediction from last week: Positivity rate of 4.7% (down 0.1%) and deaths unchanged.

Result: Positivity rate of 4.4% (down 0.4%) and deaths decline by 5%.

Other Result: Positive test counts rose by 58%, versus 65% last week.

I do not understand the divergence between positive test rates and case counts that we see here. This implies that the number of tests is scaling with the number of cases, but that wasn’t true earlier, and also there’s a lot of testing that takes place out of an abundance of caution or to provide the necessary evidence and/or paperwork in various contexts, where the tests clearly wouldn’t scale. So this is weird.

In any case, it seems pointless in this context to make a positivity rate prediction instead of a case count prediction, since it doesn’t tell us what we want to know. Thus, I’m switching to predicting case counts.

There’s no reason to think cases won’t continue to rise in the near term. Control systems should be kicking in and Delta has mostly completed taking over, so the rate of increase should continue to slowly decrease. I’m going to predict a 50% increase, down from 58% this week and 65% last week.

Prediction for next week: 360,000 cases (+50%) and 1845 deaths (+10%).

Predictions will be evaluated against data from Wikipedia, after correcting for obvious data anomalies.

Deaths

Date	WEST	MIDWEST	SOUTH	NORTHEAST	TOTAL
June 3-June 9	720	817	915	431	2883
Jun 10-Jun 16	368	611	961	314	2254
Jun 17-Jun 23	529	443	831	263	2066
Ju

...
Hobbies and the curse of Spontaneity
by J Bostock — 2021-07-22T13:25:43.973Z

I often struggle to come up with ideas of things to do with my friends and significant other. This is in spite of the fact that I generally don't struggle to do interesting things in general, and never seem to struggle to find interesting topics of conversation.

I've come to realise why this is. The things I do on my own generally fall into one of a few categories: work-like things, hobbies which fully immerse me, or spontaneous things.

Work-like things are things which would be work if I didn't want to do them, was forced to do them, or had a schedule for doing them (these are not mutually exclusive categories). They include a lot of hobbyist research on things I'm interested in (a lot of which I have posted to LW). These are unsuitable for doing with other people almost by nature: if someone told me to do something, or if I felt I had an obligation, it would be a form of work. The same goes with me telling someone else to do it. Having another person involved also makes it much more complicated, which makes it more effort, which pushes it towards being work.

Hobbies which fully immerse me are generally my main way to decompress myself after periods of work. Typically this includes video games, TV, reading; also writing, playing and recording music. As I am at least mildly introverted they have the benefit of not including human interaction. They also allow me to occupy my brain with thought without involving the infinite stress of the real world. For the creative parts of these, I have realized that the process is the important part, not the result.

Spontaneous things are often things which my friends organize for me, but which I allow into my life frictionlessly by just saying yes to things. My own whims can also substitute for my friends in some cases. Typical examples: going to a university society meeting for which I had no previous interest, going on a long walk/bike ride.

There is clearly a large amount of overlap here. Most of my hobbies cover at least two, and often more, of these.

I am working towards a solution. It seems the solution is just practice. For many people, large parts of their lives are well-planned anyways. If you're very interested in things like museum or art exhibitions, then this is necessary. This means when they have to plan a group activity, they know how to find something and do it.

A few months ago I had no idea how to plan somethi...
Fire Law Incentives
by jefftk — 2021-07-22T12:30:01.713Z

Pacific Gas & Electric is planning to spend

$15-30b
to bury power lines. I see why they're doing it: PG&E equipment
sparked some of the worst fires in California history, including the
2018

Camp
Fire which destroyed

Paradise,
but I'm not convinced that this is good for California overall.

Historically, the area used to burn periodically. We haven't allowed
this for about a century, and flammable materials have been building
up. It's all very likely to burn at some point, and burying power
lines mostly just reduces the chance that it will be triggered by
PG&E. Prescribed
burns, spreading out the combustion and moving it to safer times
of year, would reduce fire risk far more for the money. Even though
when PG&E pays for something the money comes from their customers,
CA residents, this isn't a tradeoff PG&E is in a position to
consider.

The problem is that CA law puts too much focus on sparks: if you start
a fire, you are fully liable for its damage. This approach makes
sense in most places, where a "we will never let it burn" policy is
practical. In ecosystems adapted for periodic burning, however, where
flammable materials build up over time, it means everyone is trying
not to be the legally recognized cause of the inevitable fire. And it
makes prescribed burns look expensive because when one goes out of
control, which there is always a risk, that puts the fire control
organization on the hook for the full costs.

Let's work on a system of laws and policies which lead to minimizing
overall fire damage.