trentmkelly/LessWrong-43k huggingface dataset:
Why everything might have taken so long
by KatjaGrace — 2018-01-01T01:00:00.441Z

I asked why humanity took so long to do anything at the start, and the Internet gave me its thoughts. Here is my expanded list of hypotheses, summarizing from comments on the post, here, and here.

Inventing is harder than it looks

Inventions are usually more ingenious than they seem. Relatedly, reality has a lot of detail.
There are lots of apparent paths: without hindsight, you have to waste a lot of time on dead ends.
People are not as inventive as they imagine. For instance, I haven’t actually invented anything – why do I even imagine I could invent rope?
Posing the question is a large part of the work. If you have never seen rope, it actually doesn’t occur to you that rope would come in handy, or to ask yourself how to make some.
Animals (including humans) mostly think by intuitively recognizing over time what is promising and not among affordances they have, and reading what common observations imply. New affordances generally only appear by some outside force e.g. accidentally. To invent a thing, you have to somehow have an affordance to make it even though you have never seen it. And in retrospect it seems so obvious because now you do have the affordance.

People fifty thousand years ago were not really behaviorally modern

People’s brains were actually biologically less functional fifty thousand years ago.
Having concepts in general is a big deal. You need a foundation of knowledge and mental models to come up with more of them.
We lacked a small number of unimaginably basic concepts that it is hard to even imagine not having now. For instance ‘abstraction’, or ‘changing the world around you to make it better’.
Having external thinking tools is a big deal. Modern ‘human intelligence’ relies a lot on things like writing and collected data, that aren’t in anyone’s brain.
The entire mental landscapes of early people was very different, as Julian Jaynes suggests. In particular, they lacked self awareness and the ability to have original thought rather than just repeating whatever they usually repeat.

Prerequisites

Often A isn’t useful without B, and B isn’t useful without A. For instance, A is chariots and B is roads.
A isn’t useful without lots of other things, which don’t depend on A, but take longer to accrue than you imagine.
Lots of ways to solve problems don’t lead to great things in the long run. ‘Crude hacks’ get you most of the way there, reducing the value of g

...
A Candidate Complexity Measure
by interstice — 2017-12-31T20:15:39.629Z

I'd like to introduce a mathematical definition that tries to capture some of our intuitive notion of 'complexity'. People use this word in lots of ways. What I mean by this word is the sort of 'complexity' that seems to have been increasing over the history of our planet for the past few billion years. By example:

--Simple self-replicators are more complex than a random sea of chemicals

--Single-celled bacteria are more complex than simple replicators

--Multicellular life is more complex than single-celled life

--Hunter-gatherer bands are more complex than most other life forms

--Modern civilizations are more complex than hunter-gatherer bands

This definition would be in terms of the underlying computer program of physics, and in the style of Kolmogorov complexity. Kolmogorov complexity itself doesn't work, because it seems that our entire universe can be generated by a short computer program, namely the laws of physics running on some simple initial state. (Yes, our universe is in fact quantum-mechanical and hence randomized. But it's still seems likely that we can sample from the distribution of possible worlds with a short program) Since our universe can be generated by a short program, it seems that any sort of 'complexity' that can be found must lie in something besides the underlying program itself.

Now, let's try to construct a formal model. We'll consider some sort of computable dynamical system with size N. For concreteness let's imagine it's a (randomized) cellular automaton. Then running it for T steps should take O(NT) steps of computation.

We're trying to predict the evolution of the system with limited computational resources. We'll build a randomized Turing machine and only let it run for m timesteps. We'll score our predictor on the summed KL divergence of its predictions at each timestep and the true distribution at that step.

Suppose we are using the best possible predictor for a given runtime. The entire history of the universe can be run in time NT simply by running physics. Then our KL divergence at each timestep will be zero.

Conversely, if we are given no time at all the best we can do is output the uniform distribution at each timestep. the KL divergence will be 1 - o(T), pretty close to 1 at each timestep. (Assuming the system continues to change at least somewha...
Musings about the AGI strategic landscape
by  — 2017-12-31T19:38:33.989Z

My model of the economy after AGI is not currently explosive. There is some acceleration and risk but the level of existential risk does not have a huge multiplier. I thought it interesting to map out my current thoughts, based on these assumptions. I'm exploring the scenario where ai-alignment looks possible, but might still lead to bad outcomes.

Threats

Threats are normally caused by asymmetry. Attackers can more easily upgrade their attack infrastructure as it is concentrated, defensive infrastructure can take longer to upgrade as it is spread out geographically and over many different security levels. The current mishmash of humans and computers is moderately insecure as it is. If bad actors gain an advantage in AGI, this insecurity can lead to a number of possible threats.

National Security Threats

If rogue states or terrorists gain an advantage they can potentially exploit the current technology and human make up of the military services. One example of something that may be possible with an AGI asymmetry is sending fake orders to military units. This could have devastating impacts.

Such things will be avoided if at all possible by national security agencies.

Political Threats

Manipulating the populace to elect bad leaders or support bad policies (such as excessive military reduction) can have a variety of political implications.

Takeover from national security

Another possible threat is national security gaining the lead in AGI and subverting their own country with it. This kind of silent coup would be very hard to detect, as the logic of the situation suggests that the national security agencies should do very similar actions whether, malign or benevolent, until AGI is created. The more covert and less oversight an operation has the greater the potential for national security based project to go rogue.

Likely response from security agencies

If the security agencies think that they are within this kind of world they will likely do what they can to make sure they come out ahead in the asymmetry so that they can increase the defensive capabilities ahead of the offensive ramp up.

This might involve trying to slow other people down or avoiding too much public knowledge about AGI to maintain their edge.

You would also hope that they would be investing massively in any defensive non-agi technologies and spreading them around through out the world. If malicious ac...
Book Review: The Elephant in the Brain
by Zvi — 2017-12-31T17:30:00.501Z

We don’t only constantly deceive others. In order to better deceive others, we also deceive ourselves. You’d pay to know what you really think.

Robin Hanson has worked tirelessly to fill this unmet need. Together with Kevin Simler, he now brings us The Elephant in the Brain.

I highly recommend the book, especially to those not familiar with Overcoming Bias and claims of the type “X is not about Y.” The book feels like a great way to create common knowledge around the claims in question, a sort of Hansonian sequence. For those already familiar with such concepts, it will be fun and quick read, and still likely to contain some new insights for you.

Two meta notes. In some places, I refer to Robin, in others to ‘the book’. This is somewhat random but also somewhat about which claims I have previously seen on Overcoming Bias. I nowhere mean any disrespect to Kevin Simler. Also, this is a long review, so my apologies for not having the time to write a shorter one, lest this linger too long past the book’s publication.

I

The book divides into two halves. In the first half, it is revealed (I’m shocked, shocked to find gambling in this establishment) that we are political animals and constant schemers that are constantly looking out for number one, but we have the decency to pretend otherwise lest others discover we are constantly scheming political animals. The easiest way to pretend otherwise is often to fool yourself first. That’s exactly what we do.

What are our real motives? Exactly what you’d think they would be. We want the loyal and strong political allies, true friends, the best sex with the most fit mates, food on the table, enforcement of our preferred norms, high status, respect and other neat stuff like that. The whole reproductive fitness package. To get these things, we must send the right signals to others, and detect the right ones in others, and so forth.

This insight is then used to shine a light on some of our most important institutions:

This book attempts to shine light on just those dark, unexamined facets of public life: venerated social institutions in which almost all participants are strategically self-deceived, markets in which both buyers and sellers pretend to transact one thing while covertly transacting another. The art scene, for example, isn’t just about “appreciating beauty”; it

...
The answer sheet
by Elo — 2017-12-31T00:26:08.144Z

Original post with better formatting on links: http://bearlamp.com.au/the-answer-sheet/

http://bearlamp.com.au/the-answer-sheet/

I always wished I had an answer sheet. A cheat sheet to a lot of my problems. Well now I do. But it's all in my head. I solved a lot of my problems by reading books and building models to understand how things work. This is a cheat sheet.

Interpersonal

If you have problems letting yourself connect with other people, making sense of why you do that, and moving past that is probably connected to vulnerability - daring greatly by Brene Brown (video). If you want more - Read The Art Of Asking by Amanda Palmer.

If you have problems communicating what you want and need from people. OR if you have problems with people demanding things from you - NVC (video) by Marshall rosenberg is what you need.

If you can't seem to say things without them getting personal - look at Concrete instructions. (It's from NVC (video))

If you need to apologise to someone, read On Apology by Aaron Lazare (video). (If you don't think you need to apologise then this isn't the right resource)

If you need help understanding how to manage emotions, read Emotional Intelligence by Daniel Goleman (plus NVC (video), guilt series).

If you want to closely connect with people in the present moment, read the Circling handbook by Marc Benetau, also probably information about mindfulness, and the book The Charisma Myth.

If you want people to feel important when around you (or you want to be a politician), read The Charisma Myth and Rethinking Narcissism.

If you want to get better at sex (any gender), read Come As You Are. Also if you are anxious about if a thing is normal (P.S. it probably is, if it doesn't hurt, if you like it - enjoy it. If you don't avoid it)

Psychology

If you are finding yourself thinking that people have fixed traits like, "Bob is smart", and "I am not good at maths" read Mindsets by Carol Dweck, then start adding "yet" to the end of every sentence that wants you to be fixed.

If you are finding yourself using fixed mindsets in your relationships like, "Dave is lazy", and you want your relationships to do substantially better than the statistics - Read John Gottman's 7 principles for making marriage work.

If you want to know how to coordinate a group of people... If they are an ano...
The True Essence of Honesty: How to Lie and Get Away With It
by Bound_up — 2017-12-30T21:39:21.524Z

"Well, you see, what I meant by that, see, is that I would do all in my power to ensure the bill would pass through Congress by being voted on, not that I would necessarily vote for it myself."

"What I meant by that..."

Is true honesty to speak the truth? Then were all ancient people despicable liars, as they were incapable of speaking the truth about the world, about which they were ignorant?

Well, then, is honesty to speak the truth as best you know it? So, the above politician, and those like them, are honest, because "what they meant" was some true statement?

Hmm...who's to say if you did "speak the truth as best you know it"? Is it based on how you privately intended the statement to be understood? Or is it based on how the majority interpret your statement? Or how the smartest interpret it? The people nearest to you? Or is it based on how you expected some particular group to interpret it?

This, I think is the real key: Honesty is to try and make your audience hear the truth.

Less important is whether or not you speak the truth. Who cares what you're saying; are you talking to yourself? The important thing is what they're hearing. So, shall we boil you up and throw you out the window if, despite your best efforts, others hear something you didn't mean for them to hear?

No, I think all we can ask is that you try your best to make them hear the truth, as best as you understand it. The honest person is this: the one who speaks words which they expect to make other people form true beliefs about the world.

You can fail to fulfill this principle in a number of ways:

You can incorrectly guess at how they'll interpret this statement. Then, you are incompetent, unskilled in truth-telling (causing-truth-to-be-heard-ing, better said, maybe (...maybe not)). People cannot trust the best interpretations of your sayings to necessarily be true, but they can trust that some (usually) reasonable interpretation thereof be true. They can also trust that, if you become more skilled over time, the best interpretations of your sayings will begin to be true ones.
Or, you can correctly guess how others will interpret your statement, and say something you know will make people form false beliefs, buuut which could also be interpreted in a way that could lead to true beliefs, the classic "Well, what I meant was...&quo

...
Methods of Phenomenology
by Gordon Seidoh Worley — 2017-12-30T18:42:03.513Z
Link: https://mapandterritory.org/methods-of-phenomenology-e2f936651ff

NB: Originally posted on Map and Territory on Medium, so some of the internal series links go there.

In the previous post we looked at phenomenology and the thinking that motivates it. We saw that it is based on taking a naive, skeptical, beginner’s view to asking “why?” and choosing to address the question using only knowledge we can obtain from experience. We also saw that experience is intentional: that it is directed from subject to object and forms an inseparable, arity 3 relation called a phenomenon. Taken together this gave us a feel for the shape of phenomenological philosophy and allowed us to glimpse some of the consequences of taking this view seriously.

We now have the context to begin exploring phenomenology’s details, and the first detail to explore is the methods of exploration themselves because phenomenology is highly integrative and our assumptions — that we know the world only through experience and that experience is intentional — determine what sorts of methods we can use. Thus if we are to approach phenomenology we must first gain some familiarity with its practices.

Now if I’m honest there is only really one method of phenomenology — the phenomenological reduction — but that’s a bit like telling you that the only method of decision making is Bayes’s Theorem: in a sense it’s true, but it’s not likely to help you understand anything in the same way that telling you that you are already enlightened doesn’t make you enlightened. To explain how phenomenologists think, it’s more useful to talk about a broad base of specific methods and through them approach the reduction proper. Luckily, there are many methods, and you are almost certainly already familiar with several of them, so let’s take a look!

Science

It might seem a little surprising to you that science is a method of phenomenology since, historically, phenomenology emerged in part from Husserl noticing the inadequacy of natural science for addressing conscious experience, and, in the latter-half of the 20th century, phenomenologically inspired thinkers in the post-modern movement sometimes took anti-scientific stances. But Husserl viewed science as part of phenomenology, and some consider phenomenology a science of consciou...
The Three Levels of Goodhart's Curse
by Scott Garrabrant — 2017-12-30T16:41:25.000Z

Note: I now consider this post deprecated and instead recommend this updated version.

Goodhart's curse is a neologism by Eliezer Yudkowsky stating that "neutrally optimizing a proxy measure U of V seeks out upward divergence of U from V." It is related to many near by concepts (e.g. the tails come apart, winner's curse, optimizer's curse, regression to the mean, overfitting, edge instantiation, goodhart's law). I claim that there are three main mechanisms through which Goodhart's curse operates.

Goodhart's Curse Level 1 (regressing to the mean): We are trying to optimize the value of V, but since we cannot observe V, we instead optimize a proxy U, which is an unbiased estimate of V. When we select for points with a high U value, we will be biased towards points for which U is an overestimate of V.

As a simple example imagine V and E (for error) are independently normally distributed with mean 0 and variance 1, and U=V+E. If we sample many points and take the one with the largest U value, we can predict that E will likely be positive for this point, and thus the U value will predictably be an overestimate of the V value.

In many cases, (like the one above) the best you can do without observing V is still to take the largest U value you can find, but you should still expect that this U value overestimates V.

Similarly, if U is not necessarily an unbiased estimator of V, but U and V are correlated, and you sample a million points and take the one with the highest U value, you will end up with a V value on average strictly less than if you could just take a point with a one in a million V value directly.

Goodhart's Curse Level 2 (optimizing away the correlation): Here, we assume U and V are correlated on average, but there may be different regions in which this correlation of stronger or weaker. When we optimize U to be very high, we zoom in on the region of very large U values. This region could in principle have very small V values.

As a very simple example imagine U is integer uniform between 0 and 1000 inclusive, and V is equal to U mod 1000. Overall, U and V are correlated. The point where U is 1000 and V is 0 is an outlier, but it is only one point and does not sway the correlation that much. However, when we apply a lot of optimization pressure, we through away all the points with low U values, and left with a small number of extreme points. Since this is a

...
Goodhart Taxonomy
by Scott Garrabrant — 2017-12-30T16:38:39.661Z

Goodhart’s Law

states that "any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." However, this is not a single phenomenon. I propose that there are (at least) four different mechanisms through which proxy measures break when you optimize for them.

The four types are Regressional, Causal, Extremal, and Adversarial. In this post, I will go into detail about these four different Goodhart effects using mathematical abstractions as well as examples involving humans and/or AI. I will also talk about how you can mitigate each effect.

Throughout the post, I will use V to refer to the true goal and use U to refer to a proxy for that goal which was observed to correlate with V and which is being optimized in some way.

Quick Reference

Regressional Goodhart - When selecting for a proxy measure, you select not only for the true goal, but also for the difference between the proxy and the goal.

Model

: When U is equal to V+X, where X is some noise, a point with a large U value will likely have a large V value, but also a large X value. Thus, when U is large, you can expect V to be predictably smaller than U.
Example: height is correlated with basketball ability, and does actually directly help, but the best player is only 6'3", and a random 7' person in their 20s would probably not be as good

Causal Goodhart - When there is a non-causal correlation between the proxy and the goal, intervening on the proxy may fail to intervene on the goal.

Model

: If V causes U (or if V and U are both caused by some third thing), then a correlation between V and U may be observed. However, when you intervene to increase U through some mechanism that does not involve V, you will fail to also increase V.
Example: someone who wishes to be taller might observe that height is correlated with basketball skill and decide to start practicing basketball.

Extremal Goodhart - Worlds in which the proxy takes an extreme value may be very different from the ordinary worlds in which the correlation between the proxy and the goal was observed.

Model

: Patterns tend to break at simple joints. One simple subset of worlds is those worlds in which U is very large. Thus, a strong correlation between U and V observed for naturally occuring U values may not transfer to worlds in which U is very large. Further, since there may be relatively fe

... ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,