trentmkelly/LessWrong-43k huggingface dataset:
Musings on Cumulative Cultural Evolution and AI
by calebo — 2019-07-07T16:46:44.658Z

This post might be interesting to you if you want a conceptual model of cumulative cultural evolution and/or you’re curious how cumulative cultural evolution impacts AI forecasting and development.

In particular, I’ll argue that cumulative cultural evolution makes one argument for the discontinuity of AI progress more plausible and sketches out at least two possible paths of development that are worth further investigation.

Cumulative Cultural Evolution

Humans have altered more than one-third of the earths’ land surface. We cycle more nitrogen than all other terrestrial life forms combined and have now altered the flow of two-thirds of the earth’s rivers. Our species uses 100 times more biomass than any large species that has ever lived. If you include our vast herds of domesticated animals, we account for more than 98% of terrestrial vertebrate biomass. — Joseph Henrich

Cumulative cultural evolution is often framed as an answer to the general question: what makes humans so successful relative to other apes? Other apes do not occupy as many continents, are not as populous, and do not shape or use environment to the extent that humans do.

There are a smorgasbord of different accounts of this, referencing:

human capability for deception ratcheted up human sociality and intelligence via arms race like effects.
runaway signaling.
human’s general intelligence or improvisational intelligence
humans’ prosociality
a package of instinctual modules, such as language, intelligence, social learning, folk physics and biology
human’s cultural learning abilities

In this section, I’ll explain the account that refers to human’s cultural learning abilities. Cultural learning typically refers to a subset of social learning abilities such as mindreading, imitation, and teaching. I’ll use cultural and social learning interchangeably here.

Muthukrishna, Doebeli, Chudek, and Henrich’s recent paper “

The Cultural Brain Hypothesis: How culture drives brain expansion, sociality, and life history

” contains one of the best conceptual models of the cumulative cultural evolution story.

In the paper, Muthukrishna and co are in the business of making mathematical models which they can use to simulate the impact of brain size, sociality, mating structures, and life history. I’ll go the conceptual features of the primary model here and motivate it’s plausibility.

The components of the model are:

assumptio

...
Black hole narratives
by Alexei — 2019-07-07T04:07:11.835Z

Related to: Self-consciousness wants to make everything about itself (by jessicata), UNIVERSAL LOVE, SAID THE CACTUS PERSON (by SSC) and Ms. Blue, meet Mr. Green

Edit (Jul 17, 2019): The concept I'm describing is basically cognitive fusion.

There's an idea that I think is really important to understand and to try to communicate. I think Jessica in her post touched on it a little, but that touch inspired me to give it a shot myself. However, instead of focusing on social dynamics, I'm going to focus on an individual's internal world and interpretation.

Here are a few examples of mental narratives that I'm going to dissect. (I borrowed a few from Jessica's post and a few from the SSC post.) See if you can find the ones that happen to you. Or see if you can understand the pattern to find ones that are not on the list.

1. This thing that's happening is bad. I wish it wouldn't happen.

2. Am I a bad person for participating in this? I think something's wrong with me.

3. I'm not a bad person! I'm definitely in the right here.

4. I'm feeling attacked. I feel like people are against me.

5. I don't feel safe here. I think they'll hurt me.

6. I feel like I'm the odd one out. I don't think I belong here.

7. I don't understand this at all. What am I missing? There's a clue here somewhere. If I can find it, then I'll understand it.

Lenses

To try to understand the pattern and what to do with it, I'll provide four different lenses.

Physical sensations

What does it feel like to experience these narratives? Look and see.

For me, it feels constraining. It feels like sensing a wall and then pushing away from it. Or pushing into it. It feels like a river that has hit the bank and has to turn. My body feels tighter. My awareness feels more narrow or disappears entirely. The thought grows bigger in my mind until it dominates. Until, just for a moment, it feels like my entire mental reality is described by the circumstance. By how I feel attacked. Or by how I feel like I don't belong. It stops being a thought and becomes reality.

So how does it feel when the narrative ends? To me it feels like someone was hugging me super tight and then let go. It feels like a release. An expansion. An exhale. It feels like freedom. Freedom to reconsider. To review. To remember other sides of the problem. Free...
Thank You, Old Man Jevons, For My Job
by Closed Limelike Curves — 2019-07-06T18:37:48.146Z
Link: https://thelimelike.wordpress.com/2019/07/06/thank-you-old-man-jevons-for-my-job/

[Epistemic status: Half-truth.]
[CW: Microeconomics]

Sometime in the 1800s, some random English economist named William Stanley Jevons made an amazing discovery: The demand for coal power is elastic.

This may sound boring as fuck. That’s because it is. A lot of things have price-elastic demand curves. This is probably just another one of those things that should go in the Journal of Things Nobody Has Ever Cared About But May Be Useful to Some Guy with a PhD in a Bizarrely Specific Field. Anyways, that’s why when steam engines became more efficient, British coal consumption shot through the roof.

wat

So anyways, let’s take a look at this graph, taken (With some small edits) from Wikipedia. It’s the main reason why you haven’t lost your job to automation yet, and your only hope for surviving the oncoming robot apocalypse:

The economy: Now with 20% more jobs!

So, that graph makes perfect sense, right? No? OK, fair. Let’s go over what everything means.

This is a supply and demand graph, showing the employment before and after a new machine is introduced which allows more work to be done by fewer people. Let’s say it’s some new steam-powered textile machinery, just to make any 19th-century Luddites reading this very angry.

The curve labeled S is the supply curve for labor. It shows how many man-hours of work you can hire at any particular hourly wage. Offer a wage of $20 an hour, and you’ll get more people showing up and offering to work than if you only offered $10, so it’s always upward-sloping — higher wages mean you get more people willing to work.

HR is the hourly revenue curve. It takes as an input how many man-hours are already being used, and represents how many dollars an employer would make if they paid for an extra man-hour of work.1 Importantly, though, this is also the demand curve for labor: An employer will refuse to hire anyone for more than this amount, because it’ll lose them money, and they will agree to hire anyone willing to work for less than this price.2

If you know any microeconomics, you’re going to be taking a look at the point where the two curves intersect. That’s the point the economy natural settles on, the “Equilibrium point.” I’ll write more on this in a future post, but for now, think about what it means if the HR curve is above the supply curve: It means that the employer is willing to hire for a wage higher than what the potential employe...
87,000 Hours or: Thoughts on Home Ownership
by romeostevensit — 2019-07-06T08:01:59.092Z

Edited to add epistemic status and summary:

The basic thesis of this post is that, for many reasons, the volatility (riskiness) of buying a house are significantly higher than generally believed.

The considerations explored here are the result of discussions with a financial advisor and plugging various pieces of publicly available data about housing markets into adhoc models of long run performance. Ultimately, predictions about asset classes usually come down to reference class forecasting, as any appeal to 'over' or 'under' priced is relative to some trend, and choice of the timescale on which trend following is expected can be chosen to tell a story. E.g. two people arguing can have one arguing for a reversion to a long run historical trend while the other argues for a reversion to a short run trend. They may both be right on different time scales going forward too. This post attempts to surface reasons why buying may be more expensive than a popular model one might encounter, which may have been generated by people in the pro real estate camp. Thus it argues that people's confidence intervals, especially to the downside, are too narrow when modeling home buying. Losses in the housing market are hard to get quantified data on since they are private and often vary enormously due to imputed rents and opportunity cost being idiosyncratic between individuals. A small thumb on the sale at each step in a multiplicative planning space can add up to a lot by the time you reach the output of your model.

The main parameters looked at in this post if you want to do you your own homework (which obviously you should for an individual purchasing decision) are as follows:

Rent-to-Buy ratio (see linked post at bottom for problems with the default parameters of popular rent to buy calculators)

Average time spent in purchased homes

Historical house price indexes

Historical rent price indexes

Vanguard's Balanced Index investment fund (leveraging a balanced fund handily beats all-stock allocations btw)

Salary and savings considerations over a career

Home ownership tax benefit calculators

I'll add any more that occur to me.

Beware extrapolating averages to unusual situations. Outlier markets usually have various reasons that have driven metrics to extremes. Failure to understand the casual mechanisms will prevent one from predicting when such trends...
Learning biases and rewards simultaneously
by Rohin Shah — 2019-07-06T01:45:49.651Z

I’ve finally uploaded to arXiv our work on inferring human biases alongside IRL, which was published at ICML 2019.

Summary of the paper

The IRL Debate

Here’s a quick tour of the debate about inverse reinforcement learning (IRL) and cognitive biases, featuring many of the ideas from the first chapter of the Value Learning sequence:

I had the intuition that the impossibility theorem was like the other no-free-lunch theorems in ML: not actually relevant for what ML could do in practice. So we tried to learn and correct for systematic biases in IRL.

The idea behind the algorithms

The basic idea was to learn the planning algorithm by which the human produces demonstrations, and try to ensure that the planning algorithm captured the appropriate systematic biases. We used a Value Iteration Network to give an inductive bias towards “planners” but otherwise did not assume anything about the form of the systematic bias. [1] Then, we could perform IRL by figuring out which reward would cause the planning algorithm to output the given demonstrations. The reward would be “debiased” because the effect of the biases on the policy would already be accounted for in the planning algorithm.

How could we learn the planning algorithm? Well, one baseline method is to assume that we have access to some tasks where the rewards are known, and use those tasks to learn what the planning algorithm is. Then, once that is learned, we can infer the rewards for new tasks that we haven’t seen before. This requires the planner to generalize across tasks.

However, it’s kind of cheating to assume access to ground truth rewards, since we usually wouldn’t have them. What if we learned the planning algorithm and rewards simultaneously? Well, the no-free-lunch theorem gets us then: maximizing the true reward and minimizing the negative of the true reward would lead to the same policy, and so you can’t distinguish between them, and so the output of your IRL algorithm could be the true reward or the negative of the true reward. It would be really bad if our IRL algorithm said exactly the opposite of what we want. But surely we can at least assume that humans are not expected utility minimizers in order to eliminate this possibility.

So, we make the assumption that the human is “near-optimal”. We initialize the planning algorithm to be optimal, and then optimize for a planning algorithm that is “near” the opt...
Thoughts on The Replacing Guilt Series⁠ — pt 1
by dragohole — 2019-07-05T19:35:58.053Z

I’m currently going through Nate’s series on caring. He starts with the issue of listless guilt. First post is about the stamp collector, the second one is about allowing yourself to fight for something.

I want to log my thoughts on this, because it seems that there’s enough to chew on here.

First, a bit on the stamp collector.

I agree with the general message. It’s true, people aren’t optimized solely for pleasure-seeking. There are other things. Evolutionary psychology has a sufficient enough explanation, I believe. The heuristic “people are ultimately hedonists” is simply wrong, its predictive power is weak.

Now, about the second post.

Yes, you can care about things in the world. You can only access it through your map, but this doesn’t postulate wireheading as the final goal.

I, too, think that the word “want” is loaded. Let’s dissect.

People are embedded in their environments. Their motivational systems are complex. There’s the primal “draw” towards certain things, and it’s mediated by different aspects of motivational systems. For example, it’s generally easier to feel this primal “draw” towards concrete things, which impinge on our senses (let’s not talk PP here).

But we also can use our executive functions in tandem with the ebb-and-flow of our primal motivations. You can “force” yourself to start working on something, and the short bursts of reinforcement for achieving measurable (micro-)milestones will do the rest.

What’s important though is the link between the “forceful” and the “primal”, S2 and S1. There’s only so much you can realistically do purely through S2 control. Usually it’s both S2 and S1 that affect your productivity.

(correct me if I’m wrong here; it might be possible to run on S2 for a long time, but I believe that at some point your local productivity will tank so hard it won’t be worth to continue anymore; furthermore, if you try to abuse your S1 with S2 in this way for longer periods of time, S1 will start resisting, and you’ll probably burn out and\or ruin the rest of your life; alright, I just stated a bunch of obvious, but potentially important stuff to someone who haven't thought about it before; you might be one of them, and if you are, here's your insight: you don't actually need motivation to do stuff, thanks to your S2. go now, try it.)

I think it’s important to also tell yourself that you’re allowed to care about exte...
Here Be Epistemic Dragons
by DirectedEvolution — 2019-07-04T22:31:44.061Z

In Rhetoric, Aristotle said that in an argument, “...if the decisions of judges are not what they ought to be, the defeat must be due to the speakers themselves… the underlying facts do not lend themselves equally well to the contrary views. No; things that are true and things that are better are, by their nature, practically always easier to prove and easier to believe in.” Yet we might argue most about those rare topics where we do encounter difficulties in proving or believing what’s true or best; perhaps the operative word in the quote is practically. To capture the hero’s journey quality of inquiry, let’s call these exceptional cases epistemic dragons. We have to deal with more research than ever before, which constantly pushes the boundaries of what is proven or believable and spawns epistemic dragons at a historically extraordinary rate.

Although we can take a sociological or anthropological view of the scientific, media, and political industries that generate so many of these epistemic dragons, each worker in these industries is an individual jonesing for truth. Alfréd Rényi said, "a mathematician is a machine for turning coffee into theorems," but all our industry has not yet devised a way to mass-produce standardized rational minds, although we are trying.

It’s best practice to base beliefs on the preponderance of evidence, especially controlled and well-replicated experiments that look at a problem from many angles. That’s the paved road through safe territory. A science progresses faster when its frontiers can be expanded that way, but this is often not possible. Science should converge at a border drawn precisely at the point where lie the limits of our ability to conduct controlled experiments. Here be dragons.

Adventures in this dangerous territory aren’t often motivated by the pure desire to consume an entire body of subject literature for its own sake. Our human values, a puzzle-solving itch, intellectual wanderlust, or industrial and professional objectives often lead to a study that is of necessity rather shallow, amateurish, and interdisciplinary. Various broad forms of consensus, logical solutions and storytelling based on descriptive models, direct perception, forecasts and prices, and durability over time are some of the instruments we navigate by.

I want to shift gears for a minute and talk about what it’s like to read. When I w...
What product are you building?
by Raemon — 2019-07-04T19:08:01.694Z

Epistemic status: It appears like it works for me. Not meant to be a hard/fast rule.

A frame I frequently examine conversations through is "are we building a product together, or not?"

Many conversations (online or in person) are "just sorta hanging out." Just-sorta-hanging-out can be quite important, both because it's fun, and as part of building up friendships, etc. But if one or more participants isn't having fun, it's more likely that the conversation should end, or change.

"Building a product" conversations have a goal, and the goal is "create something that someone is going to use someday." Here's a few types of products you might be building:

A literal product

i.e. programming a website or designing a widget

A relationship

you and a friend, or a romantic partner, are trying to improve the foundations of trust or communication

A party

you and your roommates are planning a surprise birthday bash for a friend

A new set of norms

you and your friends, or your coworkers, all agree that something about the current social equilibrium is off, and should be replaced by something new

A fun idea

you're doing random brainstorming on something kinda crazy, but you still want it to be the best something-kinda-crazy that it can be. Maybe it'll later turn into a literal product, or maybe it'll turn into a fun story to tell.

A felt sense

Maybe the object level discussion isn't about the product. Maybe you're exploring a philosophical idea. But your goal isn't to output a useful philosophical idea. Your goal might instead be:

develop a sense of what a particular flavor of confusion feels like
develop some shared intuitions that underlie the philosophical idea

In some of those cases, there are "just hanging out" conversations you might also have about it. You can talk about "a new set of norms" that's less goal directed and more just meandering and exploratory. A fun idea can certainly be nothing more than "a fun idea."

But, I find product-building conversations way more interesting, most of the time. In particular, I'm willing to invest a lot more effort into a conversation if it seems like it's about building something.

Product-building-conversations don't have to be rushed, or laser-focused. Sometimes the best way to design a product is to have a...
How to handle large numbers of questions?
by Raemon — 2019-07-04T18:22:18.936Z

Within a few weeks, I expect to have gotten the Related Questions feature up to a standard where it makes sense to start using it a lot.

Even if I were the *only* one using it extensively, I'd expect to generate very large numbers of questions. (For two frames of reference, imagine if every answer to the Open Problems in Human Rationality had been a related question. For another, see Ruby's Space Colonization: What can we do and how do we know? post (which has 8 related questions)

The currently planned tools will let people set whether each related question appears on the home page or not (but, to have all related questions appear in recent discussion).

I feel sorta differently-worried about each way to turn this knob. Either related questions won't get enough visibility for people to actually see the ones that are interesting to them, or it'll get so much visibility that it'll drown out the site. The "optionally make related questions on home page" helps somewhat, but still leaves me with a sense that a lot of related questions will end up not getting seen by people most inclined to answer them.
Let's Read: an essay on AI Theology
by Yuxi_Liu — 2019-07-04T07:50:51.277Z

Follow me as I read through an essay on AI Theology and give my comments. Good Shepherds (O’Gieblyn, 2019)

It’s not a human move,” said one former champion. “I’ve never seen a human play this move.” Even AlphaGo’s creator could not explain the algorithm’s choice. But it proved decisive.

Obligatory reference to AlphaGo. Could have mentioned AlphaZero, but it was not nearly as popular as AlphaGo.

While Elon Musk and Bill Gates maunder on about AI nightmare scenarios—self-replication, the singularity—the smartest critics recognize that the immediate threat these machines pose is not existential but epistemological.

By "immediate" means "now". Reasonable. AI apocalypse is not scheduled until perhaps 100 years later.

Noah Harari has argued that the religion of “Dataism” will soon undermine the foundations of liberalism. “Just as according to Christianity we humans cannot understand God and His plan,” he writes, “so Dataism declares that the human brain cannot fathom the new master algorithms.”... Our role as humans is not to question the algorithmic logic but to submit to it.

"Dataism" does not necessarily undermine liberalism. It is possible to augment human thought to keep up with the data deluge, and preserve (trans)human liberalism. Although this is not guaranteed.

Job assumes the role of a prosecuting attorney and demands a cosmic explanation for his suffering. God dutifully appears in court, but only to humiliate Job with a display of divine supremacy. Where were you when I laid the foundation of the earth? he thunders, then poses a litany of questions that no human can possibly answer. Job is so flummoxed, he denounces his own ability to reason. “Therefore I have declared that which I did not understand,” he says, “Things too wonderful for me, which I did not know.”

The problem of Theodicy is amusing. And the trials of God can be total dark humor. My favorite is

In a concentration camp, one evening after work, a rabbi called together three of his colleagues and convoked a special court. Standing with his head held high before them, he spoke as follows: “I intend to convict God of murder, for he is destroying his people and the law he gave to them ... I have irrefutable proof in my hands. Judge without fear or sorrow or prejudice. Whatever you have to lose has long since been taken away.” The trial proceeded in due legal form, with witnesses for bo

...