trentmkelly/LessWrong-43k huggingface dataset:
Are long-term investments a good way to help the future?

by Dacyn — 2018-04-30T14:41:56.640Z

[Epistemic status: I am very confused, possibly about very basic Econ 101 things. Probably there is more research I could do to resolve my confusion but I don't have the time or motivation for that right now.]

[Update: Paul Christiano's comment resolved a reasonable part of my confusion, and gjm's comment seems to indicate this is probably the wrong discussion to be having.]

Recently there has been some discussion regarding the question of whether the fact that most people don't make long-term investments to help the future means that people don't care much about the future. To me this argument seems kind of silly: EAs have a hard enough time already getting people interested in ideas of how to help other people without one more weird idea in the mix. But I am more interested in the object-level question: is long-term investment actually a good way to help the future?

The basic argument in favor is simple. The power of compound interest means that if you invest, you will end up with more money (inflation-adjusted) than you started with, which in turn means that you can help more people. If we value all people equally, then this means a higher utility.

There are many possible objections but to me, the most obvious one is that it's not clear making $1000 by investing in the stock market means that you have added $1000 worth of value to the world, it could just mean that you have taken it from someone else. I do know enough Econ 101 to know the obvious counterargument: if you assume rational decision-making together with no negative externalities, then you have to have added $1000 worth of value to the world, because every deal that happens as a result of your investment was rationally consented to by the parties making the deal, and so can be assumed not to hurt them (according to the rationality assumption) nor anyone else (according to the assumption of no negative externalities). So everyone else is better off or at least not worse off than they were before, and you are $1000 better off. So $1000 of value has been added to the world. [Edit to clarify: The point of this argument is not supposed to be that the assumptions are always satisfied (they aren't), but that maybe they are satisfied often enough that we can expect a generic investment that makes $1000 to have added close to $1000 of real value to the world.]

Those seem li...
Soon: a weekly AI Safety prerequisites module on LessWrong
by  — 2018-04-30T13:23:15.136Z

(edit: we have a study group running a week ahead of this series that adds important content. It turns out that to get that content ready on a weekly basis, we would have to cut corners. We prefer quality over speed. We also like predictability. So we decided to cut us some slack and publish every 2 weeks instead for the time being)

Hey there! It's been about 6 weeks since RAISE doubled down on it's mission to make learning AI Safety as convenient as possible.

We've been geared towards STEM majors, but the grand vision was to eventually lay out a learning path that any high-school graduate could take.

So a prerequisites track was on our wish list. Little did we know that such a track had already been constructed, and abandoned, in 2015.

We met it's creators, Erik Istre and Trent Fowler, and we decided to collaborate. There is already at least 20 weeks of content to work with, and they're going to extend it further. Many thanks to them!

For what it's worth: the track was shown to various leading figures in AIS, and the reception has thus far been uniformly positive. To get a sneak peek, register on our platform and have a look at the column called "Prerequisites: Fundamentals of Formalization". The first two levels are already uploaded.

A module will be published every Friday, starting with "level 1: Basic Logic" on May 4th. Let's get some momentum going here! If you complete the track in it's entirety, you should be ready to understand most of the work in AI Safety.

Each module is a set of references to textbooks explaining important topics like Logic, Set theory, Probability and Computability theory. The intention is to 80/20 a bachelor's degree: by covering 20% of the material, you should learn 80% of the relevant concepts. At the end of each module, we made some exercises of our own. Those are made not for practice, but to validate your knowledge. If you think you already know a subject, you can use these to verify it.

All but 2 of the quoted textbooks are available online for free. The other ones will be announced on time. You won't need them before week 3.

We hope that this will help some of you learn AI Safety!

Warm regards,

Toon, Veerle, Johannes, Remmelt, Ofer

The RAISE team

PS: If you'd like to generally stay up to date with RAISE, join our Facebook group or visit our website.
Research: Rescuers during the Holocaust
by Martin Sustrik — 2018-04-30T06:15:40.659Z

Cross-posting from 250bpm.com

The goal

People who helped Jews during WWII are intriguing. They appear to be some kind of moral supermen. Observe how they had almost nothing to gain and everything to lose. Jewish property was confiscated early on and any portable assets Jews may have had have evaporated quickly. Helping Jews, after a short initial period, wasn't a way to get rich. Hoping for compensation after the war didn't work either. At the time it was not obvious that Nazis will lose. Until last couple of years of WWII it was even probable that they will win. And even if they had lost, hoping for some kind of future appreciation from their anti-Semitic compatriots would be naive. On the other hand, by helping Jews, one put oneself into grave danger. In the eastern Europe, if you were found helping Jews, you were most likely to be shot. And your family as well. In western Europe the consequences were a bit less brutal, but still, going to Nazi jail or concentration camp is pretty bad in itself. Moreover, if one decided to hide Jews, it was no temporary thing. As already said, it was entirely plausible that Nazis will win and with that in mind, a person deciding to shelter a Jew was basically signing up for the life in danger for ever after.

Thus, the obvious questions are: Who were these moral supermen? How did they differ from the general population? And: Can we do anything to get more of such people today?

I finally got some free time to research the topic. In this post I am going to summarize what I've learned. However, I am not an expert and my research was superficial at best. Take it with a grain of salt and if you know better, please do correct me.

The sources

Unlike with other genocides we do have some source material on the topic. A lot of it comes from Yad Vashem's "Righteous among the Nations" programme. It's an institution tasked to document the help to the Jews during the Holocaust and to award medals to the rescuers.

Let me insert a personal note here: How cool is that? A nation that was recently slaughtered by millions have created an institution for rewarding people from perpetrator nations, people who haven't participated in the atrocities but rather turned against their compatriots and done what was right. I don't think anything like that was ever done before. I am, for example, unaware of any similar undertaking in the context of the earli...
Give praise
by  — 2018-04-29T21:00:42.003Z

The dominant model about status in LW seems to be one of relative influence. Necessarily, it's zero-sum. So we throw up our hands and accept that half the community is just going to run a deficit.

Here's a different take: status in the sense of worth. Here's a set of things we like, or here's a set of problems for you to solve, and if you do, you will pass the bar and we will grant you personhood and take you seriously and allow you onto the ark when the world comes crumbling. Worth is positive-sum.

I think both models are useful, but only one of these models underlies the emotional need we call status. I think it's the latter.

Another assumption: humans are satisficers. Those that claim to the contrary have never been satisfied. An unsatisfied satisficer acts like a maximizer. I think that Maslov got pretty close with his hierarchy of needs. Not the specific set of needs, not necessarily their order, but the idea of humans working on one need at the time, until satisfaction, so that the next need comes up.

It seems to me that many of us are stuck at the status level, and I think getting past it makes us surely happier and plausibly more effective.

How is worth generated? Quite simply, by giving praise. You either find your behavior exceeding a standard that the community agreed on, or someone actually tells you you're doing well. The latter seems more powerful.

I've asked around, and it even seems to be considered "normal" in most (non-systematizing) communities to habitually give praise. It's even apparently something people regard as necessary for proper psychological health. But honestly, apart from volunteering at CFAR, I can't recall getting much praise for anything I've done for the community. As a result I never quite feel like I'm doing enough, edging on burnout a few times. Reminds me of pica. Does working on AI Safety ever get me the sense of worth I'm looking for, or should I give up?

So I'd like to suggest we try for Giving Praise as a rationalist virtue. It might just be a staple of group rationality.
Does Thinking Hard Hurt Your Brain?
by Raemon — 2018-04-29T18:56:35.325Z

This is a "typical mind fallacy check" post. Curious how much (within and without the rationalsphere) people's experience varies.

I generally experience "thinking hard" to be some combination of stressful, headache inducing, and energy draining (sometimes I feel like I actually am burnt out of energy, sometimes I just need to switch tasks). I've talked to other rationalists and they often don't have this experience, and I'm trying to figure out what's going on here.

I specifically experience this when doing conscious, deliberate thought. In particular if it strains the bounds of my current skills, or my working memory. This most often involves thinking strategically in a careful way (i.e. I get this when playing Chess).

When I'm writing characters or imagining talking to people I know (i.e. simulating another person in my head, or pretending to be another person) I also get a headache if I keep it up for over an hour.

I've only talked to a few people about this, so not sure how wide the spread of experience is. But at the very least this varies a bit among people-I-know.

Some hypotheses (or partial hypotheses) so far:

A. Deliberate Practice is straining. Doing deliberate-practice thinking (i.e. straining at the edge of your competence of a skill) is energy draining, and people vary on a) what things strain the edge of their competence, and b) how often they do such things.

B. Straining is wasted motion. Someone I know recently argued that thinking shouldn't drain willpower or otherwise cost resources other than time, and that any time you do that, it's because you're doing some wasted motion. This initially sounded wrong to me. After experimenting a bit I at least believe that much of "draining" thinking is wasted motion that you can learn to skip.

C. People vary in raw cognitive power. Some people may just think faster naturally, or have a higher bandwidth of working memory.

For now, mostly interested in getting a sense of what diversity of experience we have of this on LessWrong. I also suspect there's some research into this somewhere and am curious if anyone either knows about that, or has further thoughts on mechanisms.
The Case Against Education: Splitting the Education Premium Pie and Considering IQ
by Zvi — 2018-04-29T12:40:00.219Z

Previously: The Case Against Education: Foundations, The Case Against Education

VI

Bryan Caplan makes several cases against education. His primary case revolves around the question of where the education premium comes from – no one doubts that the better educated make more money. As we noted last time in Part V, there are three categories of explanation: Human capital (school teaches you valuable things, so you produce more value), signaling (school sends a valuable signal, so you get better opportunities and better pay), and ability bias (those who would always have earned more money attend more school, and school is taking credit for effects it didn’t cause).

As I noted in part V, it’s undeniable (despite frequently being denied) that ability bias plays a large part in the story, as does signaling, but there’s large uncertainty on how much. It’s a difficult problem, worth looking at from multiple angles.

One way is to ask, what are the relevant measures of human capital? What do we learn in school that helps us be more productive? If we can measure that, we know how much of the story is human capital, leaving the rest to signaling and ability bias.

Another way in is to start with ability bias. If we can correct our measures for underlying ability via a natural experiment or finding good proxy measures for underlying ability, the difference between the adjusted and non-adjusted numbers represents ability bias. There’s lots of types of ability, but what we can measure and correct for would represent a safe lower bound on ability bias.

If you had a good measure of human capital, you could combine these two approaches. Measure human capital before school, then measure it again after school, and compare it to the measure of those who didn’t attend. The additional increase in human capital from school represents human capital, and the starting difference is ability bias. The rest is signaling (or unmeasured abilities and capital, so one must be careful).

We could also start by measuring signaling directly. If we can find explicit signaling effects we can measure, that’s a safe lower bound on the value of signaling.

Another alternative is to compare personal benefits, which include both signaling and human capital, to societal benefits, which should include only human capital (plus the benefits of signaling by the society, ...
Levels of AI Self-Improvement 
by avturchin — 2018-04-29T11:45:42.425Z

[new draft for commenting, full formatted version is here: https://goo.gl/c5UfdX]

Abstract: This article presents a model of self-improving AI in which improvement could happen on several levels: hardware, learning, changes in code, in goals system, creating virtual organization, and each of which has several sublevels. We demonstrate that despite diminishing returns at each level and some intrinsic difficulties of recursive self-improvement—like the intelligence-measuring problem, testing problem, parent-child problem and halting risks—even non-recursive self-improvement could produce a mild form of superintelligence by combining small optimizations on different levels with the power of learning. Based on this, we analyze how self-improvement could happen on different stages of the development of AI, including the stages at which AI is boxed or hiding in the internet. AI may prefer slow low-level evolutionary self-improvement as low-level improvement has more chances to preserve AI's goal stable and prevents AI's halting risks. But if AI is limited in time of threatened, it may prefer revolutionary style of self-improvement, which promises bigger gains but also higher risks of non-alignment of the next version of AI or AI halting.

Keywords: AI, recursive self-improvement, intelligence, existential risks

Highlights:

· AI may self-improve on 6 levels: hardware, learning, writing code, goal system changes, running many AI copies and acquisition of resources, with around 30 sublevels.

· Returns from improvement on each level decline logarithmically but could be multiplicative between levels, which implies possibility of at least 2E30 improvement.

· Even without recursive self-improvement, AI may reach the level of “mild superintelligence”, where it has combined intelligence of all humanity.

· There are two styles of self-improvement: evolutionary and revolutionary; AI will more likely prefer evolutionary changes, if not threaten.

· High-level revolutionary changes provide bigger expected gains but also have more risks for AI itself and for humans, as there are higher chances of losing alignment, AI halting or AI versions war.

1. Introduction

The term “self-improving artificial intelligence” (SI-AI) is often used as “magic words” which explain everything, but the concept requires deeper analysis. Bostrom and Yudkowsky wrote that AI could start self-improving after reachi...
Inefficient Doesn’t Mean Indifferent
by Zvi — 2018-04-29T11:30:01.467Z

Many people, including Bryan Caplan and Robin Hanson, use the following form of argument a lot. It could be considered the central principle of the (excellent) The Elephant in the Brain. It goes something like:

People say they want X, and they do Y to get it.
If people did C, they would get X, and the price of C is cheap!
Therefore, people really value X at less than the price of C, so they don’t really care much about X.

There’s something very perverse going on here. We’re using people trying to get X in an inefficient way as evidence they don’t care about X, rather than as evidence that people aren’t efficient.

The trick is, there’s a lot of assumptions hidden in the above logic. In practice, they rarely hold outside of simple cases (e.g. consumption goods).

The motivating example was Bryan Caplan using this one in The Case Against Education:

People say they want smart employees, and look at school records to get it.
If people gave out IQ tests, they would get smart employees, and testing is cheap!
Therefore, people don’t really value smart employees.

In that case, I agree. Employers (most often) don’t want smart employees beyond a threshold requirement. But local validity is vital, and you can’t do that.

There are lots of reasons why one might not want to do C.

As a minimal first step, people have to believe that strategy C would work. A recent example of Robin Hanson using this technique, that violates that requirement, from How Best Help Distant Future?, could be summarized this way:

People say they want to help the future, and lobby for policies they think help.
If people saved money to help the far future, which they almost never do, they could help more, and since you get real returns from it, it’s really cheap!
Therefore, people don’t much care about the far future.

In that case, I strongly disagree. People rightfully do not have faith that saving money now to help the far future will result in the far future being helped. Perhaps it would, but there’s a lot of assumptions that case relies upon, many of which most folks disagree with – about when money will have how much impact (especially if you expect a singularity to happen), about what you can expect real returns to be especially in the worlds that need help most, about whether that money is likely to be confiscated, about whether the money if not confiscated would actually get spent in a useful way when the ti...
Program Search and Incomplete Understanding
by Diffractor — 2018-04-29T04:32:22.125Z

epistemic status: crystallizing a pattern

When trying to solve some foundational AI problem, there is a series of steps on the road to implementation that goes something like this (and steps may be skipped):

First, there is developing some philisophical solution to the problem. The early decision theory work on TDT falls in this category. Making progress on this step involves developing the ability to take an idea and find holes in it, although a lot of the thought here isn't quite precise enough to be mathematized.

Second, there is developing some solution to the problem, given infinite computing power. If there is some algorithm to solve the problem with a halting oracle or a reflective oracle, then this step is finished. Being able to translate something from philosophy to math is extremely important, and there's no official training method for this skill. Inventing AIXI for the first time would lie in this level. Often, by crystallizing some idea into math, a bunch of subtle little holes will appear that aren't findable by philisophical thought, although once they're found, it's generally possible to translate them back into intuitive philisophical terms.

Third, there is developing some sort of very impractical solution to the problem. The algorithm from the logical induction paper falls in this category, and so do brute-force approaches to solving 3SAT or playing go by building a game tree 50 moves ahead. Going from "doable with infinite computing power" to "doable with very large amounts of computing power" is generally easier than crystallizing an idea into an algorithm in the first place.

Fourth, there is developing some sort of poly-time solution to the problem. Generating an algorithm that solves practical 3SAT instances falls in this category. I'm unsure, but this seems to be the step that involves the most technical and brute-force math work, because it requires characterizing easily exploitable patterns, as well as features that make a problem hard, in order to pick as much low-hanging (polynomial) fruit as you can. Also, there generally aren't proofs that the algorithm is optimal among the set of poly-time algorithms, there are probably going to be enough advances available to keep a sub-sub-field going for a while. Computational complexity theory is the most applicable in this stage.

Fifth, ther... ,,,,,,
Meditations on the Medium
by  — 2018-04-29T02:21:35.595Z

[I explore how the nature of the essay medium presents potential challenges for rationality as a field.

First off, essays often aren’t updated and exploring new ideas is often more rewarding than revisiting old content. This means that there is not much of an incentive to create accessible textbook-like introductions to rationality.

Secondly, despite having ported over to a digital medium, essays are still largely the same. I think there’s value in exploring different norms and features in our writing.

Thirdly, I point out how, even if we figure out how to create very good online content, the way that the medium shapes our expectations might still not be good enough. Often, structure seems to be an important component, and I also don’t think we’ve pushed in this direction enough either.]

Rationality is concerned with a good number of questions. The two that I'm thinking about right now are:

Why do some attempts at behavioral modification succeed and others fail, both between individuals and within an individual?
If we had an answer to 1, what does it look like to actually make that knowledge usable, bridging the gap from just a mere description?

I think both of these queries connect to the question of how the nature of the medium affects our actual experience when trying to engage with rationality concepts. From a design and pedagogical standpoint, I think there changes we could make to potentially improve the ways that ideas advance and cash out into practicality.

Where Are All The Good Rationality Textbooks?

Let's start with something more personal.

I hate revisiting old essays.

I'm happy linking back to them, sure, but the idea of editing or rewriting them feels abhorrent. As a kid, I hated checking my work when doing math. In both cases, there's a feeling of "I just put in all this effort to get this thing done. Now, you want me to...do it all over again!?"

There's a sense of, not just our expectations being violated, but of all of our past work being essentially for naught.

For example, it's not exactly the marathon runner who is told they have 3 more miles to run when they thought they were already finished, but the marathon runner who is told they ran the race backwards and needs to start over from the actual starting line.

This sense of "not wanting to repeat things" is what I think forms a core part of why s...