trentmkelly/LessWrong-43k huggingface dataset:
Is there a recursive self-improvement hierarchy?
by PhilGoetz — 2015-10-29T02:55:00.909Z

When we talk about recursively self-improving AI, the word "recursive" there is close enough to being literal rather than metaphoric that we glide over it without asking precisely what it means.

But it's not literally recursion—or is it?

The notion is that an AI has a function optimize(X) which optimizes itself. But it's recursion in the sense of modifying itself, not calling itself. You can imagine ways to do this that would use recursion—say, the paradigmatic executable that rewrites its source code, compiles it, and exec's it—but you can imagine many ways that would not involve any recursive calls.

Can we define recursive self-improvement precisely enough that we can enumerate, explicitly or implicitly, all possible ways of accomplishing it, as clearly as we can list all possible ways of writing a recursive function? (You would want to choose one formalism to use, say lambda calculus.)

To see why this is important, consider this set of LISP functions:

[1]> (defun f(x) (g x))
F
[2]> (defun g(x) (f x))
G
[3]> (f 0)

*** - Lisp stack overflow. RESET

The composition of f and g is a recursive function, though neither f nor g is itself recursive. This is analogous to two AIs, neither of which wishes to nor can modify itself, but each of which can modify the other. I can imagine that AI #1 wants to use AI #2 for some task, and tweaks its code to improve it. Meanwhile AI #2 wants to use AI #1 for something else, and tweaks its code to improve it. And so on. The effect is something like recursive self-improvement of both AIs. Without being guided by the analogy between recursive functions and recursive self-improvement, it would be easy to overlook this possibility.

The notion of an AI which can rewrite its own source code is just one special case, which is both unlikely and poorly-defined. Unlikely, because lots of programmers already write self-modifying code, and they don't give the program a text file with its source code. Poorly defined, because many languages and computational architectures blur the distinction between code and data.

Once we try to be specific about what it means, we see there are many different ways of accomplishing it. We could provide a LISP program with a pointer to the data structure of the running program. We could write Perl code that can read and modify functions defined in the symbol ta...
What Makes the New Atheists So Charitable?
by tog — 2015-10-29T01:50:45.618Z

Authors: Joey Savoie and Tee Barnett

Before getting to know your local atheist, it’s very much worth rehashing the ABCs of non-belief that run the risk of remaining little known, especially now that the skeptic community has become more interested and active in charitable causes. Public intellectuals frequenting bestseller lists on our behalf are swift to go on the offensive, but there’s something to be said for shoring up the defense as well. A real uneasiness toward atheists and their intentions seems to flow from a very common and endlessly parroted assumption—that without belief in god, anything would be allowed. Divine reprimand and reward are ultimately credited with keeping us on the straight and narrow, and often said to have provided the moral foundation for our society. A charitable movement populated with skeptics and atheists would seem counterintuitive or even completely bananas then, but nonetheless, a number of causes under the umbrella of Effective Altruism (EA) are blossoming. How do we account for this? Should we credit our learned behavior to a society built on these heavenly mandates, or is it something else?

Spoiler alert: the answer is something else.

Whether it comes from the prosaic lips of Ivan Fyodorovitch of The Brothers Karamazov, in the more contemporary form of Dinesh D’Souza, or confronted you recently one way or another, the common argument sees morality as having originated from the outside. Without a punishing set of external pressures imposed from up high, so it goes, mankind will naturally veer off into a wilderness of undesirable behavior. If we care to reexamine the ABCs of atheism, we could start with what Elizabeth Anderson aptly describes as the atheist commitment to “the expansion and growth of the human mind.” Of course, this might seem obnoxiously smug. Who would march against personal growth?

Nobody

What Anderson is getting at is less about condescending to theists and more about illustrating where morality originates for atheists. Jean-Paul Sartre pre-empted her decades before by contending that morality comes from within and grows outwardly. Without wading too far into the philosophical thicket, we’ll leave you with Sartre illustrating how even personal decisions can radiate outwards, “In fashioning myself I fashion man.” Within each of us, over...
Meetup : LessWrong Reading Picnic
by dutchie — 2015-10-28T23:15:26.123Z

Discussion article for the meetup : LessWrong Reading Picnic

WHEN:
31 October 2015 11:00:00AM (+0000)

WHERE:
Forbury Park, Reading

It's picnic season while it's still October! Bring blankets, food, games, conversation topics and anything else you think might be fun. This being England, probably also an umbrella.

Facebook event: https://www.facebook.com/events/1922133231345631/

Discussion article for the meetup : LessWrong Reading Picnic
Does the Internet lead to good ideas spreading quicker?
by Stefan_Schubert — 2015-10-28T22:30:50.026Z

I think it does among the cognitive elite, and that this explains the rise of complex but good ideas such as applied rationality and Effective altruism. I'm less sure about other groups.

The Internet increases the speed and the convenience of communication vastly. It also makes it much easier for people with shared interests to get in contact.

This will of course lead to a tremendous increase in the amount of false or useless information. But it will also lead to an increase in true and relevant information.

Now members of the cognitive elite are, or so I claim, reasonably good at distinguishing between good and bad ideas. They do this not the least by finding reliable sources. They will quickly pass this, mostly true information, on to other members of the cognitive elite. This means that the higher pace of information dissemination will translate into a higher pace of learning true ideas, for this group.

What about non-elite groups? I'm not sure. On the one hand, they are, by definition, not as good at distinguishing between good and bad ideas. On the other hand, they are likely to be heavily influenced by the cognitive elite, especially in the longer run.

By and large, I think we have cause for optimism, though: good ideas will continue to spread quickly. How could we make them spread even quicker?The most obvious solution is to increase the reliability of information. Notice that while information technology has made it much more convenient to share information quickly, it hasn't increased the reliability of information.

There are a couple of ways of addressing this problem. One is better reputation/karma systems. That would both incentivize people to disseminate true and relevant information, and make it easier to find true and relevant information. (An alternative, and to my mind interesting, version is reputation systems where the scores aren't produced by users, but rather by verified experts.)

Another method is automatic quality-control of information (e.g. fact-checking). Google have done some work on this, but still, it is in its infancy. It'll be interesting to follow the development in this area in the years to come.
What we could learn from the frequency of near-misses in the field of global risks (Happy Bassett-Bordne day!)
by turchin — 2015-10-28T18:28:51.613Z

I wrote an article how we could use such data in order to estimate cumulative probability of the nuclear war up to now.

TL;DR: from other domains we know that frequency of close calls is around 100:1 to actual events. If approximate it on nuclear war and assume that there were much more near misses than we know, we could conclude that probability of nuclear war was very high and we live in improbable world there it didn't happen.

Yesterday 27 October was Arkhipov day in memory of the man who prevented nuclear war. Today 28 October is Bordne and Bassett day in memory of Americans who prevented another near-war event. Bassett was the man who did most of the work of preventing launch based false attack code, and Bordne made the story public.

The history of the Cold War shows us that there were many occasions when the world stood on the brink of disaster. The most famous of them being the cases of Petrov , Arkhipov and the recently opened Bordne case in Okinawa

I know of over ten, but less than a hundred similar cases of varying degrees of reliability. Other global catastrophic risk near-misses are not nuclear, but biological such as the Ebola epidemic, swine flu, bird flu, AIDS, oncoviruses and the SV-40 vaccine.

The pertinent question is whether we have survived as a result of observational selection, or whether these cases are not statistically significant.

In the Cold War era, these types of situations were quite numerous, (such as the Cuban missile crisis). However, in each case, it is difficult to say if the near-miss was actually dangerous. In some cases, the probability of disaster is subjective, that is, according to participants it was large, whereas objectively it was small. Other near-misses could be a real danger, but not be seen by operators.

We can define near-miss of the first type as a case that meets the both following criteria:

a) safety rules have been violated

b) emergency measures were applied in order to avoid disaster (e.g. emergency breaking of a vehicle, refusal to launch nuclear missiles)

Near-miss can also be defined as an event which, according to some participants of the event, was very dangerous. Or, as an event, during which a number of factors (but not all) of a possible catastrophe coincided.

Another type of near-miss is the miraculous salvation. This is a situation whereby a disaster was aver...
Meetup : Washington, D.C.: Fun & Games
by RobinZ — 2015-10-28T18:19:56.499Z

Discussion article for the meetup : Washington, D.C.: Fun & Games

WHEN:
01 November 2015 04:00:00PM (-0400)

WHERE:
National Portrait Gallery

Crossposted from mailing list. Meeting in courtyard, gathering from 3:00 to 3:30 to start at 3:30.

We'll be meeting to hang out, play games, converse, and any combination thereof.

If you would like to play a specific game, especially a long one or one with many players, feel free to send messages to the list to organize - they kick us out a little before 7 p.m.

Upcoming meetups:

Nov. 8: Book Swap

Nov. 15: Why Ad Hominem Works

Nov. 22: Fun & Games

Discussion article for the meetup : Washington, D.C.: Fun & Games
The Ultimate Testing Grounds
by Stuart_Armstrong — 2015-10-28T17:08:00.473Z

A putative new idea for AI control; index here.

EDIT: the mathematics of this approach can be found here.

In a previous post, I described a procedure for querying an AI about whether a certain outcome was desirable for it. When I did so, I didn't realise the power of the idea.

In brief, that idea allows us to query an AI with utility v as to whether an outcome X or an outcome ¬X was the one it most desired in a particular situation. By judicious use of 'false beliefs', the AI would react as if it had miraculous powers to determine X vs ¬X in ways that no-one could ever know about.

Many (including Toby Ord) has suggested variants on the 'honey pot' idea: leave something available to the AI that looks like an oversight, but is actually a trap, to see if the AI goes beyond the parameters of what we'd expect. Maybe a wire to the internet that the AI is not supposed to use, but seems to be unobserved.

The problem with these ideas is that the honey pots had to be physically constructed, physically accessible to the AI, and the AI had to be wrong about whether it was a trap or not.

But using the method above, we can go much further. We can test the AI in any scenario which we can imagine, as long as we can precisely physically describe it. We can check what would happen if the AI was given the ability to launch the world's nuclear arsenals, or divert an asteroid towards Earth. We can check what it would design if it was given unrestricted access to a DNA synthesiser. We could see what it would do if it could rewrite the text of a speech an important world leader was due to give. We could see whether it would sabotage the creation of other AIs - and if so, which ones. We could see what it did to a probe humanity planned to launch to the stars, what numbers it would call if it had access to a phone, what it would purchase if it had capital.

And, in all cases, the AI would be behaving as if its actions were undetectable and certain to succeed, while we ensured that they were (almost certainly) detectable and certain to fail. There is no "almost" in that certain to fail condition, because the AI is acting as if it had miraculous powers that it doesn't have.

Now, we can't use testing as our sole method of AI control (for reasons similar to here), but it does allow us some pretty effective ways of catching bad AIs.
Proposal for increasing instrumental rationality value of the LessWrong community
by Vlad Sitalo — 2015-10-28T15:18:52.194Z

There were some concerns here (http://lesswrong.com/lw/2po/selfimprovement_or_shiny_distraction_why_less/) regarding value of LessWrong community from the perspective of instrumental rationality.

In the discussion on the relevant topic I've seen the story about how community can help http://lesswrong.com/lw/2p5/humans_are_not_automatically_strategic/2l73 from this perspective.

And I think It's a great thing that local community can help people in various ways to achieve their goals. Also it's not the first time I hear about how this kind of community is helpful as a way of achieving personal goals.

Local LessWrong meetups and communities are great, but they have kind of different focus. And a lot of people live in places where there are no local community or it's not active/regular.

So I propose to form small groups (4-8 people). Initially, groups would meet (using whatever means that are convenient for a particular group), discuss the goals of each participant in a long and in a short term (life/year/month/etc). They would collectively analyze proposed strategies for achieving these goals. Discuss how short term goals align with long term goals. And determine whether the particular tactics for achieving stated goal is optimal. And is there any way to improve on it?

Afterwards, the group would meet weekly to:

Set their short term goals, retrospect on the goals set for previous period. Discuss how successfully they were achieved, what problems people encountered and what alterations to overall strategy follows. And they will also analyze how newly set short-term goals coincide with long-term goals.

In this way, each member of the group would receive helpful feedback on his goals and on his approach to attaining them. And also he will fill accountable, in a way, for goals, he have stated before the group and this could be an additional boost to productivity.

I also expect that group would be helpful from the perspective of overcoming different kind of fallacies and gaining more accurate beliefs about the world. Because it's easier for people to spot errors in the beliefs/judgment of others. I hope that group's would be able to develop friendly environment and so it would be easier for people to get to know about their errors and change their mind. Truth springs from argument amongst friends.

Group will reflect on it's effectiveness and procedures...
[Link] Rationality and Willpower in LifeHack
by Gleb_Tsipursky — 2015-10-27T21:49:03.599Z

Diversifying the genres of spreading rationality as part of my broader project by publishing a listicle, and getting it published in Lifehack, a major online publication. Included a link to Less Wrong in it, guessing this is the first time there's a link to LessWrong in a listicle. What do you think about the article and this genre of spreading rationality?
Meetup : West LA: Bias Bias
by abramdemski — 2015-10-27T08:53:18.843Z

Discussion article for the meetup : West LA: Bias Bias

WHEN:
28 October 2015 07:00:00PM (-0700)

WHERE:
Westside Pavilion Mall, 10850 Pico Blvd, Los Angeles, CA 90064

How to Find Us: We are meeting at the old location again this week, in the upstairs wine bar at Westside Pavillion.

Discussion: The phrase "bias bias" could mean many things. Perhaps one might employ the term to point to the tendency to accuse others of bias before oneself. Perhaps, as in this paper, it could refer to the tendency of statisticians to be overly concerned with eliminating statistical bias and under-concerned about variance. What I want to discuss is the risk that, if we are observing other decision-makers from the outside with less knowledge about the situation than them, we will almost always find predictable irregularities in their decision-making which we cannot explain via our understanding of the situation. This will, I think, tend to be true whether they're "biased" in a significant sense or not. In other words: we're very likely to have less knowledge about the situation than the people making the decisions, and this is very likely to mislead us into thinking they're making biased decisions which are harming them, if we approach the question without sufficient awareness. This doesn't mean we can't assess bias, but it does sound a note of caution in doing so. Even in cases where the reasoning from our perspective seems very clear, the decision-maker may have other considerations to take into account.

Recommended Reading: I don't know of anything written specifically on this, but the recent breaking Chesterton’s fence in the presence of bull seems relevant here.

No prior exposure to Less Wrong is required; all are welcome.

Discussion article for the meetup : West LA: Bias Bias
"How To Become Less Wrong" - Feedback on Article Request
by Gleb_Tsipursky — 2015-10-26T14:42:58.650Z

Would appreciate feedback on this article I plan to submit to a broad media publication as part of my broader project of promoting rationality and raising the sanity waterline. Can't make it much longer as I'm at word limit, so if you suggest adding something, also suggest taking something away. The article is below the black line and thanks for any feedback!

____________________________________________________________________________________________________________

Article - How I Became Less Wrong

On a sunny day in early August, my wife Agnes Vishnevkin and I came to a Rationality Dojo in Columbus, OH. Run by Max Harms, this group is devoted to growing mentally stronger through mental fitness practices. That day, the dojo’s activities focused on probabilistic thinking, a practice of assigning probabilities to our intuitive predictions about the world to improve our ability to evaluate reality accurately and make wise decisions to reach our goals. After learning the principles of probabilistic thinking, we discussed how to apply this strategy to everyday life.

We were so grateful for this practice in early September, when my wife and I started shopping for our new house. We discussed in advance the specific goals we had for the house, enabling us to save a lot of time by narrowing our options. We then spent one day visiting a number of places we liked, rating each aspect of the house important to us on a numerical scale. After visiting all these places, we sat down and discussed the probabilities on what house would best meet our goals. The math made it much easier to overcome our individual aesthetic preferences, and focus on what would make us happiest in the long run. We settled on our top choice, made a bid, and signed our contract.

This sounds like a dry and not very exciting process. Well, we were very excited!

Why? Because we were confident that we made the best decision with the information available to us. The decision to get a new house is one of the biggest financial decisions we will make in our lifetime. It felt great to know that we could not have done any better than we did through applying the principles of probabilistic thinking and other rationality-informed strategies. Of course, we could still be wrong, there are no guarantees in life. Yet we know we did the best we could - we grew less wrong.

Th...
Open thread, Oct. 26 - Nov. 01, 2015
by MrMind — 2015-10-26T08:34:29.940Z

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
Bounded Solomonoff Induction: Three Difficulty Settings
by Scott Garrabrant — 2015-10-26T05:20:49.000Z

I am starting another (probably) long sequence of posts on asymptotic logical uncertainty. Here, we will move past the Benford test to a more general goal. The purpose of this series will be to investigate the following question:
What is the correct resource bounded analogue of Solomonoff induction?

This series is a continuation of the Asymptotic Logical Uncertainty series, but I will start fresh with the notation, and try to keep it self contained.

Consider an environment, which is a Turing machine E. E takes as input a single natural number n, and outputs a bit E(n).

Our goal is to construct a Turing machine M, which takes as input an environment E and a natural number n, and outputs a probability M(n). This probability is meant to represent the probability that E outputs an 1.

Let A(n) be a fast growing function. We restrict M to run in time O(A(n)2) on input n. In all of our proposals, M will sample and simulate Turing machines which run it tame O(A(n)) on input n. We will judge M by comparing it to other turing machines which run in O(A(n)) time on input n.

We will score M based on how close its probabilities are to the true environment, E. I will talk later about different ways we can score M, but first, I want to describe three different difficulty settings for this problem:

Level 1: E(n) is guaranteed to output in time at most A(n+1).

Note that we assume that A(n+1)>>A(n)2, so this does not make the question completely trivial. However, this assumption does allow M to compute all E(i) for all i<n when asked to assign a probability to n. As we will soon see, this assumption make the problem much easier. Unfortunately, it also makes the problem much less important, as this is not a very realistic assumption to make.

Bounded Solomonoff induction is almost completely solved at this difficulty level by the "Subsequence Induction" algorithm, which I will present in detail later in the series. There may be some further work extending the existing algorithm so that M not only does better than anything that runs in time A(n), but is also well calibrated and does better than any A(n) time transformations that can be applied to its output.

Level 2: E(n) is guaranteed to output in bounded time for all inputs.

It is perhaps surprising that this difficulty level is so much harder than level 1. We still eventually get to learn every bit of the true environmen

...