trentmkelly/LessWrong-43k huggingface dataset:
Many Worlds against Simulation?
by Coacher — 2016-03-07T13:50:09.396Z

Lets assume few things:

1. Many Worlds is real.

2. All identical consciousnesses measures as 1 in anthropics . So if we have set of consciousness: 1xA,1xB and 1000000xC, it is still 1/3 chance, to perceive being C.

Now say some intelligent being (i.e. human) starts another human brain simulation on silicon chip. The operations it does are all discrete, so despite the chip splitting in to many chips in many worlds, the simulated consciousness itself remain just 1 (because of #2 assumption).

But that is not true for human who started the simulation as he differs somehow in every Everett branch and reaches billions different consciousnesses really fast.

Is there some mistake in reasoning, that real persons should heavily outweigh simulations, despite, how many of them are running, given such assumptions?
Cultivate the desire to X
by Elo — 2016-03-07T03:40:27.221Z

Recently I have found myself encouraging people to cultivate the desire to X.

Examples that you might want to cultivate interest in include:

Diet

Organise ones self

Plan for the future

be a goal-oriented thinker

build the tools

Anything else in the list of common human goals

Getting healthy sleep

Being less wrong

Trusting people more

Trusting people less

exercise

interest in a topic (cars, fashion, psychology etc.)

Why do we need to cultivate?

We don't. But sometimes we can't just "do". Lot's of reasons are reasonable reasons to not be able to just "do" the thing:

Some things are scary

Some things need planning

Some things need research

Some things are hard

Some things are a leap of faith

Some things can be frustrating to accept

Some things seem stupid (well if exercising is so great why don't I automatically want to do it)

Other excuses exist.

On some level you have decided you want to do X; on some other level you have not yet committed to doing it. Easy tasks can get done quickly. More complicated tasks are not so easy to do right away.

Well if it were easy enough to just successfully do the thing - you can go ahead and do the thing (TTYL flying to the moon tomorrow - yea nope.).

your system 1 wants to do the thing and your system 2 is not sure how.

your system 2 wants to do the thing and your system 1 is not sure it wants to do the thing.

The healthy part of you wants to diet; the social part of you is worried about the impact on your social life.

(now borrowing from Common human goals)

Your desire to live forever wants you to take a medication every morning to increase your longevity; your desire for freedom does not want to be tied down to a bottle of pills every morning.

Your desire for a legacy wants you to stay late at work; your desire for quality family time wants you to leave the office early.

The solution:

The solution is to cultivate the interest; or the desire to do the thing. From the initial point of interest or desire - you can move forward; do some research to either convince your system 2 of the benefits, or work out how to do the thing to convince your system 1 that it is possible/viable/easy enough. Or maybe after some research the thing seems impossible. I offer Cultivating the desire as a step along the way to working it out.

Short post for today; Cultivate the desire to do X.

Meta: t...
Open Thread March 7 - March 13, 2016
by Elo — 2016-03-07T03:24:22.618Z

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
Outreach Thread
by Gleb_Tsipursky — 2016-03-06T22:18:01.599Z

Based on an earlier suggestion, here's an outreach thread where you can leave comments about any recent outreach that you have done to convey rationality-style ideas broadly. The goal of having this thread is to organize information about outreach and provide community support and recognition for raising the sanity waterline. Likewise, doing so can help inspire others to emulate some aspects of these good deeds through social proof and network effects.
What is the future of nootropic drugs? Why can't there be ones more effective than ones that have existed for 15+ years?
by InquilineKea — 2016-03-06T18:45:06.910Z

So Scott Alexander's post at http://slatestarcodex.com/2016/03/01/2016-nootropics-survey-results/ shows that the most "effective" "nootropics" have still been the ones that have existed for a long time. What do these results really mean, though? Is it possible that people are just worse at noticing the subtler effects of the other drugs, or are just much worse at disciplining themselves enough to correctly use the racetams or noopept (as in, with choline)?

How much potential is there in innovation in nootropics? What is holding this innovation back, if anything? It feels like there hasn't been any real progress over the last 15 years (other than massively increased awareness), but could targeted drug discovery (along with people willing to be super-liberal with their experimentation) finally lead to some real breakthroughs?
Preference over preference
by Elo — 2016-03-06T00:51:14.520Z

Each individual person has a preference. Some preferences are strong, others are weak. For many preferences it's more complicated than that; they aren’t static, and we change our preferences all the time. Some days we don't like certain foods, sometimes we may strongly dislike a certain song then another time we may not care so much. Our preferences can change in scope, as well as intensity.

Sometimes people can have preferences over other people's preferences.

Example 1: I prefer to be surrounded by people who enjoy exercise, that way I will be motivated to exercise more.

Example 2: I prefer to be surrounded by people who don't care how they look, that way I look prettier than everyone else.

Example 3: I prefer when other people like my clothes.

Example 4: I prefer my partners to be polyamorous.

Example 5: I prefer people around me to not smoke.

The interesting thing about example 3; is that there are multiple ways to achieve that preference:

Find out what clothes people like and acquire those clothes, then wear them regularly.

Find people who already like the clothes that you have, then hang around those people regularly.

Change the preference of the people around you so that they like your clothes.

Changing someone’s preference over clothing seems pretty harmless, and that way you get to wear clothes you like, they get to like the clothes you wear, and you get to be around people who like the clothes you wear without finding new people. The scary and maybe uncomfortable thing is that the other preferences can be also achieved through these means.

Example 4:

Find out where poly people are, and hang out with them. (and ask to be their partners - etc)

Find out which of the people you know are already poly and hang out with them (and ask to be their partners - etc)

Change the preferences of your existing partner/s.

Example 1:

Find out where people who enjoy exercise hang out, and join them.

Find out which of your friends already enjoy exercise and hang out with them.

Change the preferences of those around you to also enjoy exercise.

Example 5:

Find out where people don't smoke, hang out in those places.

Figure out who already doesn't smoke and hang out with them.

Encourage people you know to not smoke.

(I think that's enough examples)

Is it wrong?

There is nothing inherently wrong with having a p...
Updating towards the simulation hypothesis because you think about AI
by SoerenMind — 2016-03-05T22:23:49.424Z

(This post is both written up in a rush and very speculative so not as rigorous and full of links as a good post on this site should be but I'd rather get the idea out there than not get around to it.)

Here’s a simple argument that could make us update towards the hypothesis that we live in a simulation. This is the basic structure:

1) P(involved in AI* | ¬sim) = very low

2) P(involved in AI | sim) = high

Ergo, assuming that we fully accept this the argument and its premises (ignoring e.g. model uncertainty), we should strongly update in favour of the simulation hypothesis.

Premise 1

Supposed you are a soul who will randomly awaken in one of at least 100 billion beings (the number of homo sapiens that have lived so far), probably many more. What you know about the world of these beings is that at some point there will be a chain of events that leads to the creation of superintelligent AI. This AI will then go on to colonize the whole universe, making its creation the most impactful event the world will see by an extremely large margin.

Waking up, you see that you’re in the body of one of the first 1000 beings trying to affect this momentous event. Would you be surprised? Given that you were randomly assigned a body, you probably would be.

(To make the point even stronger and slightly more complicated: Bostrom suggests to use observer moments, e.g. an observer-second, rather than beings as the fundamental unit of anthropics. You should be even more surprised to find yourself as an observer-second thinking about or even working on AI since most of the observer seconds in people's lives don’t do so. You reading this sentence may be such a second.)

Therefore, P(involved in AI* | ¬sim) = very low.

Premise 2

Given that we’re in a simulation, we’re probably in a simulation created by a powerful AI which wants to investigate something.

Why would a superintelligent AI simulate the people (and even more so, the 'moments’) involved in its creation? I have an intuition that there would be many reasons to do so. If I gave it more thought I could probably name some concrete ones, but for now this part of the argument remains shaky.

Another and probably more important motive would be to learn about (potential) other AIs. It may be trying to find out who its enemies are or to figure out ways of acausal trade. An AI created ...
Rationality Quotes Thread March 2016
by elharo — 2016-03-05T18:44:48.980Z

Another month, another rationality quotes thread. The rules are:

Provide sufficient information (URL, title, date, page number, etc.) to enable a reader to find the place where you read the quote, or its original source if available. Do not quote with only a name.

Post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)

Do not quote yourself.

Do not quote from Less Wrong itself, HPMoR, Eliezer Yudkowsky, or Robin Hanson. If you'd like to revive an old quote from one of those sources, please do so here.

No more than 5 quotes per person per monthly thread, please.
Informed oversight through an entropy-maximization objective
by jessicata — 2016-03-05T04:26:54.000Z

The informed oversight problem is a serious
challenge for approval-directed agents (I recommend reading the post if you haven't already). Here is one approach
to the problem that works by adding an entropy-maximization objective.

Let agent B be overseeing agent A. It seems that some of the problem is that A has many different possible strategies that B
evaluates as good. Thus, A may choose among these good-looking strategies arbitrarily. If some of the good-looking
strategies are actually bad, then A may choose one of these bad strategies.

This is not a problem if B's evaluation function B(x,⋅) has a single global maximum, and solutions significantly different from this one
are necessarily rated as worse. It would be nice to have a general way of turning a problem with multiple global maxima
into one with a unique global maximum.

Here's one attempt at going this. Given the original evaluation function b mapping strings to reals, construct a new
evaluation function v mapping distributions over strings to reals. Specifically, for some other distribution of strings f and a constant α>0, define

v(d)=−γDKL(f||d)+Ey∼d[b(y)]=γH(d)+Ey∼d[γlogf(y)+b(y)]

where the equality holds because DKL(f||d)=H(d,f)−H(d)=−Ey∼d[logf(y)]−H(d).
Observe that v is strongly concave, so it has a single global maximum and no other local maxima. This global maximum is

d(y)∝f(y)eb(y)/γ

So the optimal solution to this problem is to choose a distribution that is somewhat similar to f but overweights
y values with a high b(y) value (with the rationality parameter γ
determining how much to overweight).
The higher γ is, the more strongly concave the problem is and the more d will imitate f; the lower
γ is, the more this problem looks like the original b-maximization problem. This interpolation
is similar to quantilization, but is somewhat different mathematically.

Intuitively, optimizing v seems harder
than optimizing b: the distribution d must be able to provide all possible
good solutions to b, rather than just one. But I think standard reinforcement learning
algorithms can be adapted to optimize v. Really, you just need to optimize H(d)+Ey∼d[b(y)],
for some b, since you can wrap the γlogf(y) and b(y) terms together into a single function. So the agent must be
able to maximize the sum of some original objective b and the entropy of its own actions.

Consider Q-learning. An agent using

...
On Making Things
by Gram_Stone — 2016-03-05T03:26:02.996Z

(Content note: This is basically just a story about how I accidentally briefly made something that I find very unfun into something very fun, for the sake of illustrating how surprising it was and how cool it would be if everyone could do things like this more often and deliberately. You also might get a kick out of this story in the way that you might get a kick out of How It's Made, or many of Swimmer963's posts on swimming and nursing, or Elo's post on wearing magnetic rings. If none of that interests you, then you might consider backing out now.)

I'm learning math under the tutelage of a friend, and I go through a lot of paper. I write a lot of proofs so there can be plenty of false starts. I could fill a whole sheet of paper, decide that I only need one result to continue on my way, and switch to a blank sheet. Since this is how I go about it, I thought that a whiteboard would be a really good idea. The solution is greater surface area and practical erasure.

I checked Amazon; whiteboards are one of those products with polarized reviews. I secretly wondered if ten percent of all whiteboards manufactured don't just immediately permanently stain. Maybe I was being a little risk-averse, but I decided to hold off on buying one.

Then I remembered that I make signs for a living, and I realized that I could probably just make a whiteboard myself.

I had a good rapport with my supervisor. I have breaks and lunch time, and the boundaries are kind of fuzzy, so the time wouldn't be an issue. I didn't have to print anything, so I wouldn't be taking up time on the printers or using ink.

Maybe everyone knows what 'vinyl' is and I don't need to explain this, but the stuff that 'PVC pipes' (PVC stands for polyvinyl chloride) are made out of can be formed into thin elastic sheets. Manufacturers apply adhesive and paper backing to these sheets and sell them to people so they can pull off the paper and stick the vinyl to stuff. You can print on some of it too. It comes on long rolls, typically 54 in. or 60 in., sort of like tape or paper towels. If you ever see a vehicle that belongs to a business with all sorts of art all over it, then it's probably printed on vinyl.

It's kind of hard to print on a really short roll without everything going horribly awry, so we have tons of rolls with like 10 ft. by 54 in. sheets on them that just get thrown away.

If you scratch a ...
What does it mean for correct operation to rely on transfer learning?
by jessicata — 2016-03-05T03:24:27.000Z

Summary: Some approaches to AI value alignment rely on transfer learning. I attempt to explain this idea more clearly.

Paul and I have written previously about how some approaches to AI value alignment rely on transfer learning more than others. To make this distinction clearer, I'll consider some different situations where an AI system is relying or not relying on transfer learning for correct operation.

In the following situations, we will consider some component. This component might be something like a prediction engine or a SAT solver. It is something that we don't expect to be able to manually specify, due to its complexity; instead, we will need an AI system to do some learning to figure out what the right component is.

Situation 1: No reliance on transfer learning

We don't rely on transfer learning for the reliability of some component (e.g. a prediction engine) if the effect of the component on the rest of the system is through behavior that is easy to test.

For concreteness, consider a prediction engine used to predict a human thinking for a short time period. This prediction engine is the component. We might ask the predictor about its distribution for a human's answer to a particular question, and then sample from this distribution in order to automatically answer the question. The behavior that is easy to test is the distribution for the human's answer to the question; it's easy to test because, if two predictors make significantly different predictions of a human, we can eventually tell which is making better predictions by actually asking the human. The effect of the component on the rest of the system is the sample from its distribution. The internals can only affect this sample through behavior that is easy to test.

To more precisely specify what it means for the behavior to be "testable", we could say that, if we have 10 candidate components, only one of which is good (with no guarantees about the other 9), then we can produce a component that is about as good as the good component (with some amount of overhead).
In the prediction case, we can use active learning or selective sampling to make predictions almost as good as a good predictor in the mixture (with overhead due to having to ask the actual human some questions). Even though the predictor will not be tested on all possible questions, it could potentially be tested on any of

...
AIFoom Debate - conclusion?
by Bound_up — 2016-03-04T20:33:49.919Z

I've been going through the AIFoom debate, and both sides makes sense to me. I intend to continue, but I'm wondering if there're already insights in LW culture I can get if I just ask for them.

My understanding is as follows:

The difference between a chimp and a human is only 5 million years of evolution. That's not time enough for many changes.

Eliezer takes this as proof that the difference between the two in the brain architecture can't be much. Thus, you can have a chimp-intelligent AI that doesn't do much, and then with some very small changes, suddenly get a human-intelligent AI and FOOM!

Robin takes the 5-million year gap as proof that the significant difference between chimps and humans is only partly in the brain architecture. Evolution simply can't be responsible for most of the relevant difference; the difference must be elsewhere.

So he concludes that when our ancestors got smart enough for language, culture became a thing. Our species stumbled across various little insights into life, and these got passed on. An increasingly massive base of cultural content, made of very many small improvements is largely responsible for the difference between chimps and humans.

Culture assimilated new information into humans much faster than evolution could.

So he concludes that you can get a chimp-level AI, and to get up to human-level will take, not a very few insights, but a very great many, each one slowly improving the computer's intelligence. So no Foom, it'll be a gradual thing.

So I think I've figured out the question. Is there a commonly known answer, or are there insights towards the same?
Weekly LW Meetups
by FrankAdamek — 2016-03-04T16:59:23.733Z

This summary was posted to LW Main on March 4th. The following week's summary is here.

Irregularly scheduled Less Wrong meetups are taking place in:

Ann Arbor Meetup - Saturday, March 5, 7 PM, Amer's: 05 March 2016 07:00PM
European Community Weekend: 02 September 2016 03:35PM
San Antonio Meetup: 06 March 2016 02:00PM
San Francisco Meetup: Projects: 07 March 2016 06:15PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

London rationalish meetup - 2016-03-06: 06 March 2016 02:00PM
Melbourne Rationality Dojo, March: A deep dive into Effective Altruism: 05 March 2016 02:30PM
Raleigh, NC (RTLW) Discussion Meetup: 04 March 2027 07:30PM
Sydney Rationality Dojo - March: 06 March 2016 04:00PM
Sydney pub meetup: 09 March 2016 06:00PM
Vienna: 12 March 2016 03:00PM
Washington, D.C.: Designing Games: 06 March 2016 03:30PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Canberra, Columbus, Denver, London, Madison WI, Melbourne, Moscow, Mountain View, New Hampshire, New York, Philadelphia, Research Triangle NC, Seattle, Sydney, Tel Aviv, Toronto, Vienna, Washington DC, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers and a Slack channel for daily discussion and online meetups on Sunday night US time.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific m...