trentmkelly/LessWrong-43k huggingface dataset:
The Friendly AI Game
by bentarm — 2011-03-15T16:45:13.739Z

At the recent London meet-up someone (I'm afraid I can't remember who) suggested that one might be able to solve the Friendly AI problem by building an AI whose concerns are limited to some small geographical area, and which doesn't give two hoots about what happens outside that area. Cipergoth pointed out that this would probably result in the AI converting the rest of the universe into a factory to make its small area more awesome. In the process, he mentioned that you can make a "fun game" out of figuring out ways in which proposed utility functions for Friendly AIs can go horribly wrong. I propose that we play.

Here's the game: reply to this post with proposed utility functions, stated as formally or, at least, as accurately as you can manage; follow-up comments explain why a super-human intelligence built with that particular utility function would do things that turn out to be hideously undesirable.

There are three reasons I suggest playing this game. In descending order of importance, they are:

It sounds like fun

It might help to convince people that the Friendly AI problem is hard(*).

We might actually come up with something that's better than anything anyone's thought of before, or something where the proof of Friendliness is within grasp - the solutions to difficult mathematical problems often look obvious in hindsight, and it surely can't hurt to try

DISCLAIMER (probably unnecessary, given the audience) - I think it is unlikely that anyone will manage to come up with a formally stated utility function for which none of us can figure out a way in which it could go hideously wrong. However, if they do so, this does NOT constitute a proof of Friendliness and I 100% do not endorse any attempt to implement an AI with said utility function.

(*) I'm slightly worried that it might have the opposite effect, as people build more and more complicated conjunctions of desires to overcome the objections that we've already seen, and start to think the problem comes down to nothing more than writing a long list of special cases but, on balance, I think that's likely to have less of an effect than just seeing how naive suggestions for Friendliness can be hideously broken.
Admit your ignorance
by  — 2011-03-15T10:28:33.654Z

Reading Being a teacher made me think about my experiences tutoring university students. I'm a PhD student, so my teaching currently consists of helping first year undergraduates on problem sheets. I think I'm reasonably good- I try to appreciate what difficulties they are having and anticipate them, by explaining what they are doing means, and approaching the problem in different ways.

One constant frustration I get though is that, having explained a problem to a particular student, the student will give me a blank look, and then mutter "ok". I know what that look means, and will ask "so do you understand that?" "sort of...." "well look at it this way....".

Now some of this may come from me- I'm explaining too fast or in a way they don't understand, and my familiarity with the subject, but I suspect some comes from them. It can be difficult to admit one's ignorance, from my own experience. I, and I suspect others who go on to do university maths, was used to being the best or near the best in school, with "being smart" being part of my core identity, something that made me distinct from my more attractive or more fit peers. Getting to university and realising one is having difficulty with even basic questions can be a knock to ones identity. So I hid my ignorance, and did myself damage. I might lose the respect of a tutor, or even a lecturer by admitting my ignorance, but the alternative is to remain ignorant.

I suspect this is a problem that is common among us all. Its a lot easier to pretend we understand, and sometimes it may help- if we want to impress a potential employer we shouldn't admit ignorance (unless the alternative makes us look more ignorant)- but in general admitting ignorance helps us learn. There is (almost) always someone with more knowledge on a particular subject than you, and a failure to use that resource is a failure of rationality.
A math problem I'm trying to figure out now
by cousin_it — 2011-03-15T09:54:28.454Z

This will be unparseable to anyone except maybe ten people here. What the hell, I'm posting it anyway.

Consider game 3 from my earlier post: a possibly uncomputable sequence of bits comes in, you're trying to guess the next bit, the payoff for each correct guess is $1. Obviously any deterministic strategy can be humiliated by a sequence of bits that's chosen to make it always guess wrong. User paulfchristiano suggested that multiplicative weights could yield a randomized strategy that does at most a constant worse than any human in expectation. I noted that, unlike the Solomonoff mixture, multiplicative weights over lower-semicomputable randomized strategies is not itself lower-semicomputable (several pages of algebra omitted), so we don't have a strategy that's optimal within its own class. Then I found some slides by Hutter that include a handy table showing how unusual it is for a class do be dominated by a measure within that same class.

So here's the question: is there an analog of "dominant lower-semicomputable semimeasure" for some class of games other than the log-score game? Desiderata: I guess it must correspond to randomized strategies (because deterministic ones are not enough), it must play at most an additive constant worse than any human in expectation (or maybe a multiplicative constant that can be made arbitrarily close to 1 by parameter choice, as with multiplicative weights), and it must be good within its own class of computability (not just superior to all members of some lower class). As far as I can tell, the literature has no answer to this question, and I already spent about a week on it with no success. Anyone here have more understanding than me?
Limitless, a Nootropics-Centered Movie
by atucker — 2011-03-15T01:58:29.057Z

Limitless is a movie coming out this Friday which includes nootropics as a major plot device. I think that the way they are portrayed in the movie, and the subsequent media discussion (if any) about nootropics would be of interest here, even if the movie isn't.
From what I can tell, the movie is about a guy who uses a drug to improve his mental capabilities, uses those to radically alter his life, who is then targeted because its just that genre of movie.
Just a heads up, if anyone is interested.
Nootropics-Based Movie Coming Out
by atucker — 2011-03-15T01:48:14.019Z

Limitless is a movie coming (in the US) this Friday that people around here might be interested in, at least in how it portrays nootropics, or in how it prompts media discussion about them.
In a nutshell, it seems to be about a guy who uses nootropics to rapidly become more intelligent, charming, successful, etc. to the point that it completely changes his life. Then because it's a "thriller", people try to kill (or steal something from?) him.

Wikipedia page
On Pi day, we eat pie; On Tau day, we eat Taoists?
by  — 2011-03-15T01:43:12.323Z

I'd like to start by wishing everyone a Happy Pi day (even if for some of you it was yesterday).
Today, going about my usual Pi day celebration (which included pi(e) of the chocolate, cherry, apple, and movie variety), I stumbled across pi-protesters, who spoke of Tau-ism. For those who haven't heard, Tauist claim that Tau, represented by the Greek letter T, is the real circle constant. There's one proponent and his arguments here: http://tauday.com/
I read the article, saw the points that were made, and I've remained impartial (despite being a mathematics major). I can see Tau's usefulness; I can see why pi hasn't changed (and hardly would need to). So I decided to do something else: Present this claim to the LessWrong community, for those who are interested. What do you think?
Visualizing Bayesian Inference [link]
by Dreaded_Anomaly — 2011-03-14T20:10:02.273Z

Galton Visualizing Bayesian Inference (article @ CHANCE)

Excerpt:

What does Bayes Theorem look like? I do not mean what does the formula—

—look like; these days, every statistician knows that. I mean, how can we visualize the cognitive content of the theorem? What picture can we appeal to with the hope that any person curious about the theorem may look at it, and, after a bit of study say, “Why, that is clear—I can indeed see what is happening!”

Francis Galton could produce just such a picture; in fact, he built and operated a machine in 1877 that performs that calculation. But, despite having published the picture in Nature and the Proceedings of the Royal Institution of Great Britain, he never referred to it again—and no reader seems to have appreciated what it could accomplish until recently.

Schematics for the machine and its algorithm can be found at the link. This is a really cool design, and maybe it can aid Eliezer's and others' efforts to help people understand Bayes' Theorem.
Being a teacher
by Swimmer963 (Miranda Dixon-Luinenburg)  — 2011-03-14T20:03:27.602Z

A few weeks ago, while giving unofficial swimming lessons to an acquaintance about my age, I had an insight. It was that before you can teach something, you have to realize it’s hard.

I don’t think I noticed this before, because I thought it was obvious. Of course someone who doesn’t know how to swim isn’t going to learn perfect front crawl just by looking at yours. If I was told to watch someone else swimming a brand-new stroke that I’d never seen before, I could imitate it pretty easily, because to me it’s a trivial skill. But to someone who has nothing to refer to, it’s hard.

“You’re like the fifth person who’s tried to teach me how to swim,” my acquaintance said as I led her into the shallow end holding a foam noodle. “People just tell me to move my arms and legs, and they didn’t seem to understand why I couldn’t do it.”

There are, needless to say, a lot of different ways to move your arms and legs. Some of them resemble swimming. A subset of those actually work to keep someone’s head at the surface, and an even smaller subset of those are effective enough that they have names, like front crawl. To me, this is obvious, because I’ve watched hundreds of children in my classes flail and struggle in their front crawl, or lift their head to breathe, or turn their toes inwards in whip kick, and make the same mistakes persistently even when I corrected them, both verbally and by literally grabbing their arms/legs and moving them. I know it’s hard.

I went through this flailing/struggling phase too and have no memory of it whatsoever, having been three at the time. This is probably true of most good swimmers; the procedural memory is so embedded that it makes sense to say “move your arms and legs” because that's all you think about consciously; you forget how many other things you’re doing just to stay afloat. (Poor swimmers might have a different perspective, but they aren’t likely to use that perspective to try to teach other people how to swim.)

In order to bring a non-swimmer to the point of doing perfect front crawl, you have to teach them, one at a time, a long list of motor skills that have to be learned well enough to come naturally before you can move on. With adults, you can compress this process into a much shorter period than with restless, distractible, and lacking-fully-developed-motor-skills children, but you can’t omit it. You have

...
Science Journalism and How To Present Probabilities [Link]
by gimpf — 2011-03-14T18:30:51.721Z

I just stumbled across Language Log: Thou shalt not report odds ratios (2007-07-30), HT reddit/statistics:

(…) this finding was widely reported in the media:(…)

“Doctors are only 60% as likely to order cardiac catheterization for women and blacks as for men and whites.”

Now let't try a little test of reading comprehension. The study found that the referral rate for white men was 90.6%. What was the referral rate for blacks and women?

If you're like most literate and numerate people, you'll calculate 60% of 90.6%, and come up with .6*.906 = .5436. So, you'll reason, the referral rate for blacks and women was about 54.4 %.

But in fact, what the study found was a referral rate for blacks and women of 84.7%.

This was a failure mode of pop-sci journalism which I was not aware of (if I would happen to know enough to understand real papers, I’d definitely value pop-sci at minus-whatever in the meantime…)

On a related note this article got me remembering Understanding Uncertainty: 2845 ways to spin the Risk, which argues that certain presentations bias the understanding of probabilities:

Similarly people confronted with the statement “Cancer kills 2,414 people out of 10,000” rated cancer as more risky than those told “Cancer kills 24.14 people out of 100”. The potential influence of the size of the numerator and denominator is known as the 'ratio bias'.

I’d be quite interested if anybody could point me to further resources on good presentation of statistical facts (beside the normalization on one type of presentation), or on further pop-sci journalism failure modes.
How I Lost 100 Pounds Using TDT
by Zvi — 2011-03-14T15:50:00.695Z

Background Information: Ingredients of Timeless Decision Theory

Alternate Approaches Include: Self-empathy as a source of “willpower”, Applied Picoeconomics, Akrasia, hyperbolic discounting, and picoeconomics, Akrasia Tactics Review

Standard Disclaimer: Beware of Other-Optimizing

Timeless Decision Theory (or TDT) allowed me to succeed in gaining control over when and how much I ate in a way that previous attempts at precommitment had repeatedly failed to do. I did so well before I was formally exposed to the concept of TDT, but once I clicked on TDT I understood that I had effectively been using it. That click came from reading Eliezer’s shortest summary of TDT, which was:

The one-sentence version is: Choose as though controlling the logical output of the abstract computation you implement, including the output of all other instantiations and simulations of that computation

You can find more here but my recommendation at least at first is to stick with the one sentence version. It is as simple as it can be, but no simpler.

Utilizing TDT gave me several key abilities that I previously lacked. The most important was realizing that what I chose now would be the same choice I would make at other times under the same circumstances. This allowed me to compare having the benefits now to paying the costs now, as opposed to paying costs now for future benefits later. This ability allowed me to overcome hyperbolic discounting. The other key ability was that it freed me from the need to explicitly stop in advance to make precommitements each time I wanted to alter my instinctive behavior. Instead, it became automatic to make decisions in terms of which rules would be best to follow.

With that as background, this is how I made it happen:

I was walking home from class along my usual route I had made a habit while doing this of stopping into Famiglia Pizza and ordering garlic knots. I like garlic knots quite a bit, but I also hated being fat and the way being fat made me feel. Things weren’t quite as bad on that front as they’d been a few years before but they were still extraordinarily bad. I thought abou

...
Handedness Bias
by beriukay — 2011-03-14T14:49:11.689Z

I just read a blog post on NeuroLogica Blog that could have been a LW post, so I figured that I would bring it on over. It basically details how knowing about our biases can help us correct for them, a la the lens that sees its flaws, and then brings to light a new study (unfortunately behind a paywall... I wanted to see the methodology) that shows that the simple act of wearing a glove on your dominant hand can influence how you perceive the world.

When I learned that Dexter and Sinister were Latin words for Right and Left, respectively, I was told that it came from shield formations, and how the person on your left was a leech for using your shield protection, and the one on your right was your protector. Now that explanation sounds a bit hollow.
Rationalist Lord of the Rings fanfiction, newly translated from Russian
by Costanza — 2011-03-14T02:38:38.736Z

This may be old news to some people, especially the Russian speakers, but I didn't see an article about it here.

In 1999, Kirill Yeskov, a Russian paleontologist, wrote The Last Ringbearer, a 270-page take on Lord of the Rings from the point of view of a medic in Mordor's dying armies who is also a "skeptic and a rationalist." In fact, Mordor represents the forces of reason in this retelling of the story. As a Nazgúl (himself a former mathematician) explains, Mordor is "the little oasis of Reason in which your light-minded civilization had so comfortably nestled." Barad-dur is "that amazing city of alchemists and poets, mechanics and astronomers, philosophers and physicians, the heart of the only civilization in Middle-earth to bet on rational knowledge and bravely pitch its barely adolescent technology against ancient magic."

The story has been newly translated and is available in free PDF form -- in English and the original Russian. There's a recent review from Salon as well.
Map and Territory and "Paths"
by Goobahman — 2011-03-13T22:38:16.608Z

Hi everyone,

Recently have started a discussion group amongst some of my friends and aquaintences in an attempt to study and improve our rationality.

What I tend to do, is using the sequences as 'source' material, write up a lesson plan that attempts to be a bit more accesible to the layman and go through it with them as slowly and meticulously as possible.

Only had one meeting so far, but it's been exciting seeing it come together and people get engaged with the idea of improving their rationality.

Mm question is regarding the "Map and Territory" analogy, which is what we're planning to look at indepth next meeting.

Could you further the analogy and say that the paths that would write on your map, to get you to and from different points be considered the application of Instrumental Rationality? (As charting the map is the application of epistemic) You could point out that Instrumental Rationality also requires you to have a good map.

Any thoughts/comments would be appreciated.
Map and Territory and "Paths"
by Goobahman — 2011-03-13T22:38:15.437Z

Hi everyone,

Recently have started a discussion group amongst some of my friends and aquaintences in an attempt to study and improve our rationality.

What I tend to do, is using the sequences as 'source' material, write up a lesson plan that attempts to be a bit more accesible to the layman and go through it with them as slowly and meticulously as needed.

Only had one meeting so far, but it's been exciting seeing it come together and people get engaged with the idea of improving their rationality.

My question is regarding the "Map and Territory" analogy, which is what we're planning to look at indepth next meeting.

Could you further the analogy and say that the paths that you write on your map, to get you to and from different points be considered the application of Instrumental Rationality? (As charting the map is the application of epistemic) You could point out that Instrumental Rationality also requires you to have a good map.

Any thoughts/comments would be appreciated.
What exactly IS the overpopulation argument (in regards to immortality)?
by Raemon — 2011-03-13T18:22:59.508Z

Periodically in discussions of cryonics and related issues, people bring up "the overpopulation argument, and the counterarguments that respond to it" without actually describing those arguments in detail. Overpopulation was a big concern of mine prior to exposure to anti-deathism. And so far, it still is. I have no philosophical problem with eliminating death, but it seems to me that getting rid of death BEFORE we eliminate all the other major problems facing humanity is going to make those problems much worse. Death is an enemy that should be vanquished, but it's not an enemy I'm prepared to destroy until I'm satisfied that we'll be able to handle the consequences.

If death is solved via uploads running at high speed, I'm not too concerned. (Still a little concerned, since computers still take up space, but the issue is close enough to negligible that I'm fine ignoring it).

If we're dealing with physical humans taking up physical space requiring physical resources, then I'm worried. Either we're growing exponentially, or we've eliminated childbirth, or we have strict rules in place about people who have children being willing to die. (The latter might work but modifying central aspects of the human life cycle that we are hard-wired to value seems.... challenging, to say the least)

The sense of I've gotten around here is that "exponential growth is okay, because Space is Big". Space certainly is big, and I imagine we could expand for a long time without running into conflict. But if there's even one other alien race who solves their problems the same way (expanding whenever they run out of space or resources), then eventually there's going to be conflict. And the longer we go BEFORE that conflict, the more human suffering it might entail. I'd prefer to have achieved equilibrium as a species beforehand.

I want to expand into the universe, but I think we should do so out of *curiosity* rather than *necessity.*

I assume there's been a lot of discussions about this and I don't want to rehash them. But if someone could summarize the issues at hand and explain what the community consensus is, I'd appreciate it. ("Community Consensus" might mean "there are a few dominant schools of thought here").

Edit: FAWS pointed out that as long as, on average, each person reproduces slightly less than once, growth will not continue exponentially. That's pretty much the ...
Move meetups to the sidebar?
by Johnicholas — 2011-03-13T16:35:28.503Z

The number of meetup announcements on the main blog has been increasing. Though it's reasonable to try to get meetups high visibility to increase the chance that people who are nearby see the announcement, the posts themselves are content-free.

How difficult would it be to, instead of promoting meetup announcements, tag them "meetup" and put a "meetups" section in the sidebar, similar to "recent comments" or "recent posts"?
Organ donation versus cryonics
by Snowyowl — 2011-03-13T16:08:52.588Z

Simultaneously signing up for organ donation and cryonics versus only cryonics. Does having less organs decrease the likelihood of cryonics (including revival) working? Is it a good idea to have only your head frozen anyway, to save on electricity and storage? Do the benefits of organ donation outweigh any costs it could possibly incur, since organ donation is known to work?

Discuss.

I'm an organ donor because signing up was quick and easy. I'm not signing up for cryonics, because I anticipate that my family and close friends will have a harder time overcoming their grief if my body is not actually present at the funeral.