trentmkelly/LessWrong-43k huggingface dataset:
Leveling Up in Rationality: A Personal Journey
by lukeprog — 2012-01-17T11:02:55.284Z

See also: Reflections on rationality a year out

My favorite part of Lord of the Rings was skipped in both film adaptations. It occurs when our four hobbit heroes (Sam, Frodo, Merry and Pippin) return to the Shire and learn it has been taken over by a gang of ruffians. Merry assumes Gandalf will help them free their home, but Gandalf declines:

I am not coming to the Shire. You must settle its affairs yourselves; that is what you have been trained for... My dear friends, you will need no help. You are grown up now. Grown indeed very high...

As it turns out, the hobbits have acquired many powers along their journey — powers they use to lead a resistance and free the Shire.

That is how I felt when I flew home for the holidays this December. Minnesota wasn't ruled by ruffians, but the familiar faces and places reminded me of the person I had been before I moved away, just a few years ago.

And I'm just so much more powerful than I used to be.

And in my case, at least, many of my newfound powers seem to come from having seriously leveled up in rationality.

Power 0: Curiosity

I was always "curious," by which I mean I felt like I wanted to know things. I read lots of books and asked lots of questions. But I didn't really want to know the truth, because I didn't care enough about the truth to study, say, probability theory and the cognitive science of how we deceive ourselves. I just studied different Christian theologies — and, when I was really daring, different supernatural religions — and told myself that was what honest truth-seeking looked like.

It took 20 years for reality to pierce my comfortable, carefully cultivated bubble of Christian indoctrination. But when it finally popped, I realized I had (mostly) wasted my life thus far, and I was angry. Now I studied things not just for the pleasure of discovery and the gratifying feeling of caring about truth, but because I really wanted an accurate model of the world so I wouldn't do stupid things like waste two decades of life.

And it was this curiosity, more than anything else, that led to everything else. So long as I burned for reality, I was bound to level up.

Power 1: Belief Propagation

One factor that helped religion cling to me for so long was my ability to compartmentalize, to shield certain parts of my beliefs from attack, to app...
Q&A with Abram Demski on risks from AI
by XiXiDu — 2012-01-17T09:43:10.805Z

[Click here to see a list of all interviews]

Abram Demski is a computer science Ph.D student at the University of Southern California who has previously studied cognitive science at Central Michigan University. He is an artificial intelligence enthusiast looking for the logic of thought. He is interested in AGI in general and universal theories of intelligence in particular, but also probabilistic reasoning, logic, and the combination of the two ("relational methods"). Also, utility-theoretic reasoning.

I interviewed Abram Demski due to feedback from lesswrong. cousin_it, top contributer and research associate of the Singularity institute, wrote the following:

I'm afraid of Abram Demski who wrote brilliant comments on LW and still got paid to help design a self-improving AGI (Genifer).

Enough already, here goes....

The Interview:

Q1: Assuming no global catastrophe halts progress, by what year would you assign a 10%/50%/90% chance of the development of roughly human-level machine intelligence?

Explanatory remark to Q1:
P(human-level AI by (year) | no wars ∧ no disasters ∧ beneficially political and economic development) = 10%/50%/90%

Abram Demski:

10%: 5 years (2017).
50%: 15 years (2027).
90%: 50 years (2062).

Of course, just numbers is not very informative. The year numbers I gave are unstable under reflection, at a factor of about 2 (meaning I have doubled and halved these estimates in the past minutes while considering it). More relevant is the variance; I think the year of development is fundamentally hard to predict, so that it's rational to give a significant probability mass to within 10 years, but also to it taking another 50 years or more. However, the largest bulk of my probability mass would be roughly between 2020 and 2030, since (1) the computing hardware to simulate the human brain would become widely available and (2) I believe less than that will be sufficient, but the software may lag behind the hardware potential by 5 to 10 years. (I would estimate more lag, except that it looks like we are making good progress right now.)

Q2: What probability do you assign to the possibility of human extinction as a result of badly done AI?
Explanatory remark to Q2:
P(human extinction | badly done AI) = ?
(Where 'badly done' = AGI capable of self-modification that is not provably non-dangerous.)

Abram Demski: This is somewhat difficult. We could say ...
[SEQ RERUN] Neural Categories
by MinibearRex — 2012-01-17T05:19:13.777Z

Today's post, Neural Categories was originally published on 10 February 2008. A summary (taken from the LW wiki):

You treat intuitively perceived hierarchical categories like the only correct way to parse the world, without realizing that other forms of statistical inference are possible even though your brain doesn't use them. It's much easier for a human to notice whether an object is a "blegg" or "rube"; than for a human to notice that red objects never glow in the dark, but red furred objects have all the other characteristics of bleggs. Other statistical algorithms work differently.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Disguised Queries, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
ICONN 2012 nanotechnology conference in Perth
by Solvent — 2012-01-17T02:34:47.334Z

I won a prize and get to travel to Perth, Australia to attend the 2012 ICONN nanotechnology conference.

Is anyone else from LessWrong going to be there?

I don't actually know much about nanotechnology. Does anyone have any particular recommendations for some a good introduction?

Thanks.
Random thought: What is the optimal PD strategy under imperfect information? 
by RolfAndreassen — 2012-01-17T01:06:28.375Z

We know that Tit-for-Tat and variants do very well in iterated-Prisoner's-Dilemma tournaments. However, such tournaments are a bit unrealistic in that they give the agents instant and complete information about each other's actions. What if this signal is obscured? Suppose, for example, that if I press "Cooperate", there is a small chance that my action is reported to you as "Defect", presumably causing you to retaliate; and conversely, if I press "Defect" there is a chance that you see "Cooperate", thus letting me get away with cheating. Does this affect the optimal strategy? Does the probability of getting wrong information matter? What if it is asymmetric, ie P(observe C | actual D) != P(Observe D | actual C)?
What would you do with a financial safety net?
by faul_sname — 2012-01-16T23:38:18.978Z

In the open thread, moridinamael hypothesized that LWers would be willing to take more risks in order to become rich if they had a financial safety net. This seems like an idea worth exploring further.

What would you do if you had a financial safety net (maybe a year's worth of living expenses) to fall back on if your venture failed?
[LINK] Being No One (~50 min talk on the self-model in your brain)
by machrider — 2012-01-16T18:18:33.149Z

Summary: This is a ~50 minute talk (plus some introductory ado) by Thomas Metzinger on the problem of the experiencing, subjective self (why it exists, what it even means, how it arises). Not to be too cliché, but he attacks the problem by dissolving the question, and the solution he arrives at sounds a lot like how an algorithm feels from inside.

Using several examples from neuroscience (particularly the many illuminating failure modes of the brain), he explains how the brain models the self and its place in the center of experiential space. He discusses the limitations of our access to our own cognitive systems, and how those limitations force us to be naive realists.

I hesitate to summarize further, because there is a lot of value in hearing the entire argument. (I will say that he gets a little cute at the end, but that doesn't detract from the excellent content.)

Link: Being No One on Youtube.

(Normally I think LWers dislike the talk format because it's inherently time-consuming, but I'd say this one is information dense and well worth your time.)
link to a Feynman video (and a 2nd (distantly related) video)
by Craig_Heldreth — 2012-01-16T18:00:50.618Z

Video presentation from 1963 at Cornell. Character of Physical Law #2 The Relation of Mathematics to Physics.

This may be the best one of all the Feynman videos. He explains how physicists do mathematics and distinguishes how physicists do math from how mathematicians do math. He describes the two flavors as Greek and Babylonian.

The money quote if you don't have time to watch it: "Physicists do Babylonian mathematics and pay little attention to precise reasoning from fixed axioms."

Comment number 1; the quality of the video ain't all that great--in particular there is an annoying very small synch error which is just large enough to be definitely noticeable. Nevertheless it is absorbing because of the power of his presentation. The occasional audience shots show an audience transfixed. They also show the majority of the people wearing eyeglasses.

Comment number 2; in his earlier days his working class New York accent seemed far more obvious. I am reminded of Art Carney from The Honeymooners. (That is a four minute clip.) Feynman was born in Queens in 1918. Carney was born in Mount Vernon in 1918.
Completeness, incompleteness, and what it all means: first versus second order logic
by Stuart_Armstrong — 2012-01-16T17:38:04.771Z

First order arithmetic is incomplete. Except that it's also complete. Second order arithmetic is more expressive - except when it's not - and is also incomplete and also complete, except when it means something different. Oh, and full second order-logic might not really be a logic at all. But then, first order logic has no idea what the reals and natural numbers are, especially when it tries to talk about them.

That was about the state of my confusion, and I set out to try and clear it up. Here I'll try and share an understanding of what is really going on with first and second order logic and why they differ so radically. It will be deliberately informal, so I won't be distinguishing between functions, predicates and subsets, and will be using little notation. It'll be exactly what I wish someone had told me before I started looking into the whole field.

Meaningful Models

An old man starts talking to you about addition, subtraction and multiplication, and how they interact. You assume he was talking about the integers; turns out he means the rational numbers. The integers and the rationals are both models of addition, subtraction and multiplication, in that they obey all the properties that the old man set out. But notice though he had the rationals in mind, he didn't mention them at all, he just listed the properties, and the rational numbers turned out, very non-coincidentally, to obey them.

These models are generally taken to give meaning to the abstract symbols in the axioms - to give semantics to the syntax. In this view, "for all x,y xy=yx" is a series of elegant squiggles, but once we have the model of the integers (or the rationals) in mind, we realise that this means that multiplication is commutative.

Similarly, models can define the "truth" of sentences. Consider the following sentences:

(1) 2 has a multiplicative inverse.
(2) There exists a number that squares to -1.
(3) 2 is not equal to zero.
(4) If a+b=0, then a2=b2.
(5) No number is equal to zero.

Are these true? You and the old man would disagree on (1), with him saying yes and you saying no - your models have enabled you to attach truth-values to the statement. You would both claim (2) is false, but there are other models - such as the complex numbers - where it is true. You would both claim (3) is true, but there are other models - such as the field with two elements - w...
The lessons of a world without Hitler
by Stuart_Armstrong — 2012-01-16T16:16:18.136Z

What would the world look like without Hitler? Fiction is generally unequivocal about this: the removal of Hitler makes no difference, the world will still lurch towards a world war through some other path. WWII and the Holocaust are such major, defining events of the twentieth century, that we twist counterfactual events to ensure they still happen.

Against this, some have made the argument that Hitler was essentially sole responsible for WWII and especially for the Holocaust - no Hitler, no war, no extermination camps. The no Holocaust argument is quite solid: the extermination system was expensive, militarily counter-productive, and could only have happened given a leader lacking checks and balance and with an idée fixe that overrode everything else (general European antisemitism allowed the Holocaust, but didn't cause it). The no WWII argument points out that Hitler was both irrational and lucky: he often took great risks, on flimsy evidence, and got away with them. Certainly his decisions in the later, post-Barbarossa period of his reign belie political, military or organisational genius. And it was the height of stupidity to have gone to war, for a half of Poland, with simultaneously the world's greatest empire and what appeared to be the overwhelmingly strong French army. Yes Gamelin, the French commander in chief, did behave like a concussed duckling, and the German army outfought the French - but no-one could have predicted this, and no-one sensible would have counted on it, and hence they wouldn't have risked the war. Hitler wan't sensible, and lucked out.

Lay aside whether that argument is true, and let's explore its consequences. The counterfactual history is fascinating enough on its own - no rise of the USA and USSR as military superpowers, no Manhattan project, most likely no war between Japan and any western powers in the Pacific, the continuing occupation of China, and probably a much slower and Japan-influenced decolonisation process. Speculation and more sensible models do point towards another war: most of Europe allied against the USSR, a war that the USSR would most likely have lost.

That is all entertaining; but much more important is the fact that if WWII was an unlikely event, then the lessons we've learnt from it have been over-learnt. WWII proved that a devel...
[SEQ RERUN] Disguised Queries
by MinibearRex — 2012-01-16T04:47:17.549Z

Today's post, Disguised Queries was originally published on 09 February 2008. A summary (taken from the LW wiki):

You ask whether something "is" or "is not" a category member but can't name the question you really want answered. What is a "man"? Is Barney the Baby Boy a "man"? The "correct" answer may depend considerably on whether the query you really want answered is "Would hemlock be a good thing to feed Barney?" or "Will Barney make a good husband?"

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was The Cluster Structure of Thingspace, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Idea: Add books to an SRS without splitting them into facts
by D_Malik — 2012-01-15T08:05:37.476Z

Spaced repetition systems use math to determine the optimal way to study things. This post is about an idea I've been trying for a few months for improving SRS for some subjects.

SRSs usually use a pretty rigid system of asking questions and demanding answers. I think that for many subjects it's not very important to know specific answers, either because such answers can be looked up easily or because the gist of a subject is more important. So here's an idea: add an ebook to a spaced repetition system and read/skim each chapter or page when it's due for review. This can be used for ebooks, physical books, or articles from the internet or elsewhere.

For books or ebooks, there are two ways to do this: either add each page as an individual card (with an image of the page right on the card) or create a card for each section or chapter. The latter technique can be used for non-electronic books. If each page is its own card, you can review things more quickly because you don't have to open an ebook or book each time you review, but you'll need to convert the ebook to images first. You can also add annotations, either by editing page images, typing notes onto pages' cards, or adding annotations with your ebook-reading software.

One way to convert ebooks to images is to use imagemagick. On Linux,

convert -density 180x180 BOOK.pdf folder/imgname.png

Change the density if images are too small or too large. You'll have to convert ebooks to pdf format first. This command creates all the pages as imgname-1.png, imgname-2.png, etc. Move the images into a .media folder where your other anki decks are. Use a script to make a card for each page. For example, using python:

f = open("bookcards", "w")

for i in range(0,NUMPAGES): f.write(str(i+1) + ";<img src=\"imgname-" + str(i) + ".png\" />\n")

You probably want to review cards in the order they were created (so that you'll review due cards by page number). This option doesn't exist in anki, so you'll need to make each book a separate deck and use the patch command to apply this diff to /usr/share/anki/anki/deck.py, or wherever that file is on your computer:

64a65
> REV_CARDS_CREATED_FIRST = 4
389c390,391
< "priority desc, factId, ordinal")[self.revCardOrder]
---
> &nb

...
The Savage theorem and the Ellsberg paradox
by fool — 2012-01-14T19:06:53.147Z

Followup to: A summary of Savage's foundation for probability and utility.

In 1961, Daniel Ellsberg, most famous for leaking the Pentagon Papers, published the decision-theoretic paradox which is now named after him 1. It is a cousin to the Allais paradox. They both involve violations of an independence or separability principle. But they go off in different directions: one is a violation of expected utility, while the other is a violation of subjective probability. The Allais paradox has been discussed on LW before, but when I do a search it seems that the first discussion of the Ellsberg paradox on LW was my comments on the previous post 2. It seems to me that from a Bayesian point of view, the Ellsberg paradox is the greater evil.

But I should first explain what I mean by a violation of expected utility versus subjective probability, and for that matter, what I mean by Bayesian. I will explain a special case of Savage's representation theorem, which focuses on the subjective probability side only. Then I will describe Ellsberg's paradox. In the next episode, I will give an example of how not to be Bayesian. If I don't get voted off the island at the end of this episode.

Rationality and Bayesianism

Bayesianism is often taken to involve the maximisation of expected utility with respect to a subjective probability distribution. I would argue this label only sticks to the subjective probability side. But mainly, I wish to make a clear division between the two sides, so I can focus on one.

Subjective probability and expected utility are certainly related, but they're still independent. You could be perfectly willing and able to assign belief numbers to all possible events as if they were probabilities. That is, your belief assignment obeys all the laws of probability, including Bayes' rule, which is, after all, what the -ism is named for. You could do all that, but still maximise something other than expected utility. In particular, you could combine subjective probabilities with prospect theory, which has also been discussed on LW before. In that case you may display Allais-paradoxical behaviour but, as we will see, not Ellsberg-paradoxical behaviour. The rationalists might excommunicate you, but it seems to me you should keep your Bayesianist card.

On the other hand your behaviour could be incompatible with any subjective probability distribution. But you could ...
Meetup : Austin LW Meetup
by Vaniver — 2012-01-13T17:32:11.064Z

Discussion article for the meetup : Austin LW Meetup

WHEN:
14 January 2012 01:30:00PM (-0800)

WHERE:
2222B Guadalupe St Austin, Texas 78705

Despite my absence, the Austin meetup continues! We have two people confirmed to show up, so I hope you join them. We sit on the second floor to the left, near (or often on) the stage.

Discussion article for the meetup : Austin LW Meetup
Weekly LW Meetups: Melbourne, Austin, Atlanta, Seattle, Philadelphia, Fort Collins
by FrankAdamek — 2012-01-13T16:19:47.331Z

There are upcoming irregularly scheduled Less Wrong meetups in:

First Philadelphia Meetup of 2012: 08 January 2012 02:00PM
Fort Collins, Colorado Meetup: 11 January 2012 07:00PM
[TENTATIVE] Portland Meetup?: 14 January 2012 12:00PM
San Diego experimental meetup: 15 January 2012 01:00PM
First Sydney 2012 meetup.: 19 January 2012 06:00PM
Salt Lake City, Late January 2012
Columbus or Cincinnati Meetup: 22 January 2012 05:00PM
First Brussels meetup: 11 February 2012 11:00AM

The following meetups take place in cities with regularly scheduled meetups, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Melbourne practical rationality meetup: 06 January 2012 07:00AM
Austin, TX: 07 January 2012 01:30PM
Atlanta Meetup: 07 January 2012 06:30PM
Seattle Board Games: 08 January 2012 01:01PM

Cities with regularly scheduled meetups: Austin, Berkeley, Cambridge, MA, London, Madison, WI, Marin, CA (uses the Bay Area List), Melbourne, Mountain View, New York, Ottawa, Oxford, San Francisco, Seattle, Toronto, Washington, DC, and West Los Angeles.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview will continue to be posted on the front page every Friday. These will be an attempt to collect information on all the meetups happening in the next weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll now also have the benefit of having your meetup mentioned in a weekly overview. These overview posts will be moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing list in order to be notified when an irregular meetup is happening: Atlanta, Chicago, Helsinki, London, Pittsburgh, Southern California (Los Angeles/Orange County area), St. Louis, Vancouver.