trentmkelly/LessWrong-43k huggingface dataset:
A model of UDT with a halting oracle
by cousin_it — 2011-12-18T14:18:56.858Z

This post requires some knowledge of mathematical logic and computability theory. The basic idea is due to Vladimir Nesov and me.

Let the universe be a computer program U that can make calls to a halting oracle. Let the agent be a subprogram A within U that can also make calls to the oracle. The source code of both A and U is available to A.

Here's an example U that runs Newcomb's problem and returns the resulting utility value:

def U():
# Fill boxes, according to predicted action.
box1 = 1000
box2 = 1000000 if (A() == 1) else 0
# Compute reward, based on actual action.
return box2 if (A() == 1) else (box1 + box2)

A complete definition of U should also include the definition of A, so let's define it. We will use the halting oracle only as a provability oracle for some formal system S, e.g. Peano arithmetic. Here's the algorithm of A:

Play chicken with the universe: if S proves that A()≠a for some action a, then return a.

For every possible action a, find some utility value u such that S proves that A()=a ⇒ U()=u. If such a proof cannot be found for some a, break down and cry because the universe is unfair.

Return the action that corresponds to the highest utility found on step 2.

Now we want to prove that the agent one-boxes, i.e. A()=1 and U()=1000000. That will follow from two lemmas.

Lemma 1: S proves that A()=1 ⇒ U()=1000000 and A()=2 ⇒ U()=1000. Proof: you can derive that from just the source code of U, without looking at A at all.

Lemma 2: S doesn't prove any other utility values for A()=1 or A()=2. Proof: assume, for example, that S proves that A()=1 ⇒ U()=42. But S also proves that A()=1 ⇒ U()=1000000, therefore S proves that A()≠1. According to the first step of the algorithm, A will play chicken with the universe and return 1, making S inconsistent unsound (thx Misha). So if S is sound, that can't happen.

We see that the agent defined above will do the right thing in Newcomb's problem. And the proof transfers easily to many other toy problems, like the symmetric Prisoner's Dilemma.

But why? What's the point of this result?

There's a big problem about formalizing UDT. If the agent chooses a certain action in a deterministic universe, then it's a tru...
Request: Induction, Proof, Experimental mathematics
by snarles — 2011-12-18T13:29:04.575Z

Suppose we made an algorithm capable of forming empirical conjectures for mathematics. How might such an algorithm discover the principle of mathematical proof?

I would like to see an article on the relevant philosophy and mathematical logic background for this problem. Since I currently lack the inclination to research and write up such an article, I instead made this post.
List of machine learning researchers
by robertzk — 2011-12-18T09:54:03.221Z

This is not a long post, but I wanted to draw attention to Wikipedia's list of notable machine learning researchers, because it has done so much for me. For anyone interested in problems related to AI (to get a feel for what AI is like), I recommend reading at least the Wikipedia page of each of those people, followed by playing around on their professional home page (just put their name into Google). You will soon be overwhelmed, but that is to be expected unless you have a PhD in statistics or machine learning. (And believe me, it is much less overwhelming than delving into my world of pure math, for example!)

Nevertheless, you will run into many insightful things, such as Steve Omohundro's (who is an advisor to SIAI) Self-Aware Systems, and Ghahramani's list of topics for a qualifying exam on machine learning. I also recommend skimming the bibliographies of the research articles for those researchers who post PDFs on their webpages as to get a familiarity for what journals and foundational papers/books are important. (And coming back here to post that information!)

Know these names. When Eliezer uses the words "the programmers of Friendly AI," there is a significantly higher probability than the average person that these names have something to do with that (future graduate students, or otherwise intellectual descendants). If you are yourself a researcher in machine learning, ethics, or statistics, it would be helpful if you could post any names not listed in the above Wikipedia category. Their homepages have utility for us.

Incidentally, I recommend working through The Elements of Statistical Learning by Hastie, Tibshirani, and Friedman, where a lot of these ideas are gathered into one (certainly not comprehensive) treatise.

As a related question, is there an equivalent to the unbelievably helpful MathSciNet for other domains? (Where there are reviews of papers available as well as hyperlinks to the database entry for citations as to determine what important papers and books in the field are, etc.)

One final remark: Under Crocker's Rules, please give me suggestions how to rephrase the above sentences to have less parentheses. I am much too fond of them.
[SEQ RERUN] Absolute Authority
by MinibearRex — 2011-12-18T03:05:33.679Z

Today's post, Absolute Authority was originally published on 08 January 2008. A summary (taken from the LW wiki):

Those without the understanding of the Quantitative way will often map the process of arriving at beliefs onto the social domains of Authority. They think that if Science is not infinitely certain, or if it has ever admitted a mistake, then it is no longer a trustworthy source, and can be ignored. This cultural gap is rather difficult to cross.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was The Fallacy of Gray, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Philosophy and Machine Learning Panel on Ethics
by jsteinhardt — 2011-12-17T23:32:20.026Z

Just got back from NIPS 2011 (Neural Information Processing Systems, one of about 4 major machine learning conferences). One of the workshops there was on Philosophy and Machine Learning. An interesting statistic was that, when asked what topics we should discuss at the same workshop next year, 3 out of 10 of the discussion panelists answered "ethics". I didn't keep as close of track of the other responses, but I think they were roughly split between causality and universal learning, with a few also wanting to talk more about induction (there was some overlap there with universal learning).

The full program can be found here, although it is also linked to from the main NIPS website.

P.S. Did anyone else from LW end up going to NIPS?
Selfish reasons for FAI
by snarles — 2011-12-17T22:58:46.209Z

Let's take for granted that pursuing FAI is the best strategy for researchers interested in the future of all humanity. However, let's also assume that controlling unfriendly AI is not completely impossible. I would like to see arguments on why FAI may or may not be the best strategy for AGI researchers who are solely interested in selfish values: i.e., personal status, curiosity, well-being of their loved ones, etc.

I believe such discussion is important because i) all researchers are to some extent selfish and ii) it may be unwise to ignore researchers who fail to commit to perfect altruism. I, myself, do not know how selfish I would be if I were to become an AGI researcher in the future.

EDIT: Moved some of the original post content to a comment, since I suspect it was distracting from my main point.
[Link] Belief in religion considered harmful?
by  — 2011-12-17T22:38:45.029Z

I've recently run across this 2007 post on the blog Unqualified Reservations (archive best read here). It is written by Mencious Moldbug, who is probably familiar to some Overcoming Bias and Lesswrong readers. He is a erudite, controversial and most of all contrarian social critic and writer. In 2010 he debated Robin Hanson on the subject of Futarchy.

Why do atheists believe in religion?

Not everyone these days believes in God. But pretty much everyone believes in religion.
By "believing in religion," I mean recognizing a significant categorical distinction between "religious" phenomena, and those that are "nonreligious" or "secular."
For example, the concepts of "freedom of religion" and "separation of church and state" are dependent on the concept of "religion." If "religion" is a noninformative, unimportant, or confusing category, these concepts must also be noninformative, unimportant, or confusing.
Since most atheists, agnostics, etc, consider the First Amendment pretty important, we can assume they "believe in religion."
My question is: why? Is this a useful belief? Does it help us understand the world? Or does it confuse or misinform us? Once again, our team of crack philosophers is on the case.
Let's rule out the possibility that "religion" is noninformative. We can define "religion" as the attribution of existence to anthropomorphic paranormal entities. This definition has its fuzzy corner cases, notably some kinds of Buddhism, but it's short and it'll do for the moment.
We are left with the question: is "religion" an important or clarifying category? Or is it unimportant and confusing?
If you believe in God, obviously you have to believe in religion. Religion is an important category because your religion is true, and all other religions are false. (As Sam Harris puts it, "everyone's an atheist with respect to Zeus.")
For atheists of the all-around variety - including me - the question remains. Why do we believe in "religion?"
One obvious answer is that we have to share the planet with a lot of religious people. If you are an atheist, there is no getting around it: religion, as per Dawkins, is a delusion. Deluded people do crazy things and are often dangerous. We need to have a category for these people, just as we have a category for "large, man-eating carnivores." Certainly, religious violence has killed a lot more people lately than lions, tigers, or

...
[LINK] High school students coerced into non-optimal philanthropy via psychological warfare
by Dallas — 2011-12-17T22:24:40.075Z

From the article:

For someone who isn’t a fan of teen idol Justin Bieber, being forced to listen to one of his songs over and over again could be considered cruel and unusual punishment.

At Evanston Township High School this week, they called it a fund-raiser.

To motivate their fellow students to donate money for a struggling cafe/arts center popular with ETHS kids, seniors Charlotte Runzel and Jesse Chatz persuaded administrators to let them blast Bieber’s hit “Baby” over the school’s loudspeaker system at the end of each class period — and not stop playing the song until Runzel and Chatz had met their goal.

...

Perhaps not surprisingly, Runzel and Chatz, who were given one week to meet their goal of $1,000 for Boocoo cafe on Church Street, were able to raise the money in just three days.

“It made me smile to look at what we can do and look at the money we are raising,” said Chatz.

Link. (Chicago Sun-Times)

I wonder if this atrocity is going to go unpunished?
MINE: Free tool for detecting novel associations in large data sets
by curiousepic — 2011-12-17T15:02:28.193Z

I was waiting for someone more knowledgeable to post something about this, but it's been a couple days and thought I'd bring it to LW's attention.

The maximal information coefficient (MIC) is a measure of two-variable dependence developed with the guidelines of generality and equitability in mind. The published paper describing MIC shows that it comes very close to achieving both goals simultaneously, and that it significantly outperforms competing methods in this regard.

Video summary

Main site with access to tool
Anesthesia and Consciousness [link]
by NancyLebovitz — 2011-12-17T11:50:01.021Z

link

Engel found that at the deepest levels of anaesthesia, the primary sensory cortex was the only region to respond to the electric shock. "Long-distance communication seems to be blocked, so the brain cannot build the global workspace," says Engel, who presented the work at last year's Society for Neuroscience meeting in San Diego. "It's like the message is reaching the mailbox, but no one is picking it up."

What could be causing the blockage? Engel has unpublished EEG data suggesting that propofol interferes with communication between the primary sensory cortex and other brain regions by causing abnormally strong synchrony between them. "It's not just shutting things down. The communication has changed," he says. "If too many neurons fire in a strongly synchronised rhythm, there is no room for exchange of specific messages."
[SEQ RERUN] The Fallacy of Gray
by MinibearRex — 2011-12-17T06:11:18.254Z

Today's post, The Fallacy of Gray was originally published on 07 January 2008. A summary (taken from the LW wiki):

Nothing is perfectly black or white. Everything is gray. However, this does not mean that everything is the same shade of gray. It may be impossible to completely eliminate bias, but it is still worth reducing bias.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was But There's Still A Chance, Right?, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
[Transcript] Tyler Cowen on Stories
by Grognor — 2011-12-17T05:42:39.630Z

I was shocked, absolutely shocked, to find that Tyler Cowen's excellent TEDxMidAtlantic talk on stories had not yet been transcribed. It generated a lot of discussion in the thread about it where it was first introduced, so I went ahead and transcribed it. I added hyperlinks to background information where I thought it was due. Here you go:

Host: In normal times, a blog written by an economist might not get that much attention, but our next presenter's blog, called Marginal Revolution, is quite popular, and he writes a column for the New York Times called the Economic Scene, here to explain the world to us in terms of the Great Recession and beyond, is Tyler Cowen.

Cowen: I was told to come here and tell you all stories, but what I'd like to do is instead tell you why I'm suspicious of stories, why stories make me nervous. In fact, the more inspired a story makes me feel, very often the more nervous I get. So the best stories are often the trickiest ones. The good and bad things about stories is they're a kind of filter. They take a lot of information, and they leave some of it out, and they keep some of it in. But the thing about this filter, it always leaves the same things in. You're always left with the same few stories. There's the old saying, just about every story can be summed up as, "A stranger came to town." There's a book by Christopher Booker, he claims there are really just seven types of stories. There's monster, rags to riches, quest, voyage and return, comedy, tragedy, rebirth. You don't have to agree with that list exactly, but the point is this: if you think in terms of stories, you're telling yourself the same things over and over again.

There was a study done, we asked some people to describe their lives. And when asked to describe their lives, what's interesting is how few people said, "mess". It's probably the best answer; I don't mean that in a bad way. "Mess" can be liberating, "mess" can be empowering, "mess" can be a way of drawing upon multiple strengths. But what people wanted to say was, "My life is a journey." 51% wanted to turn his or her life into a story. 11% said, "My life is a battle." Again, that's a kind of story. 8% said, "My life is a novel," 5% "My life is a play." I don't think anyone said, "My life is a reality TV show." Again, we're imposing order on the mess we observe, and it's taking the same patterns, and when so

...
Christopher Hitchens 1949-2011
by Costanza — 2011-12-16T22:05:16.117Z

Having been diagnosed with cancer last year, writer Christopher Hitchens has died. He was known as as an outspoken atheist, which is not, in itself, identical to being a committed rationalist in any systematic way. Even so, he seemed to have the virtue of moral courage, the willingness to speak the truth as he saw it, without fear.
No one knows what Peano arithmetic doesn't know
by cousin_it — 2011-12-16T21:36:56.640Z

WARNING: this post requires some knowledge of mathematical logic and computability theory.

I was just talking with Wei Dai and something came up that seems at once obvious and counterintuitive. Though if the argument is correct, I guess it will be old news to about 50% of the people who read my posts :-)

Imagine you have an oracle that can determine if an arbitrary statement is provable in Peano arithmetic. Then you can try using it as a halting oracle: for an arbitrary Turing machine T, ask "can PA prove that there's an integer N such that T makes N steps and then halts?". If the oracle says yes, you know that the statement is true for standard integers because they're one of the models of PA, therefore N is a standard integer, therefore T halts. And if the oracle says no, you know that there's no such standard integer N because otherwise the oracle would've found a long and boring proof involving the encoding of N as SSS...S0, therefore T doesn't halt. So your oracle can indeed serve as a halting oracle.

On the other hand, if you had a halting oracle to begin with, you could use it as a provability oracle for PA: "if a program successively enumerates all proofs in PA, will it ever find a proof for such-and-such statement?"

So having a provability oracle for PA or any other consistent formal system that proves some valid arithmetic truths (like ZFC) is equivalent to having a halting oracle, and thus leads to a provability oracle for any other formal system. In other words, if you knew all about the logical implications of PA, then you would also know all about the logical implications of ZFC and all other formal systems. Hee hee.

ETA: this line leads to a nontrivial question. Is there a formal system (not talking about the standard integers, I guess) whose provability oracle is strictly weaker than the halting oracle, but still uncomputable?

ETA 2: the question seems to be resolved, see Zetetic's comment and my reply.
I started a blog: Concept Space Cartography
by jimrandomh — 2011-12-16T21:06:28.888Z

For quite a while, I've felt that my mind contains important insights which are getting held up because I don't write enough. So to encourage myself to write more, I've started a blog: Concept Space Cartography. My most recent post there, Two Definitions for Critical Thinking, is likely to be of interest to Less Wrong.

Also, every page on my blog has an embedded copy of Textcelerator, a tool I created for speed-reading. (It's normally a browser plugin, which makes it appear on every web page. But you can use it on my blog without the plugin.)
The rationalist's checklist
by robertzk — 2011-12-16T16:21:45.453Z

Doctor Peter Pronovost has managed to single-handedly reduce the infection rates in ICU facilities nationwide from numbers like fourteen percent or twenty percent to zero. His solution is idiotically simple: a checklist. In a process as complex as ICU treatment, doctors perform chained simple steps very many times, and it can be easy to forget a step. These things add up. Read the article before continuing.

In their phenomenal book, The Power of Full Engagement, Jim Loehr and Tony Schwartz discuss a pattern they have discovered among all top performers, ranging from sports to music and business. Beyond a certain level, all top performers had established positive rituals for relaxation and deliberate practice. These positive rituals were daily ingrained habits and allowed them to surpass the merely excellent performers.

It is difficult to make use of most Less Wrong posts in terms of changing one's behavior. Even if you integrate a lesson fully, you will still miss steps on occasion. I propose we suggest checklists for various recurring activities that will offload these responsibilities from conscious thought to its much more reliable brother, ingrained habit. Fortunately, we should not need many checklists.

An example of a checklist is a morning routine:

Short exercise (40 pushups, 50 situps)

Shower

Daily cosmetics (brush teeth, shave, skin moisturizer, hair forming cream, male scented lotion, deodorant)

Make breakfast, which can be depending on what day it is mod 3: Black mango tea as well as

Omelette with cheese and tomatoes

Cereal with a side of oatmeal

A piece of fruit and yoghurt

Brief non-work and non-study related reading, for example, a novel

Another example of a checklist is during a conversation with a non-rationalist on the 5-second level: If you feel strong affect (anger or annoyance) at someone's point in a debate,

Say "Let me think about this for a second."

Are you two on the same inferential level? If not, backtrack by trying to bridge it first.

Have either of you committed a cognitive bias? (This may not be so helpful, given this is a potential fifty-item checklist!) If so, inform them about them about it in a friendly manner. ("Ah, I see! You have to be careful here because..." or "Oh, oops, I had forgotten here about...")

Think of this as a re-run of the 5-second l...
Weekly LW Meetups: Atlanta, Austin, Melbourne, Pittsburgh
by FrankAdamek — 2011-12-16T15:08:23.362Z

There are upcoming irregularly scheduled Less Wrong meetups in:

Next Atlanta Less Wrong Meetup: 10 December 2011 06:00PM
Pittsburgh Holiday Meetup: 16 December 2011 08:00PM
Is there anybody from Indianapolis here?: 08 January 2012 02:00PM
First Brussels meetup: 18 February 2012 11:00AM

The following meetups take place in cities with regularly scheduled meetups, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Austin, TX: 10 December 2011 01:30PM
Melbourne social meetup: 16 December 2011 07:00PM

The Mountain View meetup is now every other week, rather than weekly.

Cities with regularly scheduled meetups: Austin, Berkeley, Cambridge, MA, Irvine, London, Madison, WI, Marin, CA (uses the Bay Area List), Melbourne, Mountain View, New York, Ottawa, Oxford, San Francisco, Seattle, Toronto, Washington, DC, and West Los Angeles.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview will continue to be posted on the front page every Friday. These will be an attempt to collect information on all the meetups happening in the next weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll now also have the benefit of having your meetup mentioned in a weekly overview. These overview posts will be moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing list in order to be notified when an irregular meetup is happening: Atlanta, Chicago, Helsinki, London, Pittsburgh, Southern California (Los Angeles/Orange County area), St. Louis, Vancouver.