trentmkelly/LessWrong-43k huggingface dataset:
What is the advantage of the Kolmogorov complexity prior?
by skepsci — 2012-02-16T01:51:26.001Z

As I understand it, Solomonoff induction works by the a procedure loosely stated as saying we start with a Kolmogorov complexity universal prior (formalized Occam's razor), then update using Bayesian inference any time we see new data.

More precisely, suppose the universe is a computable sequence X of 0s and 1s. We want to predict the nth term before it happens, using the first n-1 terms.

To do this, we use the following 'algorithm':

List all computable sequences in increasing order of the complexity of the algorithm used to generate them to get a list A1, A2, A3, ...

We start with the prior P(Ak)=1/2k.

With this prior, we predict to see a string b of observations with probability P(b)=∑k P(Ak)[b<Ak], where [b<Ak]=1 if Ak starts with the string b, and [b<Ak]=0 otherwise.

After we see a string b of observations, we update our beliefs using Bayes theorem: P(Ak|b) = P(b|Ak)P(Ak)/P(b) = [b<Ak]P(Ak)/P(b), and similarly P(bc|b) = P(b|bc)P(bc)/P(b) = P(bc)/P(b).

I'm putting the word 'algorithm' in quotes, because there is no list of all computable sequences where [b<Ak] is a computable function of k and b, let alone one where the sequences are listed in increasing Kolmogorov complexity. You can think of it as an algorithm relative to an oracle that computes such a function.

If we follow this 'algorithm', we get a nice theorem:

For all ε>0 there is some N such that if b is an initial segment of X of length at least N, then P(bc|b)>1-ε if bc is an initial segment of X, and P(bc|b)<ε, otherwise.

So in the limit, we can correctly extrapolate from what we've seen with high confidence.

Proof: Let K = {k : X=Ak}. Let p0 = ∑k in K 1/2k. Let M ≥ -log(εp0). Let b be a sufficiently long initial segment of X such that b is not an initial segment of any Am, where m≤M and m is not in K. Then if bc is an initial segment of X,

P(bc) = ∑k 1/2k[bc<Ak] ≥ p0,

but on the other hand

P(b) = ∑k 1/2k[b<Ak] ≤ p0 + ∑k>M 1/2k = p0 + 2-M ≤ (1+ε)p0.

Therefore,

P(bc|b)=P(bc)/P(b) ≥ p0/(1+ε)p0 = 1/(1+ε) > 1-ε.

What if the universe isn't actually computable? Maybe there is some true randomness in the universe, as quantum-mechani...
Meetup : Austin, TX TCDS Experiment
by Vaniver — 2012-02-15T17:28:04.683Z

Discussion article for the meetup : Austin, TX TCDS Experiment

WHEN:
18 February 2012 01:30:00PM (-0600)

WHERE:
21st & Speedway, Austin TX

The Austin, TX meetup is in a new location for just this week; we'll be meeting in CBA 4.342 on Saturday the 18th at 1:30 PM. We're going to be trying out transcranial direct current stimulation. Please bring a laptop (or some device that can implement dual n-back) and headphones; I'll have the electrodes, batteries, and other assorted materials. If you could do a session or two of dual n-back before showing up, that'd be great, but isn't necessary.

(Parking: You can either park wherever you parked for Caffe Medici and walk ~three blocks to the east, or you can part in the Brazos street garage (at Brazos and MLK) and walk west then north to CBA.)

Discussion article for the meetup : Austin, TX TCDS Experiment
Pooling resources for valuable actuarial calculations
by  — 2012-02-15T17:01:22.744Z

It occurred to me this morning that, if it's actually valuable, generating true beliefs about the world must be someone's comparative advantage. If truth is instrumentally important, important people must be finding ways to pay to access it. I can think of several examples of this, but the one that caught my attention was actuarial science.

I know next to nothing about what actuaries actually do, but Wikipedia says:

"Actuaries mathematically evaluate the likelihood of events and quantify the contingent outcomes in order to minimize losses, both emotional and financial, associated with uncertain undesirable events."

Why, that sounds right up our alley.

So what I'm wondering is: for those who can afford it, wouldn't it be worth contracting with actuaries to make important personal decisions? Not merely with regards to business, but everything else as well? My preliminary ideas include:

Lifestyle choices to reduce personal risk of death

Health and wellness decisions

Vehicle choice for economic and safety considerations

Where to send your kid to college and otherwise improve life success

Lastly, if consulting actuaries is worth doing as a wealthy individual, shouldn't it also be worth doing as a group? Couldn't we pool money to get excellent information about questions that haven't yielded answers to our research attempts?

If I am not misunderstanding the work that actuaries do, there may indeed be low-hanging fruit here.
Meetup : Fort Collins, Colorado Meetup Wedneday 7pm
by EvelynM — 2012-02-15T15:30:19.429Z

Discussion article for the meetup : Fort Collins, Colorado Meetup Wedneday 7pm

WHEN:
15 February 2012 07:00:00AM (-0700)

WHERE:
144 North College Avenue, Fort Collins, CO 80524

Come hang out with interesting people, drink tea, and maybe go for dinner.

Discussion article for the meetup : Fort Collins, Colorado Meetup Wedneday 7pm
Anyone want a LW Enhancement Suite?
by MBlume — 2012-02-15T08:48:28.367Z

Reddit Enhancement Suite

If anyone cares, I could probably port this to work on LW without too much trouble. Optimistically it'd just involve opening up the source and replacing reddit.com with lesswrong.com. More realistically, there'd probably be a lot of baked-in assumptions about DOM structure that'd need to be updated to have the UI enhancements make sense.

Anyway, this is mostly just a straw poll to see how many others would be interested in such a thing.
[SEQ RERUN] Conditional Independence, and Naive Bayes
by MinibearRex — 2012-02-15T05:09:40.712Z

Today's post, Conditional Independence, and Naive Bayes was originally published on 01 March 2008. A summary (taken from the LW wiki):

You use categorization to make inferences about properties that don't have the appropriate empirical structure, namely, conditional independence given knowledge of the class, to be well-approximated by Naive Bayes.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Searching for Bayes-Structure, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Hard philosophy problems to test people's intelligence?
by Solvent — 2012-02-15T04:57:39.960Z

I'm looking for hard philosophical questions to give to people to gauge their skill at philosophy.

So far, I've been presenting people with Newcomb's problem and the Sleeping Beauty problem. I've also been presenting them with contrarian opinions and asking them to evaluate them, and I have a higher opinion of them if they avoid just icking away from the subject.

What other problems should I use?
Avoid misinterpreting your emotions
by Kaj_Sotala — 2012-02-14T23:51:32.998Z

A couple of weeks ago, I was suffering from insomnia. Eventually my inability to fall asleep turned into frustration, which then led to feelings of self-doubt about my life in general. Soon I was wondering about whether I would ever amount to anything, whether any of my various projects would ever end up bearing fruit, and so forth. As usual, I quickly became convinced that my life prospects were dim, and that I should stop being ambitious and settle for some boring but safe path while I still had the chance.

Then I realized that there was no reason for me to believe in this, and I stopped thinking that way. I still felt frustrated about not being able to sleep, but I didn't feel miserable about my chances in life. To do otherwise would have been to misinterpret my emotions.

Let me explain what I mean by that. There are two common stereotypes about the role of emotions. The first says that emotions are something irrational, and should be completely disregarded when making decisions. The second says that emotions are basically always right, and one should follow their emotions above all. Psychological research on emotions suggests that the correct answer lies in between: we have emotions for a reason, and we should follow their advice, but not unthinkingly.

The Information Principle says that emotional feelings provide conscious information from unconscious appraisals of situations1. Your brain is constantly appraising the situation you happen to be in. It notes things like a passerby having slightly threatening body language, or conversation with some person being easy and free of misunderstandings. There are countless of such evaluations going on all the time, and you aren't consciously aware of them because you don't need to. Your subconscious mind can handle them just fine on its own. The end result of all those evaluations is packaged into a brief summary, which is the only thing that your conscious mind sees directly. That "executive summary" is what you experience as a particular emotional state. The passerby makes you feel slightly nervous and you avoid her, or your conversational partner feels pleasant to talk with and you begin to like him, even though you don't know why.

To some extent, then, your emotions will guide you to act appropriately in various situations, even when you don't know why you feel the way you do. However, it's important to intepret...
The Future of Education
by Michelle_Z — 2012-02-14T20:58:06.904Z

This morning I read an interesting post on the future of education. I thought it would be interesting to have some members of LessWrong discuss it. I know it is idealistic, but some of the points raised were interesting.

Alex Lindsay

The Future of Education
Through an anomaly in the space-time continuum, I fell into the future last week. It was an odd sensation … traveling through time. But at least I made it back. I spent the time I had there at a local school and thought I would share what I saw …
Grades are gone.
Kids aren't in grade 1 or grade 3 … which was described to me as a "rudimentary" way to "cattle" students. The admins were gentle about it, explaining to me that when school was paper based, there just wasn't the facility to customize the classroom to the student. They explained that even though it was horribly inefficient, they understood why it needed to exist. They did point out that it ran a decade too long, affecting millions … but I changed the subject before it got ugly.
Instead, kids in school have individual achievement levels, which are different for every subject. They have 0-1800 points in each subject. Each student works at their own pace through these milestones and moving forward when they get near perfect scores. A student might have 1500 in one subject and 400 in another. Because everything is online and integrated, there aren't really "grades" like A, B, C, and D … kids are just accumulating points.
When kids max out in a subject, they can spend more time on other subjects; if they are on pace (they are expected to accumulate 100 points a year), they can create independent studies. Many students work very hard to move through the point structure so they can have more free time … which is structured but still up to them. Added resources are applied to students more than 100 points behind their pace. You end up with 20% of the students passing through the system with very little help beyond the structure, 60% getting some help, and 20% getting a much larger amount of attention to move through the system.
Lectures are gone
Lectures the way we know them don't really exist. Most of school is divided into 4 processes: Movies, Games, Projects and Discussion. Movies and Games largely exist on the tablets every student has (these look like iPads but they roll up into a baton-like structure)

...
On What Selves Are - CEV sequence
by diegocaleiro — 2012-02-14T19:21:48.081Z

The CEV Sequence Summary: The CEV sequence consists of three posts tackling important aspects of Coherent Extrapolated Volition (CEV). It covers conceptual, practical and computational problems of CEV's current form. On What Selves Are draws on analytic philosophy methods in order to clarify the concept of Self, which is necessary in order to understand whose volition is going to be extrapolated by a machine that implements the CEV procedure. Troubles with CEV part1 and Troubles with CEV part2 on the other hand describe several issues that will be faced by the CEV project if it is actually going to be implemented. Those issues are not of conceptual nature. Many of the objections shown come from scattered discussions found on the web. Finally, six alternatives to CEV are considered.

On What Selves Are Summary: We start by concurring on a Hofstadterian metaphysical view of Selves. We suggest two ways in which to divide the concept of Self, admitting Selves to be mongrel concepts, and cluster concepts. We then proceed to the identification of Selves, in particular, a proposed new method for a machine to identify Self-like entities. In the spirit of Dennettian philosophy, we then ask what we demand of Selves, to better grasp what they are. In conclusion, we present some views of Selves that are worth wanting, and claim that only considering Selves in their full complexity we can truly analyze them.

Note: A draft of the first half of On What Selves Are was published in discussion here, those who read it may want to skip straight to section "Organisms, Superorganisms and Selves".

On What Selves Are

Background: Symbols Coalesce to Form Selves

Some of what is taken for granted in this text is vividly subsumed by pg 204 and 289-290 of Hofstadter's “I Am a Strange Loop”(2007). To those who are still in the struggle relating to monism, dualism, qualia, Mary the neuroscientist, epiphenomenons and ineffable qualities, it is worth it to read through his passage to understand the background metaphysical view of the universe from which it is derived. To those on the other hand who are good willed reductionists of the non-greedy, no-skyhook, no 'design only from Above' kind may skip past this section:

[What makes and “I” come seemingly out of nowhere] is ironically, an inability - namely our [...] inability to see, feel

...
[LINK] The Hacker Shelf, free books.
by  — 2012-02-14T16:52:55.141Z

Yes, this a repost from Hacker News, but I want to point out some books that are of LW-related interest.

The Hacker Shelf is a repository of freely available textbooks. Most of them are about computer programming or the business of computer programming, but there are a few that are perhaps interesting to the LW community. All of these were publicly available beforehand, but I'm linking to the aggregator in hopes that people can think of other freely available textbooks to submit there.

The site is in its beginning explosion phase; in the time it took to write this post, it doubled in size. If previous sites are any indication, it will crest in a month or so. People will probably lose interest after three months, and after a year the site will probably silently close shop.

MacKay, Information Theory, Inference, and Learning Algorithms

I really wish I had an older version of this book; the newer one has been marred by a Cambridge UP ad on the upper margin of every page. Publishers ruin everything.

The book covers reasonably concisely the basics of information theory and Bayesian methods, with some game theory and coding theory (in the sense of data compression) thrown in on the side. The style takes after Knuth, but refrains from the latter's more encyclopedic tendencies. It's also the type of book that gives a lot of extra content in the exercises. It unfortunately assumes a decent amount of mathematical knowledge — linear algebra and calculus, but nothing you wouldn't find on the Khan Academy.

Hacker Shelf review, book website.

Easley and Kleinberg, Networks, Crowds, and Markets

There's just a lot of stuff in this book, most of it of independent interest. The thread that ties the book together is graph theory, and with it they cover a great deal of game theory, voting theory, and economics. There are lots of graphs and pictures, and the writing style is pretty deliberate and slow-paced. The math is not very intense; all their probability spaces are discrete, so there's no calculus, and only a few touches of linear algebra.

Hacker Shelf review, book website.

Gabriel, Patterns of Software

This is a more fluffy book about the practice of software engineering. It's rather old, but I'm linking to it anyway because I agree with the author's feeling that the software engineering discipline has more or less misunderstood Christopher Alexander's work on pattern...
"The Book Of Mormon" or Belief In Belief, The Musical
by Raw_Power — 2012-02-14T14:48:27.919Z

This song is... beautiful... Tragically so... It's like someone took some of the sequences here and made a checklist of everything that's wrong with religion, and why it still works and why it can stir the heart of noble, brave, generous people... The main character is a Mormon missionary, he was always groomed to be a believer, and his cheering and professing was rewarded, and he was happy. Then he was sent to some warzone in Africa, to convert people there to his faith (And I believe that in 1978, God changed his mind about black people! You can be a Mormon!). Except the people really aren't interested in what he preaches, and might easily shoot him in the head for it (But "What's so scary about that?", when you have an immortal soul and know you are doing the right thing? Whatever the outcome, you win!). But, see, that's okay, as long as he believes wholeheartedly and without a shred of doubt, everything will be fine... (You cannot just believe part way, you have to believe in it all. My problem was doubting the Lord's will, instead of standing tall!). Of course, if it does not turn out fine, that's all your fault for not believing enough...

You know, I'd like to say something smart to start the discussion over, but right now I'm just feeling too emotional... You know what, I'll just post the lyrics for the song, and let you guys suggest the adequate potholes for every instance of... blatant, obvious, categorized insanitiy irrationality that this song demonstrates...

Ever since I was a child I tried to be the best
So, what happened?
My family and friends all said I was blessed
So, what happened?
It was supposed to be all so exciting to be teaching of Christ 'cross the sea,
But, I allowed my faith to be shaken.
Oh, what's the matter with me?

I've always longed to help the needy
To do the things I never dared.
This was the time for me to step up
So, then, why was I so scared?

A warlord who shoots people in the face.
What's so scary about that?
I must trust that my Lord is mightier
And always has my back.
Now I must be completely devout
I can't have even one shred of doubt...

I believe that the Lord, God, created the universe.
I believe that He sent His only Son to die for my sins.
And I believe that ancient Jews built boats and sailed to America
I am a Mormon,
And a Mormon just believes.

You cannot just believe part way,
You have to believe...
What happens when your beliefs fully propagate
by Alexei — 2012-02-14T07:53:25.005Z

This is a very personal account of thoughts and events that have led me to a very interesting point in my life. Please read it as such. I present a lot of points, arguments, conclusions, etc..., but that's not what this is about.

I've started reading LW around spring of 2010. I was at the rationality minicamp last summer (2011). The night of February 10, 2012 all the rationality learning and practice finally caught up with me. Like a water that has been building up behind a damn, it finally broke through and flooded my poor brain.

"What if the Bayesian Conspiracy is real?" (By Bayesian Conspiracy I just mean a secret group that operates within and around LW and SIAI.) That is the question that set it all in motion. "Perhaps they left clues for those that are smart enough to see it. And to see those clues, you would actually have to understand and apply everything that they are trying to teach." The chain of thoughts that followed (conspiracies within conspiracies, shadow governments and Illuminati) it too ridiculous to want to repeat, but it all ended up with one simple question: How do I find out for sure? And that's when I realized that almost all the information I have has been accepted without as much as an ounce of verification. So little of my knowledge has been tested in the real world. In that moment I achieved a sort of enlightenment: I realized I don't know anything. I felt a dire urge to regress to the very basic questions: "What is real? What is true?" And then I laughed, because that's exactly where The Sequences start.

Through the turmoil of jumbled and confused thoughts came a shock of my most valuable belief propagating through my mind, breaking down final barriers, reaching its logical conclusion. FAI is the most important thing we should be doing right now! I already knew that. In fact, I knew that for a long time now, but I didn't... what? Feel it? Accept it? Visualize it? Understand the consequences? I think I didn't let that belief propagate to its natural conclusion: I should be doing something to help this cause.

I can't say: "It's the most important thing, but..." Yet, I've said it so many times inside my head. It's like hearing other people say: "Yes, X is the rational thing to do, but..." What follows is a defense that allows them to keep the path to their goal that they are comfortable with, that they are already invested in.

Interes...
[SEQ RERUN] Searching for Bayes-Structure
by MinibearRex — 2012-02-14T06:46:54.088Z

Today's post, Searching for Bayes-Structure was originally published on 28 February 2008. A summary (taken from the LW wiki):

If a mind is arriving at true beliefs, and we assume that the second law of thermodynamics has not been violated, that mind must be doing something at least vaguely Bayesian - at least one process with a sort-of Bayesian structure somewhere - or it couldn't possibly work.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Perpetual Motion Beliefs, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.