trentmkelly/LessWrong-43k huggingface dataset:
Learn (and Maybe Get a Credential in) Data Science
by Jayson_Virissimo — 2014-02-01T18:39:25.445Z

Coursera is now offering a sequence of online courses on data science. They include:

1. The Data Scientist's Toolbox

Upon completion of this course you will be able to identify and classify data science problems. You will also have created your Github account, created your first repository, and pushed your first markdown file to your account.

2. R Programming

In this course you will learn how to program in R and how to use R for effective data analysis. You will learn how to install and configure software necessary for a statistical programming environment, discuss generic programming language concepts as they are implemented in a high-level statistical language. The course covers practical issues in statistical computing which includes programming in R, reading data into R, accessing R packages, writing R functions, debugging, and organizing and commenting R code. Topics in statistical data analysis and optimization will provide working examples.

3. Getting and Cleaning Data

Upon completion of this course you will be able to obtain data from a variety of sources. You will know the principles of tidy data and data sharing. Finally, you will understand and be able to apply the basic tools for data cleaning and manipulation.

4. Exploratory Data Analysis

After successfully completing this course you will be able to make visual representations of data using the base, lattice, and ggplot2 plotting systems in R, apply basic principles of data graphics to create rich analytic graphics from different types of datasets, construct exploratory summaries of data in support of a specific question, and create visualizations of multidimensional data using exploratory multivariate statistical techniques.

5. Reproducible Research

In this course you will learn to write a document using R markdown, integrate live R code into a literate statistical program, compile R markdown documents using knitr and related tools, and organize a data analysis so that it is reproducible and accessible to others.

6. Statistical Inference

In this class students will learn the fundamentals of statistical inference. Students will receive a broad overview of the goals, assumptions and modes of performing statistical inference. Students will be able to perform inferential tasks in highly targeted settings and will be able to use the skills devel

...
Open Problems in FAI
by abramdemski — 2014-02-01T00:22:55.741Z

Edit: Please note that this write-up is poor quality, having the style of a hastily written personal note.

It has been mentioned that there should be a better write-up of open problems in FAI, and as I understand it there is an ongoing effort to explain such open problems. My feeling has been that the recent effort has tended too hold off proposing solutions for too long. I prefer the approach in the Tiling Agents paper, which explained problems through example systems which fail in various respects. What follows is an outline of what I'd write if I spent significant time; I think it is enough to be of some use. This list very much reflects my personal interests & beliefs.

- Tarski's Undefinability Theorem

- We would like a system to be able to reason about itself (in a few critical ways), and Tarski's Theorem is one of the important obstacles. Kripke provided the first hope of progress in this area by showing that we can embed a partial truth predicate in a language if we accept a "gap" (statements which cannot be assessed as true or false within the system's self-theory). Work over the decades has "reduced the gap" (capturing an increasing number of the self-judgements we want, while always leaving a gap). There are also "glut" theories (which must assess some things as both true and false), which typically mirror gap theories. Paul Christiano provided a theory of probabilistic self-reference which intuitively reduces the "gap" to infinitesimal size: the system's knowledge about its own probabilities can be wrong, but only by an infinitesimal. (For example, if it believes X, then it may fail to believe P(X)=1, but it will still believe P(X)>c for all c<1.) (Note, this feels a bit like a "glut" theory since the system solves the problem by saying too much rather than remaining silent.)

- First Incompleteness Theorem

- Logical Uncertainty:

- First-Order ("easy"): assigning probabilities to eventual results of (halting) computations we don't have time to make. I claim this is mostly solved by the FOL prior: we can prefer simpler hypotheses about the behavior of systems, treating computations as black boxes which we use universal induction to predict, while allowing us to incorporate logical reasoning about the function's behavior via bayesian updates. It also solves somewhat more; I claim it will have better properties th...
Best of Rationality Quotes, 2013 Edition
by DanielVarga — 2014-01-31T21:35:37.593Z

Here is the 2013 edition of the Best of Rationality Quotes collection. (Here is last year's.)

Best of Rationality Quotes 2013 (400kB page, 350 quotes)
and Best of Rationality Quotes 2009-2013 (1600kB page, 1490 quotes)

The page was built by a short script (source code here) from all the LW Rationality Quotes threads so far. (We had such a thread each month since April 2009.) The script collects all comments with karma score 10 or more, and sorts them by score. Replies are not collected, only top-level comments.

As is now usual, I provide various statistics and top-lists based on the data. (Source code for these is also at the above link, see the README.) I added these as comments to the post:

Top quote contributors by total karma score collected

Top quote contributors by karma score collected in 2013

Top quote contributors of 2013 by statistical significance level (See this comment for a description of this metric.)

Top original authors by number of quotes

Top original authors by total karma score collected

Best short quotes 2009-2013
Practical Benefits of Rationality (LW Census Results)
by Unnamed — 2014-01-31T17:24:38.810Z

by Dan from CFAR

Abstract: Two measures of the practical benefits of rationality, one a self-report of the benefits of being part of the rationality community and the other a measure of how often a person adds useful techniques to their repertoire, were included on the 2013 Less Wrong survey. In-person involvement with LW/CFAR predicted both measures of benefits, with friendships with LWers and attending a CFAR workshop showing the strongest and most consistent effects. Online Less Wrong participation and background had weaker and less consistent effects. Growth mindset also independently predicted both measures of practical benefits, and on the measure of technique acquisition there was an interaction effect suggesting that in-person LW/CFAR involvement may be especially beneficial for people high in growth mindset. However, some caution is warranted in interpreting these correlational, self-report results.

Introduction

Though I first found Less Wrong through my habit of reading interesting blogs, the main reason why I've gotten more and more involved in the rationality community is my suspicion that this rationality stuff might be pretty useful. Useful not only for thinking clearly about tricky intellectual topics, but also in ways that have more directly practical benefits.
CFAR obviously has similar interests, as it aims to create a community of people who are effective at acting in the world.

The 2013 LW census/survey provided an opportunity for us to probe how the rationality community is doing so far at finding these practical benefits, as it allowed us to survey a large cross section of the Less Wrong community. Unfortunately, there is not a standard, simple measure of practical benefits which we could just stick on the survey, and we were only able to use a correlational research design, but we sought to get some relevant information by coming up with two self-report questions to include on the survey.

One question was somewhat broader than the set of practical benefits that we were interested in and the other was somewhat narrower. First, there was a broad self-report question asking people how much they had benefited from being involved in the rationality community. Second, we asked people more narrowly how often they successfully added a useful technique or approach to their repertoire. We were primarily interested in seeing whether invol...
New LW Meetups: Hamburg, Ljubljana
by FrankAdamek — 2014-01-31T17:02:38.521Z

This summary was posted to LW Main on January 24th. The following week's summary is here.

New meetups (or meetups with a hiatus of more than a year) are happening in:

Meetup: First Meetup in Hamburg, Germany: 07 February 2014 07:00PM
Ljubljana: 25 January 2014 05:00PM

Other irregularly scheduled Less Wrong meetups are taking place in:

Atlanta Meetup, Topic: How to Become Immortal: 26 January 2014 07:00PM
Frankfurt meetup: 09 February 2014 02:00PM
Moscow, Another Meet up: 26 January 2014 04:00PM
Pittsburgh Meetup 24 January 2014 06:00 PM: 24 January 2014 06:00PM
Sydney Meetup: February: 26 February 2014 06:30PM
Tel Aviv: Uses of Cryptography: 30 January 2014 08:00PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Berkeley: Talk on communication: 29 January 2014 07:00PM
Community Weekend in Berlin: 11 April 2014 04:00PM
Brussels: Morality - also cake: 08 February 2014 01:00PM
[Melbourne] February Rationality Dojo: Planning Fallacy: 02 February 2014 03:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Brussels, Cambridge, MA, Cambridge UK, Columbus, London, Madison WI, Melbourne, Mountain View, New York, Philadelphia, Research Triangle NC, Salt Lake City, Seattle, Toronto, Vienna, Washington DC, Waterloo, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscr...
Meetup (West LA) : Virtual Evidence
by abramdemski — 2014-01-31T05:20:33.359Z

Discussion article for the meetup : Virtual Evidence

WHEN: 05 February 2014 07:00:00PM (-0800)

WHERE: 10850 West Pico Blvd, Los Angeles, CA 90064, USA

How to get in: Go to the Westside Tavern in the upstairs Wine Bar (all ages welcome), located inside the Westside Pavillion on the second floor, right by the movie theaters. The entrance sign says "Lounge". Parking is free for 3 hours.

Discussion: If we ask an expert to judge, purely based on their observations and not on prior expectations, whether a rug is a genuine antique, and we get the answer "Based only on what I could observe, I'd say 5 to 1 it's genuine", how should we revise our beliefs? We could assume that 5/6 is the posterior probability which the expert arrives at, but this is in bad faith with respect to our request to ignore the prior and infer based on observation alone. In "Probabilistic Reasoning in Intelligent Systems", Judea Pearl discusses a solution to this problem called "virtual evidence". I'll approach this issue from a perspective of how we should treat opinions offered to us in everyday conversation, and relate it to the general idea of approximate Bayesian updates. The discussion may be more toward the mathematical side, or more toward the practical side, depending on the audience response.

No prior knowledge of or exposure to Less Wrong is necessary; this will be generally accessible.

Discussion article for the meetup : Virtual Evidence
Amanda Knox Guilty Again
by christopherj — 2014-01-31T04:12:34.529Z

Today an Italian court has declared that Amanda Knox is, once again, guilty. She did not attend that trial (is not required to in Italy), so her final verdict will be either by appeal to the Supreme Court of Italy or the US extradition court. Extradition requests might be impeded due to the fact US does not have double jeopardy.

Previously on LessWrong, in The Amanda Knox Test: How an Hour on the Internet Beats a Year in the Courtroom there was some complaint that it actually took more than an hour on the internet to thoroughly research the case. Of course, the courts have been at this since 2007...

Her co-defendant, Raffaele Sollecito, who did show up at the trial, got sentenced to 25 years, but I don't know for sure where he is now because apparently he's totally unimportant and who cares (the media's opinion, not mine). I'm fairly sure he's in Italy though. So far it seems the plan is to revoke his passport but not arrest him.

Anyone want to take their hand at making predictions?

Will the final appeal find Amanda Knox and/or Raffaele Sollecito innocent or guilty?

When will the trial end? edit: I mean the inevitable appeal

If convicted, will the US extradite Amanda Knox?
On saving the world
by So8res — 2014-01-30T20:00:05.353Z

This is the final post in my productivity sequence.

The first post described what I achieved. The next three posts describe how. This post describes why, explaining the sources of my passion and the circumstances that convinced a young Nate to try and save the world. Within, you will find no suggestions, no techniques to emulate, no new ideas to ponder. This is a rationalist coming-of-age story. With luck, you may find it inspiring. Regardless, I hope you can learn from my mistakes.

Never fear, I'll be back to business soon — there's lots of studying to do. But before then, there's a story to tell, a memorial to what I left behind.

I was raised Catholic. On my eighth birthday, having received my first communion about a year prior, I casually asked my priest how to reaffirm my faith and do something for the Lord. The memory is fuzzy, but I think I donated a chunk of allowance money and made a public confession at the following mass.

A bunch of the grownups made a big deal out of it, as grownups are like to do. "Faith of a child", and all that. This confused me, especially when I realized that what I had done was rare. I wasn't trying to get pats on the head, I was appealing to the Lord of the Heavens and the Earth. Were we all on the same page, here? This was the creator. He was infinitely virtuous, and he had told us what to do.

And yet, everyone was content to recite hymns once a week and donate for the reconstruction of the church. What about the rest of the world, the sick, the dying? Where were the proselytizers, the missionary opportunities? Why was everyone just sitting around?

On that day, I became acquainted with civilizational inadequacy. I realized you could hand a room full of people the literal word of God, and they'd still struggle to pay attention for an hour every weekend.

This didn't shake my faith, mind you. It didn't even occur to me that the grownups might not actually believe their tales. No, what I learned that day was that there are a lot of people who hold beliefs they aren't willing to act upon.

Eventually, my faith faded. The distrust remained.

Gaining Confidence

I grew up in a small village, population ~1200. My early education took place in a one-room schoolhouse. The local towns eventually rolled all their school districts into one, but even then, my graduating class barely broke 50 people....
Multiverse-Wide Preference Utilitarianism
by Brian_Tomasik — 2014-01-30T18:08:55.878Z

Summary

Some preference utilitarians care about satisfaction of preferences even when the organism with the preference doesn't know that it has been satisfied. These preference utilitarians should care to some degree about the preferences that people in other branches of our multiverse have regarding our own world, as well as the preferences of aliens regarding our world. In general, this suggests that we should give relatively more weight to tastes and values that we expect to be more universal among civilizations across the multiverse. This consideration is strongest in the case of aesthetic preferences about inanimate objects and is weaker for preferences about organisms that themselves have experiences.

Introduction

Classical utilitarianism aims to maximize the balance of happiness over suffering for all organisms. Preference utilitarianism focuses on fulfillment vs. frustration of preferences, rather than just at hedonic experiences. So, for example, if someone has a preference for his house to go to his granddaughter after his death, then it would frustrate his preference if it instead went to his grandson, even though he wouldn't be around to experience negative emotions due to his preference being thwarted.

Non-hedonic preferences

In practice, most of people's preferences concern their own hedonic wellbeing. Some also concern the wellbeing of their children and friends, although often these preferences are manifested through direct happiness or suffering in oneself (e.g., being on the edge of your seat with anxiety when your 14-year-old daughter hasn't come home by midnight).

However, some preferences are beyond hedonic experience by oneself. This is true of preferences about how the world will be after one dies, or whether the money you donated to that charity actually gets used well even if you wouldn't find out either way. It's true of many moral convictions. For instance, I want to actually reduce expected suffering rather than hook up to a machine that makes me think I reduced expected suffering and then blisses me out for the rest of my life. It's also true of some aesthetic preferences, such as the view that it would be good for art, music, and knowledge to exist even if no one was around to experience them.

Certainly these non-hedonic preferences have hedonic effects. If I learned that I was going to be hooked up to a machine that would erase ...
Meetup : Inaugural Canberra meetup
by Solvent — 2014-01-30T17:03:10.593Z

Discussion article for the meetup : Inaugural Canberra meetup

WHEN:
12 February 2014 07:30:00PM (-0800)

WHERE:
CSIT Building, Acton ACT 2601, Australia

(I'm posting this on behalf of my lurker friend.)

WHEN: 12 February 2014 07:30:00PM(+1100)
WHERE: CSIT Building, Blg 108, North Road, ANU

This will be the first meetup in Canberra! We will get to know each other and play board games, joining the weekly board games night held by the ANU Computer Science Students' Association. Vegan-friendly snacks and board games will be provided (although if anyone has board games, it would be great if you could bring your own). I will be waiting at the main door of the building from 7:20 to 7:40 to show people to the room where we will be meeting.

Discussion article for the meetup : Inaugural Canberra meetup
Humans can drive cars
by Apprentice — 2014-01-30T11:55:45.027Z

There's been a lot of fuss lately about Google's gadgets. Computers can drive cars - pretty amazing, eh? I guess. But what amazed me as a child was that people can drive cars. I'd sit in the back seat while an adult controlled a machine taking us at insane speeds through a cluttered, seemingly quite unsafe environment. I distinctly remember thinking that something about this just doesn't add up.

It looked to me like there was just no adequate mechanism to keep the car on the road. At the speeds cars travel, a tiny deviation from the correct course would take us flying off the road in just a couple of seconds. Yet the adults seemed pretty nonchalant about it - the adult in the driver's seat could have relaxed conversations with other people in the car. But I knew that people were pretty clumsy. I was an ungainly kid but I knew even the adults would bump into stuff, drop things and generally fumble from time to time. Why didn't that seem to happen in the car? I felt I was missing something. Maybe there were magnets in the road?

Now that I am a driving adult I could more or less explain this to a 12-year-old me:

1. Yes, the course needs to be controlled very exactly and you need to make constant tiny course corrections or you're off to a serious accident in no time.

2. Fortunately, the steering wheel is a really good instrument for making small course corrections. The design is somewhat clumsiness-resistant.

3. Nevertheless, you really are just one misstep away from death and you need to focus intently. You can't take your eyes off the road for even one second. Under good circumstances, you can have light conversations while driving but a big part of your mind is still tied up by the task.

4. People can drive cars - but only just barely. You can't do it safely even while only mildly inebriated. That's not just an arbitrary law - the hit to your reflexes substantially increases the risks. You can do pretty much all other normal tasks after a couple of drinks, but not this.

So my 12-year-old self was not completely mistaken but still ultimately wrong. There are no magnets in the road. The explanation for why driving works out is mostly that people are just somewhat more capable than I'd thought. In my more sunny moments I hope that I'm making similar errors when thinking about artificial intelligence. Maybe creating a safe AGI isn't as impossible as it looks to me. Mayb...