trentmkelly/LessWrong-43k huggingface dataset:
Eliezer Yudkowsky and Michael Vassar at NYU, Thursday March 3rd
by Cosmos — 2011-03-01T04:53:15.233Z

Eliezer Yudkowsky and Michael Vassar are going to be speaking at NYU this Thursday, March 3rd, on the subject of SIAI and rationality. Any readers in the NYC area are encouraged to attend.

Details:

6-8 PM
Vanderbilt Hall, Room 220
40 Washington Square South

Food and drink will be provided. There will be a small reception at a nearby bar at the conclusion of the talk as well.

Even if you don't live here yourself, you can still show support by inviting anyone you know in the NYC area via the Facebook event.
Are Interesting Problems Useful?
by paulfchristiano — 2011-03-01T03:59:56.160Z

In a recent post I posed the question: is the common good served by directing research efforts towards theoretical problems which are interesting to researchers?
komponisto defends interesting problems, arguing that researcher's perceptions of interestingness are often better able to predict future usefulness than anyone trying deliberately to determine what will be useful. This is a plausible claim (although I disagree), and I have encountered it a number of times in the last couple of days. This claim was advanced as a defense of the status quo, but if we really believe it then we should certainly try and understand all of its consequences.
When setting out to predict the usefulness of a research program (as I suggest we should), we are not required to do it via deductive arguments which estimate the likelihood of certain applications. We can use all of the data available, including how interesting the problem seems---to us, to other researchers, to lay people, etc. If intelligent observers' notions of interestingness are substantially corellated with future usefulness, potentially in unpredictable ways, then we would be wise to take this information into account. This is precisely what komponisto and others argue, and they conclude that we should support work on the problems an investigator finds most interesting. I claim this is an example of motivated stopping: the argument was thought through just far enough to support changing nothing.
We have access to many, many indicators of interestingness for any candidate research problem. A problem can seem interesting only to a single person who understands the background in great depth; it can seem interesting to a small group of researchers in related fields; it can seem interesting to mathematicians broadly; it can seem interesting to computer scientists, to physicists, to biologists, to engineers, to laypeople. It can seem particularly interesting to professional mathematicians, or to novices with new ideas. It can invoke feelings of immediacy, of needing to know the answer; it can simply be fun to work on. Particular countries or cultures or time periods or subfields may have objectively better or worse aesthetics.
If our aim is to use interestingness as a predictor of potential usefulness then all of this variability is an asset. We have a historical record to be scoured; patterns to be evaluated. Unders...
Experimental evidence of the value of redundant oral tradition
by Raemon — 2011-03-01T01:08:28.566Z

Something I've been hearing a lot lately (specifically from Orthodox Jews, although it comes up a lot in debates about religion) is that having a large number of people telling a story makes it more likely the story is true, because multiple witnesses can call each other out for deviating from the truth.

My gut reaction is that this is extremely false. But it's a point that should be scientifically testable, and I figure that someone should have done a study on it by now. Does anyone know of such a thing?

A related issue is the argument that oral tradition meant something very different thousands of years ago, when it was the ONLY form of historical record. Oral historians were duty-bound to preserve the story. This sounds plausible. It probably ISN'T as easily testable since we can't compare oral history from pre-writing times against... well, much of anything. (Well, I guess archaeological evidence, if the events being described would have left enough archaeological evidence). Is there an official, accepted scholarly opinion on this?
Cryptography
by MinibearRex — 2011-02-28T21:35:11.824Z

Breaking encrypted messages offers a unique challenge, at least in its pure form. In most cryptography puzzles, the method of encryption is known, and so the challenge is finding the key. The most common form of encryption used for this is a simple substitution cipher. This is not a very difficult challenge. Depending on the puzzle, it can be tough, but it isn't something that will really strain your intellect to its maximum.

True cryptography occurs when you just get a message, and no other information. Then, the codebreaker has to find a way to determine the type of code, and then they have to find the key. This type of challenge is good for a rationalist, since you have to make sense of something confusing by running experimental tests, usually by analyzing the text. I've always found codebreaking in this sense to be very enjoyable and useful for training your mind. The main obstacle to doing so is the lack of any system designed for it. There is no website (that I know of) that provides this sort of cryptography challenge. Typically, if you want to do this, you have to find other people who also have an interest in it.

It occurred to me that people on Less Wrong might have an interest in doing something of this nature. Now, obviously, we probably won't be trying to break RSA ciphers, but there are a bunch of methods of encryption that were developed over the years before the invention of computer cryptography that could be used, without us requiring any participants to know how to program computers or do anything like that.

Is there any interest in something like this? I personally don't care how much you know already about cryptography. If you don't know anything I'd be happy to give you some book recommendations.

PS There is a difference between "cipher" and "code", but in practical language the two are sometimes interchangeable. For instance, "codebreaker" vs "cipherbreaker" isn't often a very important distinction to make, so I used the more common term. As long as the correct message gets across, you can use either term. Just make sure, if you are saying something that is specific to one particular type of encryption that you use the right vocabulary then.
I am anti-awareness and you should be too
by KatjaGrace — 2011-02-28T17:31:48.000Z

People seem to like raising awareness a lot. One might suspect too much, assuming the purpose is to efficiently solve whatever problem the awareness is being raised about. It’s hard to tell whether it is too much by working out how much is the right amount then checking if it matches what people do. But a feasible heuristic approach is to consider factors that might bias people one way or the other, relative to what is optimal.

Christian Lander at Stuff White People Like suggests some reasons raising awareness should be an inefficiently popular solution to other people’s problems:

This belief [that raising awareness will solve everything] allows them to feel that sweet self-satisfaction without actually having to solve anything or face any difficult challenges…

What makes this even more appealing for white people is that you can raise “awareness” through expensive dinners, parties, marathons, selling t-shirts, fashion shows, concerts, eating at restaurants and bracelets. In other words, white people just have to keep doing stuff they like, EXCEPT now they can feel better about making a difference…

So to summarize – you get all the benefits of helping (self satisfaction, telling other people) but no need for difficult decisions or the ensuing criticism (how do you criticize awareness?)…

He seems to suspect that people are not trying to solve problems, but I shan’t argue about that here. At least some people think that they are trying to effectively campaign; this post is concerned with biases they might face. Christian may or may not demonstrate a bias for these people. All things equal, it is better to solve problems in easy, fun, safe ways. However if it is easier to overestimate the effectiveness of easy, fun, safe things, we probably raise awareness too much. I suspect this is true. I will add three more reasons to expect awareness to be over-raised.

First, people tend to identify with their moral concerns. People identify with moral concerns much more than they do with their personal, practical concerns for instance. Those who think the environment is being removed too fast are proudly environmentalists while those who think the bushes on their property are withering too fast do not bother to advertise themselves with any particular term, even if they spend much more time trying to correct the problem. It’s not pa...
Singularity Institute now accepts donations via Bitcoin
by Kevin — 2011-02-28T16:03:32.243Z

Now you can donate to Singularity Institute using Bitcoin.

Currently Bitcoin mining appears to be profitable as only bubble economics can be. Already some Less Wrong users have purchased GPUs and started mining Bitcoin. Please consider sending some Bitcoins to SI at address 1HUrNJfVFwQkbuMXwiPxSQcpyr3ktn1wc9
2014 Edit: Please donate Bitcoin by using the Bitpay link on MIRI's donate page. Thanks!

.
Wikipedia: Moravec's Paradox
by cousin_it — 2011-02-28T14:13:58.676Z

Linky. Quotes:

Moravec's paradox is the discovery by artificial intelligence and robotics researchers that, contrary to traditional assumptions, high-level reasoning requires very little computation, but low-level sensorimotor skills require enormous computational resources.

[...]

We should expect the difficulty of reverse-engineering any human skill to be roughly proportional to the amount of time that skill has been evolving in animals.

The oldest human skills are largely unconscious and so appear to us to be effortless.

Therefore, we should expect skills that appear effortless to be difficult to reverse-engineer, but skills that require effort may not necessarily be difficult to engineer at all.
Mistakes and Rationality
by InquilineKea — 2011-02-28T00:27:24.541Z

So, like all of us, I've made numerous mistakes in my life. And I agree with the cliche belief - that I've learned numerous useful things from these mistakes. Mistakes are often a way for me to "test my social boundaries". And that's important, because many many potentially novel behaviors are behaviors that do test on various social boundaries, and it's important to have some intuition about where these boundaries lie, so that I can be innovative without being offensive (and also so that I can be efficient and waste as little time as possible on unnecessary social formalities).

Furthermore, past mistakes are often a strong impetus for motivation. I've tried many strategies in the past that simply didn't work. And due to all the valuable time I wasted on them, I always am able to motivate myself by reminding myself of these past mistakes that I'm still very ashamed of (mistakes such as staring at math books for hours and hours on end, while not getting anything out of them).

I've also had the nasty experience to see many of my old friendships end badly. But I've learned in those examples - I've learned how to be better to people, to not expect too much out of them, to try to be appreciative to them and to anticipate what they want, to try to care about them (if possible), and also to see through their numerous white lies. Theoretically, this could have been done if I didn't have friendships that ended badly. But it would have been harder to do without emotional destabilization, since even I am prone to psychological inertia.

Now, is it rational to make mistakes early on? There are a few things to keep in mind:

(1) Some mistakes have the potential of permanently setting us back in life. My parents, for example, often threatened to force me to get a job, which would have had the very strong potential of setting me back for life. Unfortunately, as someone with both ADD and Asperger's, I simply cannot do a job and learn at the same time, and that happened, I may never be able to go on to graduate school. I am, however, fortunate enough to have parents willing to pay my way through college. If I didn't have that option (and was in the lower classes), then yes, some of my mistakes could have forced me into a perpetual cycle of repetitive low-wage jobs effectively for life.

And that's the key thing with mistakes. Mistakes can sometimes permanently doom...
Inspiring Rationalists
by paulfchristiano — 2011-02-27T05:08:59.445Z

I am concerned with the number of rational, altruistic, smart people in the world. By my standards there are a reasonable number of smart, altruistic (in far mode) people, so a good first step is improving their rationality. To keep "rationality" grounded, I should be a little more precise about what I mean.

There are people who care about the needs of others, while they are in far mode at least, and who are good at solving hard problems. I believe many of them fail to apply their problem solving capabilities effectively to some important questions: "What do I believe about the world, and why?" "What do I value?" "Based on those beliefs and values, what should I spend my time doing?" Worse, they sometimes arrive at answers to these questions which are seriously affected by various biases which (I believe) they would eventually recognize as biases. This behavior defines irrationality.

Suppose that someone has successfully created a sequence of readings which contained everything you might need to know in order to become reasonably rational. The accessibility of these readings is not in itself adequate to make smart, altruistic people automatically rational. Based on my experience with humans, I strongly suspect that personal engagement is ultimately necessary, at least if the smart people in question are living in the same culture I am (and the best way to change culture is to change people). What needs to be accomplished with this personal engagement, so that the mere accessibility of written wisdom could do the rest?

Here is my take on the requirements.

1) They need to believe that thinking about their beliefs, values, thought process, and decision making process is valuable. They need to have a strong sense that self-improvement along these axes is valuable; they need to believe that more is possible.

2) They need to be able to act on the basis of abstract reasoning. Most people seem to have a very firm barrier between the part of them that learns things and the part that makes decisions; no matter how much the learning part believes that the decision making part should listen to it, it doesn't do any good unless the decision making part actually listens. This seems to be a rather serious problem in general.

3) They need to admit the possibility that some beliefs they hold strongly are wrong. Beliefs in which a person places complete confidence seem to be ...
Some Considerations Against Short-Term and/or Explicit Focus on Existential Risk Reduction
by multifoliaterose — 2011-02-27T04:31:03.693Z

Over the past six months I've been repeatedly going back and forth on my attitude toward the value of short-term and/or exclusive focus on existential risk. Here I'll offer some reasons why a utilitarian who recognizes the upside of preventing human extinction may refrain from a direct focus on existential risk reduction. I remain undecided on my attitude toward short-term and/or exclusive focus on existential risk - this article is not rhetorical in intent; I'm just throwing some relevant issues out there.

1. On the subject of FAI research, Prase stated that:

The whole business is based on future predictions of several tens or possibly hunderts years in advance, which is historically a very unsuccessful discipline. And I can't help but include it in that reference class.

The same can be said of much of the speculation concerning existential risk in general, not so much existential risk due to asteroid strike or Venusian global warming but rather with the higher probability but much more amorphous existential risks connected with advanced technologies (general artificial intelligence, whole brain emulation, nano weapons, genetically engineered viruses, etc.).

A principle widely held by many highly educated people is that it's virtually impossible to predict the future more than a few decades out. Now, one can attempt to quantify "virtually impossible" as a small probability that one's model of the future is correct and multiply it by the numbers that emerge as outputs of one's model of the future in Fermi calculations, but the multiplier corresponding to "virtually impossible" may be considerably smaller than one might naively suppose...

2. As AnnaSalamon said in Goals for which Less Wrong does (and doesn't) help,

conjunctions are unlikely

Assuming that A and B are independent events, the probability of their conjunction is p(A)p(B). So for example, an event that's the conjunction of n independent events each with probability 0.1 occurs with probability 10-n. As humans are systematically biased toward believing that conjunctions are more likely than their conjuncts (at least in certain setting), there's a strong possibility of exponentially overestimating probabilities in the course of Fermi calculations. This is true both of the probability that one's model is correct (given the amount of uncertainty involved in the future as reflected by historical preced...
Vassar talk in New Haven, Sunday 2/27
by alyssavance — 2011-02-27T02:49:28.919Z

Hey all. I've invited Michael Vassar, president of the Singularity Institute, to come to Yale to give a talk on AI and the Methods of Rationality. We'll be holding the talk on Sunday the 27th at 4 PM, at WLH 119 (100 Wall St., New Haven CT), with an open discussion afterwards. Everyone should come- there will be free pizza!

(Reposted to main section on request of JGWeissman).
Existential Risk Reduction Career Network
by FrankAdamek — 2011-02-27T00:27:08.597Z

Interested in donating to existential risk reduction efforts? Would you like to exchange career information with like-minded others? Then you should consider the Existential Risk Reduction Career Network! ("X Risk Network" for those short on time.) From the front page of the website:

"This network is for anyone interested in donating substantial amounts (relative to income) to non-profit organizations focused on the reduction of existential risk, such as SIAI, FHI, and the Lifeboat Foundation. [...] We are a community of people assisting each other to increase our resources available for contribution. Members discuss the strengths and weaknesses of different careers, network, share advice on job applications and career advancement, assist others with finding interviews, and occasionally look for qualified individuals to hire from within the network."

For more details, including on the process of requesting invitations, head on over to the front page at http://www.xrisknetwork.org/

Keep in mind that the network is for students as well, not just those currently on the job market. The network also has discussion of long term job strategy, school admissions, and intern possibilities.
Vassar talk in New Haven, Sunday 2/27 
by alyssavance — 2011-02-26T20:57:27.652Z

Hey all. I've invited Michael Vassar, president of the Singularity Institute, to come to Yale to give a talk on AI and the Methods of Rationality. We'll be holding the talk on Sunday the 27th at 4 PM, at WLH 119 (100 Wall St., New Haven CT), with an open discussion afterwards. Everyone should come- there will be free pizza!
When to scream "Error!"
by Dorikka — 2011-02-26T17:40:35.688Z

In Anna’s recent post, she talked about training your mind to notice when it wasn’t curious about something and scream “Error! Look for a different way to do this” in such cases. Johnicholas and TheOtherDave's list of what stupidity feels like also looks useful for this purpose. I'm creating this post to make a more comprehensive list of feelings which indicate that people should reanalyze different possible paths to make sure that the one which they're taking is the most effective one to their objective.

Please suggest additions to the list in your comments -- I'll move them up here (along with links to further explanation, if given.) Keep in mind that your description of the feeling should be as illustrative as possible. For example, "feeling stupid" is unhelpful, while "you feel like you've taken a wrong turn into a never-ending tunnel" is better. Of course, metaphors which are immediately understood by some people may not be so easily understood by others, so try to give a more detailed description of the feeling if other people express that you're probably saying more than they're hearing.

List: "Error! Look for a different way to do this" if you feel like:

your curiosity is dead

an automaton -- you're acting, albeit slowly, but your mind isn't grokking

you've taken a wrong turn into a never-ending tunnel

you're behaving clunkily, mechanically, when there's an already-formed instinct waiting to be used

what you're doing is "like watching cable, only with fewer hair replacement infomercials"

you have to avoid working out the implications of a particular line of reasoning.

you're going to have to grit your teeth and do a tedious and boring task1

you're blind and must be led around by more observant people

you're doing something because you're supposed to, rather than because it helps you achieve goals3

you just shrugged off something that seemed important

Stupidity feels like:

being bored, being in pain, being distracted, wanting to do anything else than this

being unworthy of these divine (external) ideas

blind plodding obedience

being tired all the time, even if you're not2

not having enough fingers to hold all of my thoughts in place

merging onto the highway when I can't see all the oncoming traffic

someone's playing loud distracting music that I can't hear

riding on a train with square wheels

1. Sometimes tedious/borin

...