trentmkelly/LessWrong-43k huggingface dataset:
Do the people behind the veil of ignorance vote for "specks"?
by D227 — 2011-11-11T01:26:52.017Z

The veil of ignorance as Rawls put it ..."no one knows his place in society, his class position or social status; nor does he know his fortune in the distribution of natural assets and abilities, his intelligence and strength, and the like."

The device allows for certain issues like slavery and income distribution to be determined beforehand. Would one vote for a society in which there is a chance of severe misfortune, but greater total utility? e.g, a world where 1% earn $1 a day and 99% earn $1,000,000 vs. a world where everyone earns $900,000 a day. Assume that dollars are utilons and they are linear (2 dollars indeed gives twice as much utility). What is the obvious answer? Bob chooses $900,000 a day for everyone.

But Bob is clever and he does not trust himself that his choice is the rational choice, so he goes into self-dialogue to investigate:

Q: "What is my preference, value or goal(PVG), such that, instrumental rationality may achieve it?"

A "I my preference/value/goal is for there to be a world in which total utility is less, but severe misfortune eliminated for everyone"

Q " As an agent are you maximizing your own utility by your actions of choosing a $900,000 a day world?

A " Yes, my actions are consistent with my preferences; I will maximize my utility by achieving my preference of limiting everyone's utility. This preference takes precedence.

Q: "I will now attack your position with the transitivity argument. At which point does your consistency change? What if the choices where 1% earns $999,999 and 99% earn 1,000,000?"

A: "My preference,values and goals have already determined a threshold, in fact my threshold is my PVG. Regardless the fact that my threshold may be different from everyone else's threshold, my threshold is my PVG. And achieving my PVG is rational."

Q: "I will now attack your position one last time, with the "piling argument". If every time you save one person from destitution, you must pile on the punishment on the others such that everyone will be suffering."

A: "If piling is allowed then it is to me a completely different question. Altering what my PVG is. I have one set of values for a non piling and piling scenario. I am consistent because piling and no

...
Which fields of learning have clarified your thinking? How and why?
by Arkanj3l — 2011-11-11T01:04:37.087Z

Did computer programming make you a clearer, more precise thinker? How about mathematics? If so, what kind? Set theory? Probability theory?

Microeconomics? Poker? English? Civil Engineering? Underwater Basket Weaving? (For adding... depth.)

Anything I missed?

Context: I have a palette of courses to dab onto my university schedule, and I don't know which ones to chose. This much is for certain: I want to come out of university as a problem solving beast. If there are fields of inquiry whose methods easily transfer to other fields, it is those fields that I want to learn in, at least initially.

Rip apart, Less Wrong!
Meetup : First(New?) Waterloo Meetup
by Paamayim — 2011-11-10T22:41:46.098Z

Discussion article for the meetup : First(New?) Waterloo Meetup

WHEN:
24 November 2011 08:00:00PM (-0500)

WHERE:
Meetpoint, 150 University Ave W, Waterloo, Ontario

LWers from Waterloo - of whom I know there are a few - assemble!

It looks like there have been a few meetups in the past, but nobody has been responding to my PM's, so I thought I'd take matters into my own hands. Rationality is too important to wait around for.

As a topic, I was thinking "what does LW mean to you?", but in my experience the conversation usually takes its own wings.

Hope to see everyone there!

Discussion article for the meetup : First(New?) Waterloo Meetup
Help with a (potentially Bayesian) statistics / set theory problem?
by joshkaufman — 2011-11-10T22:30:33.798Z

Update: as it turns out, this is a voting system problem, which is a difficult but well-studied topic. Potential solutions include Ranked Pairs (complicated) and BestThing (simpler). Thanks to everyone for helping me think this through out loud, and for reminding me to kill flies with flyswatters instead of bazookas.

I'm working on a problem that I believe involves Bayes, I'm new to Bayes and a bit rusty on statistics, and I'm having a hard time figuring out where to start. (EDIT: it looks like set theory may also be involved.) Your help would be greatly appreciated.

Here's the problem: assume a set of 7 different objects. Two of these objects are presented at random to a participant, who selects whichever one of the two objects they prefer. (There is no "indifferent" option.) The order of these combinations is not important, and repeated combinations are not allowed.

Basic combination theory says there are 21 different possible combinations: (7!) / ( (2!) * (7-2)! ) = 21.

Now, assume the researcher wants to know which single option has the highest probability of being the "most preferred" to a new participant based on the responses of all previous participants. To complicate matters, each participant can leave at any time, without completing the entire set of 21 responses. Their responses should still factor into the final result, even if they only respond to a single combination.

At the beginning of the study, there are no priors. (CORRECTION via dlthomas: "There are necessarily priors... we start with no information about rankings... and so assume a 1:1 chance of either object being preferred.) If a participant selects B from {A,B}, the probability of B being the "most preferred" object should go up, and A should go down, if I'm understanding correctly.

NOTE: Direct ranking of objects 1-7 (instead of pairwise comparison) isn't ideal because it takes longer, which may encourage the participant to rationalize. The "pick-one-of-two" approach is designed to be fast, which is better for gut reactions when comparing simple objects like words, photos, etc.

The ideal output looks like this: "Based on ___ total responses, participants prefer Object A. Object A is preferred __% more than Object B (the second most preferred), and ___% more than Object C (the third most preferred)."

Questions:

1. Is Bayes actually the most straightforward way of calculat...
Maximizing Cost-effectiveness via Critical Inquiry
by HoldenKarnofsky — 2011-11-10T19:25:14.904Z

I am cross-posting this GiveWell Blog post, a followup to an earlier cross-post I made. Here I provide a slightly more fleshed-out model that helps clarify the implications of Bayesian adjustments to cost-effectiveness estimates. It illustrates how it can be rational to take a "threshold" approach to cost-effectiveness, asking that actions/donations meet a minimum bar for estimated cost-effectiveness but otherwise focusing on robustness of evidence rather than magnitude of estimated impact.

We've recently been writing about the shortcomings of formal cost-effectiveness estimation (i.e., trying to estimate how much good, as measured in lives saved, DALYs or other units, is accomplished per dollar spent). After conceptually arguing that cost-effectiveness estimates can't be taken literally when they are not robust, we found major problems in one of the most prominent sources of cost-effectiveness estimates for aid, and generalized from these problems to discuss major hurdles to usefulness faced by the endeavor of formal cost-effectiveness estimation.

Despite these misgivings, we would be determined to make cost-effectiveness estimates work, if we thought this were the only way to figure out how to allocate resources for maximal impact. But we don't. This post argues that when information quality is poor, the best way to maximize cost-effectiveness is to examine charities from as many different angles as possible - looking for ways in which their stories can be checked against reality - and support the charities that have a combination of reasonably high estimated cost-effectiveness and maximally robust evidence. This is the approach GiveWell has taken since our inception, and it is more similar to investigative journalism or early-stage research (other domains in which people look for surprising but valid claims in low-information environments) than to formal estimation of numerical quantities.

The rest of this post

Conceptually illustrates (using the mathematical framework laid out previously) the value of examining charities from different angles when seeking to maximize cost-effectiveness.

Discusses how this conceptual approach matches the approach GiveWell has taken since inception.

Conceptual illustration

I previously laid out a framework for making a "Bayesian adjustment" to a cost-effectiveness estimate. I stated (and posted the mathemat...
[Draft] Poker With Lennier
by HonoreDB — 2011-11-10T17:14:01.171Z

In J. Michael Straczynski's science fiction TV show Babylon 5, there's a character named Lennier. He's pretty Spock-like: he's a long-lived alien who avoids displaying emotion and feels superior to humans in intellect and wisdom. He's sworn to always speak the truth. In one episode, he and another character, the corrupt and rakish Ambassador Mollari, are chatting. Mollari is bored. But then Lennier mentions that he's spent decades studying probability. Mollari perks up, and offers to introduce him to this game the humans call poker.

Later, we see Mollari, Lennier, and some others playing poker. Lennier squints at his hand and remarks, "Interesting. The odds of this combination are 5000:1, against." Everybody considers this revelation for a moment, then folds, conceding the hand. Mollari is exasperated, and tells him to stop doing that. Because Lennier is essentially announcing that he has a good hand, Lennier's winning far fewer chips than he should.

The other poker players, and the audience, are picturing Lennier as having a hand something like this:

This is a four of a kind, the second-best hand in most poker games. The odds against being dealt a four of a kind in a hand of five cards are 4164:1--one might, in a moment of excitement, round that up to an even five thousand.

We the audience are meant to have a hearty chuckle over how theory doesn't translate into practice. But! We never get to see Lennier's cards, which means we get to picture whatever we want. I choose to believe, and I urge you to do so as well, that Lennier had this hand:

This is one of the worst hands possible in poker: ace-high. It loses to almost everything. By causing everyone else to fold, Lennier won a hand he probably would otherwise have lost. He knew exactly what he was doing.

"Wait," I hear you say. "Lennier is sworn to always tell the truth. How could he ever make a verbal bluff in a poker game?" Well. Let's consider a few different ways we can interpret the phrase "the odds of this combination."

First of all, the specific two hands I've given above are equally likely to be dealt. Any specific set of five cards is just as likely to show up as any other. There are 2,598,960 distinct hands of poker, all created equal, so the odds against any particular one showing up are 2598959:1. That's all the hands of poker but one, lined up against that on...
Philanthropy Promotion: Bolder Giving
by jefftk — 2011-11-10T16:29:21.585Z

If you're trying to do as much good as possible, earning money to give away is a strong contender. Convincing other people to follow your example multiplies your impact. Julia and I have a profile on the philanthropy promotion website Bolder Giving. Much of the site consists of stories of people giving. I was talking to one of the co-founders, Anne Ellinger, and she would be interested in hearing from you if you give away a substantial fraction of your income or wealth, especially if you have been giving 20%+ of your income for 3+ years and are open to having your story posted on their site. I would be happy to introduce you, or you could write them yourself.
PredictionBook: A Short Note
by Jayson_Virissimo — 2011-11-10T15:10:28.480Z

New Updates

PredictionBook has been updated and the speed improvements are massive. I was considering abandoning it because navigating the site was so slow (especially on my Android smartphone), but now I'm here to stay. Since there is no news feed on the website itself, it seemed appropriate to announce the update here. Also, the layout has changed slightly (for the better IMO) and the probability assignments have been made more intuitive (which many of the newcomers were having trouble with). The updates, along with the (slowly) growing user base, make now a better time then ever to join. Thanks TrikeApps and Ivan Kozik!

My Experience So Far

I was one of those that was convinced to give PredictionBook a shot based on gwern's article. It has only been a month, so I am using up my willpower preventing myself from drawing any firm conclusions about my web of belief or sanity from such a small sample size. Ultimately, my goal is to know what 5% feels like and I already believe I have made a small step towards doing so. Even if PredictionBook turns out to be a failure for improving my calibration to the extent that I can intuitively feel what my degree of belief in a proposition is in percentage terms, it would still be worth it for the simple reason that I now have an inkling about how poorly calibrated I am. Finally, on at least three occasions, I have found myself trying to cook up some kind of rationalization in order to put off doing some task that I probably should be doing, but have felt additional pressure not to give in to akrasia because "my prediction will be judged false".

Admonition/Guilt Trip

Honestly, PredictionBook is a rationalist's dream tool. How would you even know if you are actually becoming less wrong without tracking your predictions? Eliezer Yudkowsky, Robin Hanson, Lukeprog, Yvain, Alicorn, Phil Goetz, wedrifid, Anna Salamon, Wei Dai, and cousin_it where the hell are you? Do you doubt the potential of PredictionBook to improve your calibration and diminish akrasia or is rationality really about "affiliating with an ideology or signaling one’s authority" and not about having the map reflect the territory (yes, I realize this is a false dichotomy, but it makes for better rhetoric)?

The Future of PredictionBook

I would like to see a few changes in the way PredictionBook works and a few new features. Sometimes when you are entering da...
No Basic AI Drives
by XiXiDu — 2011-11-10T13:27:34.207Z

People who think that risks from AI is the category of dangers that is most likely to be the cause of a loss of all human value in the universe often argue that artificial general intelligence tends to undergo recursive self-improvement. The reason for doing so is that intelligence is maximally instrumentally useful in the realization of almost any terminal goal an AI might be equipped with. They believe that intelligence is an universal instrumental value. This sounds convincing, so let's accept it as given.

What kind of instrumental value is general intelligence, what is it good for? Personally I try to see general intelligence purely as a potential. It allows an agent to achieve its goals.

The question that is not asked is why an artificial agent would tap the full potential of its general intelligence rather than only use the amount it is "told" to use, where would the incentive to do more come from?

If you deprived a human infant of all its evolutionary drives (e.g. to avoid pain, seek nutrition, status and - later on - sex), would it just grow into an adult that might try to become rich or rule a country? No, it would have no incentive to do so. Even though such a "blank slate" would have the same potential for general intelligence, it wouldn't use it.

Say you came up with the most basic template for general intelligence that works given limited resources. If you wanted to apply this potential to improve your template, would this be a sufficient condition for it to take over the world? I don't think so. If you didn't explicitly told it to do so, why would it?

The crux of the matter is that a goal isn't enough to enable the full potential of general intelligence, you also need to explicitly define how to achieve that goal. General intelligence does not imply recursive self-improvement, just the potential to do so, but not the incentive. The incentive has to be given, it is not implied by general intelligence.

For the same reasons that I don't think that an AGI will be automatically friendly, I don't think that it will automatically undergo recursive self-improvement. Maximizing expected utility is, just like friendliness, something that needs to be explicitly defined, otherwise there will be no incentive to do so.

For example, in what sense would it be wrong for a general intelligence to maximize paperclips in the universe by waiting for t...
Is the SIAI interview series available on youtube? 
by  — 2011-11-10T10:49:55.502Z

Or some other site? And if not, why not?

I'm especially interested in higher quality versions, as well as in a easy to share (embedding) format with a possiblity of people commenting them. At least there is convenient way to download them. Watching the interviews with Peter Norvig, Vernor Vinge and Peter Thiel, I found the current way a bit buggy, full screen option didn't work for example.

People searching for say interviews with X person on youtube might find these and later head out to the SIAI website or watch the other videos on the channel. More clicks are good right? I'm asking because there are some modified versions of the interviews on the site but I didn't find the original ones there.

Yes I know there are ways around this, but why should they put trivial inconveniences in people's way? It should be more user friendly.

Edit: Also why haven't the presentations been updated since oh 2007?
Slightly known unknowns
by NancyLebovitz — 2011-11-10T10:29:37.863Z

Inspired by What visionary project would you fund?, I'm wondering about whether there are known blank areas in our knowledge which might turn up surprising knowledge.

Once upon a time, the sun was a mystery. People had a pretty good idea of its mass, and how much chemical energy would be needed to keep it shining. I don't remember how long the sun could last running on chemical energy, but it didn't seem plausible that it could be so new.

It turned out that chemical energy wasn't the only possibility.

I don't get the impression that there was wide appreciation that most of the world was being ignored by scientists because the calculations were too difficult-- until the calculations were made easier.

Do people here expect to be surprised by whatever it takes to understand qualia?
[SEQ RERUN] Unbounded Scales, Huge Jury Awards, & Futurism
by MinibearRex — 2011-11-10T04:29:47.914Z

Today's post, Unbounded Scales, Huge Jury Awards, & Futurism was originally published on 29 November 2007. A summary (taken from the LW wiki):

Without a metric for comparison, estimates of, e.g., what sorts of punitive damages should be awarded, or when some future advance will happen, vary widely simply due to the lack of a scale.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Evaluability (And Cheap Holiday Shopping), and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
A question on rationality.
by D227 — 2011-11-10T00:20:53.331Z

My long runs on Saturdays give me time to ponder the various material at lesswrong. Recently my attention has been kept busy pondering a question about rationality that I have not yet resolved and would like to present to lesswrong as a discussion. I will try to be as succinct as possible, please correct me if I make any logical fallacies.

Instrumental rationality is defined as the art of choosing actions that steer the future toward outcomes ranked higher in your preferences/values/goals (PVGs)

Here are my questions:

1. If rationality is the function of achieving our preferences/values/goals, what is the function of choosing our PVGs to begin with, if we could choose our preferences? In other words, is there an "inherent rationality" absence of preference or values? It seems as if the definition of instrumental rationality is saying that if you have a PVG, that there is a rational way to achieve it, but there is not necessarily rational PVGs.

2.If the answer is no, there is no "inherent rationality" absence of a PVG, then what would preclude the possibility that a perfect rationalist, given enough time and resources, will eventually become a perfectly self interested entity with only one overall goal which is to perpetuate his existence, at the sacrifice of everything and everyone else?

Suppose a superintelligence visits Bob and grants him the power to edit his own code. Bob can now edit or choose his own preferences/values/goals.

Bob is a perfect rationalist.

Bob is genetically predisposed to abuse alcohol, as such he rationally did everything he could to keep alcohol off his mind.

Now, Bob no longer has to do this, he simply goes into his own code and deletes this code/PVG/meme for alcohol abuse.

Bob continues to cull his code of "inefficient" PVGs.

Soon Bob only has one goal, the most important goal, self preservation.

3. Is it rational for Bob, having these powers, to rid himself of humanity, and rewrite his code to only want to support one meme, that is the meme to ensure his existence. Everything he will do will goes to support this meme. He will drop all his relationships, his hobbies, all his wants and desires into concentrate on a single objective. How does Bob not become...