trentmkelly/LessWrong-43k huggingface dataset:
Putting in the Numbers
by Manfred — 2014-01-30T06:41:42.112Z

Followup To: Foundations of Probability

In the previous post, we reviewed reasons why having probabilities is a good idea. These foundations defined probabilities as numbers following certain rules, like the product rule and the rule that mutually exclusive probabilities sum to 1 at most. These probabilities have to hang together as a coherent whole. But just because probabilities hang together a certain way, doesn't actually tell us what numbers to assign.

I can say a coin flip has P(heads)=0.5, or I can say it has P(heads)=0.999; both are perfectly valid probabilities, as long as P(tails) is consistent. This post will be about how to actually get to the numbers.

If the probabilities aren't fully determined by our desiderata, what do we need to determine the probabilities? More desiderata!

Our final desideratum is motivated by the perspective that our probability is based on some state of information. This is acknowledged explicitly in Cox's scheme, but is also just a physical necessity for any robot we build. Thus we add our new desideratum: Assign probabilities that are consistent with the information you have, but don't make up any extra information. It turns out this is enough to let us put numbers to the probabilities.

In its simplest form, this desideratum is a symmetry principle. If you have the exact same information about two events, you should assign them the same probability - giving them different probabilities would be making up extra information. So if your background information is "Flip a coin, the mutually exclusive and exhaustive probabilities are heads and tails," there is a symmetry between the labels "heads" and "tails," which given our new desideratum lets us assign each P=0.5.

Sometimes, though, we need to pull out the information theory. Using the fact that it doesn't produce information to split the probabilities up differently, we can specify something called "information entropy" (For more thoroughness, see chapter 11 of Jaynes). The entropy of a probability distribution is a function that measures how uncertain you are. If I flip a coin and don't know about the outcome, I have one bit of entropy. If I flip two coins, I have two bits of entropy. In this way, the entropy is like the amount of information you're "missing" about the coin flips.

The mathematical expression for information entropy is that it's the sum of each probability...
Huffington Post article on DeepMind-requested AI ethics board, links back to LW [link]
by Dr_Manhattan — 2014-01-30T01:20:10.579Z

http://www.huffingtonpost.com/2014/01/29/google-ai_n_4683343.html

Not going to summarize the article content, but I think this is the highest-level publication linking to LW so far.

Also, it's appears that Shane Legg, Jaan Tallin and others at Deep Mind leveraged the acquisition and moved the friendly AI conversation to a higher level, quite possibly highest level at Google. Interesting times, these are.
Meetup : West LA Meetup-indexical and logical uncertainty
by Scott Garrabrant — 2014-01-29T22:26:20.792Z

Discussion article for the meetup : West LA Meetup-indexical and logical uncertainty

WHEN:
29 January 2014 02:32:41PM (-0800)

WHERE:
10850 West Pico Blvd, Los Angeles, CA 90064, USA

How to get in: Go to the Westside Tavern in the upstairs Wine Bar (all ages welcome), located inside the Westside Pavillion on the second floor, right by the movie theaters. The entrance sign says "Lounge".
Parking is free for 3 hours.
Discussion: We will discuss the difference between logical and indexical uncertainty, mostly inspired by my recent blog post. I expect that this will be a short discussion, and most of the time will be casual conversation. We will also discuss a new meeting place, (or maybe even new time) so if you have opinions on this, and will not be there, leave a comment. This meetup will be in less than 5 hours at the time of this post. Sorry for the short notice. No prior knowledge of or exposure to Less Wrong is necessary; this will be generally accessible.

Discussion article for the meetup : West LA Meetup-indexical and logical uncertainty
Logical and Indexical Uncertainty
by Scott Garrabrant — 2014-01-29T21:49:53.387Z

Cross-posted on By Way of Contradiction

Imagine I shot a photon at a half silvered mirror which reflects the photon with "probability" 1/2 and lets the photon pass through with "probability" 1/2.

Now, Imagine I calculated the trillionth decimal digit of pi, and checked whether it was even or odd. As a Bayesian, you use the term "probability" in this situation too, and to you, the "probability" that the digit is odd is 1/2.

What is the difference between these too situations? Assuming the many worlds interpretation of quantum mechanics, the first probability comes from indexical uncertainty, while the second comes from logical uncertainty. In indexical uncertainty, both possibilities are true in different parts of whatever your multiverse model is, but you are unsure which part of that multiverse you are in. In logical uncertainty, only one of the possibilities is true, but you do not have information about which one. It may seem at first like this should not change our decision theory, but I believe there are good reasons why we should care about what type of uncertainty we are talking about.

I present here 6 reasons why we potentially care about the 2 different types of uncertainties. I do not agree with all of these ideas, but I present them anyway, because it seems reasonable that some people might argue for them. Is there anything I have missed?

1) Anthropics

Suppose Sleeping Beauty volunteers to undergo the following experiment, which is described to her before it begins. On Sunday she is given a drug that sends her to sleep, and a coin is tossed. If the coin lands heads, Beauty is awakened and interviewed on Monday, and then the experiment ends. If the coin comes up tails, she is awakened and interviewed on Monday, given a second dose of the sleeping drug that makes her forget the events of Monday only, and awakened and interviewed again on Tuesday. The experiment then ends on Tuesday, without flipping the coin again. Beauty wakes up in the experiment and is asked, "With what subjective probability do you believe that the coin landed tails?"

People argue about whether the "correct answer" to this question should be 1/3 or 1/2. Some say that the question is malformed, and needs to be rewritten as a decision theory question. Another view is that the question actually depends on the coin flip:

If the coin flip is a indexical coin flip, then there are effectively ...
Futurism's Track Record
by lukeprog — 2014-01-29T20:27:24.738Z

It would be nice (and expensive) to get a systematic survey on this, but my impressions [1] after tracking down lots of past technology predictions, and reading histories of technological speculation and invention, and reading about “elite common sense” at various times in the past, are that:

Elite common sense at a given time almost always massively underestimates what will be technologically feasible in the future.

“Futurists” in history tend to be far more accurate about what will be technologically feasible (when they don’t grossly violate known physics), but they are often too optimistic about timelines, and (like everyone else) show little ability to predict (1) the long-term social consequences of future technologies, or (2) the details of which (technologically feasible; successfully prototyped) things will make commercial sense, or be popular products.

Naturally, as someone who thinks it’s incredibly important to predict the long-term future as well as we can while also avoiding overconfidence, I try to put myself in a position to learn what past futurists were doing right, and what they were doing wrong. For example, I recommend: Be a fox not a hedgehog. Do calibration training. Know how your brain works. Build quantitative models even if you don’t believe the outputs, so that specific pieces of the model are easier to attack and update. Have broad confidence intervals over the timing of innovations. Remember to forecast future developments by looking at trends in many inputs to innovation, not just the “calendar years” input. Use model combination. Study history and learn from it. Etc.

Anyway: do others who have studied the history of futurism, elite common sense, innovation, etc. have different impressions about futurism’s track record? And, anybody want to do a PhD thesis examining futurism’s track record? Or on some piece of it, ala this or this or this? :)

I should explain one additional piece of reasoning which contributes to my impressions on the matter. How do I think about futurist predictions of technologies that haven’t yet been definitely demonstrated to be technologically feasible or infeasible? For these, I try to use something like the truth-tracking fields proxy. E.g. very few intellectual elites (outside Turing, von Neumann, Good, etc.) in 1955 thought AGI

...
Recommendation request: Budgeting/accounting software for a non-profit
by XFrequentist — 2014-01-29T15:16:01.984Z

I'm currently working for a non-profit, and a colleague recently stated that there isn't a good software solution for non-profit accounting.

This seems unlikely. Any recommendations?

I know that's a pretty generic description, but it seems like requirements would be quite similar across US non-profits. I can seek out more info if needed.
Tricky Bets and Truth-Tracking Fields
by lukeprog — 2014-01-29T08:52:38.889Z

While visiting Oxford for MIRI’s November 2013 workshop, I had the pleasure of visiting a meeting of “Skeptics in the Pub” in the delightfully British-sounding town of High Wycombe in Buckinghamshire. (Say that aloud in a British accent and try not to grin; I dare you!)

I presented a mildly drunk intro to applied rationality, followed by a 2-hour Q&A that, naturally, wandered into the subject of why AI will inevitably eat the Earth. I must have been fairly compelling despite the beer, because at one point I noticed the bartenders were leaning uncomfortably over one end of the bar in order to hear me, ignoring thirsty customers at the other end.

Anyhoo, at one point I was talking about the role of formal knowledge in applied rationality, so I explained Solomonoff’s lightsaber and why it made me think the wave function never collapses.

Someone — I can’t recall who; let’s say “Bob” — wisely asked, “But if quantum interpretations all predict the same observations, what does it mean for you to say the wave function never collapses? What do you anticipate?” [1]

Now, I don’t actually know whether the usual proposals for experimental tests of collapse make sense, so instead I answered:

Well, I think theoretical physics is truth-tracking enough that it eventually converges toward true theories, so one thing I anticipate as a result of favoring a no-collapse view is that a significantly greater fraction of physicists will reject collapse in 20 years, compared to today.

Had Bob and I wanted to bet on whether the wave function collapses or not, that would have been an awfully tricky bet to settle. But if we roughly agree on the truth-trackingness of physics as a field, then we can use the consensus of physicists a decade or two from now as a proxy for physical truth, and bet on that instead.

This won’t work for some fields. For example, philosophy sometimes looks more like a random walk than a truth-tracking inquiry — or, more charitably, it tracks truth on the scale of centuries rather than decades. For example, did you know that one year after the cover of TIME asked “Is God dead?”, a philosopher named Alvin Plantinga launched a renaissance in Christian philosophy, such that theism and Christian particularism were more commonly defended by analytic philosophers in th...
Personal Psychiatric Analysis
by pscheyer — 2014-01-29T06:02:38.134Z

Imagine reading about the following result buried in a prestigious journal:

We administered [Drug X] to 10,000 patients 80+ years of age selected to be a statistical representation of the populace. None had exhibited any prior medical history to suggest unusual conditions, outside of the normal range of issues collected over a lifetime. 1/3 of the patients were selected as a control group, and the others were entered into a longitudinal study of [Drug X] in which they were given varying doses over a 30 year timespan. [Please read charitably and flesh this out to be a good, well run longitudinal study by your personal standards. The important thing is the number of patients involved.]

Of the patients administered [drugx] 1x/month for 10 years, we found that there was an increase of average lifespan by 1 year compared to normal actuarial tables. We are unsure of the cause of this. We also had one patient who has yet to die after 30 years and shows no signs of aging. Our drug has effectively demonstrated its properties as a medication designed to reduce cholesterol and will proceed to be approved for normal prescription.

Now, personally, reading this I would be completely uninterested in the normal result and fascinated by the one, crazy, outlier. Living to the age of 110 is abnormal enough that within 6,666 people selected as a statistical representation of the population, it is extremely unlikely that anyone would live that long, much less continue performing at the apparent health of an 80 year old.

How small would the sample size have to be before you would consider trying the drug yourself, just to see if you, too, lived forever as long as you took it? What adverse effects and hassles would you go through to try it? Would these factors interact to influence your decision (Mild headaches and a pill 4x/day in exchange for maybe apparent eternal life? Sign me up!)

This example is an oversimplification to make a point- often in clinical trials there are odd outliers in the results. Patients who went into full remission, or had a full recovery, or were cured of schizophrenia completely.

In the example above, if the sample size had been 10 people, 9 of whom had no adverse effects and one who lived forever, I would take it. I have been known to try nootropics with little or no proven effect, because there are outliers ...
Self-Study Questions Thread
by TylerJay — 2014-01-29T01:32:17.379Z

There are a lot of autodidacts on LessWrong and many of us hold scholarship to be a virtue. Learning things on your own can be tricky for a number of reasons, but there is a lot of good material on LessWrong on how to do it efficiently. We know that it's usually best to build small skills in the right order, but sometimes it's difficult to figure out just what that order is, especially across disciplines. University programs have strict prerequisites that are supposed to ensure that you can handle a course before you take it, but taking a college course catalog's prerequisites to be absolute is an easy way to get demotivated before you learn anything you're actually interested in.

I've seen a lot of people run into trouble with self-study when they really want to study topic x, but know that they have to study topic y first as a prerequisite. (For example, I once saw someone on LW who wanted to start learning programming, but thought he couldn't get started without learning a bunch of math first.) For progress to be by accumulation and not by random walk, it is sometimes necessary to do this, but it can be hard to tell.

I wanted to create a thread where would-be autodidacts with limited knowledge of their target domain can ask these kinds of questions so that they can better plan their courses of study. Whether it's questions about specific texts, recommended prerequisites, MOOCs, course overlap, whatever.

Any general discussion about self-study is welcome too.
What was that article named?
by Adam Zerner — 2014-01-28T02:31:15.011Z

I'm recalling a Less Wrong post about how rationality only leads to winning if you "have enough of it". Like if you're "90% rational", you'll often "lose" to someone who's only "10% rational". I can't find it. Does anyone know what I'm talking about, and if so can you link to it?

I'm not sure if this is appropriate to ask. If not, let me know.
New website contents
by KatjaGrace — 2014-01-28T01:39:35.000Z

Rob and Federico of GWWC have made a nice summary of metacharities, which might be useful to those of you interested in cause prioritization as a cause. For future reference, I’ve put in my online collection of useful things (just for useful things which need help with their web presence).

I’ve also put up a few ‘structured cases’ for various claims there. These are intended to be well organized collections of arguments and considerations regarding a particular question, such as ‘should I invest in opening US borders?’. The claims of interest are mostly about how resources can be used to make the world better. The arguments are in progress, and probably not in particularly good formats, nor complete. However I talk to people about these often, so it is good to have them online to point to.

I’ve also put up some puzzles and other things.
Are you a virtue ethicist at heart?
by Shmi — 2014-01-27T22:20:25.189Z

Disclaimer: I am not a philosopher, so this post will likely seem amateurish to the subject matter experts.

LW is big on consequentialism, utilitarianism and other quantifiable ethics one can potentially program into a computer to make it provably friendly. However, I posit that most of us intuitively use virtue ethics, and not deontology or consequentialism. In other words, when judging one's actions we intuitively value the person's motivations over the rules they follow or the consequences of said actions. We may reevaluate our judgment later, based on laws and/or actual or expected usefulness, but the initial impulse still remains, even if overridden. To quote Casimir de Montrond, "Mistrust first impulses; they are nearly always good" (the quote is usually misattributed to Talleyrand).

Some examples:

Eliezer in a facebook post linked the article When Doing Good Means You’re Bad, which points out that people taking commission to raise a lot of money for charity are commonly considered less moral than those who raise much less but are not paid to do so ("tainted altruism").

This was brought up at a meetup: a pregnant woman in a dire financial situation who decides to have an abortion because she does not want a burden of raising a baby is judged harsher than a woman in a similar situation whose motivation is to avoid inflicting harsh life on the prospective child.

In real-life trolley problems even the committed utilitarians (like commanders during war time) are likely to hesitate before sacrificing lives to save more.

I am not sure how to classify religious fanaticism (or other bigotry), but it seems to require a heavy dose of virtue ethics (feeling righteous), in addition to following the (deontological) tenets of whichever belief, with some consequentialism (for the greater good) mixed in.

When I try to introspect my own moral decisions (like whether to tell the truth, or to cheat on a test, or to drive over the speed limit), I can usually find a grain of virtue ethics inside. It might be followed or overridden, sometimes habitually, but it is always there. Can you?
[Link] Death, long lives, uploading - a conworlding perspective
by Vulture — 2014-01-27T16:04:33.876Z

Death, long lives, uploading

Mark Rosenfelder (aka zompist, of language construction kit fame) writes about the advantages and drawbacks of mortality and its alternatives, in fiction and real life. Rosenfelder, as an author, clearly takes Fun Theory very seriously. After discussing the mental and physical decline that age usually entail, he assumes that the most difficult to surmount of these problems will be the loss of mental flexibility and tolerance of novelty. He then uses this obstacle to offer interesting fun-theoretic arguments against uploading and cryonics:

One futuristic approach to the problem: get yourself uploaded to a computer, so you can stay alive indefinitely. I think it’d be horrible to give up food, sex, exercise, and the rest of our bodily experience, even if we posit that you can still somehow retain your visual qualia. But I can see the attraction of wanting to find out what’s next. Perhaps you could hibernate for fifty years at a time, then wake up and avidly consume all the pop culture that’s been created since last time. Avoid Sturgeon’s Law and read just the best 10% of stuff, forever!

However, I suspect the plan would fall apart in under 200 years. How much really grabs us from that long ago? We do read stuff that old, of course, but it’s only a tiny fraction of our mental diet. The past is a strange world that takes some effort to immerse ourselves in– when it doesn’t repel us with a mindset that’s now confusing, boring, or vile. 400 years ago is even harder to grok, and 1000 is an alien world. And looking back, I’d maintain, is far easier than looking forward. We’re exposed to the past as history and literature– we can read Jane Austen or Jonathan Swift or Molière far easier than they’d be able to understand us.

Imagine Jules Verne, for instance, trying to make sense of a Laundry novel. The prose itself might not be too difficult. The idea of monsters and government bureaucracies would be understood. But he’d miss the allusions to Lovecraft and spy novels, and references to the Cold War and computers would require a whole education to follow. Something like an episode of The Simpsons would probably produce complete befuddlement.

I’m no

...
Meetup : Meetup : London - Paranoid Debating 2nd Feb, plus social 9th Feb
by philh — 2014-01-27T15:01:16.132Z

Discussion article for the meetup : Meetup : London - Paranoid Debating 2nd Feb, plus social 9th Feb

WHEN:
02 February 2014 02:00:00PM (+0000)

WHERE:
Shakespeare's Head, Holborn, WC2B 6BG

The next London meetup will be a session of paranoid debating. Trust no one.

The venue will be the usual Shakespeare's head, and the time will be the usual 2pm until we get bored, after which we will probably be talking.

In addition! There will be a purely-social meetup on the 9th, at the same time and place. And the next not-purely-social meetup is likely to be the 16th, but if it's not, there will be a purely-social one then as well. We're doing weekly meetups, but to avoid spamming /r/discussion, we're currently posting only the not-purely-social meetups to LW.

If you have difficulty finding us, you can reach me on 07792009646.

We also have a Google group which you might find worth joining.

Discussion article for the meetup : Meetup : London - Paranoid Debating 2nd Feb, plus social 9th Feb