trentmkelly/LessWrong-43k huggingface dataset:
The AI-box for hunter-gatherers
by Alex Flint — 2011-04-02T12:09:42.602Z

The following is a minor curiosity that occurred to me regarding real-world analogies to the AI-box concept.

Fundamentally, the reason that we fear a randomly-chosen super-intelligent AI is twofold:

It would be smarter than us, so it could outwit us no matter what its goals.

We have no reason to expect its goals to exactly coincide with ours, so we expect its actions to be detrimental to us.

Now, let's say you're playing a game with a wide range of outcomes, over which you have a well-defined utility function. Let's say you can choose a partner from some pool of potential partners, each with a random utility function. Let's say the intelligence of each player is known to you. It would be unwise to choose a player more intelligent than yourself because

They could outwit you.

You have no reason to expect their goals to coincide with yours.

On the other hand, if you pick a less intelligent player then perhaps you could trick them into furthering your own goals, or at least ascertain whether their motives coincide with yours. At the very least you would be able to keep them from subverting your own goal-directed actions.

I conjecture that this explains why people sometimes fear those more intelligent than themselves, and also why people sometimes act dumb. Imagine a hunter-gatherer group considering inviting an outsider to join them. If the outsider's motives are uncertain then the more intelligent the outsider, the less well-advised the group would be to let em in.

In fact, a situation analogous to an AI-box could arise:

Group member 1: We should let this intelligent outsider in, but we should keep tabs on them should they act against us.

Group member 2: But the fact that they're more intelligent than us means that you *can't* expect to keep tabs on them.

Group member 1: But couldn't we set a test of their motives and only allow them in if they proves to have motives coinciding with ours?

Group member 2: No matter what test you set, we can expect them to outwit us -- that's what intelligence means.
I want a better memory.
by Alex Flint — 2011-04-02T11:36:01.147Z

I suspect that forgetfulness is the single largest hindrance to me improving my rationality. This isn't something I've seen others report on LessWrong, so I'm suspicious that I'm in some kind of self-serving spiral, or that I'm doing something obvious wrong. So, I'm seeking feedback on (a) whether the above statement is true -- whether forgetfulness is likely to really be a dominant hindrance; (b) what I can do about it; and (c) why others haven't reported this.

Ways that I suspect forgetfulness harms me:

I forget past experiences that would otherwise allow me to observe correlations and extrapolate time-series style. Did my mood improve last time I called this friend? Was I more productive during that period where I got less sleep?

I fail to recall evidence that would be salient when evaluating probabilities. This includes evidence relevant not only to coffee-time discussions on abstract topics, but also questions that occur to me about akrasia and the like.

I can't remember names of authors and papers that could be relevant to ideas that arise during my research. Searching online for papers is fine for major literature reviews, but enormously expensive to evaluate hour-by-hour conjectures that occur to me.

When I encounter the same problem multiple times, I forget how I solved the problem last time.

When a new productivity/social/rationality strategy *does* work, I forget to keep using it. This isn't only about laziness: sometimes I actually wonder explicitly whether strategy X worked, and I can't remember.

I forget names, faces, places, facts and figures. But I understand this one is quite common.

I forget all the ways that forgetfulness frustrates me.

Steps I've taken:

I keep an elaborate diary with appointments like "bring USB drive home from work" and "purchase bread en route to Sam's house".

I keep an elaborate e-notebook where I try to record my "brain state" so that I can more quickly pick up where I left off with my work.

Every time I solve a technical problem, I write it down.

I use a memory app called mnemosyne to memorize foreign language words, names of jitsu techniques, etc.

The write-everything-down strategy has helped some, but it's *orders of magnitude less effective* than recalling stuff right from my brain. It's like replacing a CPU's L1 cache with a magnetic hard drive and expecting performance not to drop.
[LINK] What’s The Most Difficult CEO Skill? Managing Your Own Psychology
by David_Gerard — 2011-04-02T10:22:27.886Z

The general application to life - ultimately, you are the person responsible for turning the crank that makes your world go round - is obvious. His methods to deal with it will ring true with LW readers.

By far the most difficult skill for me to learn as CEO was the ability to manage my own psychology. Organizational design, process design, metrics, hiring and firing were all relatively straightforward skills to master compared to keeping my mind in check. Over the years, I’ve spoken to hundreds of CEOs all with the same experience. Nonetheless, very few people talk about it and I have never read anything on the topic. It’s like the fight club of management: The first rule of the CEO psychological meltdown is don’t talk about the psychological meltdown.

[...]

This means that you will face a broad set of things that you don’t know how to do that require skills that you don’t have. Nevertheless, everybody will expect you to know how to do them, because, well, you are the CEO.

Ben Horowitz. "What’s The Most Difficult CEO Skill? Managing Your Own Psychology." Ben's Blog, 2001-04-01.
Q: What has Rationality Done for You?
by atucker — 2011-04-02T04:13:34.789Z

So after reading SarahC's latest post I noticed that she's gotten a lot out of rationality.
More importantly, she got different things out of it than I have.
Off the top of my head, I've learned...

that other people see themselves differently, and should be understood on their terms (mostly from here)

that I can pay attention to what I'm doing, and try to notice patterns to make intervention more effective.

the whole utilitarian structure of having a goal that you take actions to achieve, coupled with the idea of an optimization process. It was really helpful to me to realize that you can do whatever it takes to achieve something, not just what has been suggested.

the importance/usefulness of dissolving the question/how words work (especially great when combined with previous part)

that an event is evidence for something, not just what I think it can support

to pull people in, don't force them. Seriously that one is ridiculously useful. Thanks David Gerard.

that things don't happen unless something makes them happen.

that other people are smart and cool, and often have good advice

On top of becoming a little bit more effective at a lot of things, and with many fewer problems.

(I could post more on the consequences of this, but I'm going for a different point)

Where she got...

a habit of learning new skills

better time-management habits

an awesome community

more initiative

the idea that she can change the world

I've only recently making a habit out of trying new things, and that's been going really well for me. Is there other low hanging fruit that I'm missing?
What cool/important/useful things has rationality gotten you?
New Rationality Blog: 'Measure of Doubt'
by lukeprog — 2011-04-01T20:11:37.523Z

Measure of Doubt is a new rationality blog from Julia and Jesse Galef (a sister-brother team), aimed at a much broader audience than Less Wrong.

Julia is the author of the recently-linked article Are Intuitions Good Evidence? and many other writings. I know less about Jesse, but I can say a bit more about Julia. She co-hosts the Rationally Speaking podcast with philosopher-biologist Massimo Pigliucci. She writes quite a bit about becoming more rational, dissolving Big Questions, and wanting to change your mind. Her video on different definitions of 'rationality' distinguishes Hollywood rationality, epistemic rationality, and instrumental rationality. She explains the mirror paradox ala Gary Drescher in Good and Real. She blogged Harry Potter and the Methods of Rationality.

I suspect Less Wrong readers will enjoy Measure of Doubt, and will find it a useful place to send people who don't have the patience to read the Less Wrong Sequences and long posts about Bayesian updating and the relevance of universe-simulation arguments to decision theory.
The Good News of Situationist Psychology
by lukeprog — 2011-04-01T19:28:07.295Z

Part of the sequence: The Science of Winning at Life

In 1961, Stanley Milgram began his famous obedience experiments. He found that ordinary people would deliver (what they believed to be) excruciatingly painful electric shocks to another person if instructed to do so by an authority figure. Milgram claimed these results showed that in certain cases, people are more heavily influenced by their situation than by their internal character.

Fifty years and hundreds of studies later, this kind of situationism is widely accepted for broad domains of human action. People can inflict incredible cruelties upon each other in a prison simulation.b Hurried passersby step over a stricken person in their path, while unhurried passersby stop to help.a Willingness to help varies with the number of bystanders, and with proximity to a fragrant bakery or cofee shop.c The list goes on and on.d

Our inability to realize how powerful the effect situation has on human action is so well-known that it has a name. Our tendency to over-value trait-based explanations of others' behavior and under-value situation-based explanations of their behavior is called the fundamental attribution error (aka correspondence bias).

Recently, some have worried that this understanding undermines the traditional picture we have of ourselves as stable persons with robust characteristics. How can we trust others if their unpredictable situation may have so powerful an effect that it overwhelms the effect of their virtuous character traits?

But as I see it, situationist psychology is wonderful news, for it means we can change!

If situation has a powerful effect on behavior, then we have significant powers to improve our own behavior. It would be much worse to discover that our behavior was almost entirely determined by traits we were born with and cannot control.

For example, drug addicts can be more successful in beating addiction if they change their peer group - if they stop spending recreational time with other addicts, and spend time with drug-free people instead, or in a treatment environment.e

Improving rationality

What about improving your rationality? Situationist psychology suggests it may be wise to surround yourself with fellow rationalists. Having now been a visiting fellow with the Singularity Institute for only two days, I can already tell that almost everyo...
An Anchoring Experiment
by prase — 2011-04-01T14:19:03.444Z

The experiment is closed, for the results look here.

In recent discussion I have expressed an opinion that anchoring may, for some quantitative questions, cause the answer to lie further away from the correct value than the anchor itself. For concreteness, let's suppose that the correct value of a quantity Q is x, and the subject is asked whether Q is greater or lower than y, y > x. My hypothesis is that the anchor moves the subject's probability distribution up as a whole, including the part which already has been lying above y. Therefore the subjects will positively answer the question "Is Q > y ?" more often than their guess would exceed y if they were just asked to estimate the value of Q with no anchor given. One commenter apparently disagreed. I thought it may be interesting to resolve the disagreement experimentally. (More generally, I would like to see how well LW audience fights the standard biases, and if this experiment turns out successful - which means the number of respondents be greater than, say, five - I would think about posting more of this kind.)

How to participate:

The experiment has two parts.

First, toss a coin to decide whether you belong to the biased group I or the control group II for the first question. If you belong to the group I, look at a comment linked below, which will give you a question of form "is Q is greater or lower than y", where y is either significantly lower or significantly greater than the correct value of Q. The comment has a form of a typical LW poll. If you belong to the group II, look at different linked comment which asks "what is the value of Q", and then give your estimate in a subcomment there.

The second part is completely analogical to the first one, only with a different question. If you have participated in the first part within the group I, take part in the group II for the second part, and vice versa. Try to eliminate the irrelevant biases: switch on the anti-kibitzer before looking on the group I questions to avoid being influenced by the votes of others. Don't read the subcomments of the group II questions before writing down your own.

The hypothesis is that the percentage of the group I respondents answering incorrectly will be greater than the percentage of the group II respondents estimating on the incorrect side of the anchor.

First part: Question for the group I. Question for the group II.

Seco...
Singularity Institute featured on Philanthroper
by Louie — 2011-04-01T05:46:36.658Z

Singularity Institute is today's featured charity on Philanthroper.com

Philanthroper is a micro-giving site that profiles small charities hand-selected by their editors. Their site encourages donors to “give every day” by only requesting $1 contributions.

A group of Singularity Institute donors has stepped forward to match all donations given through Philanthroper today so I'd encourage each of you to give $1 now if you support Singularity Institute and have a US-based bank account (Philanthroper requirement). We'd like to have a healthy total raised by the end of the day. The fundraiser has already been featured on Gizmodo but please submit it to other news sites if you can.

I signed up and gave my $1.
Link: "Health Care Myth Busters: Is There a High Degree of Scientific Certainty in Modern Medicine?"
by CronoDAS — 2011-04-01T05:25:59.052Z

A feature in Scientific American magazine casts some light on the troubled state of modern medicine.

Health Care Myth Busters: Is There a High Degree of Scientific Certainty in Modern Medicine?

Short excerpt:

We could accurately say, "Half of what physicians do is wrong," or "Less than 20 percent of what physicians do has solid research to support it." Although these claims sound absurd, they are solidly supported by research that is largely agreed upon by experts.

Scientific American often gates its online articles after some time has passed, so I don't know how long it will be available.
Decision theory and partition dependence
by  — 2011-04-01T04:46:40.305Z

I'm trying to understand partition dependence in causal decision theories and I'm struggling to think of a case where an act (as opposed to simply the expected utility) is partition dependent. Some detail (very much in order of what I'm wanting to figure out):

1.) I known that Joyce's causal decision theory is partition-invariant but Sobel's and Lewis's theories aren't and require some specification of what partition is adequate. What happens if such a specification isn't provided? More specifically, what's an example of a decision problem where the acts are partition dependent if you don't ensure you use only adequate partitions?

2.) Extending this: If you do make sure to only use adequate partitions, are there still problems with partition-dependence (other than the small world/grand world problem that Joyce talks about)? In other words, do current definitions of adequate partions:

i.) Ensure that no act will be partition dependent in decision problems that can be discussed.

ii.) Allow all decision problems to be discussed.

I guess what I'm trying to figure out is what the problem of decision dependence is. Is the problem that it means you require a definition of adequate partitions (but that such a definition is easy to find and solves the problem)? Or even with such a definition, does partition dependence still cause problems? Are these problems just about small world/grand world stuff or are they about other partition related issues as well?

I can't seem to get my head around it and was hoping some concrete answers to my questions would help. Anyone able to help?
Dense Math Notation
by JK_Ravenclaw — 2011-04-01T03:37:02.357Z

I program, and am also presently working my way through some math books. I find that I often have to backtrack to look up pieces of notation like variables and operators. Unfortunately, this is very problematic. Greek and Latin letters give no indication of where they came from and are not usable search terms, even knowing the full context in which they appeared. Many authors have their own bits of idiosyncratic notation, often combinations of subscripting and line art generated by TeX macros. Since expanding equations out to their definitions is so difficult, I sometimes don't bother to investigate when one looks odd, which as you might expect leads to big trouble later when errors in understanding creep by.

This is not nearly as big a problem for me when reading code, however, because there every variable and nonstandard operator has a descriptive name wherever it's used, and documentation is never more than a few hotkeys away. The same thing could be done for math. Suppose you took a typical higher math book, and replaced every single-letter variable and operator with an appropriate identifier. For me, this would make it much more readable; I would gain a better understanding in less time. However, I don't know the effect size or how broadly this generalizes.

Do other people have this problem? Might this issue deter some people from studying math entirely? Has anyone tried the obvious controlled experiment? How about with an experimental group specifically of programmers?
Anyone read Erfworld?
by Raemon — 2011-04-01T01:48:37.732Z

Periodically people link to "rationalist" stories (or comics that are not really rational at all, that just happen to be vaguely related to AI), so I was a bit surprised to find not a single reference to Erfworld on a Lesswrong search.

Erfworld is a webcomic that essentially tells the same story as Harry Potter and the Methods of Rationality. Regular (but very smart) guy is transported to magical world. He systematically tries to understand the rules of that world and accomplishes all kinds of awesome stuff as a result. I feel like I've read the story a bunch of times, but HP:MoR and Erfworld are the only ones I recall that really did it justice on an epic scale.

While in many ways it's almost identical, it's playing with a different set of rules than Harry is, and the jokes and philosophical questions are playing around with different material. The main character is a strategy war gamer, and he finds himself summoned into a world that runs on Turn Based Strategy Rules. People can only take certain actions on their own "turn." Everyone has "stats", gain experience and level up. At first the whole thing seems like a silly gimmick, with the protagonist benefiting from genre-savviness. He starts out asking basic questions about the rules. Later on he starts challenging those rules - what is an actual law of the universe and what is merely convention that the inhabitants follow. Eventually he starts grappling with questions about how the morality of his old world plays into the morality of a world where everyone is tied ideologically to the side that they were created to serve, and people are not born - they pop into existence as adults as soon as their commander pays for them.

I think it does a better job of showing how a "real," "typical" smart person would try and understand a new, strange world. Harry is awesome, but he strains credibility in regards to how much he knows at the age he knows it.

Book 1 is done, and is a very solid, complete work that I recommend on its own. It features what is, to my recollection, the best use of the F-word in a work of fiction.

Book 2's still in progress. You can start reading here.
Link: Gizmodo discusses SIAI, matches donations
by Normal_Anomaly — 2011-03-31T20:35:50.058Z

Gizmodo, a popular technology blog, posted this artice about SIAI. It's partly tongue-in-cheek, but also apparently thinks well of the Singularity Institute, claiming they are "a research organization that's as forward-thinking as most Gizmodo readers (read: sci-fi nerds)." More importantly, they link to Philanthroper, where you can donate and see your donation be matched. File this under "cultural penetration of Singularity memes" and also as a chance to make your donation more effective.

http://gizmodo.com/#!5787599/give-1-to-stop-terminators-seriously

Edit: Better link to the above URL
Seattle Meetup Sun April 10th 5pm + new mailing list
by jsalvatier — 2011-03-31T20:29:25.878Z

The last Seattle meetup was fun, so we're having another! Same place and time:

Big Time Brewery

4133 University Way NE

Seattle, WA 98105

Sunday, April 10, 5 pm

google maps

The place is open to minors until 8 pm. We'll be there with a sign reading "LW".

There is now also a mailing list for arranging meetups and chatting to LW people in Seattle.

We will hopefully meet regularly in the future. If Sunday evenings do not work for you, say so in the comments or send an email to the mailing list.
Analyzing our discussions in other forums
by ArisKatsaris — 2011-03-31T19:22:53.421Z

I frequent several boards about several topics, mainly fandom-related ones. In the large majority of them, I've eventually felt the urgent need to inject some rationalist ideas into threads that seemed to be sorely needing them.

Even in the best of cases, those that don't devolve into flame wars and remain overall polite, I usually meet with limited success at best. So I wonder -- assuming my own experience is shared by others here -- should we perhaps share links for some of those threads, and discuss here how we could have handled the conversation better, possibly how we could have had more success injecting rationality than we actually did?

The most recent such discussion I had was today, in the forums of Gunnerkrigg Court. I made a post very early on in the thread in question, but my real contribution began here after another poster (who actually seemed science-minded in desire, if not in practice!) asked "What if the soul is the energy?" I had to cringe at that, and tried to explain how meaningless a statement that was -- but I don't seem to have succeeded much, even though I don't know how I could have done better.

Please say what you think about my idea of discussing our threads in other forums in general, and/or any ideas you have about how I could have handled better that specific thread in particular...
Examine success as much as failure
by gwern — 2011-03-31T17:29:23.350Z

Harvard Business Review has posted something right up our alley: "Why Leaders Don't Learn From Success"

Also, the HBR essay links to a similar discussion of how Pixar avoids being brainwashed by its own success (something I had always wondered about - they seem too consistently successful): "How Pixar Fosters Collective Creativity".