trentmkelly/LessWrong-43k huggingface dataset:
The Featherless Biped
by Annoyance — 2009-09-02T17:47:00.573Z

The classical understanding of categories centers on necessary and sufficient properties. If a thing has X, Y, and Z, we say that it belongs to class A; if it lacks them, we say that it does not. This is the model of how humans construct and recognize categories that philosophers have held since the days of Aristotle.

Cognitive scientists found that the reality isn't that simple.

Human categorization is not a neat and precise process. When asked to explain the necessary features of, say, a bird, people cannot. When confronted with collections of stimuli and asked to determine which represent examples of 'birds', people find it easy to accept or reject things that have all or none of the properties they associate with that concept; when shown entities that share some but not all of the critical properties, people spend much more time trying to decide, and their decisions are tentative. Their responses simply aren't compatible with binary models.

Concepts are associational structures. They do not divide the world clearly into two parts. Not all of their features are logically necessary. The recognition of features produces an activation, the strength of which depends not only on the degree to which the feature is present but a weighting factor. When the sum of the activations crosses a threshold, the concept becomes active and the stimulus is said to belong to that category. The stronger the total activation, the more clearly the stimulus can be said to embody the concept.

Does this sound familiar? It should for us - we have the benefit of hindsight. We can recognize that pattern - it's how neural networks function. Or to put it another way, it's how neurons work.

But wait! There's more!

Try applying that model to virtually every empirical fact we've acquired regarding how people produce their conclusions. For example, our beliefs about how seriously we should take a hypothetical problem scenario depend not on a rigorous statistical analysis, but a combination of how vividly we feel about the scenario and how frequently it appears in our memory. People are convinced not only by the logical structure of an argument but the traits of the entities presenting it and the specific way in which the arguments are made. And so on, and so forth.

Most human behavior deriv...
Decision theory: Why we need to reduce “could”, “would”, “should”
by AnnaSalamon — 2009-09-02T09:23:34.936Z

(This is the second post in a planned sequence.)

Let’s say you’re building an artificial intelligence named Bob. You’d like Bob to sally forth and win many utilons on your behalf. How should you build him? More specifically, should you build Bob to have a world-model in which there are many different actions he “could” take, each of which “would” give him particular expected results? (Note that e.g. evolution, rivers, and thermostats do not have explicit “could”/“would”/“should” models in this sense -- and while evolution, rivers, and thermostats are all varying degrees of stupid, they all still accomplish specific sorts of world-changes. One might imagine more powerful agents that also simply take useful actions, without claimed “could”s and “woulds”.)
My aim in this post is simply to draw attention to “could”, “would”, and “should”, as concepts folk intuition fails to understand, but that seem nevertheless to do something important for real-world agents. If we want to build Bob, we may well need to figure out what the concepts “could” and “would” can do for him.*

Introducing Could/Would/Should agents:
Let a Could/Would/Should Algorithm, or CSA for short, be any algorithm that chooses its actions by considering a list of alternatives, estimating the payoff it “would” get “if” it took each given action, and choosing the action from which it expects highest payoff.
That is: let us say that to specify a CSA, we need to specify:

A list of alternatives a_1, a_2, ..., a_n that are primitively labeled as actions it “could” take;

For each alternative a_1 through a_n, an expected payoff U(a_i) that is labeled as what “would” happen if the CSA takes that alternative.

To be a CSA, the algorithm must then search through the payoffs for each action, and must then trigger the agent to actually take the action a_i for which its labeled U(a_i) is maximal.

Note that we can, by this definition of “CSA”, create a CSA around any made-up list of “alternative actions” and of corresponding “expected payoffs”.

The puzzle is that CSAs are common enough to suggest that they’re useful -- but i...
The puzzle continues
by KatjaGrace — 2009-09-02T09:00:47.000Z

A puzzle from ages ago:

What do these things have in common? Nerves, emotions, morality, prices.

They all send signals from distant parts of a coordinating system to a part which makes decisions. The signals are not just information, but costs and benefits to the decision maker so that the decision maker’s interests align with those of the whole. This allows cooperation of larger systems in space and time.

Nerves mean that damage to your toes translates to pain in your mind. This makes your mind act in the interest of your toes (which is of course in the interest of your mind eventually). If your foot is numb your mind is not taking into account the costs and benefits your foot faces, so eventually your foot often becomes injured. Nerves allow larger bodies to coordinate.

Emotions sometimes mean that failure or success of my future self translates to positive or negative feelings now. This makes my current self act in the interests of my future self. If something bad might happen I am scared. If my long term prospects look good I am happy. If your emotions are numb you can make decisions that are bad for your long term wellbeing. Some emotions allow temporally longer humans to coordinate.

Morality means that costs or benefits I cause to others lead to harm or good for me, either in the currency of moral feelings or terms in my calculated decisions (I make no claims here about how people do morality). This is the source of altruism, and of the complaints that it isn’t really altruism. If I donate money to charity I feel good (or calculatedly note that I have increased utility). If I hurt you I feel guilty. If your morality is numb you can hurt other people. Morality allows larger groups of people to coordinate.

Prices are the celebrated example; they mean that the costs and benefits to others across the economy feed into mine when I make choices that affect others. This makes me act efficiently if all goes well. I leave my house if someone else wants it more than I and eat foods that are easier for someone across the world to make. If the farming industry is numb nobody knows when they are doing harm or good regarding it, nor care so much, and we cause dead weight losses. Prices allow even larger groups of people to coordinate.

Can you think of more?
Knowing What You Know
by  — 2009-09-02T05:24:11.734Z

From Kahneman and Tversky:

"A person is said to employ the availability heuristic whenever he estimates frequency or probability by the ease with which instances or associations could be brought to mind"

I doubt that's news to any LessWrong readers - the availability heuristic isn't exactly cutting edge psychology. But the degree to which our minds rely on mental availability goes far beyond estimating probabilities. In a sense, every conscious thought is determined by how available it is - by whether it pops into our heads or not. And it's not something we even have the illusion of control over - if we knew where we were going, we'd already be there. (If you spend time actually looking directly at how thoughts proceed through your head, the idea of free will becomes more and more unrealistic. But I digress). What does and doesn't come to mind has an enormous impact on our mental functioning - our higher brain functions can only process what enters our working memory.

Whether it's called salience, availability, or vividness, marking certain things important relative to other things is key to the proper functioning of our brains. Schizophrenia (specifically psychosis) has been described as "a state of aberrant salience", where the brain incorrectly assigns importance to what it processes. And a quick perusal of the list of cognitive biases reveals a large number directly tied to mental availability - there's the obvious availability heuristic, but there's also the simulation heuristic (a special case of the availability heuristic), base-rate neglect (abstract probabilities aren't salient, so aren't taken into account), hyperbolic discounting (the present is more salient than the future), the conjunction fallacy (ornate, specific descriptions make something less likely but more salient), the primacy/recency bias, the false consensus effect, the halo effect, projection bias, etc etc etc. Even consciousness seems to be based on things hitting a certain availability threshold and entering working memory. And it's not particularly surprising that A) we tend to process only what we mark as important and B) our marking system is flawed. Our minds are efficient, not perfect - a "good enough" solution to the problem of finding food and avoiding lions.

That doesn't mean that availability isn't a complex system. It doesn't seem to be a fixed number

...
LW/OB Quotes - Fall 2009
by thomblake — 2009-09-01T15:11:01.113Z

This is a monthly thread for posting any interesting rationality-related quotes you've seen on LW/OB.

Please post all quotes separately (so that they can be voted up/down separately) unless they are strongly related/ordered.

Do not quote yourself.

Do not post quotes that are NOT comments/posts on LW/OB - there is a separate thread for this.

No more than 5 quotes per person per thread, please.

"this thread is insanely incestuous" - Z_M_Davis
Rationality Quotes - September 2009
by thomblake — 2009-09-01T15:06:57.167Z

A monthly thread for posting any interesting rationality-related quotes you've seen recently on the Internet, or had stored in your quotesfile for ages.

Please post all quotes separately (so that they can be voted up/down separately) unless they are strongly related/ordered.

Do not quote yourself.

Do not quote comments/posts on LW/OB - there is a separate thread for it.

No more than 5 quotes per person per monthly thread, please.

"A witty saying proves nothing." -- Voltaire
Do fetching models stave off shallowness?
by KatjaGrace — 2009-09-01T06:00:53.000Z

Prevalent advertising portraying an inaccurate proportion of humanity as too attractive is blamed for misinforming people on the value of attractiveness.

Unattractive females, both self rated and judged externally, find features of male physical attractiveness, such as facial masculinity and voice, less appealing than attractive females do.

This study suggests such behavior is to avoid wasting effort on guys who won’t be interested. That hypothesis beats the others because women’s preferences adapt fast to circumstances. In the study women saw pictures of other women who were more or less attractive. As their own perceived attractiveness went down or up accordingly, their preferences for male facial masculinity did too.

Do physically unattractive people actually believe one another to be hot? This study suggests not:

When less attractive people accept less attractive

dates, do they persuade themselves that the people they

choose to date are more physically attractive than others

perceive them to be? Our analysis of data from the popular

Web site HOTorNOT.com suggests that this is not the

case: Less attractive people do not delude themselves into

thinking that their dates are more physically attractive

than others perceive them to be.

When less attractive people accept less attractive dates, do they persuade themselves that the people they choose to date are more physically attractive than others perceive them to be? Our analysis of data from the popular Web site HOTorNOT.com suggests that this is not the case…

Instead, same study suggests that less attractive people claim to put greater weight on other characteristics than attractiveness in mate choice:

..participants’ own attractiveness was significantly correlated with their standardized weights for physical attractiveness (r 5 .60, prep 5 .98), but negatively correlated with their standardized weights for sense of humor (r 5 .44, prep 5 .91). Overall, these results suggest that more attractive people and less attractive people consider different criteria in date selection: Less attractive people tend to place less weight on physical attractiveness and greater weight on non-attractiveness-related attributes such as sense of humor.

Together these pieces of research suggests that encouraging everyone to feel less attractive should decrease the adoration people (at least claim to) direct at ...
Optimal Strategies for Reducing Existential Risk
by FrankAdamek — 2009-08-31T15:52:53.822Z

I've been talking to a variety of people about this recently, and it was suggested that people (including myself) might benefit from a LessWrong discussion on the topic. I've been thinking about it on my own for a year, which took me through Neuroscience, Computer Science, and International Security Policy. I'm hoping and finding that through discussion, a much greater variety of options can be proposed and considered, and those with particular experience or observations can have others benefit from their knowledge. I've been very happy to find there are a number of people seriously working towards this already (still far fewer than we might need), and their deliberations and learning would be particularly valuable.

This is primarily about careers and other long term focused efforts (academic research and writing on the side, etc), not smaller incremental tools such as motivation and akrasia discussions. Where you should be applying your efforts, now how (much). Unless there's a lot of interest, it might also be good to otherwise avoid discussions on self-improvement in general and how to best realize these long term concerns, bringing those up elsewhere or in a seperate post.

A few initial thoughts:

What's the marginal value of one's own contribution? How much less would the "next person" be able to accomplish, i.e. the person who would get that job if you didn't?

What are efforts that provide compounding benefit over time, rather than a constant static contribution? Most things are likely to provide some compounding benefit, but where is this most significant?

We may not need or desire to do a lot of discussion on this, but it's worth keeping in mind that as time goes on and we learn more, collectively and individually, charitable donations could become much more effective. This suggests that it may be best save money until better applications are discovered, to be balanced of course by the compounding effects earlier donations can have. See Alan Dawrst's paper on the value of making money, which besides being a worthwhile read has a point about this in the 3rd note. If others know of more thorough treatments of this, please mention them.

What are the particular existential risks that have the largest expected marginal reduction for a given amount of effort? What dangers are particularly underserved, by society in general and by t

...
Great post on Reddit about accepting atheism
by cousin_it — 2009-08-30T20:56:22.927Z

Go see for yourself.

To reject heaven and accept atheism - is not merely about science, facts, beliefs, etc - it is about accepting the reality of all those who have died - being really dead. It is accepting the same reality about everyone you love NOW one day being - really dead. It is accepting the same reality about YOU one day.

The older you are, the more dear loved ones have passed away, the harder it will be to reject the notions of religion. To reject religion requires the re-mourning of everyone who you love who has died.

The complete post is quite long and just as good as this quote.
Argument Maps Improve Critical Thinking
by Johnicholas — 2009-08-30T17:34:09.150Z

Charles R. Twardy provides evidence that a course in argrument mapping, using a particular software tool improves critical thinking. The improvement in critical thinking is measured by performance on a specific multiple choice test (California Critical Thinking Skills Test). This may not be the best way to measure rationality, but my point is that unlike almost everybody else, there was measurement and statistical improvement!

Also, his paper is the best, methodologically, that I've seen in the field of "individual rationality augmentation research".

To summarize my (clumsy) understanding of the activity of argument mapping:

One takes a real argument in natural language. (op-eds are a good source of short arguments, philosophy is a source of long arguments). Then elaborate it into a tree structure, with the main conclusion at the root of the tree. The tree has two kinds of nodes (it is a bipartite graph). The root conclusion is a "claim" node. Every claim node has approximately one sentence of english text associated. The children of a claim are "reasons", which do NOT have english text associated. The children of a reason are claims. Unless I am mistaken, the intended meaning of the connection from a claim's child (a reason) to the parent is implication, and the meaning of a reason is the conjunction of its children.

In elaborating the argument, it's often necessary to insert implicit claims. This should be done abiding by the "Principle of Charity", that you should interpret the argument in such a way as to make it the strongest argument possible.

There are two syntactic rules which can easily find flaws in argument maps:

The Rabbit Rule: Informally, "You can't conclude something about rabbits if you haven't been talking about rabbits". Formally, "Every meaningful term in the conclusion must appear at least once in every reason."

The Holding Hands Rule: Informally, "We can't be connected unless we're holding hands". Formally, "Every meaningful term in one premise of a reason must appear at least once in another premise of that reason, or in the conclusion".

I have tried the Rationale tool, and it seems afflicted with creeping featurism. My guess is the open-source tool Freemind could support argument mapping as described in Twardy's article, if the user is disciplined about it.

I'd love comments offering alternative rationality-improvement tool...
Cookies vs Existential Risk
by FrankAdamek — 2009-08-30T03:56:31.701Z

I've been thinking for a while now about the possible trade-offs between present recreation and small reductions in existential risk, and I've finally gotten around to a (consequentialist) utilitarian analysis.

ETA: Most of the similar mathematical treatments I've seen assume a sort of duty to unrealized people, such as Bostrom's "Astronomical Waste" paper. In addition to avoiding that assumption, my aim was to provide a more general formula for someone to use, in which they can enter differing beliefs and hypotheses. Lastly I include 3 examples using widely varying ideas, and explore the results.

Let's say that you've got a mind to make a batch of cookies. That action has a certain amount of utility, from the process itself and/or the delicious cookies. But it might lessen (or increase) the chances of you reducing existential risk, and hence affect the chance of existential disaster itself. Now if these cookies will help x-risk reduction efforts (networking!) and be enjoyable, the decision is an easy one. Same thing if they'll hurt your efforts and you hate making, eating, and giving away cookies. Any conflict arises when cookie making/eating is in opposition to x-risk reduction. If you were sufficiently egoist then risk of death would be comparable to existential disaster, and you should consider the two risks together. For readability I’ll refer simply to existential risk.

The question I'll attempt to answer is: what reduction in the probability of existential disaster makes refraining from an activity an equally good choice in terms of expected utility? If you think that by refraining and doing something else you would reduce the risk at least that much, then rationally you should pursue the alternative. If refraining would cut risk by less than this value, then head to the kitchen.

*ASSUMPTIONS: For simplicity I'll treats existential disaster as an abstract singular event, which we’ll survive or not. If we do, it is assumed that we do so in a way such that there are no further x-risks. Further I'll assume the utility realized past that point is not dependent on the cookie-making decision in question, and that the utility realized before that point is not dependent on whether existential disaster will occur. The utility calculation is also unbounded, being easier to specify. It is hoped that those not seeking to approximate having such ...
Some counterevidence for human sociobiology
by taw — 2009-08-29T02:08:10.855Z

I love seeing counter-evidence for everything. I estimate that while most of my beliefs are true (otherwise I wouldn't believe them in the first place), a small percentage is almost certainly completely false - and I don't really have any reliable way of telling the two apart.

Indiscriminatingly looking for counter-evidence for all of them can be very rewarding - the ones that are true are much more likely to sustain the assault of it than the ones that aren't. Yes, I might ignore counter-evidence of something that's false, or accept it for something that's true, ending up worse off, but it seems plausible that on average it should improve quality of my beliefs.

For example some of the standard beliefs about human sociobiology that seemed to be extremely widely held here are:

Men have lower chances of having any kids than women

Richer people, especially men, are more likely to have kids, and have more kids

Charting Parenthood: Statistical Portrait of Fathers and Mothers in America disagrees with them.

It's true that young men are less likely to have children than young women, but it reverses at old age, and total chance of having children during lifetime is - for people over 45 - 84% for men, and 86% for women. As some of childless men might still have children between 45 and their death (quite a few according to data), but almost no woman will, the difference must get smaller by the time of death, or it might even reverse. This is pretty convincing evidence against a major gender difference in chance of having children, at least as far as modern America is concerned.

The chance of having children is highest for people between 100% and 200% of poverty line (poverty line, not median income, these are all poorer than average people). For women going either lower or higher reduces chances of having children considerably. For men getting poorer reduces chance of having children considerably, while getting richer reduces it but only slightly. However - younger people are much more likely to fall below poverty line, and men tend to reproduce later, so even that can easily be an artifact of age-income relationship. The data is fully compatible with both poverty and wealth being negatively correlated with chance of having children in both genders.

These are not direct tests of sociobiological claims, so what we have is not exactly what we would like to, but I fin...