trentmkelly/LessWrong-43k huggingface dataset:
Decision Theories: A Semi-Formal Analysis, Part I
by orthonormal — 2012-03-24T16:01:33.295Z

Or: The Problem with Naive Decision Theory

Previously: Decision Theories: A Less Wrong Primer

Summary of Sequence: In the context of a tournament for computer programs, I give almost-explicit versions of causal, timeless, ambient, updateless, and several other decision theories. I explain the mathematical considerations that make decision theories tricky in general, and end with a bunch of links to the relevant recent research. This sequence is heavier on the math than the primer was, but is meant to be accessible to a fairly general audience. Understanding the basics of game theory (and Nash equilibria) will be essential. Knowing about things like Gödel numbering, quining and Löb's Theorem will help, but won't be required.

Summary of Post: I introduce a context in which we can avoid most of the usual tricky philosophical problems and formalize the decision theories of interest. Then I show the chief issue with what might be called "naive decision theory": the problem of spurious counterfactual reasoning. In future posts, we'll see how other decision theories get around that problem.

In my Decision Theory Primer, I gave an intuitive explanation of decision theories; now I'd like to give a technical explanation. The main difficulty is that in the real world, there are all sorts of complications that are extraneous to the core of decision theory. (I'll mention more of these in the last post, but an obvious one is that we can't be sure that our perception and memory match reality.)

In order to avoid such difficulties, I'll need to demonstrate decision theory in a completely artificial setting: a tournament among computer programs.

You're a computer programmer entering a tournament for spectacular stakes. But you won't be competing in person: instead, you'll be submitting code for a program to represent you, and that program will be competing one-on-one with other programs in a series of games.

You don't know what the specific games are, but the contest has specified some ground rules:

In each game, your program and its opponent will have to choose separately between several options. (There are independent sources of randomness available to each program, so mixed strategies are legal.)

The games are pure strategic conflicts: the expected payouts to each programmer depend on the outputs of both programs, and can be calculated simply if you know those outputs

...
A Primer On Risks From AI
by XiXiDu — 2012-03-24T14:32:42.166Z

The Power of Algorithms

Evolutionary processes are the most evident example of the power of simple algorithms [1][2][3][4][5].

The field of evolutionary biology gathered a vast amount of evidence [6] that established evolution as the process that explains the local decrease in entropy [7], the complexity of life.

Since it can be conclusively shown that all life is an effect of an evolutionary process it is implicit that everything we do not understand about living beings is also an effect of evolution.

We might not understand the nature of intelligence [8] and consciousness [9] but we do know that they are the result of an optimization process that is neither intelligent nor conscious.

Therefore we know that it is possible for an physical optimization process to culminate in the creation of more advanced processes that feature superior qualities.

One of these qualities is the human ability to observe and improve the optimization process that created us. The most obvious example being science [10].

Science can be thought of as civilization-level self-improvement method. It allows us to work together in a systematic and efficient way and accelerate the rate at which further improvements are made.

The Automation of Science

We know that optimization processes that can create improved versions of themselves are possible, even without an explicit understanding of their own workings, as exemplified by natural selection.

We know that optimization processes can lead to self-reinforcing improvements, as exemplified by the adaptation of the scientific method [11] as an improved evolutionary process and successor of natural selection.

Which raises questions about the continuation of this self-reinforcing feedback cycle and its possible implications.

One possibility is to automate science [12][13] and apply it to itself and its improvement.

But science is a tool and its bottleneck are its users. Humans, the biased [14] effect of the blind idiot god that is evolution.

Therefore the next logical step is to use science to figure out how to replace humans by a better version of themselves, artificial general intelligence.

Artificial general intelligence, that can recursively optimize itself [15], is the logical endpoint of various converging and self-reinforcing feedback cycles.

Risks from AI

Will we be able to build an artificial general intelligence? Yes, sooner or later.

Even the unin...
Is my name deceptive?
by katydee — 2012-03-24T10:43:26.526Z

So, my current username often reads as female. This is not intentional, and I am not female. Some on IRC have pointed out that this username could be considered misleading or deceptive. I have no real attachment to it, but I do have a few articles that I've been discussing and planning on posting here, and I think it would be confusing for others if I changed usernames, making it difficult for people to follow the genesis of certain ideas. However, I also don't want to mislead anybody.

Therefore, I pose the question to the community: is my current username deceptive or otherwise misleading, and should I change it?
[SEQ RERUN] Zombies! Zombies?
by MinibearRex — 2012-03-23T23:03:31.126Z

Today's post, Zombies! Zombies? was originally published on 04 April 2008. A summary (taken from the LW wiki):

Don't try to put your consciousness or your personal identity outside physics. Whatever makes you say "I think therefore I am", causes your lips to move; it is within the chains of cause and effect that produce our observed universe.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Reductive Reference, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Memory in the microtubules
by Richard_Kennaway — 2012-03-23T20:42:33.647Z

A recent article in PloS Computational Biology suggests that memory is encoded in the microtubules. "Signaling and encoding in MTs and other cytoskeletal structures offer rapid, robust solid-state information processing which may reflect a general code for MT-based memory and information processing within neurons and other eukaryotic cells."

They argue that synaptic connections are transient compared with the lifetime of memories, and therefore memories cannot be stored in them, but in some more persistent structure. The structure they suggest is the phosphorylation state of sites on microtubule lattices within neurons. And that's about as much of the technical detail as I feel able to summarise. It's not all speculation, they report technical work on the structures of these cellular components. Total memory capacity would be somewhere upwards of 10^20 bits (or in more everyday units, 10 million terabytes), depending on the encoding, of which they suggest several schemes.

Journalistic writeup here.

Note that Stuart Hameroff, one of the authors, is known for his proposals for microtubules as the mechanism of consciousness through quantum effects (and with Penrose, quantum gravitational effects). The present paper, however, is solely about memory and does not touch on quantum coherence or consciousness.
Meetup : Fort Collins, Colorado Meetup Wedneday 7pm
by EvelynM — 2012-03-23T16:37:03.163Z

Discussion article for the meetup : Fort Collins, Colorado Meetup Wedneday 7pm

WHEN:
28 March 2012 07:00:00PM (-0600)

WHERE:
144 North College Avenue, Fort Collins, CO 80524

It's spring! Come meet smart, interesting people.

Discussion article for the meetup : Fort Collins, Colorado Meetup Wedneday 7pm
Nonmindkilling open questions
by Scott Alexander — 2012-03-23T16:23:13.393Z

When I explain to people how beliefs should be expressed in probabilities, I would like to use an example like "Consider X. Lots of intelligent people believe X, but lots of equally intelligent people believe not-X. It would be ridiculous to say you are 100% sure either way, so even if you have a strong opinion about X, you should express your belief as a probability."
Trouble is, I'm having a hard time thinking of an example to plug into X. For an example to work, it would need the following properties:
Factual question. So no value-laden questions like "Is abortion morally acceptable?" or counterfactual questions like "Would the US have won a war with the Soviet Union in 1960?"
Popular and important question. The average person should be aware it's an issue and care about the answer. So no "My aunt's middle name is Gladys" or "P = NP."
High uncertainty. Reasonable people should be reluctant to give a probability >90% or <10%
No opportunity to gain status by signaling overwhelming support for one side. So cryonics is out, because it's too easy to say "That's stupid, I'm 100% sure cryonics won't work and no intelligent person could believe it." I'm assuming in any debate where you can gain free status by assigning crazy high probabilities to the "responsible" position, people will do just that - so no psi effects, Kennedy assassination, or anything in that cluster. I need a question with no well-accepted "responsible" position.
Minimal mindkilling effect. My previous go-to example has been global warming, but I keep encountering people who say that global warming 100% for sure exists and the only people who could possibly doubt it are oil company shills. Or if I were to try the existence of God, I predict half the population would say it's 100% certain God exists, and the other half would say the opposite.
So what are the important questions that average (or somewhat-above-average) people will likely agree are complicated open questions where both sides have good points? And if there aren't many such questions, what does that say about us?
Weekly LW Meetups: Atlanta, Brussels, Fort Collins, Ohio
by FrankAdamek — 2012-03-23T15:22:56.666Z

There are upcoming irregularly scheduled Less Wrong meetups in:

Brussels meetup: 17 March 2012 11:15AM
Atlanta: 17 March 2012 05:30PM
Fort Collins Meetup Saturday 17th: 17 March 2012 05:00PM
[Ohio/Washington DC] Interest in Reason Rally meetup?: 24 March 2012 04:14PM

The following meetups take place in cities with regularly scheduled meetups, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Austin, TX: 24 March 2012 01:30PM
Ohio Monthly: 17 March 2012 03:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Cambridge, MA, Cambridge UK, London, Madison WI, Melbourne, Mountain View, New York, Ohio, Ottawa, Oxford, Portland, Seattle, Toronto, Waterloo, and West Los Angeles.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview will continue to be posted on the front page every Friday. These will be an attempt to collect information on all the meetups happening in the next weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll now also have the benefit of having your meetup mentioned in a weekly overview. These overview posts will be moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing list in order to be notified when an irregular meetup is happening: Atlanta, Chicago, Helsinki, London, Marin CA, Pittsburgh, Salt Lake City, Southern California (Los Angeles/Orange County area), St. Louis, Vancouver, Washington, DC.

If your meetup has a mailing list that you'd like mentioned here, or has become regular and isn't listed as such, let me know!

Want to help out the common good? If one of the meetups listed as regular has become inactive, let me know so we can present more accurate information to newcomers.
Deadlines and AI theory
by Dmytry — 2012-03-23T10:57:39.679Z

The AI is a real-time algorithm - it has to respond to situation in the real time. The real-time systems have to trade time for accuracy, and/or face deadlines.

The straightforward utility maximization may look viable for multiple choice questions, but for write-in problems, such as technological innovation, the number of choices is so huge (1000 variables with 10 values each, 101000) , that the AI of any size - even galaxy spanning civilization of Dyson spheres - has to employ generative heuristics. Same goes for utility maximization in presence of 1000 unknowns that have 10 values each - if the values are to interact non-linearly, all the combinations, or a representative number thereof, have to be processed. There one has to trade accuracy of processing utility of a case for number of cases processed.

In general, the AIs of any size (excluding the possibility of unlimited computational power within finite time and space) will have to trade accuracy of it's adherence to it's goals, for time, and thus have to implement methods that have different goals, but are faster computationally, whenever those goals are reasoned to increase expected utility taking into consideration the time constraints.

Note that in a given time, the algorithm with lower big-O complexity is able to process dramatically larger N, and the gap increases with the time allocated (and with CPU power). For example, you can bubblesort number of items proportional to square root of the number of operations, but you can quicksort the number of items proportional to t/W(t) where W is the product-log function and t is the number of operations; this grows approximately linearly for large t. So for the situations where exhaustive search is not possible, gaps between implementations increases with extra computing power; the larger AIs benefit more from optimizing themselves.

The constraints get especially hairy when one is to think of massively parallel system that is operating with speed-of-light lag between the nodes, and where the time of retrieval is O(n1/3) .

This seems to be a big issue for FAI going FOOM. The FAI may, with perfectly friendly motives, abandon the proved-friendly goals for the simpler to evaluate, simpler to analyze goals that may (with 'good enough' confidence that needs not necessarily be >0.5) produce friendliness as instrumental, if that increases the expected utility given ...
Meetup : Brussels meetup
by Axel — 2012-03-23T10:50:34.865Z

Discussion article for the meetup : Brussels meetup

WHEN:
14 April 2012 12:00:00PM (+0200)

WHERE:
Museum of Natural Sciences Rue Vautier 29 B-1000 Brussels

During this meetup we will continue discussing our beliefs, using puzzles to improve creative thinking, and brainstorming about what makes an effective group. Should you arrive late we will have moved to the cafeteria, just go straight once you're past the entrance, you can't miss it. If you are in the neighborhood, consider dropping by. (getting there: http://www.naturalsciences.be/information/visitor/access)

Discussion article for the meetup : Brussels meetup
Open question on the certain 'hot' global issue of importance to FAI
by Dmytry — 2012-03-23T08:20:46.752Z

A question: why anything about global warming gets downvoted, even popularly readable explanation of the fairly mainstream scientific consensus? edit: Okay, this is loaded. I should put it more carefully: why is the warming discussion generally considered inappropriate here? That seems to be the case; and there are pretty good reasons for this. But why can't AGW debate be invoked as example controversy? The disagreement on AGW is pretty damn unproductive, and so it is a good example of argument where productivity may be improved.

The global warming is a pretty damn good reason to build FAI. It's quite seriously possible that we won't be able to do anything else about it. Even mildly superhuman intelligence, though, should be able to eat the problem for breakfast. Even practical sub-human AIs can massively help with the space based efforts to limit this issue (e.g. friendly space-worthy von Neumann machinery would allow to almost immediately solve the problem). We probably will still have extra CO2 in atmosphere, but that is overall probably not a bad thing - it is good for plants.

For that to be important it is sufficient to have 50/50 risk of global warming Even probabilities less than 0.5 for the 'strong' warning scenarios still are a big factor - in terms of 'expected deaths' and 'expected suffering' considering how many humans on this planet lack access to air conditioning. I frankly am surprised that the group of people fascinated with AI would have such a trouble with the warming controversy, as to make it too hot of a topic for an example of highly unproductive arguments.

I do understand that LW does not want political controversies. Politics is a mind killer. But this stuff matters. And I trust it has been explained here that non-scientists are best off not trying to second guess the science, but relying on the expert opinion. The global warming is our first example of the manmade problems which are going to kill us if there is no AI. The engineered diseases, the gray goo, that sort of stuff comes later, and will likely be equally controversial. For now we have coal.

The uFAI risk also is going to be extremely controversial as soon as those with commercial interests in the AI development take notice - way more controversial than AGW, for which we do have fairly solid science. If we cannot discuss AGW now, we won't be able to discuss AI risks once Google ...
[SEQ RERUN] Reductive Reference
by MinibearRex — 2012-03-23T04:58:38.715Z

Today's post, Reductive Reference was originally published on 03 April 2008. A summary (taken from the LW wiki):

Virtually every belief you have is not about elementary particle fields, which are (as far as we know) the actual reality. This doesn't mean that those beliefs aren't true. "Snow is white" does not mention quarks anywhere, and yet snow nevertheless is white. It's a computational shortcut, but it's still true.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Brain Breakthrough! It's Made of Neurons!, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Advice for visiting the Bay Area?
by curiousepic — 2012-03-23T01:57:59.974Z

So I'm visiting the Bay Area from April 17th-23rd, though I will be mostly busy with work from the 17th-19th. This will be my first visit to the area, and I'm trying to optimize my free time for potential LW/SI events or visits.

Brief background: I am a SI donor, and also a LW meetup organizer for the Research Triangle area in North Carolina (which has been on hiatus since last summer), through which I was introduced to Alicorn. I don't have a technical background, but am just starting to learn programming. I'm also entertaining the notion of transferring to a position with my company in Palo Alto, with the eventual potential for starting or joining a game company (or anything else that seems more interesting and profitable).

I will be staying at and working near a Hotel in Palo Alto at least until the 19th, and might try to find a nice place to stay through AirBnB (or a hostel) for the remaining time. Of course I might be more interested if you or someone you know has space to crash.

Of course, feel free to PM me instead of commenting.

Specific questions:

What meetups are potentially planned for this time? Will there be a Rationality Bootcamp? (I joined the Bay Area and Tortuga Google Groups and am watching out there, but wanted to ask here as well)

Would it make sense to visit any of the "SI/LW houses", even if just to soak up the aura?

Where should I try to find a room? SF seems to make sense if I'm making daily trips to either Mountain View or Berkeley.

Should I rent a car?

What are your recommendations for less "touristy", underground sightseeing? I found this which looks awesome. More like it!

Many thanks!
Are we infantile introspectors?
by KatjaGrace — 2012-03-23T01:26:48.000Z

Some plausible premises that have been kicking around in my head for a while, and lately found each other:

When we don’t have concepts for things, we can hardly think about them. When I learn new concepts, I often notice them applying everywhere where before I didn’t even notice anything missing. This is a partial explanation sometimes given for us having few memories from childhood; we don’t have the concepts to think explicitly about our experiences when we are very young, and without that it is hard to record them in memory.

We don’t invent our own concepts very much; we mostly inherit them from our society. How many concepts that you have did you invent? If it is very few, this is probably not just because society has already found all the useful concepts and given them to you. If you lived a thousand years ago, my guess is that society wouldn’t have given you concepts like ‘subjective probability’ or ‘tragedy of the commons’ or ‘computation’. And no matter how nerdy you are, you probably wouldn’t have made them up. After all, a whole bunch of people did live then, and they didn’t make them up.

It’s hard to share your feelings in much detail with other people. We can all learn to use the same word ‘angry’ when a person has marked external symptoms of anger. But it’s hard for an angry person to tell you anything very nuanced about how exactly they feel.

I find it hard to remember what emotions and things like that feel like. e.g. right now I’m not really sure what anger feels like. I know what it would be in response to, and that it’s kind of bad, but sometimes enjoyable. And that it might cause snide remarks or energetic ball kicking and that sort of thing.

Hypothesis: We have relatively few concepts for the world inside our heads, because it’s not very shared, and we get concepts mostly from other people. This means it is hard to think about the world inside our heads, and so also hard to remember. (This is all relative to the world outside our heads, and relative to how we would be if we could show one another inside our heads more).