trentmkelly/LessWrong-43k huggingface dataset:
What is the difference between rationality and intelligence?
by Wei Dai — 2014-08-13T11:19:53.062Z

Or to ask the question another way, is there such a thing as a theory of bounded rationality, and if so, is it the same thing as a theory of general intelligence?

The LW Wiki defines general intelligence as "ability to efficiently achieve goals in a wide range of domains", while instrumental rationality is defined as "the art of choosing and implementing actions that steer the future toward outcomes ranked higher in one's preferences". These definitions seem to suggest that rationality and intelligence are fundamentally the same concept.

However, rationality and AI have separate research communities. This seems to be mainly for historical reasons, because people studying rationality started with theories of unbounded rationality (i.e., with logical omniscience or access to unlimited computing resources), whereas AI researchers started off trying to achieve modest goals in narrow domains with very limited computing resources. However rationality researchers are trying to find theories of bounded rationality, while people working on AI are trying to achieve more general goals with access to greater amounts of computing power, so the distinction may disappear if the two sides end up meeting in the middle.

We also distinguish between rationality and intelligence when talking about humans. I understand the former as the ability of someone to overcome various biases, which seems to consist of a set of skills that can be learned, while the latter is a kind of mental firepower measured by IQ tests. This seems to suggest another possibility. Maybe (as Robin Hanson recently argued on his blog) there is no such thing as a simple theory of how to optimally achieve arbitrary goals using limited computing power. In this view, general intelligence requires cooperation between many specialized modules containing domain specific knowledge, so "rationality" would just be one module amongst many, which tries to find and correct systematic deviations from ideal (unbounded) rationality caused by the other modules.

I was more confused when I started writing this post, but now I seem to have largely answered my own question (modulo the uncertainty about the nature of intelligence mentioned above). However I'm still interested to know how others would answer it. Do we have the same understanding of what "rationality" and "intelligence" mean, and know what distinction someone is trying ...
[LINK] 2014 Fields Medals and Nevanlinna Prize anounced
by Sarunas — 2014-08-13T10:58:15.736Z

http://www.mathunion.org/general/prizes/2014

On August 13, 2014, at the opening ceremony of the [International Congress of Mathematicians](http://www.icm2014.org)) the Fields Medals, the Nevanlinna Prize and several other prizes were announced.
A full list of awardees with short citations:
Fields medals:
Artur Avila

is awarded a Fields Medal for his profound contributions to dynamical systems theory, which have changed the face of the field, using the powerful idea of renormalization as a unifying principle.

Quanta Magazine on Artur Avila
Manjul Bhargava

is awarded a Fields Medal for developing powerful new methods in the geometry of numbers, which he applied to count rings of small rank and to bound the average rank of elliptic curves.

Quanta Magazine on Manjul Bhargava
Martin Hairer

is awarded a Fields Medal for his outstanding contributions to the theory of stochastic partial differential equations, and in particular for the creation of a theory of regularity structures for such equations.

Quanta Magazine on Martin Hairer
Maryam Mirzakhani

is awarded the Fields Medal for her outstanding contributions to the dynamics and geometry of Riemann surfaces and their moduli spaces.

Quanta Magazine on Maryam Mirzakhani
Nevalinna prize:
Subhash Khot

is awarded the Nevanlinna Prize for his prescient definition of the “Unique Games” problem, and leading the effort to understand its complexity and its pivotal role in the study of efficient approximation of optimization problems; his work has led to breakthroughs in algorithmic design and approximation hardness, and to new exciting interactions between computational complexity, analysis and geometry.

Quanta Magazine on Subhash Khot

Gauss Prize:
Stanley Osher

is awarded the Gauss Prize for his influential contributions to several fields in applied mathematics, and for his far-ranging inventions that have changed our conception of physical, perceptual, and mathematical concepts, giving us new tools to apprehend the world.

Chern Medal Award:
Phillip Griffiths

is awarded the 2014 Chern Medal for his groundbreaking and transformative development of transcendental methods in complex geometry, particularly his seminal work in Hodge theory and periods of algebraic varieties.

Leelavati Prize:
Adrián Paenza

is awarded the Leelavati Prize for his decisive contributions to changing the mind of a who

...
Truth vs Utility
by Qwake — 2014-08-13T05:45:55.426Z

According to Eliezer, there are two types of rationality. There is epistemic rationality, the process of updating your beliefs based on evidence to correspond to the truth (or reality) as closely as possible. And there is instrumental rationality, the process of making choices in order to maximize your future utility yield. These two slightly conflicting definitions work together most of the time as obtaining the truth is the rationalists' ultimate goal and thus yields the maximum utility. Are there ever times when the truth is not in a rationalist's best interest? Are there scenarios in which a rationalist should actively try to avoid the truth to maximize their possible utility? I have been mentally struggling with these questions for a while. Let me propose a scenario to illustrate the conundrum.

Suppose Omega, a supercomputer, comes down to Earth to offer you a choice. Option 1 is to live in a stimulated world where you have infinite utility (on this world there is no, pain, suffering, death, its basically a perfect world) and you are unaware you are living in a stimulation. Option 2 is Omega will answer one question on absolutely any subject truthfully pertaining to our universe with no strings attached. You can ask about the laws governing the universe, the meaning of life, the origin of time and space, whatever and Omega will give you a absolutely truthful, knowledgeable answer. Now, assuming all of these hypotheticals are true, which option would you pick? Which option should a perfect rationalist pick? Does the potential of asking a question whose answer could greatly improve humanity's knowledge of our universe outweigh the benefits of living in a perfect simulated world with unlimited utility? There is probably a lot of people who would object outright to living in a simulation because it's not reality or the truth. Well lets consider the simulation in my hypothetical conundrum for a second. It's a perfect reality and has unlimited utility potential, and you are completely unaware you are in a simulation on this world. Aside from the unlimited utility part, that sounds a lot like our reality. There are no signs of our reality of being a simulation and all (most) of humanity is convinced that our reality is not a simulation. There for, the only difference that really matters between the simulation in Option 1 and our reality is the unlimi...
Meetup : MIRIxAtlanta - Decision Theory 2
by Adele_L — 2014-08-13T03:30:08.463Z

Discussion article for the meetup : MIRIxAtlanta - Decision Theory 2

WHEN:
17 August 2014 06:00:00PM (-0400)

WHERE:
2388 Lawrenceville Hwy. Unit L. Decatur, GA 30033

We'll be talking about Updateless Decision Theory: http://lesswrong.com/lw/15m/towards_a_new_decision_theory/

Last time, we looked at the results from the Robust Cooperation - and this time we will see how this fits in the framework of UDT. This will give us a direction for generalizing those results.

Discussion article for the meetup : MIRIxAtlanta - Decision Theory 2
Questions on the human path and transhumanism.
by HopefullyCreative — 2014-08-12T20:34:10.746Z

I had a waking nightmare: I know some of you reading this just went "Oh great, here we go..." but bear with me. I am a man who loves to create and build, it is what I have dedicated my life to. One day because of the Less Wrong community I was prompted to ask "What if they are successful in creating an artificial general intelligence whose intellect dwarfs our own?"
My mind raced and imagined the creation of an artificial mind designed to be creative, subservient to man but also anticipate our needs and desires. In other words I imagined if current AGI engineers accomplished the creation of the greatest thing ever. Of course this machine would see how we loathe tiresome repetitive work and design and build for us a host of machines to do it for us. However then the horror at the implication of this all set in. The AGI will become smarter and smarter through its own engineering and soon it will anticipate human needs and produce things no human being could dream of. Suddenly man has no work to do, there is no back breaking labor to be done nor even the creative glorious work of engineering, exploring and experimentation. Instead our army of AGI has robbed us of that.
At this moment I certainly must express that this is not a statement amounting to "Lets not make AGI" for we all know AGI are coming. Then what is my point in expressing this? To express a train of thought that results in questions that have yet to be answered in the hopes that in depth discussion may shed some light.

I realized that the only meaning for man in a world run by AGI would actually be to order the AGI to make man himself better. Instead of focusing on having the AGI design a world for us, use that intellect that we could not before modification compare with to design a means to put us on its own level. In other words, the goal of creating an AGI should not to be to create an AGI but to make a tool so powerful we can use it to command man to be better. Now, I'm quite certain the audience present here is well aware of transhumanism. However, there are some important questions to be answered on the subject:
Mechanical or Biological modification? I know many would think "Are you stupid?! Of course cybernetics would be better than genetic alteration!" Yet the balance of advantages is not as clear as one would think. Lets consider cybernetics for a moment: Many would require ma...
Meetup : Melbourne Social Meetup: Boardgames In The Pub
by Maelin — 2014-08-12T14:43:07.187Z

Discussion article for the meetup : Melbourne Social Meetup: Boardgames In The Pub

WHEN:
15 August 2014 06:30:00PM (+1000)

WHERE:
Alchemist's Refuge, 328 Little Lonsdale St, Melbourne VIC 3000

IMPORTANT: CHANGE OF VENUE
IMPORTANT: Due to a date mixup the trivia is NOT going ahead. We are instead going to play games at Games Lab / Alchemist's Refuge
For August's social LW meetup, we will be going to Alchemist's Refuge, a quiet basement bar that is boardgame friendly.
We'll meet there starting from 6:30pm. If we have any large groups that would like to play games, there is plenty of space upstairs in Games Lab.
If you have any issues you can call me on 0421 231 789.

Discussion article for the meetup : Melbourne Social Meetup: Boardgames In The Pub
Meetup : The Prisoner's Dilemma
by Scott Garrabrant — 2014-08-12T04:58:30.768Z

Discussion article for the meetup : The Prisoner's Dilemma

WHEN:
13 August 2014 07:00:00PM (-0700)

WHERE:
11066 Santa Monica Blvd, Los Angeles, CA

How to Find Us: Go into this Del Taco. We will be in the back room.

Parking is free in the lot out front or on the street nearby.

Discussion: This week, we will talk all about the Prisoner's Dilemma. I will start with a lesson on matrix game theory in general, and how to find a Nash equilibrium. We will then talk about the prisoner's dilemma and why the best output is not a Nash equilibrium. I will explain exactly what makes a matrix game a type of prisoner's dilemma, and what a true prisoner's dilemma might look like in real life. We will also discuss what it takes to cooperate in finitely and infinitely repeated prisoner's dilemma, and the history of prisoner's dilemma tournaments. We will talk about methods for cooperating in the prisoner's dilemma when playing with someone who is either sufficiently similar to you or someone that you can simulate. Hopefully this will inspire everyone to come up with an entry for the upcoming program equilibrium prisoner's dilemma tournament.

No prior exposure to Less Wrong is required; this will be generally accessible.

Discussion article for the meetup : The Prisoner's Dilemma
Meetup : Washington, D.C.: Fun & Games Meetup
by RobinZ — 2014-08-12T01:57:03.753Z

Discussion article for the meetup : Washington, D.C.: Fun & Games Meetup

WHEN:
17 August 2014 03:00:00PM (-0400)

WHERE:
National Portrait Gallery

We will be meeting in the Kogod Courtyard of the National Portrait Gallery (8th and F Sts or 8th and G Sts NW, go straight past the information desk from either entrance) to hang out, play games, and engage in fun conversation.

Discussion article for the meetup : Washington, D.C.: Fun & Games Meetup
Open thread, 11-17 August 2014
by David_Gerard — 2014-08-11T10:12:57.465Z

Previous open thread

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one.

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
Meetup : The Psychology of Video Games
by Anders_H — 2014-08-11T04:58:57.663Z

Discussion article for the meetup : The Psychology of Video Games

WHEN:
17 August 2014 03:30:00PM (-0400)

WHERE:
Cambridge, MA

Kelly MacNeill will talk about the psychology of video games, starting at 4pm at MIT's Building 66

Cambridge/Boston-area Less Wrong meetups start at 3:30pm, and have an alternating location:

1st Sunday meetups are at Citadel in Porter Sq, at 98 Elm St, apt 1, Somerville.

3rd Sunday meetups are in MIT's building 66 at 25 Ames St, room 156. Room number subject to change based on availability; signs will be posted with the actual room number.

(We also have last Wednesday meetups at Citadel at 7pm.)

Our default schedule is as follows:

—Phase 1: Arrival, greetings, unstructured conversation.

—Phase 2: The headline event. This starts promptly at 4pm, and lasts 30-60 minutes.

—Phase 3: Further discussion. We'll explore the ideas raised in phase 2, often in smaller groups.

—Phase 4: Dinner.

Discussion article for the meetup : The Psychology of Video Games
Ethical Choice under Uncertainty
by Anders_H — 2014-08-10T22:13:38.756Z

Ethical Choice under Uncertainty:

Most discussions about utilitarian ethics are attempt to determine the goodness of an outcome. For instance, discussions may focus on whether it would be ethical to increase total utility by increasing the total number of individuals but reducing their average utility. Or, one could argue about whether we should give more weight to those who are worst off when we aggregate utility over individuals.

These are all important questions. However, even if they were answered to everyone's satisfaction, the answers would not be sufficient to guide the choices of agents acting under uncertainty. To elaborate, I believe textbook versions of utilitarianism are unsatisfactory for the following reasons:

Ethical theories that don't account for the agent's beliefs will have absurd consequences such as claiming that it is unethical to rescue a drowning child if the child goes on to become Hitler. Clearly, if we are interested in judging whether the agent is acting ethically, the only relevant consideration is his beliefs about the consequences at the time the choice is made. If we define "ethics" to require him to act on information from the future, it becomes impossible in principle to act ethically.

In real life, there will be many situations where the agent makes a bad choice because he has incorrect beliefs about the consequences of his actions. For most people, if they were asked to judge the morality of a person who has pushed a fat man to his death, it is important to know whether the man believed he could save the lives of five children by doing so. Whether the belief is correct or not is not ethically relevant: There is a difference between stupidity and immorality.

The real choices are never of the type "If you choose A, the fat man dies with probability 1, whereas if you choose B, the five children die with probability 1". Rather, they are of the type "If you choose A, the fat man dies with probability 0.5, the children die with probability 0.25 and they all die with probablity 0.25". Choosing between such options will require a formalization of the concept of risk aversion as an integral component of the ethical theory.

I will attempt to fix this by providing the following definition of ethical choice, which is based on the...
Meditations on Löb's theorem and probabilistic logic [LINK]
by Quinn — 2014-08-10T21:41:36.243Z

A post on my own blog following a MIRIx workshop from two weekends ago.

http://qmaurmann.wordpress.com/2014/08/10/meditations-on-l-and-probabilistic-logic/

Reproducing the intro:

This post is a second look at The Definability of Truth in Probabilistic Logic, a preprint by Paul Christiano and other Machine Intelligence Research Institute associates, which I first read and took notes on a little over one year ago.

In particular, I explore relationships between Christiano et al’s probabilistic logic and stumbling blocks for self-reference in classical logic, like the liar’s paradox (“This sentence is false”) and in particular Löb’s theorem.

The original motivation for the ideas in this post was an attempt to prove a probabilistic version of Löb’s theorem to analyze the truth-teller sentences (“This sentence is [probably] true”) of probabilistic logic, an idea that came out of some discussions at a MIRIx workshop that I hosted in Seattle.
Every Paul needs a Jesus
by PhilGoetz — 2014-08-10T19:13:04.694Z

My take on some historical religious/social/political movements:

Jesus taught a radical and highly impractical doctrine of love and disregard for one's own welfare. Paul took control of much of the church that Jesus' charisma had built, and reworked this into something that could function in a real community, re-emphasizing the social mores and connections that Jesus had spent so much effort denigrating, and converting Jesus' emphasis on radical social action into an emphasis on theology and salvation.

Marx taught a radical and highly impractical theory of how workers could take over the means of production and create a state-free Utopia. Lenin and Stalin took control of the organizations built around those theories, and reworked them into a strong, centrally-controlled state.

Che Guevara (I'm ignorant here and relying on Wikipedia; forgive me) joined Castro's rebel group early on, rose to the position of second in command, was largely responsible for the military success of the revolution, and had great motivating influence due to his charisma and his unyielding, idealistic, impractical ideas. It turned out his idealism prevented him from effectively running government institutions, so he had to go looking for other revolutions to fight in while Castro ran Cuba.

The best strategy for complex social movements is not honest rationality, because rational, practical approaches don't generate enthusiasm. A radical social movement needs one charismatic radical who enunciates appealing, impractical ideas, and another figure who can appropriate all of the energy and devotion generated by the first figure's idealism, yet not be held to their impractical ideals. It's a two-step process that is almost necessary, to protect the pretty ideals that generate popular enthusiasm from the grit and grease of institution and government. Someone needs to do a bait-and-switch. Either the original vision must be appropriated and bent to a different purpose by someone practical, or the original visionary must be dishonest or self-deceiving.

There are exceptions to this pattern that, I think, prove the rule when you look at them more closely:

Hitler was simultaneously an idealist and a highly-pragmatic Machiavellian. His plan, spelled out in Mein Kampf, was to use hatred of the Jews to tap the energy of the German people, implement the reforms he desired, and stamp out the J

...
Meetup : Cognitive Biases
by immasix — 2014-08-10T14:15:59.150Z

Discussion article for the meetup : Cognitive Biases

WHEN:
23 August 2014 02:00:26PM (+0200)

WHERE:
Utrecht

We have biweekly meetups in Film Café Oskar, Slachtstraat 5, Utrecht, near Central Station. This time we will discuss cognitive biases.
For details, please look at meetup.com, which is supposed to be up-to-date.
http://www.meetup.com/LWEANL/events/200065492/
Address:

Discussion article for the meetup : Cognitive Biases
Tarski's truth sentences and MIRI's AI
by halcyon — 2014-08-09T19:28:37.839Z

(Disclaimer: I have no training in or detailed understanding of these subjects. I first heard of Tarski from the Litany of Tarski, and then I Googled him.)

In his paper The Semantic Conception of Truth, Tarski says that he analyzes the claim, '"Snow is white" is true if and only if snow is white' as being expressed in two different languages. The whole claim in single quotes is expressed in a metalanguage, while "snow is white" is in another language.

For Tarski's proof to succeed, it is (if I understood him correctly) both necessary and sufficient for the metalanguage to be logically richer than the other language in certain ways. What these ways are is, according to Tarski, difficult to make general statements about without actually following his very involved technical proof.

If I remember correctly, this implies that the two languages cannot be identical. Tarski seems to be of the opinion that for a given language satisfying specific conditions, concepts of truth, synonymy, meaning, etc. can be defined for it in a metalanguage that is richer than it in logical devices, establishing a hierarchy of truth defining languages.

My main question is, since MIRI aims to mathematically prove Friendliness in recursively self-improving AI, is "essential richness" in language handling ability something we should expect to see increasing in the class of AIs MIRI is interested in, or is that unnecessary for MIRI's purposes? I understand that semantically defining truth and meaning may not be important either way. My principal motive is curiosity.
Why humans suck: Ratings of personality conditioned on looks, profile, and reported match
by PhilGoetz — 2014-08-09T18:48:17.021Z

The recent OKCupid blog, which gwern mentioned in Media Open Thread, investigated the impact of three different factors on users' perceptions of each other: authority (reported match %), profile text (present or absent), and looks.

On the bright side, the authority versus reality match-up came out tied:

If you don't consider that a good outcome, you're not yet sufficiently cynical.

If a picture is worth a thousand words, what are a thousand words worth?

... Approximately nothing.

And the winner is...
The greatest good for the greatest number - starting soonest, or ending last, or lasting longest?
by  — 2014-08-09T02:15:55.580Z

The first greatest good for the greatest number for the greatest number will start "first" (by whatever measurement is applied) but ends before the second greatest good ends and doesn't last as long (in total) as the third greatest good.
The second greatest good for the greatest number will start end "last" (by whatever measurement is applied), but does not last as long as the third greatest good (in total)and doesn't start as soon as the first greatest good.
The third greatest good for the greatest number lasts the longest (in total), but ends before the second greatest good ends and starts after the first greatest good starts.
What within utilitarianism allows for selecting between these three greatest good for the greatest number?