trentmkelly/LessWrong-43k huggingface dataset:
Generalizing from One Trend
by katydee — 2013-01-18T01:21:43.206Z

Related: Reference Class of the Unclassreferenceable, Generalizing From One Example

Many people try to predict the future. Few succeed.

One common mistake made in predicting the future is to simply take a current trend and extrapolate it forward, as if it was the only thing that mattered-- think, for instance, of the future described by cyberpunk fiction, with sinister (and often Japanese) multinational corporations ruling the world. Where does this vision of the future stem from?

Bad or lazy predictions from the 1980s, when sinister multinational corporations (and often Japanese ones) looked to be taking over the world.[1]

Similar errors have been committed by writers throughout history. George Orwell thought 1984 was an accurate prediction of the future, seeing World War II as inevitably bringing socialist revolution to the United Kingdom and predicting that the revolutionary ideals would then be betrayed in England as they were in Russia. Aldous Huxley agreed with Orwell but thought that the advent of hypnosis and psychoconditioning would cause the dystopia portrayed in 1984 to evolve into that he described in Brave New World. In today's high school English classes, these books are taught as literature, as well-written stories-- the fact that the authors took their ideas seriously would come as a surprise to many high school students, and their predictions would look laughably wrong.

Were such mistakes confined solely to the realm of fiction, they would perhaps be considered amusing errors at best, reflective of the sorts of mishaps that befall unstudied predictions. Unfortunately, they are not. Purported "experts" make just the same sort of error regularly, and failed predictions of this sort often have negative consequences in reality.

For instance, in 1999 two economists published the book Dow 36,000, predicting that stocks were about to reach record levels; the authors of the book were so wrapped up in recent gains to the stock market that they assumed that such gains were in fact the new normal state of affairs, that the market hadn't corrected for this yet, and that once stocks were correctly perceived as safe investments the market would skyrocket. This not only did not happen, but the dot-com bubble burst shortly after the book was published.[2] Anyone following the market advice from this book lost big.

In 1968, the biologist Pa...
Outline of Possible Sources of Values
by Wei Dai — 2013-01-18T00:14:49.866Z

I don't know what my values are. I don't even know how to find out what my values are. But do I know something about how I (or an FAI) may be able to find out what my values are? Perhaps... and I've organized my answer to this question in the form of an "Outline of Possible Sources of Values". I hope it also serves as a summary of the major open problems in this area.

External

god(s)

other humans

other agents

Behavioral

actual (historical/observed) behavior

counterfactual (simulated/predicted) behavior

Subconscious Cognition

model-based decision making

ontology

heuristics for extrapolating/updating model

(partial) utility function

model-free decision making

identity based (adopt a social role like "environmentalist" or "academic" and emulate an appropriate role model, actual or idealized)

habits

reinforcement based

Conscious Cognition

decision making using explicit verbal and/or quantitative reasoning

consequentialist (similar to model-based above, but using explicit reasoning)

deontological

virtue ethical

identity based

reasoning about terminal goals/values/preferences/moral principles

responses (changes in state) to moral arguments (possibly context dependent)

distributions of autonomously generated moral arguments (possibly context dependent)

logical structure (if any) of moral reasoning

object-level intuitions/judgments

about what one should do in particular ethical situations

about the desirabilities of particular outcomes

about moral principles

meta-level intuitions/judgments

about the nature of morality

about the complexity of values

about what the valid sources of values are

about what constitutes correct moral reasoning

about how to explicitly/formally/effectively represent values (utility function, multiple utility functions, deontological rules, or something else) (if utility function(s), for what decision theory and ontology?)

about how to extract/translate/combine sources of values into a representation of values

how to solve ontological crisis

how to deal with native utility function or revealed preferences being partial

how to translate non-consequentialist sources of values into utility function(s)

how to deal with moral principles being vague and incomplete

how to deal with conflicts between different sources of values

how to deal with lack of certainty in one's intuitions/judgments

whose intuition/judgment ought to be ap

...
Update Then Forget
by royf — 2013-01-17T18:36:48.373Z

Followup to: How to Be Oversurprised

A Bayesian update needs never lose information. In a dynamic world, though, the update is only half the story. The other half, where the agent takes an action and predicts its result, may indeed "lose" information in some sense.

We have a dynamical system which consists of an agent and the world around it. It's often useful to describe the system in discrete time steps, and insightful to split them into half-steps where both parties (agent and world) take turns changing and affecting each other.

The agent's update is the half-step in which the agent changes. It takes in an observation Ot of the world (t is for time), and uses it to improve its understanding of the world. In doing so, its internal changeable parts change from some configuration (memory state) Mt-1 that they had before, to a new configuration Mt.

The other half-step is when the world changes, and the agent predicts the result of that change. The change from a previous world state Wt to a new one Wt+1 may depend on the action At that the agent takes.

In changing itself, the agent cares about information. There's a clear way - the Bayesian Way - to do it optimally, keeping all of the available information about the world.

In changing the world, the agent cares about rewards, the one it will get now and the ones to come later, possibly much later. The need to make long-term plans with only partial information about the world makes it very hard to be optimal.

Last week we quantified how much information is gained in the update half-step:

I(Wt;Mt) - I(Wt;Mt-1)

(where I is the mutual information). Quantifying how much information is discarded in the prediction half-step is complicated by the action: at the same time that the agent predicts the next world state, it also affects it. The agent can have acausal information about the future by virtue of creating it.

So last week's counterpart

I(Wt;Mt) - I(Wt+1;Mt)

is interesting, but not what we want to study here. To understand what kind of information reduction we do mean here, let's put aside the issue of prediction-through-action, and ask: why would the agent lose any information at all when only the world changes, not the agent itself?

The agent may have some information about the previous world state that simply no longer applies to the changed sta...
Meetup : Ohio LessWrong in Cincinnati
by RolfAndreassen — 2013-01-17T18:21:52.051Z

Discussion article for the meetup : Ohio LessWrong in Cincinnati

WHEN:
20 January 2013 02:00:00PM (-0500)

WHERE:
Taft Museum of Art, 316 Pike Street

We will meet at the Taft Museum entrance. Dinner afterwards will include highly-intellectual discussion of the art. Be sure you have something interesting, intelligent, and rational to say! No pressure, of course.

Discussion article for the meetup : Ohio LessWrong in Cincinnati
[SEQ RERUN] Dunbar's Function
by MinibearRex — 2013-01-17T04:38:47.751Z

Today's post, Dunbar's Function was originally published on 31 December 2008. A summary (taken from the LW wiki):

Robin Dunbar's original calculation showed that the maximum human group size was around 150. But a typical size for a hunter-gatherer band would be 30-50, cohesive online groups peak at 50-60, and small task forces may peak in internal cohesiveness around 7. Our attempt to live in a world of six billion people has many emotional costs: We aren't likely to know our President or Prime Minister, or to have any significant influence over our country's politics, although we go on behaving as if we did. We are constantly bombarded with news about improbably pretty and wealthy individuals. We aren't likely to find a significant profession where we can be the best in our field. But if intelligence keeps increasing, the number of personal relationships we can track will also increase, along with the natural degree of specialization. Eventually there might be a single community of sentients that really was a single community.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Amputation of Destiny, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Meetup : London Meetup 27th Jan
by philh — 2013-01-17T00:22:10.628Z

Discussion article for the meetup : London Meetup 27th Jan

WHEN:
27 January 2013 02:00:00PM (+0000)

WHERE:
Holborn, London

A meetup in the Shakespeare's Head pub by Holborn tube station. Everyone is welcome.

We also have a Google group.

Discussion article for the meetup : London Meetup 27th Jan
Meetup : Durham: Calibration Exercises
by evand — 2013-01-16T21:31:02.538Z

Discussion article for the meetup : Durham: Calibration Exercises

WHEN:
17 January 2013 07:00:00PM (-0500)

WHERE:
Francesca's, 706 9th Street, Durham, NC

Let's do some calibration exercises!

I'll talk a bit about what a well-calibrated estimate is, how it differs from an accurate estimate, and why it's useful. Then we'll do some exercises to test the calibration of attendees, and hopefully improve said calibration!

Discussion article for the meetup : Durham: Calibration Exercises
Quantifying wisdom
by Shmi — 2013-01-16T20:54:26.629Z

So we know that many smart people make stupid (at least in retrospect) decisions. What these people seem to be lacking, at least at the moment they make a poor decision, is wisdom ("judicious application of knowledge"). More from Wikipedia:

It is a deep understanding and realization of people, things, events or situations, resulting in the ability to apply perceptions, judgements and actions in keeping with this understanding. It often requires control of one's emotional reactions (the "passions") so that universal principles, reason and knowledge prevail to determine one's actions.

From Psychology Today:

It can be difficult to define Wisdom, but people generally recognize it when they encounter it. Psychologists pretty much agree it involves an integration of knowledge, experience, and deep understanding that incorporates tolerance for the uncertainties of life as well as its ups and downs. There's an awareness of how things play out over time, and it confers a sense of balance.

Wise people generally share an optimism that life's problems can be solved and experience a certain amount of calm in facing difficult decisions. Intelligence—if only anyone could figure out exactly what it is—may be necessary for wisdom, but it definitely isn't sufficient; an ability to see the big picture, a sense of proportion, and considerable introspection also contribute to its development.

From SEP:

(1) wisdom as epistemic humility, (2) wisdom as epistemic accuracy, (3) wisdom as knowledge, and (4) wisdom as knowledge and action.

Clearly, if one created a human-level AI, one would want it to "choose wisely". However, as human examples show, wisdom does not come for free with intelligence. Actually, we usually don't trust intelligent people nearly as much as we trust wise ones (or appearing to be wise, at any rate). We don't trust them to make good decisions, because they might be too smart for their own good. Speaking of artificial intelligence, one (informal) quality we'd expect an FAI to have is that of wisdom.

So, how would one measure wisdom? Converting the above description ("ability to apply perceptions, judgements and actions in keeping with this understanding") into a more technical form, one can interpret wisdom, in part, as understanding one's own limitations ("running on corrupt hardware", in the local parlance) and calibrating on...
Assessing Kurzweil: the results
by Stuart_Armstrong — 2013-01-16T16:51:50.511Z

Predictions of the future rely, to a much greater extent than in most fields, on the personal judgement of the expert making them. Just one problem - personal expert judgement generally sucks, especially when the experts don't receive immediate feedback on their hits and misses. Formal models perform better than experts, but when talking about unprecedented future events such as nanotechnology or AI, the choice of the model is also dependent on expert judgement.

Ray Kurzweil has a model of technological intelligence development where, broadly speaking, evolution, pre-computer technological development, post-computer technological development and future AIs all fit into the same exponential increase. When assessing the validity of that model, we could look at Kurzweil's credentials, and maybe compare them with those of his critics - but Kurzweil has given us something even better than credentials, and that's a track record. In various books, he's made predictions about what would happen in 2009, and we're now in a position to judge their accuracy. I haven't been satisfied by the various accuracy ratings I've found online, so I decided to do my own assessments.

I first selected ten of Kurzweil's predictions at random, and gave my own estimation of their accuracy. I found that five were to some extent true, four were to some extent false, and one was unclassifiable

But of course, relying on a single assessor is unreliable, especially when some of the judgements are subjective. So I started a call for volunteers to get assessors. Meanwhile Malo Bourgon set up a separate assessment on Youtopia, harnessing the awesome power of altruists chasing after points.

The results are now in, and they are fascinating. They are...

Ooops, you thought you'd get the results right away? No, before that, as in an Oscar night, I first want to thank assessors William Naaktgeboren, Eric Herboso, Michael Dickens, Ben Sterrett, Mao Shan, quinox, Olivia Schaefer, David Sønstebø and one who wishes to remain anonymous. I also want to thank Malo, and Ethan Dickinson and all the other volunteers from Youtopia (if you're one of these, and want to be thanked by name, let me know and I'll add you).

It was difficult deciding on the MVP - no actually it wasn't, that title and ma...
[Link] Selfhood bias
by  — 2013-01-16T16:05:59.765Z

Related: The Blue-Minimizing Robot , Metaethics

Another good article by Federico on his blog studiolo, which he titles Selfhood bias. It reminds me quite strongly of some of the content he produced on his previous (deleted) blog, I'm somewhat sceptical that “Make everyone feel more pleasure and less pain” is indeed the most powerful optimisation process in his brain but besides that minor detail the article is quite good.

This does seems to be shaping up into something well worth following for an aspiring rationalist. I'll add him to the list blogs by LWers even if he doesn't have an account because he has clearly read much if not most of the sequences and makes frequent references to them in his writing. The name of the blog is a reference to this room.

Yvain argues, in his essay “The Blue-Minimizing Robot“, that the concept “goal” is overused.

[long excerpt from the article]

This Gedankenexperiment is interesting, but confused.

I reduce the concept “goal” to: optimisation-process-on-a-map. This is a useful, non-tautological reduction. The optimisation may be cross-domain or narrow-domain. The reduction presupposes that any object with a goal contains a map of the world. This is true of all intelligent agents, and some sophisticated but unintelligent ones. “Having a map” is not an absolute distinction.

I would not say Yvain’s basic robot has a goal.

Imagine a robot with a turret-mounted camera and laser. Each moment, it is programmed to move forward a certain distance and perform a sweep with its camera. As it sweeps, the robot continuously analyzes the average RGB value of the pixels in the camera image; if the blue component passes a certain threshold, the robot stops, fires its laser at the part of the world corresponding to the blue area in the camera image, and then continues on its way.

The robot optimises: it is usefully regarded as an object that steers the future in a predictable direction. Equally, a heliotropic flower optimises the orientation of its petals to the sun. But to say that the robot or flower “failed to achieve its goal” is long-winded. “The robot tries to shoot blue objects, but is actually hitting holograms” is no more concise than, “The robot fires towards clumps of blue pixels in its visual field”. The latter is strictly more informa

...
Meetup : LessWrong Montreal - Social Resilience
by Paul_G — 2013-01-16T09:47:38.615Z

Discussion article for the meetup : LessWrong Montreal - Social Resilience

WHEN:
21 January 2013 06:30:00PM (-0500)

WHERE:
655 Ave. Du President-Kennedy, Montreal, Quebec, Canada

The next weekly meeting of the Montreal LessWrong group, we're going to be working on improving our Social Resilience.

We will be upstairs at the Cheesecake Factory.

See you then!

Discussion article for the meetup : LessWrong Montreal - Social Resilience
Meetup : West LA Meetup - Type 1 to Type 2 Bridging
by  — 2013-01-16T06:35:35.109Z

Discussion article for the meetup : West LA Meetup - Type 1 to Type 2 Bridging

WHEN:
16 January 2013 07:00:00PM (-0800)

WHERE:
10850 West Pico Blvd, Los Angeles, CA 90064

Orion will be hosting this week's meetup, and asked me to post the following:

When: 7:00pm Wednesday, January 16th.

Where: The Westside Tavern in the upstairs Wine Bar (all ages welcome), located inside the Westside Pavillion on the second floor, right by the movie theaters. The entrance sign says "Lounge".

Parking is free for 3 hours.

Lecture/Discussion: This meetup will be oriented around building one of two systems: either escape hatches to require ourselves to fallback to deliberate decision making (Type 2 thinking) when it matters, or rebuilding our first-glance judgments (Type 1 thinking) to be heuristics that work better for our goals. Particularly of note are resources on the habit-forming, which while somewhat tangential, are generally interesting. See BJ Fogg on habits and Roy Baumeister on ego depletion for a general idea of the discussion topics.

No foreknowledge or exposure to Less Wrong is necessary; this will be generally accessable and useful to anyone who values thinking for themselves. There will be open general conversation until 7:30, and that's always a lot of good, fun, intelligent discussion!

Nick will bring a whiteboard with Bayes' Theorem written on it, and I'll generally swoop in to rescue anyone feeling particularly embarrassed about asking strangers if they're from Less Wrong (which is not at all embarrassing, so don't worry about asking!)

Discussion article for the meetup : West LA Meetup - Type 1 to Type 2 Bridging
[SEQ RERUN] Amputation of Destiny
by MinibearRex — 2013-01-16T06:02:46.960Z

Today's post, Amputation of Destiny was originally published on 29 December. A summary (taken from the LW wiki):

C. S. Lewis's Narnia has a problem, and that problem is the super-lion Aslan - who demotes the four human children from the status of main characters, to mere hangers-on while Aslan does all the work. Iain Banks's Culture novels have a similar problem; the humans are mere hangers-on of the superintelligent Minds. We already have strong ethical reasons to prefer to create nonsentient AIs rather than sentient AIs, at least at first. But we may also prefer in just a fun-theoretic sense that we not be overshadowed by hugely more powerful entities occupying a level playing field with us. Entities with human emotional makeups should not be competing on a level playing field with superintelligences - either keep the superintelligences off the playing field, or design the smaller (human-level) minds with a different emotional makeup that doesn't mind being overshadowed.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Can't Unbirth a Child, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Is there an automatic Chrome-to-Anki-2 extension or solution?
by Mark_Eichenlaub — 2013-01-16T05:26:04.900Z

I'd like to be able to click unfamiliar words in Chrome and automatically create notes in Anki 2 using an online dictionary. It'd also be nice to have an automatic method for sending text and images to Anki notes straight from Chrome. For example, if I read an article here that I want to remember, I'd be able to highlight the title, send it to Anki, and when I review, I'd see the title on the card's front with the reverse being a link to the source if I forgot what the post was about.

I found some Chrome extensions that purport to do this sort of thing, but didn't get any of them to work with Anki 2. Is anyone currently doing this, and if so, what is the solution?