trentmkelly/LessWrong-43k huggingface dataset:
[SEQ RERUN] Einstein's Speed
by MinibearRex — 2012-05-13T08:48:00.713Z

Today's post, Einstein's Speed was originally published on 21 May 2008. A summary (taken from the LW wiki):

Albert was unusually good at finding the right theory in the presence of only a small amount of experimental evidence. Even more unusually, he admitted it - he claimed to know the theory was right, even in advance of the public proof. It's possible to arrive at the truth by thinking great high-minded thoughts of the sort that Science does not trust you to think, but it's a lot harder than arriving at the truth in the presence of overwhelming evidence.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Faster Than Science, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Neuroimaging as alternative/supplement to cryonics?
by Wei Dai — 2012-05-12T23:26:28.429Z

Paul Christiano recently suggested that we can use neuroimaging to form a complete mathematical characterization of a human brain, which a sufficiently powerful superintelligence would be able to reconstruct into a working mind, and the neuroimaging part is already possible today, or close to being possible.

In fact, this project may be possible using existing resources. The complexity of the human brain is not as unapproachable as it may at first appear: though it may contain 1014 synapses, each described by many parameters, it can be specified much more compactly. A newborn’s brain can be specified by about 109 bits of genetic information, together with a recipe for a physical simulation of development. The human brain appears to form new long-term memories at a rate of 1-2 bits per second, suggesting that it may be possible to specify an adult brain using 109 additional bits of experiential information. This suggests that it may require only about 1010 bits of information to specify a human brain, which is at the limits of what can be reasonably collected by existing technology for functional neuroimaging.

Paul was using this idea as part of an FAI design proposal, but I'm highlighting it here since it seems to have independent value as an alternative or supplement to cryonics. That is, instead of (or in addition to) trying to get your body to be frozen and then preserved in liquid nitrogen after you die, you periodically take neuroimaging scans of your brain and save them to multiple backup locations (1010 bits is only about 1 gigabyte), in the hope that a friendly AI or posthuman will eventually use the scans to reconstruct your mind.

Are there any neuroimaging experts around who can tell us how feasible this really is, and how much such a scan might cost, now or in the near future?

ETA: Given the presence of thermal noise and the fact that a set of neuroimaging data may contain redundant or irrelevant information, 1010 bits ought to be regarded as just a rough lower bound on how much data needs to be collected and stored. Thanks to commenters who pointed this out.
Practical tools and agents
by private_messaging — 2012-05-12T21:42:37.409Z

Presently, the 'utility maximizers' work as following: given a mathematical function f(x) , a solver finds the x that corresponds to a maximum (or, typically, minimum) of f(x) . The x is usually a vector describing the action of the agent, the f is a mathematically defined function which may e.g. simulate some world evolution and compute the expected worth of end state, given action x, as in f(x)=h(g(x)) where h computes worth of world state g(x), and g computes the world state at some future time assuming that action x was taken.

For instance, the f may represent some metric of risk, discomfort, and time, over a path chosen by a self driving car, in a driving simulator (which is not reductionist). In this case this metric (which is always non-negative) is to be minimized.

In a very trivial case, such as finding the cannon elevation at which the cannonball will land closest to the target, in vacuum, the solution can be found analytically.

In more complex cases multitude of methods are typically employed, combining iteration of potential solutions with analytical and iterative solving for local maximum or minimum. If this is combined with sensors and the model-updater, and actuators, an agent like a self driving car can be made.

Those are the utility functions as used in the field of artificial intelligence.

A system can be strongly superhuman at finding maximums to functions, and ultimately can be very general purpose, allowing it's use to build models which are efficiently invertible into a solution. However it must be understood that the intelligent component finds mathematical solutions to, ultimately, mathematical relations.

The utility functions as known and discussed on LW seem entirely different in nature. Them are defined on the real word, using natural language that conveys intent, and seem to be a rather ill defined concept for which the bottom-up formal definition may not even exist. The implementation of such concept, if at all possible, would seem to require a major breakthrough in the philosophy of mind.

This is an explanation of an important technical distinction mentioned in Holden Karnofsky's post.

On the discussion in general: It may well be the case that it is very difficult or impossible to define a system such as self driving car in terms of the concepts that are used on LW to talk about intelligences. In particular, the LW's notion of "utility"...
[Book Suggestions] Summer Reading for Younglings.
by Karmakaiser — 2012-05-12T16:57:51.953Z

I bought my niece a Kindle that just arrived and I'm about to load it up with books to give it to her tomorrow for her birthday. I've decided to be a sneaky uncle and include good books that can teach better abilities to think or at least to consider science cool and interesting. She is currently in the 4th Grade with 5th coming after the Summer.

She reads basically at her own grade level so while I'm open to stuffing the Kindle with books to be read when she's ready, I'd like to focus on giving her books she can read now. Ender's Game will be on there most likely. Game of Thrones will not.

What books would you give a youngling? Her interests currently trend toward the young mystery section, Hardy Boys and the like, but in my experience she is very open to trying new books with particular interest in YA fantasy but not much interest in Sci Fi (if I'm doing any other optimizing this year, I'll try to change her opinion on Sci Fi).
Tool for maximizing paperclips vs a paperclip maximizer
by private_messaging — 2012-05-12T07:38:58.960Z

To clarify some point that is being discussed in several threads here, tool vs intentional agent distinction:

A tool for maximizing paperclips would - for efficiency purposes - have a world-model which it has god's eye view of (not accessing it through embedded sensors like eyes), implementing/defining a counter of paperclips within this model. Output of this counter is what is being maximized by a problem solving portion of the tool. Not the real world paperclips

No real world intentionality exist in this tool for maximizing paperclips; the paperclip-making-problem-solver would maximize the output of the counter, not real world paperclips. Such tool can be hooked up to actuators, and to sensors, and made to affect the world without human intermediary; but it won't implement real world intentionality.

An intentional agent for maximizing paperclips is the familiar 'paperclip maximizer', that truly loves the real world paperclips and wants to maximize them, and would try to improve it's understanding of the world to know if it's paperclip making efforts are successful.

The real world intentionality is ontologically basic in human language and consequently there is very strong bias to describe the former as the latter.

The distinction: the wireheading (either direct or through manipulation of inputs) is a valid solution to the problem that is being solved by the former, but not by the latter. Of course one could rationalize and postulate tool that is not general purpose enough as to wirehead, forgetting that the issue being feared is a tool that's general purpose to design better tool or self improve. That is an incredibly frustrating feature of rationalization. The aspects of problem are forgotten when thinking backwards.

The issues with the latter: We do not know if humans actually implement real world intentionality in such a way that it is not destroyed under full ability to self modify (and we can observe that we very much like to manipulate our own inputs; see art, porn, fiction, etc). We do not have single certain example of such stable real world intentionality, and we do not know how to implement it (that may well be impossible). We also are prone to assuming that two unsolved problems in AI - general problem solving and this real world intentionality - are a single problem, or are solved necessarily together. A map compression issue.
[SEQ RERUN] Faster Than Science
by MinibearRex — 2012-05-12T02:13:35.457Z

Today's post, Faster Than Science was originally published on 20 May 2008. A summary (taken from the LW wiki):

Is it really possible to arrive at the truth faster than Science does? Not only is it possible, but the social process of science relies on scientists doing so - when they choose which hypotheses to test. In many answer spaces it's not possible to find the true hypothesis by accident. Science leaves it up to experiment to socially declare who was right, but if there weren't some people who could get it right in the absence of overwhelming experimental proof, science would be stuck.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Changing the Definition of Science, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Most transferable skills?
by kalla724 — 2012-05-11T21:58:01.712Z

So, transferable skills: skills that, upon improvement, increase your ability in other areas (and also improve other, higher-level skills).

A basic example would be reading/writing. Knowing how to read and write allows one to access a huge amount of other skills and resources which are otherwise unavailable. A less obvious example would be clear speech (enunciation). Ability to speak clearly improves one's prospects in a lot of different areas (e.g. professional advancement, dating, etc.).

I'm looking for additional examples. Which skills did you find to be most transferable? Did you become proficient in X, and then found this helped you in many other areas of your life? Please share.

(I tried to find whether this was discussed before, and failed; if it was, I would appreciate the link.)
[Requesting Advice] Applying Instrumental Rationality to College Course Selection Dilemma
by  — 2012-05-11T17:15:49.163Z

I'm faced with a dilemma and need a big dose of instrumental rationality. I'll describe the situation:
This fall, I'm entering my first semester of college. I'm aiming to graduate in 3-4 years with a Mathematics B.S. In order for my course progression to go smoothly, I need to take Calculus I Honors this fall and Calculus II in the spring. These two courses serve as a prerequisite bottleneck. They prevent me from taking higher level math courses.
My SAT scores have exempted me from all placement tests, including the math. But without taking a placement test, the highest any math SAT score can place me into is Pre-Calculus Honors, which is one level below what I want to take in the fall. The course progression goes Pre-Calculus Honors to Calc I Honors to Calc II Honors.
So in order to take Calc I Honors in the fall, I either need to:
(1) Score high enough on a College-Level Math placement test or
(2) Forgo the test and take Pre-Calc Honors for 9 weeks this summer
I've taken both pre-calculus and calculus in high school. I've also been studying precalculus material over the past few days, relearning a lot of what I've either forgotten or wasn't taught in class. If I decide to take the test, I'm pretty confident I'll place into Calculus I. I'd estimate that chance being within 0.8, plus or minus 0.1. If I pass the test, I'll save 9 weeks of studying in the summer and use them to prepare for classes I'll be taking in the fall. I'd also free me up to take another summer class worth 4 credits and fulfill a prerequisite.

But if I decide to forgo the test and take Precalc this summer, I'm also pretty confident I'll do very well in the class. I'd confidently wager above a 90%. The class would ensure I've got the material down better than the placement test and would also give me my first six credits.

The questions going through my mind right now include: How can I best decide between these two options? How can I compare the heterogeneous benefits/costs? Are there any other relevant factors that I'm leaving out?
Advice would be greatly appreciated.

Edit: Writing this post, as well as reading and responding to the comments, has clarified the situation for me. Unless there is something else important I've missed, I'll take the test, place into Calc I, spend the summer taking a different summer class and preparing for fall classes. Thanks to everyone who helped me out.
Weekly LW Meetups: Cambridge, Philadelphia, Salt Lake City, Vancouver
by FrankAdamek — 2012-05-11T15:04:35.244Z

There are upcoming irregularly scheduled Less Wrong meetups in:

Philadelphia Meetup: Introduction to drawing: 05 May 2012 02:00PM
Vancouver Fake Utility Meetup: 06 May 2012 01:00PM
Washington DC meetup: 13 May 2012 03:00PM
Brussels meetup: 19 May 2012 12:00PM
Less Wrong Sydney - Rational Acting: 21 May 2012 06:00PM
First Berlin meetup: 05 June 2012 07:30PM
Phoenix, Arizona: 15 June 2012 07:00PM

The following meetups take place in cities with regularly scheduled meetups, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

SLC Meetup: Social Hacking Presentation: 05 May 2012 01:49PM
Cambridge, MA First Sunday Meetup: 06 May 2012 02:00PM
Cambridge, MA Third Sunday Meetup: 20 May 2012 02:20PM

Locations with regularly scheduled meetups: Austin, Berkeley, Cambridge, MA, Cambridge UK, Chicago, London, Madison WI, Melbourne, Mountain View, New York, Ohio, Ottawa, Oxford, Portland, Salt Lake City, Seattle, Toronto, Waterloo, and West Los Angeles.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview will continue to be posted on the front page every Friday. These will be an attempt to collect information on all the meetups happening in the next weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll now also have the benefit of having your meetup mentioned in a weekly overview. These overview posts will be moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irregularly, consider subscribing to one or more city-specific mailing list in order to be notified when an irregular meetup is happening: Atlanta, Berlin, Helsinki, London, Marin CA, Pittsburgh, Southern California (Los Angeles/Orange County area), St. Louis, Vancouver, Washington, DC.

If your meetup has a mailing list that you'd like mentioned here, or has become regular and isn't listed as such, let me know!

Want to help out the common good? If one ...
Holden Karnofsky's Singularity Institute critique: Is SI the kind of organization we want to bet on?
by Paul Crowley — 2012-05-11T07:25:56.637Z

The sheer length of GiveWell co-founder and co-executive director Holden Karnofsky's excellent critique of the Singularity Institute means that it's hard to keep track of the resulting discussion. I propose to break out each of his objections into a separate Discussion post so that each receives the attention it deserves.

Is SI the kind of organization we want to bet on?

This part of the post has some risks. For most of GiveWell's history, sticking to our standard criteria - and putting more energy into recommended than non-recommended organizations - has enabled us to share our honest thoughts about charities without appearing to get personal. But when evaluating a group such as SI, I can't avoid placing a heavy weight on (my read on) the general competence, capability and "intangibles" of the people and organization, because SI's mission is not about repeating activities that have worked in the past. Sharing my views on these issues could strike some as personal or mean-spirited and could lead to the misimpression that GiveWell is hostile toward SI. But it is simply necessary in order to be fully transparent about why I hold the views that I hold.

Fortunately, SI is an ideal organization for our first discussion of this type. I believe the staff and supporters of SI would overwhelmingly rather hear the whole truth about my thoughts - so that they can directly engage them and, if warranted, make changes - than have me sugar-coat what I think in order to spare their feelings. People who know me and my attitude toward being honest vs. sparing feelings know that this, itself, is high praise for SI.

One more comment before I continue: our policy is that non-public information provided to us by a charity will not be published or discussed without that charity's prior consent. However, none of the content of this post is based on private information; all of it is based on information that SI has made available to the public.

There are several reasons that I currently have a negative impression of SI's general competence, capability and "intangibles." My mind remains open and I include specifics on how it could be changed.

Weak arguments. SI has produced enormous quantities of public argumentation, and I have examined a very large proportion of this information. Yet I have never seen a clear response to any of the three basic objections I listed in t

...
Holden Karnofsky's Singularity Institute critique: other objections
by Paul Crowley — 2012-05-11T07:22:13.699Z

The sheer length of GiveWell co-founder and co-executive director Holden Karnofsky's excellent critique of the Singularity Institute means that it's hard to keep track of the resulting discussion. I propose to break out each of his objections into a separate Discussion post so that each receives the attention it deserves.

Other objections to SI's views

There are other debates about the likelihood of SI's work being relevant/helpful; for example,

It isn't clear whether the development of AGI is imminent enough to be relevant, or whether other risks to humanity are closer.

It isn't clear whether AGI would be as powerful as SI's views imply. (I discussed this briefly in Karnofsky/Tallinn 2011.)

It isn't clear whether even an extremely powerful UFAI would choose to attack humans as opposed to negotiating with them. (I find it somewhat helpful to analogize UFAI-human interactions to human-mosquito interactions. Humans are enormously more intelligent than mosquitoes; humans are good at predicting, manipulating, and destroying mosquitoes; humans do not value mosquitoes' welfare; humans have other goals that mosquitoes interfere with; humans would like to see mosquitoes eradicated at least from certain parts of the planet. Yet humans haven't accomplished such eradication, and it is easy to imagine scenarios in which humans would prefer honest negotiation and trade with mosquitoes to any other arrangement, if such negotiation and trade were possible.)

Unlike the three objections I focus on, these other issues have been discussed a fair amount, and if these other issues were the only objections to SI's arguments I would find SI's case to be strong (i.e., I would find its scenario likely enough to warrant investment in).
Holden Karnofsky's Singularity Institute Objection 3
by Paul Crowley — 2012-05-11T07:19:18.688Z

The sheer length of GiveWell co-founder and co-executive director Holden Karnofsky's excellent critique of the Singularity Institute means that it's hard to keep track of the resulting discussion. I propose to break out each of his objections into a separate Discussion post so that each receives the attention it deserves.

Objection 3: SI's envisioned scenario is far more specific and conjunctive than it appears at first glance, and I believe this scenario to be highly unlikely.

SI's scenario concerns the development of artificial general intelligence (AGI): a computer that is vastly more intelligent than humans in every relevant way. But we already have many computers that are vastly more intelligent than humans in some relevant ways, and the domains in which specialized AIs outdo humans seem to be constantly and continuously expanding. I feel that the relevance of "Friendliness theory" depends heavily on the idea of a "discrete jump" that seems unlikely and whose likelihood does not seem to have been publicly argued for.

One possible scenario is that at some point, we develop powerful enough non-AGI tools (particularly specialized AIs) that we vastly improve our abilities to consider and prepare for the eventuality of AGI - to the point where any previous theory developed on the subject becomes useless. Or (to put this more generally) non-AGI tools simply change the world so much that it becomes essentially unrecognizable from the perspective of today - again rendering any previous "Friendliness theory" moot. As I said in Karnofsky/Tallinn 2011, some of SI's work "seems a bit like trying to design Facebook before the Internet was in use, or even before the computer existed."

Perhaps there will be a discrete jump to AGI, but it will be a sort of AGI that renders "Friendliness theory" moot for a different reason. For example, in the practice of software development, there often does not seem to be an operational distinction between "intelligent" and "Friendly." (For example, my impression is that the only method programmers had for evaluating Watson's "intelligence" was to see whether it was coming up with the same answers that a well-informed human would; the only way to evaluate Siri's "intelligence" was to evaluate its helpfulness to humans.) "Intelligent" often ends up getting defined as "prone to take actions that seem all-around 'good...