trentmkelly/LessWrong-43k huggingface dataset:
Building toward a Friendly AI team
by lukeprog — 2012-06-06T18:57:27.904Z

Series: How to Purchase AI Risk Reduction

A key part of SI's strategy for AI risk reduction is to build toward hosting a Friendly AI development team at the Singularity Institute.

I don't take it to be obvious that an SI-hosted FAI team is the correct path toward the endgame of humanity "winning." That is a matter for much strategic research and debate.

Either way, I think that building toward an FAI team is good for AI risk reduction, even if we decide (later) that an SI-hosted FAI team is not the best thing to do. Why is this so?

Building toward an SI-hosted FAI team means:

Growing SI into a tighter, larger, and more effective organization in general.

Attracting and creating people who are trustworthy, altruistic, hard-working, highly capable, extremely intelligent, and deeply concerned about AI risk. (We'll call these people "superhero mathematicians.")

Both (1) and (2) are useful for AI risk reduction even if an SI-hosted FAI team turns out not to be the best strategy.

This is because: Achieving part (1) would make SI more effective at whatever it is doing to reduce AI risk, and achieving part (2) would bring great human resources to the cause of AI risk reduction, which will be useful to a wide range of purposes (FAI team or otherwise).

So, how do we accomplish both these things?

Growing SI into a better organization

Like many (most?) non-profits with less than $1m/yr in funding, SI has had difficulty attracting the top-level executive talent often required to build a highly efficient and effective organization. Luckily, we have made rapid progress on this front in the past 9 months. For example we now have (1) a comprehensive donor database, (2) a strategic plan, (3) a team of remote contractors used to more efficiently complete large and varied projects requiring many different skillsets, (4) an increasingly "best practices" implementation of central management, (5) an office we actually use to work together on projects, and many other improvements.

What else can SI do to become a tighter, larger, and more effective organization?

Hire a professional bookkeeper, implement additional bookkeeping and accounting best practices. (Currently underway.)

Create a more navigable and up-to-date website. (Currently underway.)

Improve our fundraising strategy, e.g. by creating a deck of slides for major donors which explains what we're d

...
[Link] SMBC on choosing your simulations carefully
by RolfAndreassen — 2012-06-06T17:41:58.133Z

Link

I'm increasingly impressed by the power of Zach Wiener's comic to demonstrate in a few images why hard problems are hard. It would be a vast task, but perhaps it would be useful to create an index of such problem-demonstrating comics to add to the Wiki, giving us something to point newbies at which would be less intimidating than formal Sequence postings. I get the impression that a common hurdle is just to get people to accept that problems of AI (and simulation, ethics, what have you) are actually difficult.
Strategic research on AI risk
by lukeprog — 2012-06-06T17:02:54.980Z

Series: How to Purchase AI Risk Reduction

Norman Rasmussen's analysis of the safety of nuclear power plants, written before any nuclear accidents had occurred, correctly predicted several details of the Three Mile Island incident in ways that that previous experts had not (see McGrayne 2011, p. 180). Had Rasmussen's analysis been heeded, the Three Mile Island incident might not have occurred.

This is the kind of strategic analysis, risk analysis, and technological forecasting that could help us to pivot the world in important ways.

Our AI risk situation is very complicated. There are many uncertainties about the future, and many interacting strategic variables. Though it is often hard to see whether a strategic analysis will pay off, the alternative is to act blindly.

Here are some examples of strategic research that may help (or have already helped) to inform our attempts to shape the future:

FHI's Whole Brain Emulation roadmap and SI's WBE discussion at the Summit 2011 workshop.

Nick Bostrom's forthcoming book on machine superintelligence.

Global Catastrophic Risks, which locates AI risk in the context of other catastrophic risks.

A model of AI risk currently being developed in MATLAB by Anna Salamon and others.

A study of past researchers who abandoned certain kinds of research when they came to believe it might be dangerous, and what might have caused such action. (This project is underway at SI.)

Here are some additional projects of strategic research that could help inform x-risk decisions, if funding were available to perform them:

A study of opportunities for differential technological development, and how to actually achieve them.

A study of microeconomic models of WBEs and self-improving systems.

A study of which research topics should and should not be discussed in public for the purposes of x-risk prevention. (E.g. we may wish to keep AGI discoveries secret for the same reason we'd want to keep the DNA of a synthetically developed supervirus secret, but we may wish to publish research on safe AGI goals because they are safe for a broader community to work on. But it's often difficult to see whether a subject fits into one category or the other.)

I'll note that for as long as FHI is working on AI risk, FHI probably has an advantage over SI in producing actionable strategic research, given past successes like the WBE roadmap and the ...
Help please!
by Michelle_Z — 2012-06-06T15:51:15.873Z

Yesterday my mom noticed (at a funeral) that I wasn't praying or participating in the mass. She confronted me about it, and I told her that no, I am not Catholic. Apparently it's sinking in and she's a bit hysterical... crying and screaming that she doesn't know me anymore.

What do I do? I don't know how to react/behave when she's doing this. It's like she wants me to feel like I'm doing something wrong, but it isn't working, so she's getting hysterical.

*edit*

I gave her a hug when she calmed down and told her I love her. That seemed to help, a little. Based on her previous behavior in situations where I've done something "wrong," she will (in the future) make barbs and slight passes at my beliefs. (Already she made one: insisting my love of science is causing my social anxiety disorder.) The advice given in the comments is really helpful. I plan on making the most of it.
[Link] FreakoStats and CEV
by Filipe — 2012-06-06T15:21:31.259Z

I would like to point you guys to the blog FreakoStats, by Garth Zietsman, who according to his profile, "Scored an IQ of 185 on the Mega27 and has a degree in psychology and statistics and 25 years experience in psychometrics and statistics." The main concept discussed there is "The Smart Vote", whose essence, in the author’s words, is as follows:

"If there is no difference of opinion by intelligence then reason is not relevant in deciding between them and none of the opinions being considered is more correct than any of the others. However if opinions do differ systematically with intelligence then relatively more correct or better alternatives probably do exist, and that they are those relatively more favoured by the more intelligent. Statistical differences in the independent opinions of people of different intellectual ability point to the most reasonable responses to controversies”

Many of his posts are based on the choice of relevant, if controversial, topics , and his analysis of the direction and the proportionality of which an opinion on it is related to intelligence, as measured by IQ scores.

An obvious objection would be that smart people would have in many cases common interests, and this could bias the results simply to their interests, in detriment to the interests of the less smart.

Zietsman answers it with the statistical fact that people many times don’t vote selfishly, and that he can (and will) control for some of their interests anyway:

"My first response is that this isn’t always a factor. Political research shows that people frequently don’t vote their narrow selfish interests e.g. the elderly are less likely to vote for social security than the young and women are less likely to support abortion on demand than men. However there are enough cases where narrow interests obviously do play a role for it to be taken seriously. Fortunately this possibility can be dealt with by controlling for interest differences. For example we could control for class when looking at musical taste, income when looking at welfare, age when looking at social security policy, race when looking at affirmative action, etc."

This seems somewhat related to the notorious concept of Coherent Extrapolation Volition (CEV) of humankind. To have a clue of the direction it might take, I believe it is a nice i...
[Link] Thick and thin 
by  — 2012-06-06T12:08:53.635Z

A new interesting entry on Gregory Cochran's and Henry Harpending's well known blog (West Hunter). For me the information I gained from the LessWrong articles on inferential distances complemented it nicely. Link to source.

There is a spectrum of problem-solving, ranging from, at one extreme, simplicity and clear chains of logical reasoning (sometimes long chains) and, at the other, building a picture by sifting through a vast mass of evidence of varying quality. I will give some examples. Just the other day, when I was conferring, conversing and otherwise hobnobbing with my fellow physicists, I mentioned high-altitude lighting, sprites and elves and blue jets. I said that you could think of a thundercloud as a vertical dipole, with an electric field that decreased as the cube of altitude, while the breakdown voltage varied with air pressure, which declines exponentially with altitude. At which point the prof I was talking to said ” and so the curves must cross!”. That’s how physicists think, and it can be very effective. The amount of information required to solve the problem is not very large. I call this a ‘thin’ problem’.

At the other extreme, consider Darwin gathering and pondering on a vast amount of natural-history information, eventually coming up with natural selection as the explanation. Some of the information in the literature wasn’t correct, and much key information that would have greatly aided his quest, such as basic genetics, was still unknown. That didn’t stop him, anymore than not knowing the cause of continental drift stopped Wegener.

In another example at the messy end of the spectrum, Joe Rochefort, running Hypo in the spring of 1942, needed to figure out Japanese plans. He had an an ever-growing mass of Japanese radio intercepts, some of which were partially decrypted – say, one word of five, with luck. He had data from radio direction-finding; his people were beginning to be able to recognize particular Japanese radio operators by their ‘fist’. He’d studied in Japan, knew the Japanese well. He had plenty of Navy experience – knew what was possible. I would call this a classic ‘thick’ problem, one in which an analyst needs

...
Open Problems Related to Solomonoff Induction
by Wei Dai — 2012-06-06T00:26:10.035Z

Solomonoff Induction seems clearly "on the right track", but there are a number of problems with it that I've been puzzling over for several years and have not made much progress on. I think I've talked about all of them in various comments in the past, but never collected them in one place.

Apparent Unformalizability of “Actual” Induction

Argument via Tarski’s Indefinability of Truth

Informally, the theorem states that arithmetical truth cannot be defined in arithmetic. The theorem applies more generally to any sufficiently strong formal system, showing that truth in the standard model of the system cannot be defined within the system.

Suppose we define a generalized version of Solomonoff Induction based on some second-order logic. The truth predicate for this logic can’t be defined within the logic and therefore a device that can decide the truth value of arbitrary statements in this logical has no finite description within this logic. If an alien claimed to have such a device, this generalized Solomonoff induction would assign the hypothesis that they're telling the truth zero probability, whereas we would assign it some small but positive probability.

Argument via Berry’s Paradox

Consider an arbitrary probability distribution P, and the smallest integer (or the lexicographically least object) x such that P(x) < 1/3^^^3 (in Knuth's up-arrow notation). Since x has a short description, a universal distribution shouldn't assign it such a low probability, but P does, so P can't be a universal distribution.

Is Solomonoff Induction “good enough”?

Given the above, is Solomonoff Induction nevertheless “good enough” for practical purposes? In other words, would an AI programmed to approximate Solomonoff Induction do as well as any other possible agent we might build, even though it wouldn’t have what we’d consider correct beliefs?

Is complexity objective?

Solomonoff Induction is supposed to be a formalization of Occam’s Razor, and it’s confusing that the formalization has a free parameter in the form of a universal Turing machine that is used to define the notion of complexity. What’s the significance of the fact that we can’t seem to define a parameterless concept of complexity? That complexity is subjective?

Is Solomonoff an ideal or an approximation?

Is it t...
List of Problems That Motivated UDT
by Wei Dai — 2012-06-06T00:26:00.625Z

I noticed that recently I wrote several comments of the form "UDT can be seen as a step towards solving X" and thought it might be a good idea to list in one place all of the problems that helped motivate UDT1 (not including problems that came up subsequent to that post).

decision making for minds that can copy themselves

Doomsday Argument

Sleeping Beauty

Absent-Minded Driver

Presumptuous Philosopher

anthropic reasoning for non-sentient AIs

Simulation Argument

indexical uncertainty in general

wireheading/Cartesianism (how to formulate something like AIXI that cares about an external world instead of just its sensory inputs)

How to make decisions if all possible worlds exist? (a la Tegmark or Schmidhuber, or just in the MWI)

Quantum Immortality/Suicide

Logical Uncertainty (how to formulate something like Godel machine that can make reasonable decisions involving P=NP)

uncertainty about hypercomputation (how to avoid assuming we must be living in a computable universe)

What are probabilities?

What are decisions and what kind of consequences should be considered when making decisions?

Newcomb's Problem

Smoking Lesion

Prisoner's Dilemma

Counterfactual Mugging

FAI
Ask an X
by falenas108 — 2012-06-05T20:15:59.055Z

In many previous comments, people call on professionals to answer questions about specific fields, like physicists, neuroscientists, economists, or computer scientists. There are many people in all these professions on this site willing to respond to questions, but most of the time none of them happen to read that comment.

As a way to fix this, I propose that people well-educated in certain fields volunteer to make an "Ask an X" post where they list their credentials and specialties, and anyone can ask questions about that field. Obviously, this would also be a good place to have a discussion between professionals in that field.

Another possibility is to ask people who don't mind being asked random questions to volunteer to be part of a list that can be posted to the wiki. Then, people could just PM that person directly.
Global Workspace Theory
by Douglas_Reay — 2012-06-05T17:16:23.448Z

Much research has been done upon visual perception. Humans have the illusion that they are directly aware of everything in their 'field of view', but it turns out that they actually navigate not through reality, but through a model of reality that their brain stitches together mainly from the bits the eye is directly looking at as it darts about, with the rest supplied by interpolation based upon expectations.

For more info, read The Illusion of Continuity: Active Perception and the Classical Editing System, by Berliner and Cohen.

Global Workspace Theory is the idea that our awareness of our own thought process works the same way. We have the illusion of an unbroken stream of consciousness, but what we're actually referencing is a model of what the brain thinks it has been consciously thinking about, that is stitched together from brief fragments, the way a spotlight in a theatre might move about shining on different parts of a stage, revealing actors making speeches and interacting with each other. Even when the spotlight moves on, the actors, stage hands and directors remain and keep working. When the spotlight returns, to catch a later part of the drama in that area, we interpolate what the actors would have been doing while we were paying attention elsewhere.
Meetup : Small Berkeley meetup: Nisan's house
by Nisan — 2012-06-05T12:27:21.310Z

Discussion article for the meetup : Small Berkeley meetup: Nisan's house

WHEN:
06 June 2012 07:00:00PM (-0700)

WHERE:
Berkeley, CA

This Wednesday's meetup will be at my house. See the for the address.

Feel free to arrive between 7pm and 7:30pm. At 7:30pm we'll do/play some rationality exercises/games that Eliezer presented at last month's minicamp, including Rationalist Taboo. (Maybe we can come up with fun variants.)

Discussion article for the meetup : Small Berkeley meetup: Nisan's house
Meetup : Brussels meetup
by Axel — 2012-06-05T10:43:21.278Z

Discussion article for the meetup : Brussels meetup

WHEN:
16 June 2012 12:00:00PM (+0200)

WHERE:
Museum of Natural Sciences Rue Vautier 29 B-1000 Brussels

We'll discuss personal catastrophic events and how to recover from them and trying to answer http://www.quora.com/Does-embodiment-prevent-unbiased-moral-decisions

Warning: attendance will be lower because of the exam period. We'll meet in the lobby as usual, should you arrive late we will have moved to the cafeteria (just go straight once you're past the entrance, you can't miss it) If you are in the neighborhood, consider dropping by. (getting there: http://www.naturalsciences.be/information/visitor/access)

Discussion article for the meetup : Brussels meetup
Consider a robot vacuum.
by BrandonReinhart — 2012-06-05T08:08:45.546Z

My wife and I recently acquired a robot vacuum. It has turned out to be a really great time-saving and life-improving investment. Some simple math suggests it may be worth you also considering buying one.

Let's say you spend 20 minutes a week vacuuming. That's about 17 hours of vacuuming per year. The Neato XV-11 costs about $350 bucks with basic shipping. For the purposes of our Fermi calculation we will say that your time spent vacuuming with the robot is zero. This is close to true. See below for exceptions.

At $350, if you value an hour of your time at more than about $20 you would be better off buying the robot than doing the vacuuming yourself in the first year. (17*$20=$340 - close enough for our fermi estimate)

Consider also that if you spend at least 20 minutes or less a week vacuuming, you can also instruct the robot to vacuum 20 minutes a week or more and raise the quality of your life by living in a better cared-for environment by some amount. For example, you could increase the pay-out of the robot by having it vacuum every other day.

If you have the robot do 60 minutes of work a week, then you'd only have to value your time at about $7 for the robot to be worthwhile in the first year. (52*$7=$364)

Do the calculation to see if it makes sense for you:

b = value of your time in dollars/hour

y = hours/year you spend vacuuming

350 = estimated price of a robot

x = b*y - 350

If x > 0, then the robot would save you money in time, according to how you value your time. If x < 0, then you either don't clean often enough or value your time so low that doing the work yourself makes sense. (So this is a simple model, feel free to make it more complex but the purpose of this post is to illustrate a fermi calc that seems to yield an easy choice.)

Given the cost of many upright vacuums, if you can avoid buying an upright and only buy the robot the calculation shifts drastically in favor of only getting the robot (and perhaps borrowing an upright if you really need one).

If vacuuming causes you particular disutility, you could put a dollar premium on that disutility and add it to b. On the flip side, if you really like to vacuum you'd want to discount b to reflect the extra utility you get from spending your time doing something you enjoy.

Considerations:

- The robots are claimed to be pretty good at navigating complex room layouts. Our robot rarely (but sometimes) gets...
[SEQ RERUN] Grasping Slippery Things
by MinibearRex — 2012-06-05T04:33:17.992Z

Today's post, Grasping Slippery Things was originally published on 17 June 2008. A summary (taken from the LW wiki):

An illustration of a few ways that trying to perform reductionism can go wrong.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Passing the Recursive Buck, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
Group rationality diary, 6/4/12
by cata — 2012-06-05T04:12:18.453Z

This is the public group instrumental rationality diary for the week of June 4th. It's a place to record and chat about it if you have done, or are actively doing, things like:

Established a useful new habit

Obtained new evidence that made you change your mind about some belief

Decided to behave in a different way in some set of situations

Optimized some part of a common routine or cached behavior

Consciously changed your emotions or affect with respect to something

Consciously pursued new valuable information about something that could make a big difference in your life

Learned something new about your beliefs, behavior, or life that surprised you

Tried doing any of the above and failed

Or anything else interesting which you want to share, so that other people can think about it, and perhaps be inspired to take action themselves. Try to include enough details so that everyone can use each other's experiences to learn about what tends to work out, and what doesn't tend to work out.

Thanks to everyone who contributes!

(Previously: 5/14/12, 5/21/12, 5/28/12)