trentmkelly/LessWrong-43k huggingface dataset:
Superintelligence 16: Tool AIs
by KatjaGrace — 2014-12-30T02:00:09.775Z

This is part of a weekly reading group on Nick Bostrom's book, Superintelligence. For more information about the group, and an index of posts so far see the announcement post. For the schedule of future topics, see MIRI's reading guide.

Welcome. This week we discuss the sixteenth section in the reading guide: Tool AIs. This corresponds to the last parts of Chapter Ten.

This post summarizes the section, and offers a few relevant notes, and ideas for further investigation. Some of my own thoughts and questions for discussion are in the comments.

There is no need to proceed in order through this post, or to look at everything. Feel free to jump straight to the discussion. Where applicable and I remember, page numbers indicate the rough part of the chapter that is most related (not necessarily that the chapter is being cited for the specific claim).

Reading: : “Tool-AIs” and “Comparison” from Chapter 10

Summary

Tool AI: an AI that is not 'like an agent', but more like an excellent version of contemporary software. Most notably perhaps, it is not goal-directed (p151)

Contemporary software may be safe because it has low capability rather than because it reliably does what you want, suggesting a very smart version of contemporary software would be dangerous (p151)

Humans often want to figure out how to do a thing that they don't already know how to do. Narrow AI is already used to search for solutions. Automating this search seems to mean giving the machine a goal (that of finding a great way to make paperclips, for instance). That is, just carrying out a powerful search seems to have many of the problems of AI. (p152)

A machine intended to be a tool may cause similar problems to a machine intended to be an agent, by searching to produce plans that are perverse instantiations, infrastructure profusions or mind crimes. It may either carry them out itself or give the plan to a human to carry out. (p153)

A machine intended to be a tool may have agent-like parts. This could happen if its internal processes need to be optimized, and so it contains strong search processes for doing this. (p153)

If tools are likely to accidentally be agent-like, it would probably be better to just build agents on purpose and have more intentional control over the design. (p155)

Which castes of AI are safest is unclear and de

...
How is the world different if I make a donation?
by banx — 2014-12-29T23:29:31.758Z

How is the world different if I give X dollars to the Against Malaria Foundation, where X < 100,000 USD? (In reality, giving directly to GiveWell is probably the better choice. But using AMF works well for this question.)

Here's the answer that would make me very motivated to give:

If I don't give, AMF will purchase and cause to be distributed N bed nets in the next 5 years. If I do give, AMF will purchase and cause to be distributed N+K bed nets in the next 5 years, where K is roughly X / (cost of a bed net, taking into account administrative costs, etc).

In reality, I think the answer is more like:

If I give, AMF will have X more dollars in its bank account than if I don't give. It will make decisions on whether or not to move forward with various large net distribution projects based on the amount of money it has available at the time of each decision. For each decision there will be some amount Z such that AMF will choose to go forward with the project if and only if it has at least Z dollars in the bank, holding all other factors equal. (Even if they claim there's such a cut-off point Y, but would still go forward with the project if they had slightly less than Y, there's an actual amount Z that is the real cut-off point.)

I can think of two kinds of these distribution decisions, which interact differently with this extra money. First, if AMF is presented a unique, time-sensitive distribution opportunity, the extra money slightly increases the chance that they have enough to move forward with the distribution. Second, suppose there are recurring opportunities that are mostly the same. Assuming that AMF has a steady income of donations, there is a small chance this money will cause AMF to have enough money to fund the K-1th distribution instead of the Kth distribution, thereby getting bed nets out to people who need them sooner. This one-time time advantage would not "use up" the extra money. Assuming the K-1th distribution occurred but the Kth did not, at the time of the K+1th, AMF would still have X more dollars than if I hadn't given. So the chance of the time-advantage happening again (and the chance of a unique opportunity being funded) still exists.

I'm fairly confused about the above, and haven't succeeded in rigorously defining a satisfying model from which I can compute the expected utility of a donation. My intuition is that the expected value of ...
Interesting (or semi-interesting) part time jobs? 
by Capla — 2014-12-29T23:00:32.724Z

I'm currently taking time off from school to focus on my eduction. I'm reading (a lot), mastering some skills, and finishing some projects.

It takes money to live, so I need money. I was considering what my options were for jobs that would keep me engaged, and I thought I'd ask LessWrong.

Constraints:

1. I don't yet have a bachelor's degree. I am however, an intelligent and courteous student at a prestigious university, who doesn't drink smoke or do drugs.

2. I need at least $800/month (500 for rent, internet, and bus fares; 150 for food; 150 for savings).

3. I'm looking for less than 16 hours a week, or the taking time off to focus on learning becomes sort of mute. However, that is on average; it is feasible for me to work many hours one week and than little to none the next.

Optimization criteria:

1. Something interesting, especially something where I would learn something new. This may come in all kinds of forms (for instance, puts me in close contact with the sorts of people I wouldn't usually talk to), including some that I haven't thought of yet. It may even be a new approach to a generic job that makes it challenging or engaging. Jobs that will let me just sit and read without distraction, or even just listen to audio books while I work, would be great.

2. The fewer hours I have to work, the better.

I'm currently running experiments (mostly surveys) for a decision research lab. The work itself a little boring, but I do get to spend some of my time around marketing Ph.d students who are interested in behavioral economics and I get paid $12/hour. It works, but I'm open to other options.

Any ideas?
The Rubber Hand Illusion and Preaching to the Unconverted
by Gram_Stone — 2014-12-29T12:56:24.585Z

It seems that the CFAR workshops so far have been dedicated to people who have preconceptions pretty close in ideaspace to the sorts of ideas proposed on LW and by the institutions related to it. This is not a criticism; it's easier to start out this way: as has been said, in a different context and perhaps not in so many words, we should focus on precision before tractability. We're not going to learn a thing about the effectiveness of rationality training from people who won't even listen to what we have to say. Nevertheless, there will come a day when these efforts must be expanded to people who don't already view us as high in social status, so we still have to solve the problem of people being more concerned with both our and their social status than with listening to what we have to say. I propose that the solution is to divorce the consideration of social status from the argument.

There is a lot of talk of cognitive biases on LW, and for good reason, but ultimately what we are trying to teach people is that they are prone to misinterpreting reality, and cognitive biases are only one component of this. One of the problems with trying to teach people about biases is that people feel personally responsible for being biased; many people have a conception of thinking as an 'active' process, so they feel as though it reflects upon their character. On the other hand, many people conceive of perception as a 'passive' process; no one feels personally responsible for what they perceive. So, I propose that we circumvent this fear of character assassination by demonstrating how people can misinterpret reality through perception. Enter: the rubber hand illusion.

In case you're unfamiliar with this illusion, to demonstrate the rubber hand illusion, a subject sits at a table, a rubber hand is placed in front of them, oriented relative to their body as a natural hand would be, and a partition is placed between the rubber hand and their 'real' hand such that they are unable to see the 'real' hand. Then, the experimenter simultaneously 'stimulates' both hands at random intervals (usually by stroking each hand with a paintbrush). Then, the experimenter overextends the tips of a finger on each hand, the rubber hand about 90 degrees, and the 'real' hand about 20 degrees (it's not really overextension, and it wouldn't cause pain outside of the experiment's co...
Open thread, Dec. 29, 2014 - Jan 04, 2015
by MrMind — 2014-12-29T11:10:37.187Z

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
The steering problem
by paulfchristiano — 2014-12-29T04:50:55.000Z

Most work on AI safety starts with a broad, vague problem (“How can we make an AI do good things?”) and relatively quickly moves to a narrow, precise problem (e.g. "What kind of reasoning process trusts itself?“).

Precision facilitates progress, and many serious thinkers are skeptical of imprecision. But in narrowing the problem too far we do most of the work (and have most of the opportunity for error).

I am interested in more precise discussion of the big-picture problem of AI control. Such discussion could improve our understanding of AI control, help us choose the right narrow questions, and be a better starting point for engaging other researchers. To that end, consider the following problem:

The steering problem: Using black-box access to human-level cognitive abilities, can we write a program that is as useful as a well-motivated human with those abilities?

I recently wrote this document, which defines this problem much more precisely (in section 2) and considers a few possible approaches (in section 4). As usual, I appreciate thoughts and criticism. I apologize for the proliferation of nomenclature, but I couldn’t get by without a new name.

I think the steering problem captures a large part of what most people think of as “the AI safety problem.” It certainly does not capture the entire problem; in particular, we might well introduce undesired goal-directed behavior in the process of implementing human-level capabilities (either inadvertently or because it’s the easiest way to produce human-level abilities).

Since I’ve started thinking more explicitly about the steering problem, I’ve reduced my estimate of its difficulty. This leads me to be more optimistic about AI safety, but also to suspect that the steering problem is a smaller share of the whole problem than I’d originally thought. It would be great to see a more precise statement of the rest of the problem (which would probably subsume the steering problem). I’m afraid that the rest of the problem is more closely tied to the particular techniques used to produce AI, so that we probably can't state it precisely without making some additional assumptions.

I have recently been thinking about the situation for deep learning: under the (very improbable) assumption that various deep learning architectures could yield human-level AI, could they also yield a system with human-level usefulness? I’m optimisti

...
Friendly UI
by cicatriz — 2014-12-28T15:06:44.352Z

I want to bring up some questions that I find crucial to consider about technology in the present day. To contrast with Friendly AI, these questions are about our interaction with technological tools rather than developing a technology that we trust on its own with superhuman intelligence.

1. How are computational tools affecting how we perceive, think, and act?

The inspiration for this post is Bret Victor's new talk, The Humane Representation of Thought. I highly recommend it. In particular, you may want to pause and reflect on the first part before seeing his sketch of solutions in the second. In a nutshell, we have a certain range of human capacities. The use of computing as a medium propels us to develop and value particular capacities: visual & symbolic. Others have discussed diminishing our attention span, decision-making capacity, or cultural expectations of decency. Victor's term for this is "inhumane". He argues that the default path of technological progress has certain properties, but preserving humaneness is not one of them.

The FAI discussions seem to miss both sides of the coin on this phenomenon. First that computation, even though it doesn't exist as a superintelligent entity yet, still imposes values. Second that human intelligence is not a static target: humanity can only reasonably be defined as including the tools we use (humanity without writing or humanity without agriculture are very different), so human intelligence changes along with computation.

In other words, can we design computation now such that it carries us humans to superintelligence? Or, at the very least, doesn't diminish our intelligence and life experience. What are the answers when we ask questions of technology?

2. How can humans best interact with machines with superhuman aspects of intelligence?

There are already machines with superhuman aspects of intelligence, with applications such as chess, essay grading, or image recognition. These systems are deployed without fully understanding how they work, by the very definition of superhuman intelligence. For instance we don't really understand how a machine learning algorithm reaches its conclusion with an unfathomable amount of data. Even if we can prove certain mathematical properties about the behavior, it will be impossible to empathize with the full range of a computer's decision s...
The buildup to Jaynes? 
by Capla — 2014-12-28T09:38:35.320Z

Not too long ago, I asked LessWrong which math topics to learn. Eventually, I want to ask for what the prerequisites for each of those topics are and how I should go about learning them. This is a special case of that.

I'm rereading the sequences and Eliezer seems to love E.T. Jaynes. As part of my rationality self-study, I want to work my way through his Probability Theory: the Logic of Science. What math topics do I already need to understand to prepare myself for this? I learned calculus once upon a time, but not fantastically well, and I plan to start by reviewing that.

Also,

Despite Eliezer's praise of the "thousand-year-old vampire", it there a better book to learn probability theory?

Does anyone want to learn this (or the other math from my post above) with me? I'd love to have a partner or maybe even a work group. Location is no obstacle. [Two caveats: 1. I'm busy with stuff and may not be able to get into this for a few months 2. I hard, but I am incredibly slow at computation (such that on every math test I have ever taken, it took me at least 3 times as long as the second slowest person in the class to finish). You might find that I go to slow for you.]
Meetup : New Year meetup Frankfurt
by Kendra — 2014-12-27T21:30:32.729Z

Discussion article for the meetup : New Year meetup Frankfurt

WHEN:
11 January 2015 02:00:00PM (+0100)

WHERE:
Heerstrasse Frankfurt

We're having our first meetup in the New Year!
I'd like to encourage to do some projects together this year (and look at what some or all of us want to change/improve).
Our TOP (after a bit of arrival time as usual)

1) New Year Plans and Projects (and regular Goal Tracking)

2) Talk: How can you improve your social skills?

3) Prediction Game

More talks about other topics, especially project ideas, are welcome!
I'll edit more details in later. It would be great if you could bring some food. There'll also be (vegan-friendly) food available.
If you have any problems attending (social anxiety, wheelchair accessibility, etc), please tell me, I'll try to accommodate for your needs as best as I can!
You can reach me under +4917634095760 for further details and/or join our mailing list: https://groups.google.com/forum/#!forum/less-wrong-frankfurt

Discussion article for the meetup : New Year meetup Frankfurt
Meetup : January Meetup in Munich
by Valeriy — 2014-12-27T19:14:13.456Z

Discussion article for the meetup : January Meetup in Munich

WHEN:
18 January 2015 02:00:22PM (+0100)

WHERE:
Gast, Rosenheimer Straße 5, 81667 Munich, Germany

the first meetup after the Christmas holidays. We'll be talking about the difference between frequentists and bayesianists and discuss the quantum physics sequence. Further topics are welcomed and we encourage new people to come and say 'hi'.

Discussion article for the meetup : January Meetup in Munich
Tachyon neutrinos (again)
by JoshuaZ — 2014-12-27T17:19:25.332Z

In 2012, a large amount of attention was given to the OPERA experiment's apparent sighting of faster than light neutrinos. This turned out to be erroneous due to a faulty cable, and similar experiments confirmed the same results. However, while this was occurring, a distinct point was made: some attempts to determine the mass of the electron neutrino(one of the three known neutrino types) found that the square of the mass was apparently negative, which would be consistent with an imaginary mass and thus electron neutrinos would be tachyons. While little attention was paid to at the time, a new paper by Robert Ehrlich looks again at this approach. Ehrlich points out that six different experimental results seem to yield an imaginary mass for the electron neutrino, and what is more, all the results are in close agreement, with an apparent square of the mass being close to -0.11 electron-volts squared.

There are at least two major difficulties with Ehrlich's suggestion, both of which were also issues for OPERA aside from any philosophical or metaconcerns like desire to preserve causality. First, it is difficult to reconcile with Ehrlich's suggestion is one of the same data points that apparently tripped up OPERA, that is the neutrinos from SN 1987A neutrinos. In the SN 1987A supernova (the first observed in 1987 hence the name), the supernova was close enough that we were actually able to detect the neutrinos from it. The neutrinos arrived about three hours before the light from the supernova. But that's not evidence for faster than light neutrinos, since one actually expects this to happen. In the standard way of viewing things, the neutrinos move very very close to the speed of light, but during a core-collapse supernova like SN 1987A, the neutrinos are produced in the core at the beginning of the process. They then flee the star without interacting with the matter, whereas the light produced in the core is slowed down by all the matter in the way, so the neutrinos get a few hours head start.

The problem for FTL neutrinos is that if the neutrions were even a tiny bit faster than the speed of light they should have arrived much much earlier. This is strong evidence against FTL neutrinos. In the paper in question, Ehrlich mentions SN 1987A in the context of testing his hypothesis in an alternate way using a supernova and the exact distr...
We Haven't Uploaded Worms
by jefftk — 2014-12-27T11:44:45.411Z

In theory you can upload someone's mind onto a computer, allowing them to live forever as a digital form of consciousness, just like in the Johnny Depp film Transcendence.

But it's not just science fiction. Sure, scientists aren't anywhere near close to achieving such feat with humans (and even if they could, the ethics would be pretty fraught), but now an international team of researchers have managed to do just that with the roundworm Caenorhabditis elegans.
—Science Alert

Uploading an animal, even one as simple as c. elegans would be very impressive. Unfortunately, we're not there yet. What the people working on Open Worm have done instead is to build a working robot based on the c. elegans and show that it can do some things that the worm can do.

The c. elegans nematode has only 302 neurons, and each nematode has the same fixed pattern. We've known this pattern, or connectome, since 1986. [1] In a simple model, each neuron has a threshold and will fire if the weighted sum of its inputs is greater than that threshold. Which means knowing the connections isn't enough: we also need to know the weights and thresholds. Unfortunately, we haven't figured out a way to read these values off of real worms. Suzuki et. al. (2005) [2] ran a genetic algorithm to learn values for these parameters that would give a somewhat realistic worm and showed various wormlike behaviors in software. The recent stories about the Open Worm project have been for them doing something similar in hardware. [3]

To see why this isn't enough, consider that nematodes are capable of learning. Sasakura and Mori (2013) [5] provide a reasonable overview. For example, nematodes can learn that a certain temperature indicates food, and then seek out that temperature. They don't do this by growing new neurons or connections, they have to be updating their connection weights. All the existing worm simulations treat weights as fixed, which means they can't learn. They also don't read weights off of any individual worm, which means we can't talk about any specific worm as being uploaded.

If this doesn't count as uploading a worm, however, what would? Consider an experiment where someone trains one group of worms to respond to stimulus one way and another group to respond the other way. Both groups are then scanned and simulated on the computer. If the simulated worms responded to simul...
Suggestions for 31C3 (Chaos Communication Congress)
by Gunnar_Zarncke — 2014-12-27T10:08:11.620Z

Hi, I will be attending 31C3, the 31th chaos communication congress in Hamburg today and the next days. For those who don't know about this hacker conference, here is a summary:

The 31st Chaos Communication Congress (31C3) is an annual four-day conference on technology, society and utopia. The Congress offers lectures and workshops and various events on a multitude of topics including (but not limited to) information technology and generally a critical-creative attitude towards technology and the discussion about the effects of technological advances on society.

This sounds a lot like a suitable forum for talking existencial risk from AI. But apparently no such talk is listed in the Fahrplan. As this appears to be a conference largely self-organized I might find a slot where I can present UFAI risks to a smaller audience.

Thus this is a request for material I might use to present this. My first idea was to use the MIRI executive summary. What else can I use? Some quick slides? Where might I link to best? What could I use as a quickl handout/flyer?

Yes this is on short notice but I admit that I didn't draw this connection earlier. For me the 31C3 was 'just' a hacker conference to meet people at.

ADDED: And that's what it ended to be. I met Joscha Bach and took part in a discussion about consciousness as computation. I met Oliver Habryka (good luck in California) and also people from the Berlin (Hi Tristan) and Frankfurt Meetups (Hi Kai). Overall a cool event though personally I didn't get that much out of the talks.