trentmkelly/LessWrong-43k huggingface dataset:
Failures of an embodied AIXI
by So8res — 2014-06-15T18:29:23.442Z

Building a safe and powerful artificial general intelligence seems a difficult task. Working on that task today is particularly difficult, as there is no clear path to AGI yet. Is there work that can be done now that makes it more likely that humanity will be able to build a safe, powerful AGI in the future? Benja and I think there is: there are a number of relevant problems that it seems possible to make progress on today using formally specified toy models of intelligence. For example, consider recent program equilibrium results and various problems of self-reference.

AIXI is a powerful toy model used to study intelligence. An appropriately-rewarded AIXI could readily solve a large class of difficult problems. This includes computer vision, natural language recognition, and many other difficult optimization tasks. That these problems are all solvable by the same equation — by a single hypothetical machine running AIXI — indicates that the AIXI formalism captures a very general notion of "intelligence".

However, AIXI is not a good toy model for investigating the construction of a safe and powerful AGI. This is not just because AIXI is uncomputable (and its computable counterpart AIXItl infeasible). Rather, it's because AIXI cannot self-modify. This fact is fairly obvious from the AIXI formalism: AIXI assumes that in the future, it will continue being AIXI. This is a fine assumption for AIXI to make, as it is a very powerful agent and may not need to self-modify. But this inability limits the usefulness of the model. Any agent capable of undergoing an intelligence explosion must be able to acquire new computing resources, dramatically change its own architecture, and keep its goals stable throughout the process. The AIXI formalism lacks tools to study such behavior.

This is not a condemnation of AIXI: the formalism was not designed to study self-modification. However, this limitation is neither trivial nor superficial: even though an AIXI may not need to make itself "smarter", real agents may need to self-modify for reasons other than self-improvement. The fact that an embodied AIXI cannot self-modify leads to systematic failures in situations where self-modification is actually necessary. One such scenario, made explicit using Botworld, is explored in detail below.

In this game, one agent will require another agent to precommit to a trade by modifyi...
Willpower Depletion vs Willpower Distraction
by Academian — 2014-06-15T18:29:01.962Z

I once asked a room full of about 100 neuroscientists whether willpower depletion was a thing, and there was widespread disagreement with the idea. (A propos, this is a great way to quickly gauge consensus in a field.) Basically, for a while some researchers believed that willpower depletion "is" glucose depletion in the prefrontal cortex, but some more recent experiments have failed to replicate this, e.g. by finding that the mere taste of sugar is enough to "replenish" willpower faster than the time it takes blood to move from the mouth to the brain:

Carbohydrate mouth-rinses activate dopaminergic pathways in the striatum–a region of the brain associated with responses to reward (Kringelbach, 2004)–whereas artificially-sweetened non-carbohydrate mouth-rinses do not (Chambers et al., 2009). Thus, the sensing of carbohydrates in the mouth appears to signal the possibility of reward (i.e., the future availability of additional energy), which could motivate rather than fuel physical effort.

-- Molden, D. C. et al, The Motivational versus Metabolic Effects of Carbohydrates on Self-Control. Psychological Science.

Stanford's Carol Dweck and Greg Walden even found that hinting to people that using willpower is energizing might actually make them less depletable:

When we had people read statements that reminded them of the power of willpower like, “Sometimes, working on a strenuous mental task can make you feel energized for further challenging activities,” they kept on working and performing well with no sign of depletion. They made half as many mistakes on a difficult cognitive task as people who read statements about limited willpower. In another study, they scored 15 percent better on I.Q. problems.

-- Dweck and Walden, Willpower: It’s in Your Head? New York Times.

While these are all interesting empirical findings, there’s a very similar phenomenon that’s much less debated and which could explain many of these observations, but I think gets too little popular attention in these discussions:

Willpower is distractible.

Indeed, willpower and working memory are both strongly mediated by the dorsolateral prefontal cortex, so “distraction” could just be the two functions funging against one another. To use the terms of Stanovich popularized by Kahneman in Thinking: Fast and Slow, "System 2" can only override so m...
June Monthly Bragging Thread
by elharo — 2014-06-15T13:26:10.079Z

Your job, should you choose to accept it, is to comment on this thread explaining the most awesome thing you've done this month. You may be as blatantly proud of yourself as you feel. You may unabashedly consider yourself the coolest freaking person ever because of that awesome thing you're dying to tell everyone about. This is the place to do just that.

Remember, however, that this isn't any kind of progress thread. Nor is it any kind of proposal thread. This thread is solely for people to talk about the awesome things they have done. Not "will do". Not "are working on". Have already done. This is to cultivate an environment of object level productivity rather than meta-productivity methods.

So, what's the coolest thing you've done this month?
Total Utility is Illusionary
by PlatypusNinja — 2014-06-15T02:43:50.049Z

(Abstract: We have the notion that people can have a "total utility" value, defined perhaps as the sum of all their changes in utility over time. This is usually not a useful concept, because utility functions can change. In many cases the less-confusing approach is to look only at the utility from each individual decision, and not attempt to consider the total over time. This leads to insights about utilitarianism.)

Let's consider the utility of a fellow named Bob. Bob likes to track his total utility; he writes it down in a logbook every night.

Bob is a stamp collector; he gets +1 utilon every time he adds a stamp to his collection, and he gets -1 utilon every time he removes a stamp from his collection. Bob's utility was zero when his collection was empty, so we can say that Bob's total utility is the number of stamps in his collection.

One day a movie theater opens, and Bob learns that he likes going to movies. Bob counts +10 utilons every time he sees a movie. Now we can say that Bob's total utility is the number of stamps in his collection, plus ten times the number of movies he has seen.

(A note on terminology: I'm saying that Bob's utility function is the thing that emits +1 or -1 or +10, and his total utility is the sum of all those emits over time. I'm not sure if this is standard terminology.)

This should strike us as a little bit strange: Bob now has a term in his total utility which is mostly based on history, and mostly independent of the present state of the world. Technically, we might handwave and say that Bob places value on his memories of watching those movies. But Bob knows that's not actually true: it's the act of watching the movies that he enjoys, and he rarely thinks about them once they're over.

If a hypnotist convinced Bob that he had watched ten billion movies, Bob would write down in his logbook that he had a hundred billion utilons. (Plus the number of stamps in his stamp collection.)

Let's talk some more about that stamp collection. Bob wakes up on June 14 and decides that he doesn't like stamps any more. Now, Bob gets -1 utilon every time he adds a stamp to his collection, and +1 utilon every time he removes one. What can we say about his total utility? We might say that Bob's total utility is the number of stamps in his collection at the start of June 14, ...
New organization - Future of Life Institute (FLI)
by Vika — 2014-06-14T23:00:08.492Z

As of May 2014, there is an existential risk research and outreach organization based in the Boston area. The Future of Life Institute (FLI), spearheaded by Max Tegmark, was co-founded by Jaan Tallinn, Meia Chita-Tegmark, Anthony Aguirre and myself.

Our idea was to create a hub on the US East Coast to bring together people who care about x-risk and the future of life. FLI is currently run entirely by volunteers, and is based on brainstorming meetings where the members come together and discuss active and potential projects. The attendees are a mix of local scientists, researchers and rationalists, which results in a diversity of skills and ideas. We also hold more narrowly focused meetings where smaller groups work on specific projects. We have projects in the pipeline ranging from improving Wikipedia resources related to x-risk, to bringing together AI researchers in order to develop safety guidelines and make the topic of AI safety more mainstream.

Max has assembled an impressive advisory board that includes Stuart Russell, George Church and Stephen Hawking. The advisory board is not just for prestige - the local members attend our meetings, and some others participate in our projects remotely. We consider ourselves a sister organization to FHI, CSER and MIRI, and touch base with them often.

We recently held our launch event, a panel discussion "The Future of Technology: Benefits and Risks" at MIT. The panelists were synthetic biologist George Church, geneticist Ting Wu, economist Andrew McAfee, physicist and Nobel laureate Frank Wilczek and Skype co-founder Jaan Tallinn. The discussion covered a broad range of topics from the future of bioengineering and personal genetics, to autonomous weapons, AI ethics and the Singularity. A video and transcript are available.

FLI is a grassroots organization that thrives on contributions from awesome people like the LW community - here are some ways you can help:

If you have ideas for research or outreach we could be doing, or improvements to what we're already doing, please let us know (in the comments to this post, or by contacting me directly).

If you are in the vicinity of the Boston area and are interested in getting involved, you are especially encouraged to get in touch with us!

Support in the form of donations is much appreciated. (We are grateful for seed funding provided by Jaan Tallinn and Matt Wage.)

Mo

...
Turing Test and Machine Intelligence
by  — 2014-06-14T15:07:20.884Z

The recent news of Eugene Goostman passing the turing test has raised all kinds of debate around what counts as passing the turing tests and whether or not the chatbots are attempting to do it properly.

The Turing Test is not a good criterion for machine intelligence.

The computer's capability to trick humans via imitation in a conversation is an amazing prediction. I think it's important to discern what it means.

For a machine to be able to imitate human language and reasoning succesfully it would essentially need to have intelligence much above the average human. A general intelligence not specifically designed to be a fake human would need to be able to model human behavior and derive from that model communication that was misleading from the AI's "true nature".

Computers supremacy over humans in the boardgame of chess has been a common motif in AI discussion since Kasparov lost to Deep Blue in the 90's. Yet no one is trying to claim that the chesscalculators would have learned to play chess in a similar fashion to humans and would rely on a similar logic as we do. I'm not an expert on programming, AI nor chess, but it stills seems obvious that it would be improper to use thecomputers' current superiority to humans as a solid proof of their high general intelligence - one that is capable of imitating humans and play chess like humans do.

Goal structure for deception vs. Crafted set of tricks and "repeat after me"

For an AI to truly participate in the Turing Test it would need to be self-aware. In addition to being self-aware a goalstructure would be required, and that should include incentive to deceive humans to think that the AI is a human too. More specifically cognitively pretending not to be you, would require self-awareness. This would be very sophisticated and subtle. It's hard for many humans to pretend being someone else - though some excel at it - despite us having a built-in capacity for empathy and already having nearly identical brains. To do the same with an internal "mental" structure that might not be anything like ours would in my opinion require an intelligence on level above the average human or a designed set of tricks.

Are the "Artificial Intelligences" that attempt to pass the Turing Test intelligent at all? To me it seems that the chatbots are essentially one-trick-ponies that merely "repeat after me". Somebody carefully designs an a...
[tangential] Bitcoin: GHash just hit 51%
by David_Gerard — 2014-06-14T08:31:50.695Z

And apparently the sky is falling. From Ittay Eyal and Emin Gün Sirer at Hacking, Distributed:

But the fact is, this is a monumental event. The Bitcoin narrative, based on decentralization and distributed trust, is no more. True, the Bitcoin economy is about as healthy as it was yesterday, and the Bitcoin price will likely remain afloat for quite a while. But the Bitcoin economy and price are trailing indicators. The core pillar of the Bitcoin value equation has collapsed.

They note previous bad behaviour from GHash (which GHash attributed to a rogue employee).

Their proposal is a hard fork, with different parameters (to make huge mining pools no longer an economically rational choice), but respecting the blockchain to date so they can reasonably keep calling it "Bitcoin".
What resources have increasing marginal utility?
by Qiaochu_Yuan — 2014-06-14T03:43:14.195Z

Most resources you might think to amass have decreasing marginal utility: for example, a marginal extra $1,000 means much more to you if you have $0 than if you have $100,000. That means you can safely apply the 80-20 rule to most resources: you only need to get some of the resource to get most of the benefits of having it.

At the most recent CFAR workshop, Val dedicated a class to arguing that one resource in particular has increasing marginal utility, namely attention. Initially, efforts to free up your attention have little effect: the difference between juggling 10 things and 9 things is pretty small. But once you've freed up most of your attention, the effect is larger: the difference between juggling 2 things and 1 thing is huge. Val also argued that because of this funny property of attention, most people likely undervalue the value of freeing up attention by orders of magnitude.

During a conversation later in the workshop I suggested another resource that might have increasing marginal utility, namely trust. A society where people abide by contracts 80% of the time is not 80% as good as a society where people abide by contracts 100% of the time; most of the societal value of trust (e.g. decreasing transaction costs) doesn't seem to manifest until people are pretty close to 100% trustworthy. The analogous way to undervalue trust is to argue that e.g. cheating on your spouse is not so bad, because only one person gets hurt. But cheating on spouses in general undermines the trust that spouses should have in each other, and the cumulative impact of even 1% of spouses cheating on the institution of marriage as a whole could be quite negative. (Lots of things about the world make more sense from this perspective: for example, it seems like one of the main practical benefits of religion is that it fosters trust.)

What other resources have increasing marginal utility? How undervalued are they?
New LW Meetup: Bangalore
by FrankAdamek — 2014-06-13T15:49:32.205Z

This summary was posted to LW Main on June 6th. The following week's summary is here.

New meetups (or meetups with a hiatus of more than a year) are happening in:

Bangalore meetup: 29 June 2014 04:40PM

Irregularly scheduled Less Wrong meetups are taking place in:

Atlanta June meetup - Hacking Motivation: 08 June 2014 07:03PM
Atlanta June meetup - Hacking Motivation: 08 June 2014 10:34PM
Christchurch, NZ Meetup - Games & Discussion: 01 June 2014 04:30PM
Houston, TX: 14 June 2014 02:00PM
Moscow meet up: 08 June 2014 04:00PM
Southeast Michigan Meetup 6/8: 08 June 2014 02:00PM

The remaining meetups take place in cities with regular scheduling, but involve a change in time or location, special meeting content, or simply a helpful reminder about the meetup:

Austin, TX: 06 June 2025 01:30PM
Brussels - Neuroatypicality: 14 June 2014 07:40PM
Canberra: Decision Theory: 14 June 2014 06:00PM
London Social Meetup (possibly) in the Sun: 08 June 2014 02:00PM
Sydney Social Meetup - June (Games night): 12 June 2014 06:30PM
Sydney Meetup - June: 25 June 2014 07:00PM
West LA: 11 June 2025 07:00PM

Locations with regularly scheduled meetups: Austin, Berkeley, Berlin, Boston, Brussels, Buffalo, Cambridge UK, Canberra, Columbus, London, Madison WI, Melbourne, Mountain View, New York, Philadelphia, Research Triangle NC, Salt Lake City, Seattle, Sydney, Toronto, Vienna, Washington DC, Waterloo, and West Los Angeles. There's also a 24/7 online study hall for coworking LWers.

If you'd like to talk with other LW-ers face to face, and there is no meetup in your area, consider starting your own meetup; it's easy (more resources here). Check one out, stretch your rationality skills, build community, and have fun!

In addition to the handy sidebar of upcoming meetups, a meetup overview is posted on the front page every Friday. These are an attempt to collect information on all the meetups happening in upcoming weeks. The best way to get your meetup featured is still to use the Add New Meetup feature, but you'll also have the benefit of having your meetup mentioned in a weekly overview. These overview posts are moved to the discussion section when the new post goes up.

Please note that for your meetup to appear in the weekly meetups feature, you need to post your meetup before the Friday before your meetup!

If you check Less Wrong irre...
Sugar and motivation
by NancyLebovitz — 2014-06-13T15:46:03.078Z

I reread Are Wireheads Happy?[1], which is about the difference between liking and wanting.

I've noticed that too much simple carbs (not terribly much-- three twenty ounce Cokes on consecutive days will do it) severely damages my desire to do much of anything, and the effects take approximately two low-simple carbs days to clear out.

This is obviously something physiological, even though it looks like an emotional problem. Failing to remember clue to avoid simple carbs (a dessert once a week might not be a problem) might be an emotional issue. Too much sugar used to lead to an internal voice saying "I don't care". This time, it didn't, but if I thought of something I might do, there was a clear feeling of "no reward there" and a sense that it was too much effort. I was capable of enjoying things, but not of anticipating that I would like them.

I'm wondering if anything is known about simple carbs, motivation, and/or serotonin/dopamine.

[1]Recommended here.
Science/rationality subjects to teach
by Axel — 2014-06-13T12:31:30.143Z

I recently got a great new job. In short, I organize short (3-9hour) courses on pretty much any subject I want and people can pay to take them. If not enough people reserve a spot, the course is cancelled. If this happens to much I get in trouble with my boss.

This setup gives me a golden opportunity to raise the sanity waterline from the bottom up as most of our customers are ordinary people of the street. I have a few projects I am working on already but would like your input: What are some easy to learn subjects that will get people interested in science/rationality or at least more sane?

Some things to keep in mind: First, it has to grab their interest or they simply won’t come, something that they encounter in their daily lives, something the worry about, etc.
Second, it has to be concrete. Purely theoretical subjects have no success. They come in with clear questions and issues and want them answered, not because of some inherent curiosity about science/rationality. They need to walk out the door feeling they have learned something valuable an useful to them.

So, thoughts?

Edit: My first courses are planned for October, I will report on their succes here on LW if the subject was relevant.
List a few posts in Main and/or Discussion which actually made you change your mind
by Shmi — 2014-06-13T02:42:59.433Z

To quote the front page

> Less Wrong users aim to develop accurate predictive models of the world, and change their mind when they find evidence disconfirming those models, instead of being able to explain anything.

So, by that logic, one interesting metric of the forum quality would be how often what is posted here makes people change their minds. Of course, most of us change their minds almost all the time, but mostly on some mundane topics and in very small amounts, probably too small to pay attention too. But if something comes to mind, feel free to link a thread or two. Depending on the response, we can even try to measure how influential newer posts are vs. older ones.

EDIT: Feel free to mention the Sequence posts, as well, could be a useful benchmark.

EDIT2: Why specifically changing your mind and not just learning something new? Because unlearning is much harder than initial learning, and we, to generalize from one example, tend to forget the unlearned and relapsed into old ways of thinking and doing. (Links welcome). Probably because the patterns etched in the System 1 are not easily erased, and just knowing something intellectually does not remove the old habits. So, successfully unlearning something and internalizing a different view or concept or a way of doing things is indicative of a much more significant impact than "just" learning something for the first time.
Meetup : July Rationality Dojo: Disagreement
by Ruby — 2014-06-12T14:23:04.899Z

Discussion article for the meetup : July Rationality Dojo: Disagreement

WHEN:
06 July 2014 03:00:00PM (+1000)

WHERE:
491 King Street, West Melbourne

[ATTN: The dojo roster is has all free slots starting from next month, if you would like to present at a future Dojo or suggest a topic, please fill it in on the Rationality Dojo Roster: http://is.gd/dojoroster]

The Less Wrong Sunday Rationality Dojos are crafted to be serious self-improvement sessions for those committed to the Art of Rationality and personal growth. Each month a community member will run a session involving a presentation of content, discussion, and exercises.

Continuing the succession of immensely successful dojos, James will run a session on disagreement. How can two epistemic peers, equally knowledgeable and equally competent, ever feel certain about their view when their peer disagrees?

As always, we will review the personal goals we committed to at the previous Dojo (I will have done X by the next Dojo). Scott Fowler recorded the commitments, if you didn't make it but would like to add your own goal to the records, send him a message (shokwave.sf@gmail.com).

The Dojo is likely to run for 2-3 hours, after which some people will get dinner together.

If you have any trouble finding the venue or getting in, call me on 0425-855-124.

Discussion article for the meetup : July Rationality Dojo: Disagreement