trentmkelly/LessWrong-43k huggingface dataset:
Reinforcement, Preference and Utility
by royf — 2012-08-08T06:23:25.793Z

Followup to: Reinforcement Learning: A Non-Standard Introduction

A reinforcement-learning agent is interacting with its environment through the perception of observations and the performance of actions.

We describe the influence of the world on the agent in two steps. The first is the generation of a sensory input Ot based on the state of the world Wt. We assume that this step is in accordance with the laws of physics, and out of anyone's hands. The second step is the actual changing the agent's mind to a new state Mt. The probability distributions of these steps are, respectively, σ(Ot|Wt) and q(Mt|Mt-1,Ot).

Similarly, the agent affects the world by deciding on an action At and performing it. The designer of the agent can choose the probability distribution of actions π(At|Mt), but not the natural laws p(Wt+1|Wt,At) saying how these actions change the world.

So how do we choose q and π? Let's first assume that it matters how. That is, let's assume that we have some preference over the results of the process, the actual values that all the variables take. Let's also make some further assumptions regarding this preference relation:

1. The first assumption will be the standard von Neumann-Morgenstern rationality. This is a good opportunity to point out a common misconception in the interpretation of that result. It is often pointed out that humans, for instance, are not rational in the VNM sense. That is completely beside the point.

The agent doesn't choose the transitions q and π. The agent is these transitions. So it's not the agent that needs to be rational about the preference, and indeed it may appear not to be. If the agent has evolved, we may argue that evolutionary fitness is VNM-rational (even if the local-optimization process leads to sub-optimal fitness). If humans design a Mars rover to perform certain functions, we may argue that the goal dictates a VNM-rational preference (even if we, imperfect designers that we are, can only approximate it).

2. The second assumption will be that the preference is strictly about the territory Wt, never about the map Mt. This means that we are never interested in merely putting the memory of the agent in a "good" state. We need it to be followed up by "good" actions.

You may think that an agent that generates accurate predictio...
Group rationality diary, 8/6/12
by cata — 2012-08-08T05:58:52.441Z

This is the public group instrumental rationality diary for the week of August 6th. It's a place to record and chat about it if you have done, or are actively doing, things like:

Established a useful new habit

Obtained new evidence that made you change your mind about some belief

Decided to behave in a different way in some set of situations

Optimized some part of a common routine or cached behavior

Consciously changed your emotions or affect with respect to something

Consciously pursued new valuable information about something that could make a big difference in your life

Learned something new about your beliefs, behavior, or life that surprised you

Tried doing any of the above and failed

Or anything else interesting which you want to share, so that other people can think about it, and perhaps be inspired to take action themselves. Try to include enough details so that everyone can use each other's experiences to learn about what tends to work out, and what doesn't tend to work out.

Thanks to everyone who contributes!

Last week's diary; archive of prior diaries.

(Sorry for being late this week -- I'm on vacation and got distracted :-)
Natural Laws Are Descriptions, not Rules
by pragmatist — 2012-08-08T04:27:28.227Z

Laws as Rules

We speak casually of the laws of nature determining the distribution of matter and energy, or governing the behavior of physical objects. Implicit in this rhetoric is a metaphysical picture: the laws are rules that constrain the temporal evolution of stuff in the universe. In some important sense, the laws are prior to the distribution of stuff. The physicist Paul Davies expresses this idea with a bit more flair: "[W]e have this image of really existing laws of physics ensconced in a transcendent aerie, lording it over lowly matter." The origins of this conception can be traced back to the beginnings of the scientific revolution, when Descartes and Newton established the discovery of laws as the central aim of physical inquiry. In a scientific culture immersed in theism, it was unproblematic, even natural, to think of physical laws as rules. They are rules laid down by God that drive the development of the universe in accord with His divine plan.

Does this prescriptive conception of law make sense in a secular context? Perhaps if we replace the divine creator of traditional religion with a more naturalist-friendly lawgiver, such as an ur-simulator. But what if there is no intentional agent at the root of it all? Ordinarily, when I think of a physical system as constrained by some rule, it is not the rule itself doing the constraining. The rule is just a piece of language; it is an expression of a constraint that is actually enforced by interaction with some other physical system -- a programmer, say, or a physical barrier, or a police force. In the sort of picture Davies presents, however, it is the rules themselves that enforce the constraint. The laws lord it over lowly matter. So on this view, the fact that all electrons repel one another is explained by the existence of some external entity, not an ordinary physical entity but a law of nature, that somehow forces electrons to repel one another, and this isn't just short-hand for God or the simulator forcing the behavior.

I put it to you that this account of natural law is utterly mysterious and borders on the nonsensical. How exactly are abstract, non-physical objects -- laws of nature, living in their "transcendent aerie" -- supposed to interact with physical stuff? What is the mechanism by which the constraint is applied? Could the laws of nature have been different, so that they forced ele...
The High Impact Network (THINK) - Launching Now
by Raemon — 2012-08-08T02:29:02.541Z

THINK, The High Impact Network, is going live this week.

We're a network of Effective Altruists (EAs), looking to do the most good for the most people1 as efficiently as possible. We aren't bound by a central cause or ethical framework, but rather by a process, and a commitment to rigor and rationality as we try to make the world a better place.

THINK meetups are forming around the world. Some are functioning as student groups at prominent universities, others are general meetups for people of all ages who want to make effective altruism a part of their life. As I write this, 20 meetups are getting ready to launch in the fall, and discussions are underway for an additional 30. If you'd like to connect with other EA-types, see if a meetup's forming in your area, or run your own meetup, send us an e-mail here, or visit our website.

We're putting together a collection of meetup modules, which newly formed groups can use for content at weekly meetups. These fall into roughly two categories:

Introductory materials, designed to teach the basics of Effective Altruism to newcomers.

Self Improvement tools, helping newcomers and veterans to become strong enough to tackle the difficult problems ahead.

Five sample modules are available on our website, and more are coming. If you have ideas for a module and would like to create you own, e-mail us at modules@thehighimpactnetwork.org.

But most importantly - we want bright, enthusiastic people who care deeply about the world to collaborate with each other on high impact projects.

Optimal Philanthropy. Effective Altruism.

Less Wrong veterans will recognize the basics of Optimal Philanthropy, although we consider avenues beyond traditional charity. (The phrase "effective altruism" was settled on after much deliberation). For those unfamiliar, a brief overview.

Over the past decade, important changes have begun to take root in the philanthropy/altruist sector:

Organizations like Givewell, as well as a growing number of foundations like the Gates Foundation, are shifting the discussion of giving towards efficiency and evidence.

Groups like Giving What We Can and Bolder Giving are encouraging people to incorporate philanthropy into their lifestyle. You can donate 10% or more of your income and still be among the richest people on the planet, living a satisfying life.

The organization 80,000 H

...
Meetup : Washington DC Social Meetup
by rocurley — 2012-08-08T00:37:04.917Z

Discussion article for the meetup : Washington DC Social Meetup

WHEN:
12 August 2012 03:00:00PM (-0400)

WHERE:
Washington DC, USA

This meetup is the first of a new schedule of meetup, which will alternate "serious", discussion oriented meetups, with "social", more activity based meetups.

This meetup will be a social meetup; we will be playing the game of thrones board game.

Meetup will be at a private residence, please PM or email me for the address.

Discussion article for the meetup : Washington DC Social Meetup
Meetup : Washington DC Social Meetup
by rocurley — 2012-08-08T00:37:03.789Z

Discussion article for the meetup : Washington DC Social Meetup

WHEN:
12 August 2012 03:00:00PM (-0400)

WHERE:
Washington DC, USA

This meetup is the first of a new schedule of meetup, which will alternate "serious", discussion oriented meetups, with "social", more activity based meetups.

This meetup will be a social meetup; we will be playing the game of thrones board game.

Meetup will be at a private residence, please PM or email me for the address.

Discussion article for the meetup : Washington DC Social Meetup
Transcript: "Choice Machines, Causality, and Cooperation"
by Randaly — 2012-08-07T22:15:46.402Z

Gary Drescher's presentation at the 2009 Singularity Summit, "Choice Machines, Causality, and Cooperation," is online, at vimeo. Drescher is the author of Good and Real, which has been recommended many times on LW. I've transcribed his talk, below.

My talk this afternoon is about choice machines: machines such as ourselves that make choices in some reasonable sense of the word. The very notion of mechanical choice strikes many people as a contradiction in terms, and exploring that contradiction and its resolution is central to this talk. As a point of departure, I'll argue that even in a deterministic universe, there's room for choices to occur: we don't need to invoke some sort of free will that makes an exception to the determinism, no do we even need randomness, although a little randomness doesn't hurt. I'm going to argue that regardless of whether our universe is fully deterministic, it's at least deterministic enough that the compatibility of choice and full deterministic has some important ramifications that do apply to our universe. I'll argue that if we carry the compatibility of choice and determinism to its logical conclusions, we obtain some progressively weird corollaries: namely, that it sometimes makes sense to act for the sake of things that our actions cannot change and cannot cause, and that that might even suggest a way to derive an essentially ethical prescription: an explanation for why we sometimes help others even if doing so causes net harm to our own interests.

[1:15]

An important caveat in all this, just to manage expectations a bit, is that the arguments I'll be presenting will be merely intuitive- or counter-intuitive, as the case may be- and not grounded in a precise and formal theory. Instead, I'm going to run some intuition pumps, as Daniel Dennett calls them, to try to persuade you what answers a successful theory would plausibly provide in a few key test cases.

[1:40]

Perhaps the clearest way to illustrate the compatibility of choice and determinism is to construct or at least imagine a virtual world, which superficially resembles our own environment and which embodies intelligent or somewhat intelligent agents. As a computer program, this virtual world is quintessentially determinist: the program specifies the virtual world's initial conditions, and specifies how to calculate everythin...
Depth of Knowledge
by billswift — 2012-08-07T17:39:14.254Z

Depth of Knowledge refers to how thoroughly you know and understand what you have been studying. You need to decide how deeply you need to go into a particular subject.
Recognizing information seen before but not memorized. Information is familiar and you can relate parts when both are presented separately. Very useful for covering a lot of ground quickly and you may not need the information in the future. It really helps when you are trying to look up some particular piece of information by providing context. This is what is mostly tested by multiple-choice tests. Never mistake scoring well on most multiple-choice texts as real learning.
Remembering information, memorizing lists & facts - building up your internal data bank. Working from memory you can present key facts and relationships that you have studied. This is a necessary foundation for deeper learning. If you cannot remember details, then you cannot reason from them. Too often this step is skimped in modern schooling; rote memorization is often necessary for real learning to take place. The most important thing to remember here is that you must PAY ATTENTION to what you are trying to remember. Repeated exposure is necessary for memorization - but you also have to pay attention to the material. The better you pay attention, the less it will need to be repeated for real learning to take place. Having a theoretical framework for the material to fit will also help since we remember mostly by association, this helps recall that. Recall is best tested by fill-in-the-blank type tests and short essay-type questions.
Because memorization is so time and effort intensive, don't waste your time memorizing things that are not important. First, you must remember the basics - arithmetic, spelling, vocabulary, and grammar. Next, you must remember things that you may need without time or references available when you need them, first aid and various other emergency skills. While learning, you need to memorize things that further learning will depend on, the basic facts and relationships in whatever field you are studying. Finally, when actually working in a field, you need to remember frequently used information that it would waste considerable time looking up repeatedly. These are also important...
A question about Singularity Summit 2012
by UngnsCobra — 2012-08-07T12:13:25.300Z

I am planning on going to the Singularity Summit this year, I applied for a student discount earlier on - approximately 3 weeks ago. Still haven't heard back. I am curious to hear if anyone else has applied for student discount and got a reply. I am studying in the UK, so really want to wrap my logistics issues up quickly! Hence, anyone else in the same boat?
Draft: Get Lucky
by atucker — 2012-08-07T06:26:13.809Z

Here's a draft of an article that I want to post soon, but I figured I might as well get some feedback before I go ahead. If anyone else has exercises or experience with this I'd love to hear about it.

The popular science article version of the research mentioned can be found here: http://www.richardwiseman.com/resources/The_Luck_Factor.pdf

There's no fundamental propensity to good outcomes, but it looks like Luck is a thing.

There were some experiments done by Richard Wiseman (of 59 seconds fame/recommendation) investigating luck, and they found that there was a statistically significant factor which led to some people being more likely to receive unexpected benefits.

He took two groups of people, one self-proclaimed “very lucky” and the other normal, and asked them to do simple tasks, like to count the number of times a photograph appeared in a newspaper. He didn't tell them was that he had rigged the magazine to have a few convenient conveniences, like a giant sentence telling them that there were 43 photographs, or large text about how if the reader pointed this out to the researchers they could get $250.

Still, to a statistically significant degree, the “lucky” people noticed them more. What gives?

Have you ever missed an important detail because you were focused on something else? Once you're looking for something, you throw out details that fit. If you're looking for things that fit into the steps of a plan, then you're more likely to throw out stuff that's not already included.

If you weren't trying to count the number of times a word appeared, you probably would have caught the sentences. Planning on counting makes you miss details like it already being done for you.

Engineers bump into this sort of problem a lot. How many times have you spent a long time trying to code something before you found out that it was included in a library? Or tried to write a piece of code that doesn't actually wind up being used because what it accomplishes isn't actually needed?

This failure mode is child of lost causes and priming. If you try to do X in order to get G, then you're primed to miss Y even if it's relevant to G. If the bottom line is written, then you're going to leave stuff out.

Insofar as Luck exists, it seems to be the ability to use unexpected but useful information.

...
[SEQ RERUN] You Provably Can't Trust Yourself
by MinibearRex — 2012-08-07T05:06:32.238Z

Today's post, You Provably Can't Trust Yourself was originally published on 19 August 2008. A summary (taken from the LW wiki):

Lob's theorem provides, by analogy, a nice explanation for why you really can't trust yourself. Don't trust thoughts because you think them, trust them because they were generated by trustworthy rules.

Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was The Cartoon Guide to Löb's Theorem, and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.
SI/CFAR Are Looking for Contract Web Developers
by Malo — 2012-08-06T22:47:43.332Z

The Singularity Institute and the Center for Applied Rationality are looking to expand their web development team. As per usual we'd like to post this opportunity to the LW community first. All work will be on a contract basis (you quote us your hourly rate).

Perks:

Make money while contributing to organizations you care about.

Age and credentials are irrelevant; only product matters.

Requirements:

WordPress and PHP skills (some features require PHP coding).

Good design sense.

Ability to start work on tasks/features quickly and deliver them in a timely manner (i.e., responsiveness).

If you're interested, apply here!
Meetup : Bratislava, Slovakia - the first LW meetup
by BarbaraB — 2012-08-06T10:36:22.781Z

Discussion article for the meetup : Bratislava, Slovakia - the first LW meetup

WHEN: 18 August 2012 05:00:00PM (+0200)

WHERE: Bratislava, city center

Dear Lesswrongians, we cordially invite You to the first Lesswrong meetup in Bratislava ! We just were at the Rationality minicamp in California, met Eliezer and other awesome LessWrongians (and introduced the word "awesome" into our active vocabulary :-) ), and we would finally like to meet the Lesswrong fans in our area. For questions, write in the discussion under this announcement.

We are looking forward to see You !

BarbaraB and Viliam_Bur

Edit: I removed the contact data after the meetup, because the meeting was in my home.

Edit 2: The lesswrongians in our area can still contact me at barbarabrezna@gmail.com. We will try another meetup in October.

Discussion article for the meetup : Bratislava, Slovakia - the first LW meetup
Suber, "The Ethics of Deep Self-Modification"
by lukeprog — 2012-08-06T06:37:47.527Z

Peter Suber is the Director of the Harvard Open Access Project, a Senior Researcher at SPARC (not CFAR's SPARC), a Research Professor of Philosophy at Earlham College, and more. He also created Nomic, the game in which you "move" by changing the rules, and wrote the original essay on logical rudeness.

In "Saving Machines From Themselves: The Ethics of Deep Self-Modification" (2002), Suber examines the ethics of self-modifying machines, sometimes quite eloquently:

If you had the power to modify your deep structure, would you trust yourself to use it? If you had the power to give this power to... an AI, would you do it?

We human beings do have the power to modify our deep structure, through drugs and surgery. But we cannot yet use this power with enough precision to make deep changes to our neural structure without high risk of death or disability. There are two reasons why we find ourselves in this position. First, our instruments of self-modification are crude. Second, we have very limited knowledge about where and how to apply our instruments to get specific desirable effects...

It's conceivable that we might one day overcome both limitations. Even if we do, however, it is very likely that we'll acquire precise tools of self-modification long before we acquire precise knowledge about how to apply them. This is simply because manipulating brain components is easier than understanding brains. When we reach this stage, then we'll face the hard problems of self-modification: when is deep self-modification worth the risk of self-mutilation, and who should be free to make this judgment and take the risk?

Intelligent machines are likely to encounter these ethical questions much sooner in their evolution than human beings. The deep structure of an AI is a consequence of its code... All its cognitive properties and personal characteristics supervene on its code, and modifying the code can be done with perfect precision. A machine's power of self-modification can not only be more precise than ours, but can finally be sufficiently precise to make some deep self-enhancements worth the risk of self-mutilation. At least some machines are likely to see the balance of risks that way.

...Machines with [the ability to self-modify] might look with condolence and sympathy on beings like ourselves who lack it, much as Americans might have looked on Canadians prior to 1982 whe

...