trentmkelly/LessWrong-43k huggingface dataset:
Ten Commandments for Aspiring  Superforecasters
by Evan_Gaensbauer — 2018-04-25T04:55:37.642Z

Cross-posted to the Effective Altruism Forum

In the last several years, political scientist and forecasting research pioneer Philip Tetlock has made waves for the success of his research program in geopolitical forecasting, published in the form of the popular book Superforecasting. It's been discussed much in the rationality community, and reviewed by Slate Star Codex. It turns out the skills of forecasting, such as the importance of taking the outside view into account, Fermi estimates, and Bayesian updating, will be familiar to aspiring rationalists. A lot of the value from the book, then, for readers here would be a summary of insights, so I've reproduced the appendix from the book that does just that below.

(1) Triage

Focus on questions where your hard work is likely to pay off. Don’t waste time either on “clocklike” questions (where simple rules of thumb can get you close to the right answer) or on impenetrable “cloud-like” questions (where even fancy statistical models can’t beat the dart-throwing chimp). Concentrate on questions in the Goldilocks zone of difficulty, where effort pays off the most.

For instance, “Who will win the presidential election, twelve years out, in 2028?” is impossible to forecast now. Don’t even try. Could you have predicted in 1940 the winner of the election, twelve years out, in 1952? If you think you could have known it would be a then-unknown colonel in the United States Army, Dwight Eisenhower, you may be afflicted with one of the worst cases of hindsight bias ever documented by psychologists.

Of course, triage judgment calls get harder as we come closer to home. How much justifiable confidence can we place in March 2015 on who will win the 2016 election? The short answer is not a lot but still a lot more than we can for the election in 2028. We can at least narrow the 2016 field to a small set of plausible contenders, which is a lot better than the vast set of unknown (Eisenhower-ish) possibilities lurking in 2028.

Certain cases of outcomes have well-deserved reputations for being radically unpredictable (e.g., oil prices, currency markets). But we usually don’t discover how unpredictable outcomes are until we have spun our wheels for a while trying to gain analytical traction. Bear in mind the two basic errors it is possible to make here. We could fail to try to predict the potentially predictable or we could ...
Don't Believe Wrong Things
by Jacob Falkovich — 2018-04-25T03:36:09.557Z

This is cross-posted from Putanumonit.com, you can jump in the discussion in either place.

LessWrong has a reputation for being a place where dry and earnest people write dry and earnest essays with titles like “Don’t Believe Wrong Things”. A casual visitor wouldn’t expect it to host lively discussions of prophets, of wizards, and of achieving enlightenment. And yet, each of the above links does lead to LessWrong, and each post (including mine) has more than a hundred comments.

The discussion often turns to a debate that rages eternal in the rationalist community: correctness vs. usefulness. Rationality is about having true beliefs, we are told, but rationalists should also win. Winning, aka instrumental rationality, sure sounds a lot more fun than just believing true things (epistemic rationality). People are tempted to consider it as the primary goal of rationality, with the pursuit of truth being secondary.

Mentions of the “useful but incorrect”, which is how I see Jordan Peterson, invite comments like this:

A correct epistemological process is likely to assign very low likelihood to the proposition of Christianity being true at some point. Even if Christianity is true, most Christians don’t have good epistemics behind their Christianity; so if there exists an epistemically justifiable argument for ‘being a Christian’, our hypothetical cradle-Christian rationalist is likely to reach the necessary epistemic skill level to see through the Christian apologetics he’s inherited before he discovers it.

At which point he starts sleeping in on Sundays; loses the social capital he’s accumulated through church; has a much harder time fitting in with Christian social groups; and cascades updates in ways that are, given the social realities of the United States and similar countries, likely to draw him toward other movements and behavior patterns, some of which are even more harmful than most denominations of Christianity, and away from the anthropological accumulations that correlate with Christianity, some of which may be harmful but some of which may be protecting against harms that aren’t obvious even to those with good epistemics. Oops! Is our rationalist winning?

[…]

epistemic rationality is important because it’s important for instrumental rationality. But the thing we’re interested in is instrumental rationality, not epistemic rationality. If the instrumental bene

...
Wrongology 101
by sarahconstantin — 2018-04-25T00:00:00.991Z

Epistemic Status: Speculative.

Jean-Paul Sartre’s Anti-Semite and Jew, available online here, is a really interesting study of the psychology of anti-Semitism, written in a time (1940’s France) when it was common for people to talk overtly about how much they hated Jews. Sartre, being Gentile and from a culture where anti-Semitism was much more common than it is in 21st century America, had an opportunity to observe these people that I do not. So while he paints an extremely unflattering picture of anti-Semites, one that’s almost hard to believe, I take it seriously.

What are anti-Semites like, according to Sartre?

They are lazy. Sartre gives the example of a man who believes Jews are given unfair advantages in passing an exam he failed, but readily admits that he didn’t study for it.

They are people-oriented rather than thing-oriented. “They behave toward social facts like primitives who endow the wind and the sun with little souls. Intrigues, cabals, the perfidy of one man, the courage and virtue of another — that is what determines the course of their business, that is what determines the course of the world.”

They are impulsive. “the anti‐Semite understands nothing about modern society. He would be incapable of conceiving of a constructive plan; his action cannot reach the level of the methodical; it remains on the ground of passion. To a long‐term enterprise he prefers an explosion of rage analogous to the running amuck of the Malays.”

They are bullies. “He has chosen also to be terrifying. People are afraid of irritating him. No one knows to what lengths the aberrations of his passion will carry him — but he knows, for this passion is not provoked by something external. He has it well in hand; it is obedient to his will: now he lets go of the reins and now he pulls back on them.”

They are conformists. “This man fears every kind of solitariness, that of the genius as much as that of the murderer; he is the man of the crowd. However small his stature, he takes every precaution to make it smaller, lest he stand out from the herd and find himself face to face with himself. He has made himself an anti‐Semite because that is something one cannot be alone. T...
Computational Morality (Part 3) - Similarity of Proposals
by David Cooper — 2018-04-24T22:12:29.095Z

In part 1 ( https://www.lesswrong.com/posts/Lug4n6RyG7nJSH2k9/computational-morality ) I set out a proposal for a system of machine ethics to govern the behaviour of AGI (in which you simply imagine that you are all the people and other sentiences involved in any situation and seek to minimise harm to yourself on that basis, though without eliminating any components of harm which are necessary as a means of accessing enjoyment which you calculate outweighs that harm, because that kind of harm is cancelled out by the gains). People assured me in the comments underneath that it was wrong, though their justifications for doing so appear to be based on faulty ideas which they seemed unable to explore, but it's also hard to gauge how well they took in the idea in the first place. One person, for example, suggested that my proposal might be Rule-Utilitarianism, and the following link was provided: https://en.wikipedia.org/wiki/Rule_utilitarianism . Here is the key part:-

"For rule utilitarians, the correctness of a rule is determined by the amount of good it brings about when followed. In contrast, act utilitarians judge an act in terms of the consequences of that act alone (such as stopping at a red light), rather than judging whether it faithfully adhered to the rule of which it was an instance (such as, "always stop at red lights"). Rule utilitarians argue that following rules that tend to lead to the greatest good will have better consequences overall than allowing exceptions to be made in individual instances, even if better consequences can be demonstrated in those instances."

This is clearly not my proposal at all, but it's interesting none the less. The kind of rules being discussed there are really just general guidelines that lead to good decisions being made in most situations without people having to think too deeply about what they're doing, but there can be occasions when it's moral to break such rules and it may even be immoral not to. Trying to build a system of morality based on a multitude of such rules is doomed from the outset because most (if not all) of those rules will be incorrect in some or many circumstances. The way we judge whether such a rule is right in one situation and wrong in another is by applying a higher-level moral rule (or set of rules), and all of morality is contained at that higher lev...
[Alert] Running Final Migration (possible downtime)
by Raemon — 2018-04-24T20:23:23.659Z

[Update: lesswrong.com is now running as its own container and you should be able to use either URL again. There are some remaining tasks to be done that might result in small snags]

[Update 2: You can log in via google, facebook or github on lesswrong.com now, but be aware this will create a new user, not allow you to log into your old user account]

Today, we're doing a final bit of migration from lesserwrong.com to lesswrong.com. (Currently, lesswrong.com works to access the site, but is not actually the primary domain. This results in a few issues, including our Google and Github logins not working properly)

So today at 3pm, I'll be doing the last steps of the migration, and during a 10 minute window, you will not be able to use lesswrong.com, but will be able to continue using lesserwrong.com.

Afterwards, all lesserwrong.com links will redirect to lesswrong.com, and hopefully you will be able to log in via Google, Github, and Facebook without issue.

Apologies for any inconvenience.
Reward function learning: the value function
by Stuart_Armstrong — 2018-04-24T16:29:32.971Z

I've written quite a few posts about the problems with agents learning values/rewards, and manipulating the learning process. I won't be linking to those past posts, because the aim of this post and the next one is to present a complete, clear, and easy(er) to understand overview of the whole situation. They should stand alone, and serve as an introduction to the subject. This first post will present the framework, and the value function for learnt reward functions; the next one will be looking at the properties of the learning function. I'll be using variants on a single running example throughout, to try and increase understanding.

Feel free to comment on ways the ideas could be made clearer.

0 The main points

The central insight of these posts may seem trivial by now: that a learning process that the agent can influence, does not have the same properties as one that it can't. Moving from "if you don't know what is right, look it up on this read-only list" to "if you don't know what is right, look it up on this read-write list (or ask the programmer)" is a huge change. Especially when read-only lists can easily become read-write in practice when the agent becomes more powerful.

Why write long posts and papers about this idea, though? First of all, because it is easy, in practice, not to notice that shift. And secondly, because it is easy to define a "learning" process that doesn't behave like we'd expect.

So by defining the value function and learning function of an ideal learning process, this allows us to know when we are facing an ideal uninfluenceable, unmanipulable learning process, and when we are not.

Furthermore, many learning processes cannot be easily made unmanipulable - especially those that involve human feedback, feedback conditional on the agent's actions. By specifying the ideal case, and seeing examples of what goes wrong in the non-ideal case, this can help develop learning processes where a small amount of manipulation is allowed, traded off against a large amount of desirable learning.

As a minor example of this, in the next post, we'll see that though "uninfluenceable" is the ideal property, the weaker property of "unriggable" (which I previously called "unbiasable") is enough to get many of the desirable properties.

1 Framework and Ex

... ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
Community Page Mini-Guide
by Ben Pace — 2018-04-24T15:04:53.641Z

Here's how to make a meetup event and/or group on the community page.

1. Login to your (or create a) LessWrong account.

2. Navigate to the community page via the top-left hamburger menu or the community tab on the frontpage.

3. In the centre (underneath the 'local groups' header), click the 'create new event' or 'create new local group' button, depending on your goals. Then fill out the info and you have yourself a meetup!

Info on the page layout:

On the map, green circles are groups (e.g. communities, group-houses, organisations, etc) and blue pointers are events.
Under the map, groups and events are listed based on how far away from you (according to where your browser thinks you are).

If you have any other confusions with the community page, please tell me them in the comments here.
Reward function learning: the learning process
by Stuart_Armstrong — 2018-04-24T12:56:04.793Z

In the previous post, I introduced the formalism for reward function learning, and presented the expected value function for a learning agent:

V(μ,ρ,π,hn)=∑hm∈Hm∑R∈Rm∑i=1Pπ,μ(hm∣hn)ρ(R;π,hm)R(hmi).

I'll assume that people reading this are familiar with the concepts, the notations and the example of that post. I'll now look at the desirable properties for the learning function ρ.

1 Rigging the learning process

1.1 The flaws of general learning agents

First of all, are there any problems with a general ρ? There are three major ones:

1) An agent learning to maximise V with a general ρ has to use the whole of the episode to assess the value of any one action. Thus, it can learn as a Monte-Carlo agent, but not as Q-learning or Sarsa agent.
2) An agent maximising V with a general ρ can take actions the don't increase reward, but are taken purely to "justify" its past actions.
3) An agent maximising V with a general ρ can sometimes pick a policy that is sub-optimal, with certainty, for all rewards R in R.

All of these points can be seen by considering our robot cooking/washing example. In that case, it can be seen that the optimal behaviour for that robot is N,S,E,E,E; this involves cooking the two pizzas, then going East to push the lever onto the cooking choice, and then ending the episode.

Thus ρ(Rc;π,{N,S,E,E,E})=1, so the final reward is Rc, and the agent earns a reward of 2/2−4/20=0.8.

Why must Q-learning fail here? Because the reward for the first N, at the point the agent does it, is 1/2, not 1; this is because, at this point, ρ(Rc;π,{N}) is still 1/2. Thus the reward component in the Q-learning equation is incorrect.

Also note that the rest of the policy, S,E,E,E, serve no purpose to get rewards, they just "justify" the reward from the first action N.

Let us now compare this policy with the policy N,N: go North, cook, end the episode. For the value learning function, this has a value of only 1/2−1/20=0.45, since the final reward is 1/2Rc+1/2Rw. However, under the reward of Rc, this would give a reward of 0.95, more than the 0.8 that Rc gets here. And under the reward of Rw, this would get a reward of −0.05, more than the −0.5 that Rw gets under N,S,E,E,E. Thus the optimal policy for the value learner is worse for both Rc and Rw that the N,N policy.

1.2 Riggable learning processes

The problem with the ρ used in the robot example is t...
On Emotional Responsibility & Abuse
by devas — 2018-04-24T12:50:11.562Z
Link: http://landofwindandskyscrapers.tumblr.com/post/173257968172/on-emotional-responsibility-abuse

Crossposted from my personal Tumblr. This is how one does cross posts, right?

--

This is going to be a pretty serious article.

I very recently listened to a long video on youtube, discussing a case of emotional abuse in close detail, through chat logs and publicly available info on the behaviour of each member of the couple.

I won’t link directly to the video-it’s tangentially about a fantastically controversial affair, and I’d prefer not to catch the attention of a segment of the internet which could and would gleefully ruin my life-but the observation raised during it feel to me very valid. If you hold a different opinion, please let me know, being exposed to different ways of interpreting the same facts is something I value very much.

Anyway, the video brought up several times the fact that in the relationship in question, between person A and person Z, where person Z is the abuser and person A is the victim, Z acted as if A’s action were responsible for Z’s feelings, state of mind and emotions.

Things like “I cheated on you because you made me feel unloved” “You’re devoting more resources to caring about your friend than me, which makes me feel shitty, so stop seeing them entirely”.

And in the moment, I was confused.

Why was the video saying that Z had responsibility over their emotions? One thing I’ve learned throughout my life, although I can’t point to when or where, is that people have no immediate control over their emotions, that they can’t help how they feel about a particular thing when it happens.

This is a very important concept; with it, people can still have their emotions validated, even if the immediate cause for it is trivial or subjective, and it blocks others from shaming them and using their vulnerability to hurt them (in theory).

Think of how a pregnant woman might have heavy mood swings, crying like a fountain because their taco doesn’t have onions, or a football fan might fly in a rage at seeing someone dispute a referee’s decision.

It’s also a concept very important in relationship; to give a hypothetical example, someone can say to their lover: “I’m sorry, I don’t love you. I’m breaking up with you” “But I bought you flowers, gave you chocolates, complimented you, spent all my free time with you...you can’t not love me!” “The things that you did do not make me love you” and the conversation can end there.

(This is a concept that a depressing n...
The many ways AIs behave badly
by Stuart_Armstrong — 2018-04-24T11:40:51.043Z

EDIT: This has been previously posted here. Vika is now maintaining a centralized list of such examples.

I had a previous post about some of the ways AIs behave badly. But now there is a new paper, looking at many examples of (mis)behaviour, within the evolutionary programming design. A video summary of some of the results is here.

So note that these are ways that current agents already (mis)behave; theses are not theoretical arguments about what might happen with a future superintelligence.

These behaviours include:

solving the proxy/heuristic but not the proper problem (eg spinning while falling to get the highest score on a "jump" objective),
cheating on the test (eg playing dumb on a test so that they could get a higher score afterwards),
exploiting bugs in the environment (eg quickly twistching body parts to accumulate errors in the physics simulator and thus get "free energy" to propel themselves fast through virtual water),
agents deliberately crashing other agents (requesting absurdly distant moves on an unbounded tic-tac-toe game, causing the other agents to dynamically expand their memory too much and then crash)
unexpectedly elegant "impossible" solutions (crawling on its elbows when the percentage of time its feet could touch the ground was sent to 0%), and
parasitism (in Tierra, an artificial life system, not only were there parasites, but parasites of parasites).

Abstract:

Biological evolution provides a creative fount of complex and subtle adaptations, often surprising the scientists who discover them. However, because evolution is an algorithmic process that transcends the substrate in which it occurs, evolution's creativity is not limited to nature. Indeed, many researchers in the field of digital evolution have observed their evolving algorithms and organisms subverting their intentions, exposing unrecognized bugs in their code, producing unexpected adaptations, or exhibiting outcomes uncannily convergent with ones in nature. Such stories routinely reveal creativity by evolution in these digital worlds, but they rarely fit into the standard scientific narrative. Instead they are often treated as mere obstacles to be overcome, rather than results that warrant study in their own right. The stories themselves are traded among researchers through oral tradition, but that mode of information transmission is inefficient a

...
LW Update 4/23/2018 – Small Fixes
by Raemon — 2018-04-24T03:38:52.539Z

Sequence Editor Images

Creating a sequence no longer requires you to add an image (a few people had complained that this was a bit obnoxious, esp since the images had to be very large). If you want to find images for your sequence you're welcome to, and if not, admins will touch base and help come up with a suitable header.

Cleanup Search Styling

Searching for posts previously involved some janky looking styles that many people thought was a bug (it wasn't a bug per se... just... really badly styled). It's now been cleaned up a bit.

Highlight Fixes

Post highlights are now a bit longer, so that it's practical to write a short post and have the whole thing just fit on the front page. (Part of an overall build towards encouraging shortform content).

Various small things re: highlights have also been cleaned up (they include the links to linkposts, and the "continue reading" button at the end should no longer sometimes go to an undefined page).

Quick link to the git repo

Commit: 970165cba126af5d2811db8e282839bbb4aa2c56
Welcome to San Diego Rationalist Meetups 
by EmpressCelestia — 2018-04-23T21:14:16.546Z

What kind of events does your group usually run? What does it usually do? How frequently does your group organize events or meet?

--------------------------------------------------------

We typically do three events a month with at least one meetup for just discussion and a board game night. Recently we've started doing regular hikes in the area. Talks are also very common.

Past Meetups:

Earth Day Hiking at Cuyamaca State Park

Introduction to Black Holes and Hawking Radiation (In Memory of Stephen Hawking)

Petrov Day Celebration

Who would be a good fit for you group? Should they have any particular skills or have done some specific background reading?

-------------------------------------------------

A lot of us are associated with UCSD in one way or another and that I think pushes discussions towards academic topics. Having a math and/or CS background probably helps but is in no way required. Honestly if you're reading this, and the above sounded fun, you should stop by!

The Google group that will give you access to event announcements and the calendar (hopefully) is here: https://groups.google.com/forum/#!forum/san-diego-ratadj

There's also a FB group: https://www.facebook.com/groups/1944288489124568/