trentmkelly/LessWrong-43k huggingface dataset:
Bayes Academy: Development report 1
by Kaj_Sotala — 2014-11-19T22:35:15.475Z

Some of you may remember me proposing a game idea that went by the name of The Fundamental Question. Some of you may also remember me talking a lot about developing an educational game about Bayesian Networks for my MSc thesis, but not actually showing you much in the way of results.

Insert the usual excuses here. But thanks to SSRIs and mytomatoes.com and all kinds of other stuff, I'm now finally on track towards actually accomplishing something. Here's a report on a very early prototype.

This game has basically two goals: to teach its players something about Bayesian networks and probabilistic reasoning, and to be fun. (And third, to let me graduate by giving me material for my Master's thesis.)

We start with the main character stating that she is nervous. Hitting any key, the player proceeds through a number of lines of internal monologue:

I am nervous.

I’m standing at the gates of the Academy, the school where my brother Opin was studying when he disappeared. When we asked the school to investigate, they were oddly reluctant, and told us to drop the issue.

The police were more helpful at first, until they got in contact with the school. Then they actually started threatening us, and told us that we would get thrown in prison if we didn’t forget about Opin.

That was three years ago. Ever since it happened, I’ve been studying hard to make sure that I could join the Academy once I was old enough, to find out what exactly happened to Opin. The answer lies somewhere inside the Academy gates, I’m sure of it.

Now I’m finally 16, and facing the Academy entrance exams. I have to do everything I can to pass them, and I have to keep my relation to Opin a secret, too.

???: “Hey there.”

Eep! Someone is talking to me! Is he another applicant, or a staff member? Wait, let me think… I’m guessing that applicant would look a lot younger than staff members! So, to find that out… I should look at him!

[You are trying to figure out whether the voice you heard is a staff member or another applicant. While you can't directly observe his staff-nature, you believe that he'll look young if he's an applicant, and like an adult if he's a staff member. You can look at him, and therefore reveal his staff-nature, by right-clicking on the node representing his apperance.]

Here is our very first Bayesian Network! Well, it'...
Meetup : Moscow Meetup: Codename Bilderberg
by Alexander230 — 2014-11-19T21:34:40.508Z

Discussion article for the meetup : Moscow Meetup: Codename Bilderberg

WHEN:
23 November 2014 02:00:00PM (+0300)

WHERE:
Russia, Moscow, ulitsa L'va Tolstogo 16

This meetup will be in semi-closed format! If you hadn't come to our meetups before, please wait for next open meetup!

Here's our plan:

Time and space, and why are they don't exist.

About everyday calibration experience.

When don't you need psychology, or the most efficient ways to get hurt on psychology.

Details and schedule:

https://lesswrong-ru.hackpad.com/-23--xGdpSHSnXdn

Also there will be yudcoins and positive reinforcement, as usual. We will publish more detailed information about time and place in our Google group not long before the meeting:

https://groups.google.com/forum/#!forum/rationality-in-moscow

You will feel more comfortable in our meetup if you read the Core sequences:

http://wiki.lesswrong.com/wiki/Sequences#Core_Sequences

or in Russian:

http://lesswrong.ru/

If you are not interested to read that, you will be probably not interested in our talk contents.

If you want to make a talk on our meetup, please write your offer to our Hackpad:

https://lesswrong-ru.hackpad.com/--8bwR9y6GbfN

We will examine and consider it.

Discussion article for the meetup : Moscow Meetup: Codename Bilderberg
I Want To Believe: Rational Edition
by 27chaos — 2014-11-18T20:00:21.531Z

Relevant: http://lesswrong.com/lw/k7h/a_dialogue_on_doublethink/
I would like this conversation to operate under the assumption that there are certain special times when it is instrumentally rational to convince oneself of a proposition whose truth is indeterminate, and when it is epistemically rational as well. The reason I would like this conversation to operate under this assumption is that I believe questioning this assumption makes it more difficult to use doublethink for productive purposes. There are many other places on this website where the ethics or legitimacy of doublethink can be debated, and I am already aware of its dangers, so please don't mention such things here.
I am hoping for some advice. "Wanting to believe" can be both epistemically and instrumentally rational, as in the case of certain self-fulfilling prophecies. If believing that I am capable of winning a competition will cause me to win, believing that I am capable of winning is rational both in the instrumental sense that "rationality is winning" and in the epistemic sense that "rationality is truth".
I used to be quite good at convincing myself to adopt beliefs of this type when they were beneficial. It was essentially automatic, I knew that I had the ability and so applying it was as trivial as remembering its existence. Nowadays, however, I'm almost unable to do this at all, despite what I remember. It's causing me significant difficulties in my personal life.
How can I redevelop my skill at this technique? Practicing will surely help, and I'm practicing right now so therefore I'm improving already. I'll soon have the skill back stronger than ever, I'm quite confident. But are there any tricks or styles of thinking that can make it more controllable? Any mantras or essays that will help my thought to become more fluidly self-directed? Or should I be focused on manipulating my emotional state rather than on initiating a direct cognitive override?
I feel as though the difficulties I've been having become most pronounced when I'm thinking about self-fulfilling prophecies that do not have guarantees of certainty attached. The lower my estimated probability that the self-fulfilling prophecy will work for me, the less able I am to use the self-fulfilling prophecy as a tool, even if the estimated gains from the bet are large. How might I deal with this problem, specifically?
Meetup : Saint Petersburg, Russia meetup: "Low hanging fruit"
by efim — 2014-11-18T18:44:34.117Z

Discussion article for the meetup : Saint Petersburg, Russia meetup: "Low hanging fruit"

WHEN:
28 November 2014 07:00:00PM (+0300)

WHERE:
Россия, Санкт-Петербург, Московский пр. 65

We will gather for at least three hours, in the same free venue.
There will be small report on chapter of Chialdini's book on self-image and its modifications. Afterward probably discussion and then - exercise.
If we'll have enough time we will run a social skill exercise - hot seat.
And otherwise chat and have fun.
You can reach me by phone - +7-(911)-843-56-44 or mail: efim.wool@gmail.com or vk: vk.com/timetravel, also vk.com/lw_spb

Discussion article for the meetup : Saint Petersburg, Russia meetup: "Low hanging fruit"
Meetup : MelbLW: November Social Meetup
by MelbourneLW — 2014-11-18T12:37:59.602Z

Discussion article for the meetup : MelbLW: November Social Meetup

WHEN:
21 November 2014 06:30:00PM (+0800)

WHERE:
Bull & Bear Tavern, 347 Flinders Lane, Melbourne

November's social meetup is scheduled for this Friday (21st November) as usual. This month, we will be trying out a new venue, at the Bull & Bear Tavern near Flinders Street Station.

Our social meetups are relaxed, informal events where we chat and often play games. The start and finish times are very loose - people will be coming and going throughout the night, so don't worry if you are coming later or have to leave early.

Where? Bull & Bear Tavern, 347 Flinders Lane, Melbourne (on Flinders Lane between Queen and Elizabeth)

When? From 6:30pm until late, Friday November 21st

Contact? If you have any questions, just text or call Richard on 0421231789

Dinner? The Bull & Bear do serve food. At about 11pm, we will most likely go for our traditional post-meetup souvlakis at Stalactites

Games? We will have our own section of the Bull & Bear and can bring board games. If you'd like to play something, bring it along and you should be able to rustle up a group.

To organise similar events, please send an email to melbournelw@gmail.com

Discussion article for the meetup : MelbLW: November Social Meetup
[Link] Chalmers on Computation: A first step From Physics to Metaethics?
by john_ku — 2014-11-18T10:39:48.493Z

A Computational Foundation for the Study of Cognition by David Chalmers

Abstract from the paper:

Computation is central to the foundations of modern cognitive science, but its role is controversial. Questions about computation abound: What is it for a physical system to implement a computation? Is computation sufficient for thought? What is the role of computation in a theory of cognition? What is the relation between different sorts of computational theory, such as connectionism and symbolic computation? In this paper I develop a systematic framework that addresses all of these questions.
Justifying the role of computation requires analysis of implementation, the nexus between abstract computations and concrete physical systems. I give such an analysis, based on the idea that a system implements a computation if the causal structure of the system mirrors the formal structure of the computation. This account can be used to justify the central commitments of artificial intelligence and computational cognitive science: the thesis of computational sufficiency, which holds that the right kind of computational structure suffices for the possession of a mind, and the thesis of computational explanation, which holds that computation provides a general framework for the explanation of cognitive processes. The theses are consequences of the facts that (a) computation can specify general patterns of causal organization, and (b) mentality is an organizational invariant, rooted in such patterns. Along the way I answer various challenges to the computationalist position, such as those put forward by Searle. I close by advocating a kind of minimal computationalism, compatible with a very wide variety of empirical approaches to the mind. This allows computation to serve as a true foundation for cognitive science.

See my welcome thread submission for a brief description of how I conceive of this as the first step towards formalizing friendliness.
Financial Effectiveness Repository
by Gunnar_Zarncke — 2014-11-18T09:57:45.295Z

Follow-Up to: A Guide to Rational Investing Financial Planning Sequence (defunct) The Rational Investor

What are your recommendations and ideas about financial effectiveness?

This post is created in response to a comment on this Altruistic Effectiveness post and thus may have a slight focus on EA. But it is nonetheless meant as a general request for financial effectiveness information (effectiveness as in return on invested time mostly). I think this could accumulate a lot of advice and become part of the Repository Repository (which surprisingly has not much advice of this kind yet).

I seed this with a few posts about this found on LessWrong in the comments. What other posts and links about financial effectiveness do you know of?

Rules:

Each comment should name a single recommendation.

You should give the effectiveness in percent per period or absolute if possible.

Advice should be backed by evidence as usual.

General Advice (from Guide to Rational Investing):

Capital markets have created enormous amounts of wealth for the world and reward disciplined, long-term investors for their contribution to the productive capacity of the economy. Most individuals would do well to invest most of their wealth in the capital market assets, particularly equities. Most investors, however, consistently make poor investment decisions as a result of a poor theoretical understanding of financial markets as well as cognitive and emotional biases, leading to inferior investment returns and inefficient allocation of capital. Using an empirically rigorous approach, a rational investor may reasonably expect to exploit inefficiencies in the market and earn excess returns in so doing.

So what are your recommendations? You may give advanced as well as simple advice. The more the better for this to become a real repository. You may also repeat or link advice given elsewere on LessWrong.
Superintelligence 10: Instrumentally convergent goals
by KatjaGrace — 2014-11-18T02:00:26.375Z

This is part of a weekly reading group on Nick Bostrom's book, Superintelligence. For more information about the group, and an index of posts so far see the announcement post. For the schedule of future topics, see MIRI's reading guide.

Welcome. This week we discuss the tenth section in the reading guide: Instrumentally convergent goals. This corresponds to the second part of Chapter 7.

This post summarizes the section, and offers a few relevant notes, and ideas for further investigation. Some of my own thoughts and questions for discussion are in the comments.

There is no need to proceed in order through this post, or to look at everything. Feel free to jump straight to the discussion. And if you are behind on the book, don't let it put you off discussing. Where applicable and I remember, page numbers indicate the rough part of the chapter that is most related (not necessarily that the chapter is being cited for the specific claim).

Reading: Instrumental convergence from Chapter 7 (p109-114)

Summary

The instrumental convergence thesis: we can identify 'convergent instrumental values' (henceforth CIVs). That is, subgoals that are useful for a wide range of more fundamental goals, and in a wide range of situations. (p109)

Even if we know nothing about an agent's goals, CIVs let us predict some of the agent's behavior (p109)

Some CIVs:

Self-preservation: because you are an excellent person to ensure your own goals are pursued in future.

Goal-content integrity (i.e. not changing your own goals): because if you don't have your goals any more, you can't pursue them.

Cognitive enhancement: because making better decisions helps with any goals.

Technological perfection: because technology lets you have more useful resources.

Resource acquisition: because a broad range of resources can support a broad range of goals.

For each CIV, there are plausible combinations of final goals and scenarios under which an agent would not pursue that CIV. (p109-114)

Notes

1. Why do we care about CIVs?
CIVs to acquire resources and to preserve oneself and one's values play important roles in the argument for AI risk. The desired conclusions are that we can already predict that an AI would compete strongly with humans for resources, and also than an AI once turned on will go to great lengths to stay on and intact.

2. Related work
Steve Omohun...
Neo-reactionaries, why are you neo-reactionary?
by Capla — 2014-11-17T22:31:05.760Z

Through LessWrong, I've discovered the no-reactionary movement. Servery says that there are some of you here.

I'm curious, what lead you to accept the basic premises of the movement? What is the story of your personal "conversion"? Was there some particular insight or information that was important in convincing you? Was it something that just "clicked" for you or that you had always felt in a vague way? Were any of you "raised in it"?

Feel free to forward my questions to others or direct me towards a better forum for asking this.

I hope that this is in no way demeaning or insulting. I'm genuinely curious and my questioning is value free. If you point me towards compelling evidence of the neo-reactionary premise, I'll update on it.
Is this dark arts and if it, is it justified?
by Gleb_Tsipursky — 2014-11-17T19:00:39.448Z

I'd like the opinion of Less Wrongers on the extent to which it is appropriate to use Dark Arts as a means of promoting rationality.

I and other fellow aspiring rationalists in the Columbus, OH Less Wrong meetup have started up a new nonprofit organization, Intentional Insights, and we're trying to optimize ways to convey rational thinking strategies widely and thus raise the sanity waterline. BTW, we also do some original research, as you can see in this Less Wrong article on "Agency and Life Domains," but our primary focus is promoting rational thinking widely, and all of our research is meant to accomplish that goal.

To promote rationality as widely as possible, we decided it's appropriate to speak the language of System 1, and use graphics, narrative, metaphors, and orientation toward pragmatic strategies to communicate about rationality to a broad audience. Some example are our blog posts about gaining agency, about research-based ways to find purpose and meaning, about dual process theory and other blog posts, as well as content such as videos on evaluating reality and on finding meaning and purpose in life.

Our reasoning is that speaking the language of System 1 would help us to reach a broad audience who are currently not much engaged in rationality, but could become engaged if instrumental and epistemic rationality strategies are presented in such a way as to create cognitive ease. We think the ends of promoting rationality justify the means of using such light Dark Arts - although the methods we use do not convey 100% epistemic rationality, we believe the ends of spreading rationality are worthwhile, and that once broad audiences who engage with our content realize the benefits of rationality, they can be oriented to pursue more epistemic accuracy over time. However, some Less Wrongers disagreed with this method of promoting rationality, as you can see in some of the comments on this discussion post introducing the new nonprofit. Some commentators expressed the belief that it is not appropriate to use methods that speak to System 1.

So I wanted to bring up this issue for a broader discussion on Less Wrong, and get a variety of opinions. What are your thoughts about the utility of using light Dark Arts of the type I described above if the goal is to promote rationality - do the ends justify the means? How much Dark Arts, if any, is it appropriate to use...
Stable self-improvement as a research problem
by paulfchristiano — 2014-11-17T17:51:02.000Z

“Stable self-improvement" seems to be a primary focus of MIRI’s work. As I understand it, the problem is "How do we build an agent which rationally pursues some goal, is willing to modify itself, and with very high probability continues to pursue the same goal after modification?"

The key difficulty is that it is impossible for an agent to formally "trust" its own reasoning, i.e. to believe that "anything that I believe is true." Indeed, even the natural concept of "truth" is logically problematic. But without such a notion of trust, why should an agent even believe that its own continued existence is valuable?

I agree that there are open philosophical questions concerning reasoning under logical uncertainty, and that reflective reasoning highlights some of the difficulties. But I am not yet convinced that stable self-improvement as an especially important problem; I think it would be handled correctly by a human-level reasoner as a special case of decision-making under logical uncertainty. This suggests that (1) it will probably be resolved en route to human-level AI, (2) it can probably be "safely" delegated to a human-level AI. I would prefer for energy to be used on other aspects of the AI safety problem.

Consider an agent A which shares our values and is able to reason "as well as we are"---for any particular empirical or mathematical quantity, A's estimate of its expectation is as good as ours. For notational convenience, suppose that A's preferences are the same as "our" preferences, and let U be the associated utility function.

Now suppose that A is thinking about an outcome including the existence of an agent B. (Perhaps B is a new AI that A is considering designing; perhaps B is a version of A that has made some further observations; whatever.) We'd like the agent to evaluate this outcome on its merits. It should think about how good the existence of B is. If B also maximizes U, then A should correctly understand that B's existence will tend to be good.

The expected value of U conditioned on this outcome is just another empirical quantity. If A is as good at estimation as humans, then it won't predictably over- or under-estimate this quantity. And so it will weigh B's existence correctly when considering the consequences of its actions.

So if we really had a “human-level” reasoner in the sense I assumed at the outset, our problem would be solved. There

...
AI caught by a module that counterfactually doesn't exist
by Stuart_Armstrong — 2014-11-17T17:49:11.075Z

EDIT: Whenever I use colloquial phrases like "the AI believes a (false) X" I mean that we are using utility indifference to accomplish that goal, without actually giving the AI false beliefs.

A big problem with most AI control ideas is that the AI will route around them. Suppose you try to ensure that the AI stays friendly by - say - requiring it to ensure that the world's stock markets continue to function. Then a badly motivated AI will cause the destruction it was going to cause anyway - except it will also create some system that technically counts as "the world's stock markets" and ensure that it "continues to function".

I think we can use counterfactuals and corrigibility (AI-acceptable motivation changes) to partially get around the problem, in some circumstances. Suppose we had a candidate AI, with utility U, that we thought was friendly - but that we weren't sure. We have a subgoal, say SG="ensuring that the world's stock markets continue to function". We're pretty sure that if the AI is friendly, then SG will happen. We're pretty sure that if the AI were unfriendly, and the AI did not have SG added as an objective, then SG wouldn't happen. I deliberately chose a rather rough subgoal to illustrate this.

So we then add a module M to the AI. The task of M, which it has chosen to accept, is to look into the future, using the AI's predictive abilities, and see if SG is accomplished. Crucially, it will do so under the assumption that the module M does not exist (or wasn't activated, or will soon be overwritten, or will fail to do anything, or so on). If M finds that SG is not accomplished, it will turn the AI off (in a way compatible with the AI's motivations - see corrigibility).

How is this different from the usual setup? Well, a generic FAI would accomplish SG (by assumption - we may need to work a bit on this part). A generic UFAI without M wouldn't accomplish SG. A generic UFAI with M would accomplish SG, as this prevents it from being turned off.

But a generic UFAI with M which thought that M would have no impact, would behave exactly as a generic UFAI without M - and wouldn't aim to accomplish SG, and thus would be turned off by M.

Note that M can only be used as a last test for a putatively friendly AI. We cannot use it if we are continually refining AIs (either through evolution or simply redesign) as then we are optimising for SG, and SG is...