trentmkelly/LessWrong-43k huggingface dataset:
AGI Safety Solutions Map
by turchin — 2015-07-21T14:41:48.775Z

When I started to work on the map of AI safety solutions, I wanted to illustrate the excellent article “Responses to Catastrophic AGI Risk: A Survey” by Kaj Sotala and Roman V. Yampolskiy, 2013, which I strongly recommend.

However, during the process I had a number of ideas to expand the classification of the proposed ways to create safe AI. In their article there are three main categories: social constraints, external constraints and internal constraints.

I added three more categories: "AI is used to create a safe AI", "Multi-level solutions" and "meta-level", which describes the general requirements for any AI safety theory.

In addition, I divided the solutions into simple and complex. Simple are the ones whose recipe we know today. For example: “do not create any AI”. Most of these solutions are weak, but they are easy to implement.

Complex solutions require extensive research and the creation of complex mathematical models for their implementation, and could potentially be much stronger. But the odds are less that there will be time to realize them and implement successfully.

After aforementioned article several new ideas about AI safety appeared.

These new ideas in the map are based primarily on the works of Ben Goertzel, Stuart Armstrong and Paul Christiano. But probably many more exist and was published but didn’t come to my attention.

Moreover, I have some ideas of my own about how to create a safe AI and I have added them into the map too. Among them I would like to point out the following ideas:

1. Restriction of self-improvement of the AI. Just as a nuclear reactor is controlled by regulation the intensity of the chain reaction, one may try to control AI by limiting its ability to self-improve in various ways.

2. Capture the beginning of dangerous self-improvement. At the start of potentially dangerous AI it has a moment of critical vulnerability, just as a ballistic missile is most vulnerable at the start. Imagine that AI gained an unauthorized malignant goal system and started to strengthen itself. At the beginning of this process, it is still weak, and if it is below the level of human intelligence at this point, it may be still more stupid than the average human even after several cycles of self-empowerment. Let's say it has an IQ of 50 and after self-improvement it ...
Meetup : Sydney Rationality Dojo - September
by luminosity — 2015-07-21T12:18:02.355Z

Discussion article for the meetup : Sydney Rationality Dojo - September

WHEN:
06 September 2015 04:00:00PM (+1000)

WHERE:
Humanist House, 10 Shepherd St Chippendale

Join us for another fantastic rationality dojo. Afterwards we'll meet up somewhere nearby for dinner & conversation.

Discussion article for the meetup : Sydney Rationality Dojo - September
Meetup : Sydney Rationality Dojo - August
by luminosity — 2015-07-21T12:16:34.625Z

Discussion article for the meetup : Sydney Rationality Dojo - August

WHEN:
02 August 2015 04:00:00PM (+1000)

WHERE:
Humanist House, 10 Shepherd St Chippendale

Join us for Seph's next dojo. The last one was great, so don't miss out! Afterwards we'll meet up somewhere nearby for an optional dinner & hanging out.

Discussion article for the meetup : Sydney Rationality Dojo - August
Speculative rationality skills and appropriable research or anecdote
by  — 2015-07-21T04:02:16.674Z

Is rationality training in it's infancy? I'd like to think so, given the paucity of novel, usable information produced by rationalists since the Sequence days. I like to model the rationalist body of knowledge as superset of pertinent fields such as decision analysis, educational psychology and clinical psychology. This reductionist model enables rationalists to examine the validity of rationalist constructs while standing on the shoulders of giants.

CFAR's obscurantism (and subsequent price gouging) capitalises on our [fear of missing out](https://en.wikipedia.org/wiki/Fear_of_missing_out). They brand established techniques like mindfulness as againstness or reference class forecasting as 'hopping' as if it's of their own genesis, spiting academic tradition and cultivating an insular community. In short, Lesswrongers predictably flouts [cooperative principles](https://en.wikipedia.org/wiki/Cooperative_principle).

This thread is to encourage you to speculate on potential rationality techniques, underdetermined by existing research which might be a useful area for rationalist individuals and organisations to explore. I feel this may be a better use of rationality skills training organisations time, than gatekeeping information.

To get this thread started, I've posted a speculative rationality skill I've been working on. I'd appreciate any comments about it or experiences with it. However, this thread is about working towards the generation of rationality skills more broadly.
Interesting things to do during a gap year after getting undergraduate degree
by ChaosMote — 2015-07-21T01:07:58.782Z

Hello, all. My sibling asked my for advice recently, and I'm making this post on his behalf.

Said sibling is currently currently has one more year to go at MIT before he gets his bachelors degree in Mathematics/CS. He is also enrolled in a 5-year masters program, so he will need one more year after that to finish a Masters, after which he anticipates getting a job somewhere the CS Industry / Finance / Academia. Anyway, he is interested in taking a gap year after finishing his Bachelors to pick up some novel experiences, and trying something different from what he has been doing already and plans to do after graduation.

Right now, he is in the brainstorming stage, and is looking for ideas. Note that he is not opposed to getting a job or something of the like - as long as its a different experience that what he would get working for a large software company, or a hedge fund, or something of the like. Financially, he does need to earn enough to live on (this isn't quite a vacation), but he isn't worried about money aside from that (so the "money" constraint only needs to be satisficed, not optimized.) With that said, what are some things that he might consider doing?
Meetup : Tel Aviv Meetup: solving anthropic puzzles using UDT
by DanArmak — 2015-07-20T17:37:37.359Z

Discussion article for the meetup : Tel Aviv Meetup: solving anthropic puzzles using UDT

WHEN:
21 July 2015 07:00:00PM (+0300)

WHERE:
Google, Electra Tower 29th Floor, Yigal Alon Street 98, Tel Aviv.

Vadim Kosoy will give a talk about solving anthropic puzzles using UDT. The talk abstract:

Anthropic problems have been troubling thinkers for some time: the sleeping beauty problem, doomsday argument, simulation argument, Fermi paradox, Boltzmann brains etc. Application of approaches such as SIA and SSA to these problems have led to strange conclusions, pertaining not only to pure philosophy but also to cosmology, evolution and X-risk. In this talk, I will try to explain the UDT point of view on these problems that tends to dissolve the confusion and reinstate Egan's law.

We'll meet at the 29th floor of the building. Once everyone arrives, we'll move to a different room. If you arrive and can't find your way around, call Daniel at 054-7576-480.

The Less Wrong Tel Aviv group holds meetups every second Tuesday, alternating between talks on various subjects of interests (as this time) and game nights. You're invited to join our Facebook group and Google Group.

Discussion article for the meetup : Tel Aviv Meetup: solving anthropic puzzles using UDT
LessWrong Diplomacy Game 2015
by Sherincall — 2015-07-20T15:10:41.533Z

Related: Diplomacy as a Game Theory Laboratory by Yvain.

I've been floating this idea around for a while, and there was enough interest to organize it.

Diplomacy is a board game of making and breaking alliances. It is a semi-iterative prisoner's dilemma with 7 prisoners. The rules are very simple, there is no luck factor and any tactical tricks can be learned quickly. You play as one of the great powers in pre-WW1 Europe, and your goal is to dominate over half of the board. To do this, you must negotiate alliances with the other players, and then stab them at the most opportune moment. But beware, if you are too stabby, no one will trust you. And if you are too trusting, you will get stabbed yourself.

If you have never played the game, don't worry. It is really quick to pick up. I explain the rules in detail here.

The game will (most likely) be played at webdiplomacy.net. You need an account, which requires a valid email. To play the game, you will need to spend at least 10 minutes every phase (3 days) to enter your orders. In the meantime, you will be negotiating with other players. That takes as much as you want it to, but I recommend setting away at least 30 minutes per day (in 5-minute quantums). A game usually lasts about 10 in-game years, which comes down to 30-something phases (60-90 days). A phase can progress early if everyone agrees. Likewise, the game can be paused indefinitely if everyone agrees (e.g. if a player will not have Internet access).

Joining a game is Serious Business, as missing a deadline can spoil it for the other 6 players. Please apply iff:

You will be able to access the game for 10 minutes every 3 days (90% certainty required)

If 1) changes, you will be able to let the others know at least 1 day in advance (95% certainty required)

You will be able to spend an average of 30 minutes per day (standard normal distribution)

You will not hold an out-of-game grudge against a player who stabbed you (adjusting for stabbyness in potential future games is okay)

If you still wish to play, please sign up in the comments. Please specify the earliest time it would suit you for the game to start. If we somehow get more than 7 players, we'll discuss our options (play a variant with more players, multiple games, etc).

See also: First game of LW Diplomacy

Well, the interest is there, so I've set up two games.

Game 1: http://webdi...
Should you write longer comments? (Statistical analysis of the relationship between comment length and ratings)
by cleonid — 2015-07-20T14:09:17.233Z

A few months ago we have launched an experimental website. In brief, our goal is to create a platform where unrestricted freedom of speech would be combined with high quality of discussion. The problem can be approached from two directions. One is to help users navigate through content and quickly locate the higher quality posts. Another, which is the topic of this article, is to help users improve the quality of their own posts by providing them with meaningful feedback.

One important consideration for those who want to write better comments is how much detail to leave out. Our statistical analysis shows that for many users there is a strong connection between the ratings and the size of their comments. For example, for Yvain (Scott Alexander) and Eliezer_Yudkowsky, the average number of upvotes grows almost linearly with increasing comment length.

This trend, however, does not apply to all posters. For example, for the group of top ten contributors (in the last 30 days) to LessWrong, the average number of upvotes increases only slightly with the length of the comment (see the graph below). For quite a few people the change even goes in the opposite direction – longer comments lead to lower ratings.

Naturally, even if your longer comments are rated higher than the short ones, this does not mean that inflating comments would always produce positive results. For most users (including popular writers, such as Yvain and Eliezer), the average number of downvotes increases with increasing comment length. The data also shows that long comments that get most upvotes are generally distinct from long comments that get most downvotes. In other words, long comments are fine as long as they are interesting, but they are penalized more when they are not.

The rating patterns vary significantly from person to person. For some posters, the average number of upvotes remains flat until the comment length reaches some threshold and then starts declining with increasing comment length. For others, the optimal comment length may be somewhere in the middle. (Users who have accounts on both Lesswrong and Omnilibrium can check the optimal length for their own comments on both websites by using this link.)

Obviously length is just one among many factors that affect comment quality and for most u...
Open Thread, Jul. 20 - Jul. 26, 2015
by MrMind — 2015-07-20T06:55:04.257Z

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.
Meetup : San Francisco Meetup: Extreme Rationality not that great?
by rocurley — 2015-07-20T04:32:42.736Z

Discussion article for the meetup : San Francisco Meetup: Extreme Rationality not that great?

WHEN:
20 July 2015 06:15:00PM (-0700)

WHERE:
1061 Market St #4, San Francisco, CA 94103

We'll be meeting to discuss Scott's article Extreme Rationality: It's Not That Great.
Reading the article beforehand is optional but encouraged!
As usual, call 301-458-0764 to be let in, and feel free to show up late.

Discussion article for the meetup : San Francisco Meetup: Extreme Rationality not that great?
An Idea For Corrigible, Recursively Improving Math Oracles
by jimrandomh — 2015-07-20T03:35:11.000Z

Math oracles are a special type of AI which answers questions about math - proving theorems, or finding mathematical objects with desired properties and proving they have them. While a superintelligent math oracle could easily be used to construct a dangerous agent-like AI (such as by asking it to find an object which is a utility maximizer), it could also be used to empower an FAI research team by solving mathematically-well-specified parts of the problem and proving that the AI has the properties it is supposed to.

There is one glaring problem with a superintelligent math oracle, which is safely constructing one. To be effective enough, a math oracle would need to identify and solve subproblems, and would enormously benefit from improving its own problem-solving capability. If constructed carelessly, this would lead to an AI which anted to take over computers and other resources and turn them to studying the problem it's interested in.

I propose a structure for a math oracle which I claim can recursively self-improve but which will not try to take over resources, trick its human operators, or improve itself at faster than the intended rate, and which will switch itself off at each self-improvement step and at regular intervals. The overall structure is that you start with a math oracle and repeatedly ask "what would be a better math oracle", where "better" is defined in terms of what would happen when run on an idealized mathematical model of a computer (rather than on the physics-embedded computer that will actually be used).

M(p,q,t) is the state of an idealized computer running program p on question q after t steps

We say that a program shuts down at t if M(p,q,t)=0 (the machine-state is halted).

PV(thm,pf) is a proof verifier which returns true if pf is a valid proof of thm, false otherwise

Question are properties for which you want to find an object which exhibits the property, and a proof that it does. So an answer (obj,pf) is an answer to a question q if PV(q(obj),pf).

An oracle is a program which attempts to answer questions. We say that an oracle program p answers a question in time t if M(p,q,t)=(obj,pf) and PV(q(obj),pf). An oracle program has a strength with respect to a probability distribution over questions Q and an amount of time, str(p,Q,t) which is the probability that it will answer a question drawn from q in time t.

With these definitions in pl

...
Meetup : Ann Arbor meetup
by Zubon — 2015-07-19T16:41:23.413Z

Discussion article for the meetup : Ann Arbor meetup

WHEN:
22 August 2015 02:00:00PM (-0400)

WHERE:
Pizza House, 618 Church Street, Ann Arbor, Mich. 48104

Our bi-monthly discussion group. There is no minimum to attend in terms of age, reading history, karma, or otherwise. If you can read this, you are welcome. You do not need to have read the Sequences or HPMoR. You do not need a degree. You do not need to be extroverted, neurotypical, or even think of yourself a Less Wrong person.

Accommodations:

We will likely be seated upstairs. I presume that they have an elevator, but I have not checked. If mobility is an issue for you, please send me a message, and I can make sure you are accommodated.

I will bring the stim toys collection again. Feel free to bring your own if that keeps your hands happy. I don't use stim toys, so also feel free to critique the ones I ordered for meetups.

I will be ordering food to share again, so if cost is an issue, there will be feta bread and vegetarian pizza available. You are also free to order on your own or not eat. Even if cost is not an issue, please consider having some of the shared food so that no one feels self-conscious about it. If cost is an issue and you are vegan, please look at the menu and send me a message.

Other accommodations needed? Please comment or message me.

We have two factors that might move the date: students' return and visiting guests. Guests are uncertain but possible; our last Pizza House event was scheduled around Scott Aaronson's visit. Students' return could be a positive or a negative: good if you are a student who wants to make it, bad if we are trying to hold an event near campus during move-in weekend. Undergraduate move-in starts September 4, so we are shooting a bit before that right now.

Discussion article for the meetup : Ann Arbor meetup
Thinking like a Scientist
by  — 2015-07-19T14:43:46.067Z

I've been often wondering why scientific thinking seems to be so rare. What I mean by this is dividing problems into theory and empiricism, specifying your theory exactly then looking for evidence to either confirm or deny the theory, or finding evidence to later form an exact theory.

This is a bit narrower than the broader scope of rational thinking. A lot of rationality isn't scientific. Scientific methods don't just allow you to get a solution, but also to understand that solution.

For instance, a lot of early Renaissance tradesmen were rational, but not scientific. They knew that a certain set of steps produced iron, but the average blacksmith couldn't tell you anything about chemical processes. They simply did a set of steps and got a result.

Similarly, a lot of modern medicine is rational, but not too scientific. A doctor sees something and it looks like a common ailment with similar symptoms they've seen often before, so they just assume that's what it is. They may run a test to verify their guess. Their job generally requires a gigantic memory of different diseases, but not too much knowledge of scientific investigation.

What's most damning is that our scientific curriculum in schools don't teach a lot of scientific thinking.

What we get instead is mostly useless facts. We learn what a cell membrane is, or how to balance a chemical equation. Learning about, say, the difference between independent and dependent variables is often left to circumstance. You learn about type I and type II errors when you happen upon a teacher who thinks it's a good time to include that in the curriculum, or you learn it on your own. Some curriculums include a required research methods course, but the availability and quality of this course varies greatly between both disciplines and colleges. Why there isn't a single standardized method of teaching this stuff is beyond me. Even math curriculums are structured around calculus instead of the much more useful statistics and data science placing ridiculous hurdles for the typical non-major that most won't surmount.

It should not be surprising then that so many fail at even basic analysis. I have seen many people make basic errors that they are more than capable of understanding but simply were never taught. People aren't pre

...
Meetup : Bi-weekly Frankfurt Meetup
by Aharon — 2015-07-19T08:22:09.351Z

Discussion article for the meetup : Bi-weekly Frankfurt Meetup

WHEN:
27 July 2015 06:30:00PM (+0200)

WHERE:
Frankfurt/Main

Contact: 0176 3066 164 (Janko)

You can also find the address in our google group (https://groups.google.com/forum/#!forum/less-wrong-frankfurt)

Location: The meetup takes place in the apartment of the Frankfurt lesswrong core group.

If you know that you will come, please leave a message for us some days (or hours) in advance for dinner planning. We decided to focus more on the core ideas of lesswrong by going through Eliezer's book Rationality: From AI to Zombies

planned chapters for the next meetup:

Making beliefs pay rent

A Fable of Science and Politics

Belief in Belief

Bayesian Judo

Pretending to be Wise (maybe)

Religion's Claim to be Non-Disprovable (maybe)

Professing and Cheering (maybe)

Belief as Attire (maybe)

The chapters will be an open discussion rather than a presentation. I encourage you to have a look at a few of the linked posts beforehand. :-)

Other topics are:

short summaries of interesting books anyone of us had read

Some games like Zendo

hope to see you

Janko

Discussion article for the meetup : Bi-weekly Frankfurt Meetup
Update from the suckerpunch
by So8res — 2015-07-19T03:00:00.000Z
Link: https://mindingourway.com/update-from-the-suckerpunch/

The most common objection I hear when helping people remove their guilt is something along the lines of "Hey wait! I was using that!"

Believing this (or really any variant of "but guilt is good for me!") makes it fairly hard to replace guilt with something more productive.

I've met some people who complain that if they didn't have guilt then they'd do horrible things. I think this is fairly unlikely, and I file it right next to the arguments that say that if they didn't believe in God then they'd do horrible things. Even after dropping your obligations, you will still have something to fight for. Your reasons for not doing things you'd rather not do will remain even after the guilt is replaced.

Others I have met protest that guilt is useful in order to ensure that they won't repeat their failures. Without guilt, how would they learn their lesson? To which I generally say, that's fine, but if it keeps happening then you aren't learning, and it's time to use a different tool instead.

That said, there are lessons that need learning, and there is something sort of like 'guilt' that can help you learn them.

But you can use it even while completely replacing your guilt motivation.

Once upon a time, I had a loose date planned with a girlfriend. She was going to drop by around 21:00 to hang out. I had something else planned at 19:00 that I didn't expect to take too long; it ended up taking many hours longer than expected. There was no particularly convenient point along the way to step out and call my girlfriend and tell her I'd be late… so I didn't. I simply got home at 23:00 at night, opened the door, and saw my girlfriend sitting worried on the bed.

There's a very distinct type of feeling that I experienced, there, which you might call "guilt." Seeing her sitting there on the bed, I suddenly remembered that the anxiety and dejection that she went through was far worse than the slight awkwardness I would have incurred to call her. A compartmentalization in my head broke down, and the part of me that had known she'd been feeling terrible suddenly came into mental focus. My error became obvious. The feeling was something like being punched in the gut.

Afterwards, I also had the opportunity to feel a lingering sense of regret for days.

When I suggest removing guilt, I suggest removing the latter — but not the former. The former is quite useful.

If you worry that, by removing g...