trentmkelly/LessWrong-43k huggingface dataset:
Concrete Ways You Can Help Make the Community Better
by sapphire — 2017-06-17T03:03:33.317Z

There is a TLDR at the bottom

Lots of people really value the lesswrong community but aren't sure how to contribute. The rationalist community can be intimidating. We have a lot of very smart people and the standards can be high. Nonetheless there are lots of concrete ways a normal rationalist can help improve the community. I will focus on two areas - engaging with content and a list of shovel ready projects you can get involved in. I will also briefly mention some more speculative ideas at the end of the post.

1) Engaging with Content:

I have spoken to many people I consider great content creators (ex: Zvi, Putanumonit, tristanm). It’s very common to wish their articles got more comments and engagement. The easiest thing you can do is make a lesswrong account and use the upvote button. Seeing upvotes really does motivate good writers. This only works for lesswrong/reddit but it makes a difference. I can think of several lw articles with less upvotes than people who have personally told me the article was great (ex: norm-one-principle by tristanm [1]).

Good comments tend to be even more appreciated than upvotes, and comments can be left on blog posts. If a post has few comments, then almost any decent quality comment is likely to be appreciated by the author. If you have a question or concern, just ask. Many great authors read all their comments, at least those left in the first few days, and often respond to them. Lots of readers comment very rarely, if at all. 95.1% of people who took the SSC survey comment less than once a month and 73.6% never comment at all [2]. The survey showed that survey takers were a highly engaged group who had read lots of posts. If a blog has very few comments I think you should update heavily towards “it’s a good idea for me to post my comment”.

However, what is most lacking in the rational-sphere is positive engagement with non-controversial content you enjoyed. Recently the SSC sub-reddit found that about 95% of recent content was either in the culture-war thread or contained in a few threads the community considered low quality (based on vote counts) [3]. You can see a similar effect on lesswrong by considering the Dragon Army post [4]. Most good articles posted recently to lesswrong get around 10 comments or less. The Dragon Army post got over 550. I am explicitly not asking people to avoid po...
Announcing AASAA - Accelerating AI Safety Adoption in Academia (and elsewhere)
by  — 2017-06-15T18:55:06.306Z

AI safety is a small field. It has only about 50 researchers, and it’s mostly talent-constrained. I believe this number should be drastically higher.

A: the missing step from zero to hero

I have spoken to many intelligent, self-motivated people that bear a sense of urgency about AI. They are willing to switch careers to doing research, but they are unable to get there. This is understandable: the path up to research-level understanding is lonely, arduous, long, and uncertain. It is like a pilgrimage.

One has to study concepts from the papers in which they first appeared. This is not easy. Such papers are undistilled. Unless one is lucky, there is no one to provide guidance and answer questions. Then should one come out on top, there is no guarantee that the quality of their work will be sufficient for a paycheck or a useful contribution.

Unless one is particularly risk-tolerant or has a perfect safety net, they will not be able to fully take the plunge.
I believe plenty of measures can be made to make getting into AI safety more like an "It's a small world"-ride:

Let there be a tested path with signposts along the way to make progress clear and measurable.

Let there be social reinforcement so that we are not hindered but helped by our instinct for conformity.

Let there be high-quality explanations of the material to speed up and ease the learning process, so that it is cheap.

B: the giant unrelenting research machine that we don’t use

The majority of researchers nowadays build their careers through academia. The typical story is for an academic to become acquainted with various topics during their study, pick one that is particularly interesting, and work on it for the rest of their career.

I have learned through personal experience that AI safety can be very interesting, and the reason it isn’t so popular yet is all about lack of exposure. If students were to be acquainted with the field early on, I believe a sizable amount of them would end up working in it (though this is an assumption that should be checked).

AI safety is in an innovator phase. Innovators are highly risk-tolerant and have a large amount of agency, which allows them to survive an environment with little guidance, polish or supporting infrastructure. Let us not fall for the typical mind fallacy, expecting less risk-tolerant people to move into AI safety all ...
Announcing AASAA - Accelerating AI Safety Adoption in Academia (and elsewhere)
by  — 2017-06-15T18:50:05.816Z

AI safety is a small field. It has only about 50 researchers, and it’s mostly talent-constrained. I believe this number should be drastically higher.

A: the missing step from zero to hero

I have spoken to many intelligent, self-motivated people that bear a sense of urgency about AI. They are willing to switch careers to doing research, but they are unable to get there. This is understandable: the path up to research-level understanding is lonely, arduous, long, and uncertain. It is like a pilgrimage.

One has to study concepts from the papers in which they first appeared. This is not easy. Such papers are undistilled. Unless one is lucky, there is no one to provide guidance and answer questions. Then should one come out on top, there is no guarantee that the quality of their work will be sufficient for a paycheck or a useful contribution.

Unless one is particularly risk-tolerant or has a perfect safety net, they will not be able to fully take the plunge.
I believe plenty of measures can be made to make getting into AI safety more like an "It's a small world"-ride:

Let there be a tested path with signposts along the way to make progress clear and measurable.

Let there be social reinforcement so that we are not hindered but helped by our instinct for conformity.

Let there be high-quality explanations of the material to speed up and ease the learning process, so that it is cheap.

B: the giant unrelenting research machine that we don’t use

The majority of researchers nowadays build their careers through academia. The typical story is for an academic to become acquainted with various topics during their study, pick one that is particularly interesting, and work on it for the rest of their career.

I have learned through personal experience that AI safety can be very interesting, and the reason it isn’t so popular yet is all about lack of exposure. If students were to be acquainted with the field early on, I believe a sizable amount of them would end up working in it.

AI safety is in an innovator phase. Innovators are highly risk-averse and have a large amount of agency, which allows them to survive an environment with little guidance or supporting infrastructure. Let us not fall for the typical mind fallacy, expecting less risk-tolerant people to move into AI safety all by themselves. Academia can provide that supporting infrastructur...
Announcing AASAA - Accelerating AI Safety Adoption in Academia (and elsewhere)
by  — 2017-06-15T18:45:16.910Z

AI safety is a small field. It has only about 50 researchers, and it’s mostly talent-constrained. I believe this number should be drastically higher.

A: the missing step from zero to hero

I have spoken to many intelligent, self-motivated people that bear a sense of urgency about AI. They are willing to switch careers to AIS research, but they are unable to get there. This is understandable: the path up to research-level understanding is lonely, arduous, long, and uncertain. It is like a pilgrimage.

One has to study concepts from the papers in which they first appeared. This is not easy. Such papers are undistilled. Unless one is lucky, there is no one to provide guidance and answer questions. Then should one come out on top, there is no guarantee that the quality of their work will be sufficient for a paycheck or a useful contribution.

Unless one is particularly risk-tolerant or has a perfect safety net, they will not be able to fully take the plunge.
I believe plenty of measures can be made to make getting into AI safety more like an "It's a small world"-ride:

Let there be a tested path with signposts along the way to make progress clear and measurable.

Let there be social reinforcement so that we are not hindered but helped by our instinct for conformity.

Let there be high-quality explanations of the material to speed up and ease the learning process, so that it is cheap.

B: the giant unrelenting research machine that we don’t use

- The majority of researchers nowadays build their careers through academia. The typical story is for an academic to become acquainted with various topics during their study, pick one that is particularly interesting, and work on it for the rest of their career.

I have learned through personal experience that AI safety can be very interesting, and the reason it isn’t so popular yet is all about lack of exposure. If students were to be acquainted with the field early on, I believe a sizable amount of them would end up working in it (though checking that assumption is something to be done).

- AI safety is in an innovator phase. Innovators are highly risk-averse and have a large amount of agency, which allows them to survive an environment with little guidance or supporting infrastructure. Let us not fall for the typical mind fallacy, expecting risk-averse people to move into AI safety all by themselve...
Thought experiment: coarse-grained VR utopia
by cousin_it — 2017-06-14T08:03:20.276Z

I think I've come up with a fun thought experiment about friendly AI. It's pretty obvious in retrospect, but I haven't seen it posted before.

When thinking about what friendly AI should do, one big source of difficulty is that the inputs are supposed to be human intuitions, based on our coarse-grained and confused world models. While the AI's actions are supposed to be fine-grained actions based on the true nature of the universe, which can turn out very weird. That leads to a messy problem of translating preferences from one domain to another, which crops up everywhere in FAI thinking, Wei's comment and Eliezer's writeup are good places to start.

What I just realized is that you can handwave the problem away, by imagining a universe whose true nature agrees with human intuitions by fiat. Think of it as a coarse-grained virtual reality where everything is built from polygons and textures instead of atoms, and all interactions between objects are explicitly coded. It would contain player avatars, controlled by ordinary human brains sitting outside the simulation (so the simulation doesn't even need to support thought).

The FAI-relevant question is: How hard is it to describe a coarse-grained VR utopia that you would agree to live in?

If describing such a utopia is feasible at all, it involves thinking about only human-scale experiences, not physics or tech. So in theory we could hand it off to human philosophers or some other human-based procedure, thus dealing with "complexity of value" without much risk. Then we could launch a powerful AI aimed at rebuilding reality to match it (more concretely, making the world's conscious experiences match a specific coarse-grained VR utopia, without any extra hidden suffering). That's still a very hard task, because it requires solving decision theory and the problem of consciousness, but it seems more manageable than solving friendliness completely. The resulting world would be suboptimal in many ways, e.g. it wouldn't have much room for science or self-modification, but it might be enough to avert AI disaster (!)

I'm not proposing this as a plan for FAI, because we can probably come up with something better. But what do you think of it as a thought experiment? Is it a useful way to split up the problem, separating the complexity of human values from the complexity of non-human nature?
Lloyd's of London and less-than-catastrophic risk
by fortyeridania — 2017-06-14T02:49:58.254Z

I recently found that Lloyd's has a number of interesting resources on risk. One is the City Risk Index, the methodology for which comes from Cambridge's Judge Business School.

The key metric is something they call GDP@Risk. Despite the name, it is not simply an application of Value@Risk to GDP. Instead, it is simply the sum of the expected damage from a given threat (or from a set of threats) during a given time period. In this case, the time period is 2015-2015. The threats considered include manmade ones (e.g., cyber attack, oil price shock) and natural ones (e.g., drought, solar storm). The site includes brief case studies for the threats. For example, the "plant epidemic" study focuses on the demise of the Gros Michel banana:

Event: Panama disease outbreak, 1950s

Location: Latin America

Economic cost: Estimated losses across Latin America were around $400m ($2.3bn today) although this figure does not include any of the economic losses caused by unemployment, abandoned villages and unrealised income in the affected region.

Description: The Fusarium oxysporum cubense fungus was first diagnosed in Panama but quickly travelled across Central America.

Damage: The disease wiped out the Gros Michel banana, the principal cultivar at the time, from plantations across the region. Between 1940 and 1960, around 30,000 hectares of Gros Michel plantations were lost in the Ulua Valley of Honduras, and in a decade 10,000 hectares were lost in Suriname and the Quepos area of Costa Rica.

Insight: Gros Michel was replaced in the 1960s by Cavendish, a variety thought to be resistant to the disease. However, a new strain of the pathogen was found to be attacking Cavendish plantations in Southeast Asia in the early 1990s. It has since spread, destroying tens of thousands of hectares across Indonesia and Malaysia, and costing more than $400m in the Philippines alone. There is concern that it could reach Central America and destroy up to 85% of the world’s banana crop. Solutions to contain the disease could include increasing genetic diversity among banana cultivars and developing hybrid varieties with stronger resistance.

As the name implies, the site quantifies risks from these threats at the city level. So which cities are the most at risk from a plant epidemic? They're all in APAC:

Hong Kong ($3.83b)

Shanghai

...
Persona 5: Spoiler-Free Review
by Zvi — 2017-06-13T22:32:14.000Z

Epistemic Status: Spent 101 hours finishing the game, and similar amounts on finishing Persona 3 and Persona 4.

Short Version: Strongly endorse the game, the best Persona experience yet, but in the most important ways Persona 3 is still the best. If you would consider playing a Persona game, you should play one.

There are some minor spoilers here because it’s impossible to review a game without them, but I have done my best to make sure none of them impact the game experience. The tips at the end strike a balance between minor spoilers and helping you get the most out of the game.

Persona in General (Skip if you have played Persona 3, 4 or 5)

Playing a Persona game is a big commitment. A lot of what the games do is immerse you in their world and what it feels like to inhabit the life of the main character, and build his relationships with other characters. The passage of time, both in game time and real world clock time, is integral to that. Persona 5 does not need 100 hours to make its point (and I could have cut 10 of those out by doing less unnecessary grinding), but I think it genuinely does need 50.

The basic mechanic of Persona is that you decide how to spend each afternoon and each evening. You can spend that time in a variety of ways, all of them beneficial, and the game’s key events happen on their own schedule no matter what you do, so your time is precious. The two key types of actions are going into dungeons (the game can call them whatever it wants, they’re still dungeons), and doing everything else, where you are preparing for those dungeons.

In the dungeons, which in this game are called palaces, you are trying to accomplish as much as possible each trip. You want to gain as many levels and make as much money as possible, and get through as much of the palace as possible, so you don’t waste extra days you can use for something else. The combat is a lot like other RPGs, with the unique mechanic being the main character’s personas.

You get various personas in combat (and can pay to get back one you have had in the past), and can then combine them to make new personas. Your persona determines your attributes, which elements you are strong and weak to, and what skills (basically spells) you can use. You will be trying to keep your personas strong, which means continuously combining them into higher level personas, while keep...
Mathematical System For Calibration
by DragonGod — 2017-06-13T12:01:08.536Z

I am working on an article titled "You Can Gain Information Through Psychoanalysing Others", with the central thesis being with knowledge of the probability someone assigns a proposition, and their calibration, you can calculate a Bayesian probability estimate for the truthhood of that proposition.

&nbsp;

For the article, I would need a rigorously mathematically defined system for calculating calibration given someone's past prediction history. I thought of developing one myself, but realised it would be more prudent to inquire if one has already been invented to avoid reinventing the wheel.

&nbsp;

Thanks in advance for your cooperation. :)

&nbsp;

#Disclaimer

I am chronically afflicted with a serious and invariably fatal epistemic disease known as narcissist bias (this is a misnomer as it refers a broad family of biases). No cure is known yet for narcissist bias, and I’m currently working on cataloguing and documenting the disease in full using myself as a test case. This disease affects how I present and articulate my points—especially in written text—such that I assign a Pr of > 0.8 that a somebody would find this post condescending, self-aggrandising, grandiose or otherwise deluded. This seems to be a problem with all my writing, and a cost of living with the condition I guess. I apologise in advance for any offence received, and inform that I do not intend to offend anyone or otherwise hurt their sensibilities.
The Rationalistsphere and the Less Wrong wiki
by Deku-shrub — 2017-06-12T23:29:30.722Z

Hi everyone!

For people not acquainted with me, I'm Deku-shrub, often known online for my cybercrime research, as well as fairly heavy involvement in the global transhumanist movement with projects like the UK Transhumanist Party and the H+Pedia wiki.

For almost 2 years year now on and off I have been trying to grok what Less Wrong is about, but I've shirked reading all the sequences end to end, instead focused on the most popular ideas transmitted by Internet cultural osmosis. I'm an amateur sociologist and understanding Less Wrong falls within my wider project of understanding the different trends within the contemporary and historical transhumanist movement.

I'm very keen to pin down today's shape of the rationalistsphere and its critics, and the best place I have found do this is on the wiki. Utilising Cunningham's Law at times, I've been building some key navigational and primer articles on the wiki. However with the very lowest hanging fruit now addressed I ask - what next for the wiki?

Distillation of Less Wrong

There was a historical attempt to summerise all major Less Wrong posts, an interesting but incomplete project. It was also approach without a usefully normalised approach. Ideally, every article would have its own page which could be heavily tagged up with metadata such a themes, importance, length, quality, author and such. Is this the goal of the wiki?

Outreach and communications

Another major project is to fully index the Diaspora across Twitter, Blogs, Tumblr, Reddit, Facebook etc and improve the flow of information between the relevant sub communities.

You'll probably want to join one of the chat platforms if you're interested in getting involved. Hell, there are even a few memes and probably more to collect.

Rationalist research

I'll admit I'm ignorant of the goal of Arbital, but I do love me a wiki for research. Cross referencing and citing ideas, merging, splitting, identifying and fully capturing truly interesting and useful ideas from fanciful and fleeting ones is how I've become an expert in a number fields, just by being the first to assemble All The Things.

Certain ideas like the Paper clip maximizer have some popularity beyond just Less Wrong, but Murder Gandhi doesn't - yet. Polishing these ideas with existing and external references (and maybe blogging about them?) is a great way for the community discussion of yore to make its way ...
Am I/Was I a Cultist?
by DragonGod — 2017-06-12T23:21:32.999Z

I have been accused repeatedly of being a cultist whenever I wage the rationalist crusade online, and naturally I refute such allegations. However, I cannot deny that I take whatever arguments Yudkowsky (makes whose reasonability I can not ascertain for myself as by default true; an example is the Many Worlds Interpretation of quantum mechanics whose Science is far above my head, but I nonetheless took it as truth—the probabilistic variety and not the absolute kind as such honour I confer only to Mathematics—and was later enlightened that MWI is not as definitive as Yudkowsky makes it out to be, and is far from a consensus in the Scientific community). I was surprised at my blunder considering that Yudkowsky is far from an authority figure on Physics, and even if he was I was not unaware of Huxley's maxim:

The improver of natural knowledge cannot accept authority as such; for them scepticism is the highest of virtues—blind faith the one unpardonable sin.

This was the first warning flag. FUrthermore, around the time after I was introduced to RAZ (and the lesswrong website) I started following RAZ with more fervour than I ever did the Bible; I went as far as to—on multiple occasions—proclaim:

Rationality: From AI to Zombies is my Quran, and Eliezer Yudkowsky my Muhammed.

Someone who was on the traditional rationality side of the debate repeatedly described me as "lapping up Yudkowsky's words like a cultist on koolaid." I was warned by a genuinely good meaning friend that I should never let a single book influence my entire life so much, and I must admit; I never was sceptical towards Yudkowsky's words.

Perhaps the biggest alarm bell, was when I completely lost my shit and told the traditional rationalist that I would put him on permanent ignore if he "ever insults the Lesswrong community again. I am in no way affiliated with Eliezer Yudkowsky or the Lesswrong community and would not tolerate insults towards them". That statement was very significant because of its implications:
1. I was willing to tolerate insults towards myself, but not towards Yudkowsky or Lesswrong.
2. I was defensive about Yudkowsky in a way I'd only ever been about Christianity.
3. I elevated Yudkowsky far above my self and put him on a pedestal; when I was a Christian, I believed that I was the best thing since John the Baptist, and would ...