trentmkelly/LessWrong-43k huggingface dataset:
First steps of a rationality skill bootstrap
by hamnox — 2019-02-20T00:57:41.661Z

Epistemic status: slightly more advanced than mixing stuff in a pot and throwing it at a wall to see what sticks

This is approximately the first step in a rationality bootstrap system I threw together for myself which I am sharing because I think it's a useful exercise in its own right. It's not without justification—I have spent a while building up and distilling insights from self-help and psych and neuro—buuuut I'm really bad at explaining the big vector field of evidence as it exists in my head so you just get the output! This implementation notably draws on KonMari, Soares' replacing guilt series, bewelltuned.com, this tumblr post on the basics of reasoning, and Kaj Sotala's IFS steelman.

tldr: An introduction to different parts of your brain and how to get them to work together: sensory realism, the output of associative-reactive learning algorithms, and abstracted models. Say hi, let them poke each other for a bit, and give them some guidelines for how to mesh.

Overall Guidelines

Please read through all the instructions before starting.
Have a vision of what you want out of doing it.

Imagine some weeks after trying the exercise your life was reasonably improved. What happens that week? For each of the senses (sight, sound, touch, taste, smell, body sensation, etc.) name several moments you would expect to experience in that world.
Imagine you didn't finish the exercise, and/or didn't find helpful. What happened? For each of the senses, name several moments you would expect to experience in that world.
Talk these expected moments out with a friend or in a journal, *then* commit.
Avoid exaggerating predictions—trying to trick yourself into improving rational thought is really unlikely to go well. Regardless of how this exercise goes, the coming weeks will probably be very similar to the weeks before it.

Commit to completing the whole thing at once
For all your tools, consider

Does this have a high signal to noise ratio? Seek plentiful sources of accurate info so you can build rapid feedback cycles.
Is this high stake? (High cost or high reward?) You want to do the most important thing first.
Does this spark joy or curiosity? Sometimes you need to trust your gut. Not to mention, you have comparative advantage in areas you enjoy engaging with because you will more readily practice and improve in them than for areas you don't.

Express thanks or gratitude for the things y

...
Impact Prizes as an alternative to Certificates of Impact
by ozziegooen — 2019-02-20T00:46:25.912Z
Link: https://forum.effectivealtruism.org/posts/2cCDhxmG36m3ybYbq/impact-prizes-as-an-alternative-to-certificates-of-impact

I posted this one to the EA Forum, because it seemed a bit more relevant there. I think that LessWrong users will also find it interesting.

TLDR Example

An EA donor puts up a $50k prize for distribution in 2022. In 2022, several projects that have started since 2019 apply. Their net EA impacts are estimated, and these estimates (vs. the total value estimate of all submissions) are eventually used to give them corresponding proportional amounts of the $50k.

Back in 2019, several projects sell “rights” to their prize, and these get sold around. It’s expected that $1M in estimated total value will apply, so the market value of the claim of every $10 of estimated impact is ​$0.50. One project sets up an estimation service where they publicly estimate the eventual evaluation of every project, to help make the market more efficient, with the goal of themselves getting part of the prize.
[Link] OpenAI on why we need social scientists
by ioannes — 2019-02-19T16:59:32.319Z

https://distill.pub/2019/safety-needs-social-scientists/

We believe the AI safety community needs to invest research effort in the human side of AI alignment. Many of the uncertainties involved are empirical, and can only be answered by experiment. They relate to the psychology of human rationality, emotion, and biases. Critically, we believe investigations into how people interact with AI alignment algorithms should not be held back by the limitations of existing machine learning. Current AI safety research is often limited to simple tasks in video games, robotics, or gridworlds, but problems on the human side may only appear in more realistic scenarios such as natural language discussion of value-laden questions. This is particularly important since many aspects of AI alignment change as ML systems increase in capability.
Kocherga's leaflet
by Slava Matyukhin — 2019-02-19T12:06:28.257Z

Some context: Kocherga is a rationalist community hub in Moscow.

There was a large crowdfunding campaign for printing the Russian HPMoR edition last year. Kocherga was the main local pickup point in Moscow; we distributed around 5000 copies to 3000 people.

We wanted to promote Kocherga, rationality and EA among these people (most of them just know HPMoR and don't know much about LW/EA), so I designed a new tri-fold leaflet. We handed it out with each set of books.

The reaction to this leaflet was quite positive, so I thought it'd be useful to do an English translation of it and share it here:

Some notes:

Capital letters in "What's happening in Kocherga" list read "cthulhu fhtagn" in Russian, but I wasn't able to adapt it to English. This might explain why the list looks kinda disorganized.

The translation is quite crude, it took me an hour and I didn't edit it as much as I could. If the tone feels a bit off, that's probably the reason.

The labels on the Dunning-Kruger graph are «Competence» on x axis and «Test results» on y axis.

Here's an photo of the actual leaflet with HPMoR books: https://www.instagram.com/p/BqC7eQfl7T7/

If you really like the leaflet and would like to adapt it for your local community, let me know and I'll share the InDesign source file with you.

PS: We're still financially troubled, so if you like what we're doing and want to make sure we keep doing it, please consider supporting Kocherga on Patreon.
Blackmail
by Zvi — 2019-02-19T03:50:04.606Z

Epistemic Status: Surprisingly controversial

Response to: Why Blackmail Should Be Illegal (Marginal Revolution), Should Blackmail Be Legal? (David Henderson), Checkmate on Blackmail (Robin Hanson)

I notice I am confused.

Smart people are failing to provide strong arguments for why blackmail should be illegal. Robin Hanson is explicitly arguing it should be legal. He is correct that Tyler Cowen’s recent justifications of making blackmail illegal are relative weak sauce, but then states that he has reached ‘checkmate’ – that there are no reasonable consequentialist arguments against blackmail.

In his post Charity Blackmail, he lists what he says are the justifications actually made for blackmail, and it’s quite a weak list as well.

In those twenty papers, roughly a quarter of the authors think blackmail should be legal. Others offered a wide range of arguments for illegality. Robin summarized the arguments made by the papers as making only the following mix of good and bad points:

1. Your right to keep quiet is weaker than your right to speak.

2. It is stupid to pay a blackmailer; stupidity should be illegal.

3. A blackmailer’s motives, in wanting money, are immoral.

4. Saying embarrassing things about someone hurts them.

5. It is especially wrong to gain money by hurting someone.

6. The blackmailer uses third parties, without their permission, to extract gains.

7. Blackmail discourages embarrassing activities, but some things just can’t be changed.

8. Blackmailers may commit crimes to get the info, as may victims to get money.

9. Rules forbidding or requiring the telling of certain info might be good, but are less “practical” than blackmail laws.

10. If blackmail is impossible, people will instead gossip, and gossip will result in more folks knowing, and discourage embarrassing activities more.

11. Government law can optimally discourage an activity via optimal punishment and rates of detection and error. Blackmail is an out of control private law, and will get these things wrong by detecting and punishing too often.

So you see why I am confused.

Asking why blackmail should be illegal is a good idea. We need to understand what makes blackmail different from other things we might make illegal – not everything that we dislike should be illegal. Net harmful shouldn’t imply illegal. Blackmail is a place for a thought experiment, to underst...
Decelerating: laser vs gun vs rocket
by Stuart_Armstrong — 2019-02-18T23:21:46.294Z

In our paper on exploring the universe, some of our probes required huge quantities of reaction mass in order to decelerate on arrival.

This is due to the tyranny of the rocket equation: to decelerate our final mass mf by Δv, we need an initial mass m0 so that:

Δv=veln(m0mf), where ve is the exhaust velocity.

Given this equation, the initial mass m0 needed grows as exponential in the required deceleration Δv.

The relativistic rocket equation is similar but even worse, with an extra tanh term in it.

So the most effective way of allowing high speed exploration of the universe is to somehow get around the rocket equation. I was vaguely thinking about ways of using the target galaxy or solar system to do that - maybe the probe could scoop up interstellar dust or use gravity slingshots. But I realised that we can get around the rocket equation more directly.

Decelerating with guns

Suppose you are shooting through space, and you want to decelerate by pointing a gun in the direction of travel. Because of relativity, we can consider that you are at rest, and that you are accelerating by shooting a gun in an opposite direction.

You have two bullets, and you can shoot the bullets sequentially or simultaneously; imagine that you had two guns strapped together. If you shoot the bullets one after another, the first will start moving at velocity of v while you recoil at some −v′. Then when you shoot the second bullet, it will start moving at v−v′ (if we stay in the classical model for the moment). So the total backwards momentum is m(2v−v′), where m is the mass of a bullet.

If you fire the bullets simultaneously, the total backwards momentum is m(2v), however, and m(2v)>m(2v−v′). By conservation of momentum, you will therefore be recoiling faster than if you shot the bullets sequentially.

What's happened? When you shot the bullets sequentially, part of recoil of the first bullet went into moving the second bullet at the same speed as you, which you actually didn't want. When you shot both bullets together, the recoil of both went purely to moving you. Therefore simultaneous fire is more effective at accelerating/decelerating. The real tyranny of the rocket equation comes from the fact that the early fuel needs to move the later fuel that needs to move the even later fuel. And most of that momentum gain is completely wasted: we don't actually care that exhaust fue...
Epistemic Tenure
by Scott Garrabrant — 2019-02-18T22:56:03.158Z

In this post, I will try to justify the following claim (which I am not sure how much I believe myself):

Let Bob be an individual that I have a lot intellectual respect for. For example, maybe Bob had a history of believing true things long before anyone else, or Bob has a discovered or invented some ideas that I have found very useful. Now, let's say that Bob expresses a new belief that feels to me to be obviously wrong. Bob has tried to explain his reasons for the belief, and they seem to also be obviously wrong. I think I can see what mistake Bob is making, and why he is making it. I claim that I should continue to take Bob very seriously, try to engage with Bob's new belief, and give Bob a decent portion of my attention. I further claim, that many people should do this, and do it publicly.

There is an obvious reason why it is good to take Bob's belief seriously. Bob has proven to me that he is smart. The fact that Bob believes a thing is strong evidence that that thing is true. Further, before Bob said this new thing, I would not have trusted his epistemics much less than I trust my own. I don't have a strong reason to believe that I am not the one who is obviously wrong. The situation is symmetric. Outside view says that Bob might be right.

This is not the reason I want to argue for. I think this is partially right, but there is another reason I think people are more likely to miss, that I think pushes it a lot further.

Before Bob had his new bad idea, Bob was in a position of having intellectual respect. An effect of this was that he could say things, and people would listen. Bob probably values this fact. He might value it because he terminally values the status. But he also might value it because the fact that people will listen to his ideas is instrumentally useful. For example, if people are willing to listen to him and he has opinions on what sorts of things people should be working on, he could use his epistemic status to steer the field towards directions that he thinks will be useful.

When Bob has a new bad idea, he might not want to share it if he thinks it would cause him to lose his epistemic status. He may prefer to save his epistemic status up to spend later. This itself would not be very bad. What I am worried about is if Bob ends up not having the new bad idea in the first place. It is hard to have one set of beliefs, an...
A Strange Situation
by Flange Finnegan — 2019-02-18T20:38:26.432Z

Hello all,

I am a newcomer to the site, but I have been lurking and reading the various posts on this site for about 8 months. I have to tell you, life has started making a lot more sense since I read these posts. However, I am feeling somewhat directionless after reading the archives, as I cannot determine a next course of action.

The difficulty is this: I assume most of the poeple on this site are at least midle-aged; I had just turned 18 when I started browsing the site, after being vaguely dissatisfied with Rationalwiki's ability to keep me informed.

Given that, would anyone recommend a general path to follow in terms of what I should be reading from here? I have largely perused Yudkowsky's more well-known works, and have made it partially through the Codex.

Any help, recommendations, or suggestions would be appreciated, and I look forward to new horizons of knowledge as I may find them.
Implications of GPT-2
by Gurkenglas — 2019-02-18T10:57:04.720Z

I was impressed by GPT-2, to the point where I wouldn't be surprised if a future version of it could be used pivotally using existing protocols.

Consider generating half of a Turing test transcript, the other half being supplied by a human judge. If this passes, we could immediately implement an HCH of AI safety researchers solving the problem if it's within our reach at all. (Note that training the model takes much more compute than generating text.)

This might not be the first pivotal application of language models that becomes possible as they get stronger.

It's a source of superintelligence that doesn't automatically run into utility maximizers. It sure doesn't look like AI services, lumpy or no.
Is voting theory important? An attempt to check my bias.
by Jameson Quinn — 2019-02-17T23:45:57.960Z

You may have seen either or both of my two prior posts here, in which I first said I'm beginning to seek a grant for some voting theory research after I graduate, and then looked into some of the possible sources of such a grant. In this post, I'll dig into why I think that this research is important. I have plenty of ideas on that issue, and I'm using this writing as an opportunity to get them in order, so you should not expect this to be a well-organized post.

I think that voting theory is important; that is, that it is an area in which further study and the resulting insights can reasonably be expected to lead to improvements in collective welfare that are big enough to make opportunity costs essentially irrelevant. I also know that I'm biased; I've invested a significant portion of my identity into thinking about voting theory, and even if I could somehow be dispassionately rational in spite of that, an outside view would have to admit that my assessment of the importance of this area is an outlier.

I can't possibly detach myself entirely from this bias, but I should at least try. So here, I'm going to argue both sides. First, I'm going to give the reasons that I think this is important; then, I'm going to try to make the best counterarguments I can; then, I'm going to try to weigh both sides. I think that this format, where I write out one side at a time, is probably a better idea than trying to do all three at once, which seems to be what happens when I do it in my head. Not that that interior arguing in circles can't eventually lead to progress and insight; just that it seems a pretty inefficient and unreliable way of getting there.

I want to be clear about the purpose of writing this. It is, above all, to help me clarify my thinking. I'm going to post it publicly primarily because having a clear idea of my audience helps me write more clearly. I won't deny that I have other motives in publishing this; I also hope to inspire, inform, and/or impress you. But when I have a choice between spending effort clarifying a point that I'm not yet clear on, and explaining one that I am, I'll choose the former.

So, here goes:

Yes, voting theory is important.

Direct impact:

Public elections are pivotal for important stuff;
in principle, they could be done better, so as to get better immediate outcomes on average;
those improvements are possible in practice;
and a better

...
Avoiding Jargon Confusion
by Raemon — 2019-02-17T23:37:16.986Z

Previous discussion on jargon:

Against Naming Things (by whales)
Common vs Expert Jargon (by me)

If you're proposing a new jargon term that is highly specialized, and you don't want people to misuse it...

...it's important to also discuss more common concepts that people are likely to want to refer to a lot, and make sure to give those concepts their own jargon term (or refer to an existing one).

Periodically I see people introduce a new concept, only to find that:

People are motivated to use fancy words to sound smart.
People are motivated to use words to exaggerate, for rhetorical punch or political gain.
People just have multiple nearby concepts that they want to refer to, that they don't have a word for.

Jargon is useful because it lets you cache out complex ideas into simple words, which then become a building block for higher level conversation. It's less useful if the words get diluted over time.

Examples

Schelling Point

The motivating example was "Schelling Point", originally intended to mean "a place or thing people could agree on and coordinate around without communicating."

Then I observed people starting to use "Schelling Point" to mean "any place they wanted to coordinate to meet at." Initially this was a joke, or it referred to a location that probably would have been a real Schelling Point if you hadn't communicated (i.e. if you want to meet later at a park, saying 'The central fountain is the schelling point". It's true that the fountain would have been the natural place to meet if you hadn't been able to coordinate in advance)

And then people started just using it to mean any random thing, and it got harder to tell who actually knew what "Schelling Point" meant.

Affordances and Signifiers

The Design of Everyday Things is a book, originally published in 1988, which introduced a term "affordance", meaning basically "an action a design allows you to take." For example, a lightweight chair can be sat it, or moved around. A heavy chair gives less affordance for lifting.

But the author found that designers were misusing "affordance", and so in the 2013 edition of the book he introduced a second term, "signifier."

Affordances exist even if they are not visible. For designers, their visibility is critical: visible affordances p

...
Robin Hanson on Lumpiness of AI Services
by DanielFilan — 2019-02-17T23:08:36.165Z
Link: http://www.overcomingbias.com/2019/02/how-lumpy-ai-services.html

Basically, in this post Robin Hanson argues that AI systems will have multiple separate components, rather than be a big uniform lump, arguing by analogy with the multiplicity of firms in the economy. Some excerpts:

Long ago people like Marx and Engels predicted that the familiar capitalist economy would naturally lead to the immiseration of workers, huge wealth inequality, and a strong concentration of firms. Each industry would be dominated by a main monopolist, and these monsters would merge into a few big firms that basically run, and ruin, everything. (This is somewhat analogous to common expectations that military conflicts naturally result in one empire ruling the world.)...

Note that many people seem much less concerned about an economy full of small firms populated by people of nearly equal wealth. Actions seem more visible in such a world, and better constrained by competition. With a few big privately-coordinating firms, in contrast, who knows that they could get up to, and they seem to have so many possible ways to screw us...

In the area of AI risk, many express great concern that the world may be taken over by a few big powerful AGI (artificial general intelligence) agents with opaque beliefs and values, who might arise suddenly via a fast local “foom” self-improvement process centered on one initially small system. I’ve argued in the past that such sudden local foom seems unlikely because innovation is rarely that lumpy.

In a new book-length technical report, Reframing Superintelligence: Comprehensive AI Services as General Intelligence, Eric Drexler makes a somewhat similar anti-lumpiness argument. But he talks about task lumpiness, not innovation lumpiness. Powerful AI is safer if it is broken into many specific services, often supplied by separate firms. The task that each service achieves has a narrow enough scope that there’s little risk of it taking over the world and killing everyone in order to achieve that task. In particular, the service of being competent at a task is separate from the service of learning how to become competent at that task...

All these critics seem to agree with Drexler that it is harder to see and control the insides of services, relative to interfaces between them. Where they disagree is in seeing productive efficiency considerations as perhaps creating large natural service “lumps.” A big lumpy service

...