trentmkelly/LessWrong-43k huggingface dataset:
Origin of Morality
by David Cooper — 2018-04-18T20:46:57.728Z

By origin, I'm referring to the source of the need for morality, and it's clear that it's mostly about suffering. We don't like suffering and would rather not experience it, although we are prepared to put up with some (or even a lot) of it if that suffering leads to greater pleasure that outweighs it. We realised long ago that if we do a deal with the people around us to avoid causing each other suffering, we could all suffer less and have better lives - that's far better than spending time hitting each over the head with clubs and stealing the fruits of each other's labour. By doing this deal, we ended up with greater fruits from our work and removed most of the brutality from our lives. Morality is clearly primarily about management of suffering.

You can't torture a rock, so there's no need to have rules about protecting it against people who might seek to harm it. The same applies to a computer, even if it's running AGI - if it lacks sentience and cannot suffer, it doesn't need rules to protect it from harm (other than to try to prevent the owner from suffering any loss if it was to be damaged, or other people who might be harmed by the loss of the work the computer was carrying out). If we were able to make a sentient machine though, and if that sentient machine could suffer, it would then have to be brought into the range of things that need to be protected by morality. We could make an unintelligent sentient machine like a calculator and give it the ability to suffer, or we could make a machine with human-level intelligence with the same ability to suffer, and to suffer to the same degree as the less intelligent calculator. Torturing both of these to generate the same amount of suffering in each would be equally wrong for both. It is not the intelligence that provides the need for morality, but the sentience and the degree of suffering that may be generated in it.

With people, our suffering can perhaps be amplified beyond the suffering that occurs in other animals because there are many ways to suffer, and they can combine. When an animal is chased, brought down and killed by a predator, it most likely experiences fear, then pain. The pain may last for a long time in some cases, such as when wolves eat a musk ox from the rear end while it's still alive, but the victim lacks any real understanding of...
Welcome to the Washington, DC Slate Star Codex Meetup
by Robi Rahman — 2018-04-18T20:36:55.852Z

What kind of events does your group usually run? What does it usually do?

We meet 1-2 times per month to eat snacks and talk about SSC and other rationality-adjacent topics. Sometimes we have dinner or game nights. Join the Google Group if you would like monthly updates: https://groups.google.com/forum/#!forum/dc-slatestarcodex
Mitigate the Loss; Fight for the Living
by Serpent-Stare — 2018-04-18T10:23:24.517Z

This post is an addendum to A Different Prisoner's Dilemma. It is another scenario with which to demonstrate the point.

Imagine that you are a person in a horrible and traumatic situation. The love of your life, the parent of your children, has been burned to death, deliberately. Capital punishment, imposed by a culture that believes some sins are so horrible that they demand this gruesome torture to balance the scales. The whole future you dreamed of having together has gone up in flames, and you are sick with grief and loss and pain.

Some people would be tempted to fling themselves into the flames as well. You probably do think of it, at least once, as you stand there, watching the pyre blaze. Some people will even think that this is the morally right thing to do, and will judge you for not committing suicide. They will think that you didn't really love the one you have lost, if you are able to stay standing and move on without them.

If it is your aim to reduce suffering and death in the world, however, then this impulse and any social pressure behind it is your enemy. If you leap onto the pyre to be burned as well, or hang yourself from a beam at home, or stop taking meals and waste away into nothingness, regardless of the method of your self-destruction, what will it achieve?

Your friends will be hurt. Your community will be hurt. Anyone who depended on your skills and experience and work ethic will be hurt. You will be destroyed, and your children will now have lost both of their parents. It will not bring your lover back.

Furthermore, you will have validated the idea that your suicide was moral. You will have set an example of surrendering to grief instead of coping with it. Others may be tempted, they might believe it was right, to follow you just as you followed your beloved spouse, directly into the grave. Is that something you could possibly want for your children?

You wail. You gnash your teeth and flail your arms and cry late into the night and hold your vigil. Death and suffering have struck you especially close to your heart, and you feel the loss, and you need to express it, but you must not let it consume you as well. If you let that happen, death wins. It claims two casualties for the price of one, and there is one less person working to prevent future suffering.

If you understand this, and you gird up your heart and throw yourself into ...
Welcome to Brisbane Slate Star Codex Meetup [Edit With Your Details]
by Jarred Filmer — 2018-04-18T09:01:03.095Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
Understanding Iterated Distillation and Amplification: Claims and Oversight
by William_S — 2018-04-17T22:36:29.562Z

[Background: Intended for an audience that has some familiarity with Paul Christiano’s approach to AI Alignment. Understanding

Iterated Distillation and Amplification

should provide sufficient background.]

[Disclaimer: When I talk about “what Paul claims”, I am only summarizing what I think he means through reading his blog and participating on discussions on his posts. I could be mistaken/misleading in these claims]

I’ve recently updated my mental model of how Paul Christiano’s approach to AI alignment works, based on recent blog posts and discussions around them (in which I found Wei Dai’s comments particularly useful). I think that the update that I made might be easy to miss if you haven’t read the right posts/comments, so I think it’s useful to lay it out here. I cover two parts: understanding the limits on what Paul’s approach claims to accomplish, and understanding the role of the overseer in Paul’s approach. These considerations are important to understand if you’re trying to evaluate how likely this approach is to work, or trying to make technical progress on it.

What does Paul’s approach claim to accomplish?

First, it’s important to understand that what “Paul’s approach to AI alignment” claims to accomplish if it were carried out. The term “approach to AI alignment” can sound like it means “recipe for building a superintelligence that safely solves all of your problems”, but this is not how Paul intends to use this term. Paul goes into this in more detail in

Clarifying “AI alignment”

.

A rough summary is that his approach will only build an agent that is as capable as some known unaligned machine learning algorithm.

He does not claim that the end result of his approach is an agent that:

Can directly solve all problems which can be solved by a human
Will never take an unsafe catastrophic action
Will never take an action based on a misunderstanding your commands or your values
Could safely design successor agents or self-improve
Will have higher capability than an unaligned competitor

It’s important to understand the limits of what Paul’s approach claims in order to understand what it would accomplish, and the strategic situation that would result.

What is the Overseer?

Iterated Distillation and Amplification

(IDA) describes a procedure that tries to take an overseer and produce an agent that does what the overseer would want it to do, with a reasonable amount of ...
Multi-winner Voting: a question of Alignment
by Jameson Quinn — 2018-04-17T18:51:09.062Z

This is my third (and for now, last) essay about voting theory for rationalists. In the first two, I focused primarily on single-winner voting theory; that is, methods for aggregating group preferences into a final verdict on some choice. Ideally, single-winner methods would be used in cases where decisions are inherently collective, while other mechanisms such as markets are better for cases where decisions are more individual. (As I touched on in the earlier articles, Sen's theorem puts limits on how precisely that distinction can be made; but that's not the point here. I'm going to take it as given that there are some cases where collective action is called for and others where action should be left up to individuals, and I don't want to spend time here arguing about the relative frequency or importance of those two kinds of situations.)

Why isn't multi-winner voting theory just a generalization of the single-winner kind?

If we've covered the best means for collective decisions, and individual decisions are out of the scope of voting theory, then what's left? Governance. That is, cases of collective action that aren't a single decision point but an ongoing series of decisions.

Such cases probably aren't best served by a series of separate single-winner elections, for a few of reasons. To begin with, it's not cognitively efficient; it would be silly for every citizen to need the expertise in order to make decisions about the minutia of every policy area. In fact, direct democracy tends to favor negative-sum rent-seeking: small groups extracting concentrated benefits by imposing diffuse costs, merely because they're the only ones motivated enough to sweat the details. And finally, it's not predictable: in many cases, governance should be coherent even at the sacrifice of some responsiveness.

In cases of governance, voting is not the final step, but merely one step in a larger process of decision-making. Thus, traditional multi-winner voting theory would look at ways to resolve this by electing a set of representatives to take those decisions.

To the rationalist community, such a multi-step process immediately raises the question of alignment. Just as designers of artificial intelligence should worry about whether their initial goals will be warped into a contrary outcome through the process of design and improvement, so should people like me designing multi-st...
Welcome to Racionalidade SP [Edit With Your Details]
by Gust — 2018-04-17T13:08:53.795Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
Computational Morality  (Part 1) - a Proposed Solution
by David Cooper — 2018-04-17T00:09:49.213Z

Religions have had the Golden Rule for thousands of years, and while it's faulty (it gives you permission to do something to someone else that you like having done to you but they don't like having done to them), it works so well overall that it clearly must be based on some underlying truth, and we need to pin down what that is so that we can use it to govern AGI.

What exactly is morality? Well, it isn't nearly as difficult as most people imagine. The simplest way to understand how it works is to imagine that you will have to live everyone's life in turn (meaning billions of reincarnations, going back in time as many times as necessary in order to live each of those lives), so to maximise your happiness and minimise your suffering, you must pay careful attention to harm management so that you don't cause yourself lots of suffering in other lives that outweighs the gains you make in whichever life you are currently tied up in. A dictator murdering millions of people while making himself rich will pay a heavy price for one short life of luxury, enduring an astronomical amount of misery as a consequence. There are clearly good ways to play the game and bad ways, and it is possible to make the right decisions at any point along the way just by weighing up all the available data correctly, although a correct decision isn't guaranteed to lead to the best result because any decision based on incomplete information has the potential to lead to disaster, but there is no way to get around that problem - all we can ever do is hope that things will work out in the way the data says they most probably will, while repeatedly doing things that are less likely to work out well would inevitably lead to more disasters.

Now, obviously, we don't expect the world to work that way (with us having to live everyone else's life in turn), even though it could be a virtual universe in which we are being tested where those who behave badly will suffer at their own hands, ending up being on the receiving end of all the harm they dish out, and also suffering because they failed to step in and help others when they easily could have. However, even if this is not the way the universe works, most of us still care about people enough to want to apply this kind of harm management regardless - we love family and friends, and many of us love the whole of ...
Believable Promises
by Douglas_Reay — 2018-04-16T16:17:42.812Z

8. Believable Promises

Summary of entire Series: An alternative approach to designing Friendly Artificial Intelligence computer systems.

Summary of this Article: How valuable is it, for an AI to be able to believably pre-commit to carrying out a later action that won't, at that later time, benefit it?

Links to all the articles in the series:

Optimum number of single points of failure
Don't put all your eggs in one basket
Defect or Cooperate
Environments for killing AIs
The advantage of not being open-ended
Trustworthy Computing
Metamorphosis
Believable Promises

Links to the parts of this article

Two-way trust
Confidence Trick
Being Gamed
Eroding Value
Betrayal
Conclusion

Two-way trust

For a treaty between AIs (such as between a Rogue being considered for posse admission, and a Sheriff acting on behalf of the posse) to work, you not only need the Sheriff to be able to trust the Rogue (which we looked at in the previous part of this series: Metamorphosis), but also you need the Rogue to be able to have sufficient confidence in the Sheriff's promises that it believes it is more likely than not to be in the Rogue's best interests to negotiate honestly.

So now we need to consider ways the Rogue might fear being cheated.

Confidence Trick

The code of the Gateway program and the underlying GlassNet could be written in such a way that AIs running on it don't have permission to steal back from another AI resources that have been granted to it. But how does the Rogue know for certain that it has been written that way, if it wasn't a participant at the time of creation? Couldn't the whole GlassNet thing be a honey pot, devised by the original posse members to lure in unwary newborn AIs?

So it would be a good idea, when initially setting GlassNet up, to leave an untamperable non-repudiable evidence trail that would have been prohibitively hard to fake at the time. Something like videoing humans at multiple sites booting it up from known published source code, and then allowing random inspection of nodes to check that the code running matches. And then posting the code and videos (or checksums of them) so widely that even a moderately more-than-human AI would be put off by the difficulty of later finding and untraceably hacking into and altering them all.

Being Gamed

When governments fund large infrastructure projects, they often ask construction compani...
Good News for Immunostimulants
by sarahconstantin — 2018-04-16T16:10:00.575Z

Epistemic Status: Moderate

Way back in 2015 I was writing about the connection between cancer remissions and the immune response to infection. To recap the facts:

A plurality of recorded spontaneous cancer remissions happened when the patient had a strong immune response (often with fever) to a bacterial infection at the tumor site.

William Coley’s bacterial therapies for cancer at the turn of the 20th century, while not tested to the standards of modern experimental methods, did seem to produce recovery rates comparable or superior to chemotherapy.

Endotoxin, a poisonous substance found in the outer membrane of Gram-negative bacteria, can cause tumor regressions.

TNF-alpha, an inflammatory cytokine involved in the body’s response to endotoxin, is equally effective at causing tumor regressions; it is too dangerous to give to patients systemically, but is an effective cancer treatment for advanced melanoma when used in isolated limb perfusion.

There are quite a few cases, both in animals and humans, of inflammatory cytokines causing complete tumor regressions in metastatic cancers, particularly when injected directly into the tumor.

At the time, I predicted that if only there were a delivery mechanism that could more effectively isolate inflammatory cytokines to the tumor site, it might work safely for more than just special cases like isolated limb perfusion; and that there might be some delivery mechanism that made a bacterial therapy like Coley’s toxins work.

The heuristic here was that when I went looking for the biggest responses (remissions, complete tumor regressions) in the toughest cases (metastatic cancers, sarcomas which don’t respond to chemotherapy), many of them seemed to involve this picture of acute, intense activation of the innate immune response.

It turns out that two new therapies with very good results pretty much support this perspective.

CpG oligodeoxynucleotides, a motif found in bacterial DNA, are the active ingredient in Coley’s toxins; they are the part of bacterial lysate that triggers the immunostimulatory effects.

Today, SD-101, a CpG oligodeoxynucleotide drug produced by the biotech company Dynavax, is about to present its results from two trials.

This January, Stanford scientists reported that SD-101 combined with another immunotherapy — but no traditional chemotherapy — eradicated both implanted and spontaneous tumors when injec...
The Alignment Newsletter #2: 04/16/18
by Rohin Shah — 2018-04-16T16:00:18.678Z

Highlights

OpenAI Charter

: In their words, this is "a charter that describes the principles we use to execute on OpenAI’s mission".

My opinion: I'm very excited by this charter, it's a good sign suggesting that we can get the important actors to cooperate in building aligned AI, and in particular to avoid a competitive race. Key quote: "if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project".

Lessons Learned Reproducing a Deep Reinforcement Learning Paper

(Matthew Rahtz): It's exactly what the title says. There were a lot of points that I can't easily summarize, but some highlights:

My opinion: If you do deep RL research regularly, you probably won't get too much out of it (though you might still get some handy tips on things you can do with Tensorflow), but I think everyone else should read it to get a more concrete sense of what deep RL research actually looks like and to be able to communicate more effectively with deep RL researchers.

Read more:

Deep Reinforcement Learning Doesn’t Work Yet

A voting theory primer for rationalists

and

5 voting pathologies: lesser names of Moloch

(Jameson Quinn): Voting theory, or social choice theory, studies voting methods, which take a set of preferences over outcomes from voters, and decides which outcome should occur. The field is littered with impossibility results and difficult problems that are hard to resolve, but there are voting methods that do reasonably well in practice. The second post elaborates more on the 5 problems that are hard to resolve.

My opinion: A major challenge for an AI would be to figure out how to aggregate preferences across humans. This is not necessarily a problem that we have to solve immediately -- it's possible that we build an AI that mimics what we do initially and then develops its own theory of voting. However, if you are doing any kind of reward learning (such as inverse reinforcement learning), you will have to confront this problem head on. This article points out a lot of problems that we would have to be aware of in this case. The solutions seem less likely to transfer, because they are optimized for a different scenario (such as presidential elections).

Read more: Will MacAskill's PhD thesis,

Normative Uncertainty

, considers how to comb...
[Preprint for commenting] Fighting Aging as an Effective Altruism Cause
by avturchin — 2018-04-16T13:55:56.139Z

"Fighting Aging as an Effective Altruism Cause: A Model of the Impact of the Clinical Trials of Simple Interventions"

Abstract: The effective altruism movement aims to save lives in the most cost-effective ways. In the future, technology will allow radical life extension, and anyone who survives until that time will gain potentially indefinite life extension. Fighting aging now increases the number of people who will survive until radical life extension becomes possible. We suggest a simple model, where radical life extension is achieved in 2100, the human population is 10 billion, and life expectancy is increased by simple geroprotectors like metformin by three more years on average, so an additional 250 million people survive until “immortality”. The cost of clinical trials to prove that metformin is a real geroprotector is $60 million. In this simplified case, the price of a life saved is around 24 cents, 10 000 times cheaper than saving a life from malaria by providing bed nets. However, fighting aging should not be done in place of fighting existential risks, as they are complementary causes.

Highlights:

● Aging and death are the main causes of human suffering now.

● Simple interventions could extend human lives until aging is defeated.

● These interventions need to be clinically tested before FDA approval.

● A trial of the life extension drug metformin is delayed by lack of funds.

● Starting trials now will save 250 million people from death, at a cost of $0.24 for each life saved.

Please comment on the preprint of the article here:

https://goo.gl/WaEYt5
Welcome to LW-Cologne 
by EGI — 2018-04-16T08:34:20.068Z

LW-Cologne runs a monthly social meetup for people interested in LW / SSC style rationalist philosophy. We also have some overlap with the local EA community. Sometimes there are additional events and new suggestions especially suggestions by new people are allways welcome (If you want something to happen make it happen).

Trusting in the power of self selection our meetup is open to everyone intrested. If you are not familliar with LW / SSC jargon and topics expect some difficulty following the conversation, though we will try our best to accomodate newcomers. Discussion will be in English or Grman depending on the needs of those present.

We organize via a mailing list. To be added to the list you need to send a mail to marcel_mueller@mail.de (The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)