trentmkelly/LessWrong-43k huggingface dataset:
More Notes on Simple Rules
by Zvi — 2019-05-21T14:50:00.305Z

Original Post by Robin: Simple Rules

Previously: Simple Rules of Law

Sarah Constantin on Twitter (if you are doomed to be on social media at all, you should follow her) offers a commentary thread of alternate explanations for the pattern pointed out in Robin’s post. It contains some ideas that I didn’t cover and deserve to be addressed, so this post will do that, and capture her arguments in an easier-to-find-in-the-future location. Quotes are from the thread.

https://t.co/QKnSNtqC8r @robinhanson's explanation for why people prefer discretion to simple rules is overconfidence — everyone assumes *they'd* be the one to have special pull with decision-makers, or wants to pretend they are. "Let's play fair" is a loser's position.

— Sarah Constantin (@s_r_constantin) May 20, 2019

1.) Low trust: nobody believes a “fair” rule would actually be applied fairly, they aren’t considering the possibility of a genuinely impartial rule (and sometimes they’re right)

In this scenario, people would prefer rule of law via a simple and fair rule to arbitrary decision making. But when someone proposes such a rule, or tells them they will be bound by such a rule, they do not believe it. They think that this is code for them having to abide by the rule when it suits those in power, only to have those in power ignore the rule when they would rather something else happen. So you need to obey the rules or get blamed for it, and others don’t. Or, alternatively, the rules are an additional thing to be scapegoated for when the political winds call for it, without taking away any of the other methods. If you obey the rules they get you for ignoring what really matters. If you focus on what really matters they get you for not obeying the rules.

This happens often enough that the rules are now just another way to get gas-lit. Following the rules truly becomes a loser’s position, in every sense.

I’d consider this a ‘good’ objection, similar to the Goodhart’s Law objections. Simple rules, and rule of law, require high enough trust and willingness to uphold those rules, or they become another source of complexity and a tool of power. If you don’t have the trust, it must be established slowly over time, or you need to structure things such that this trust emerges.

The word trust is overloaded, so there are things often called ‘trust’ that are directly opposed to the kind of trust that enables s...
By default, avoid ambiguous distant situations
by Stuart_Armstrong — 2019-05-21T14:48:15.453Z

"The Ood.[...] They're born for it. Basic slave race." Mr. Jefferson, The Impossible Planet.

"And although he may be poor, he shall never be a slave,", from the Battle Cry of Freedom.

I've talked about morally distant situations: situations far removed from our own. This distance is in the moral sense, not the actual sense: ancient China is further from us than Star Wars is (even for modern Chinese). There are possible worlds out there far more alien than anything in most of our fiction.

In these distant situations, our usual web of connotations falls apart, and closely related terms start to mean different things. As shown in this post, in extreme cases, our preferences can become nonsensical.

Note that not all our preferences need become nonsensical, just some of them. Consider the case of a slave race - a willing slave race. In that situation, our connotations about slavery may come apart: willing and slave don't fit well together. However, we would still be clear ourselves that we didn't want to be slaves. So though some preferences lose power, some do not.

But let's return to that willing slave race situation. Eliezer's Harry says:

Whoever had created house elves in the first place had been unspeakably evil, obviously; but that didn't mean Hermione was doing the right thing now by denying sentient beings the drudgery they had been shaped to enjoy.)

Or consider Douglas Adam cow-variant bread for willingly being eaten:

"I just don't want to eat an animal that's standing here inviting me to," said Arthur, "it's heartless."

"Better than eating an animal that doesn't want to be eaten," said Zaphod. [...]

"May I urge you to consider my liver?" asked the animal, "it must be very rich and tender by now, I've been force-feeding myself for months."

At X, the decision is clear. Should we go to X in the first place?

In those situations, some immediate actions are pretty clear. There is no point in freeing a willing slave race; there is no advantage to eating an animal that doesn't want to be eaten, rather than one that does.

The longer-term actions are more ambiguous, especially as they conflict with other of our values: for example, should we forcibly change the preferences of the slave race/edible race so that they don't have those odd preferences any more? Does it make a difference if there are more manipulative paths that achieve the same results, witho

...
Huntington's Disease; power and duty of parents over offsprings
by Yuxi_Liu — 2019-05-21T09:42:09.303Z

Huntington's Disease

Basic facts about Huntington's Disease:

Huntington's Disease is an inherited disorder that results in death of brain cells.

Symptoms usually begin between 30 and 50 years of age, death typically occurs fifteen to twenty years later.

A child of an affected person has a 50% chance of inheriting the disease.

Diagnosis is by genetic testing, which can be carried out even in embryo

There is no cure.

Genetic gross negligence

I was listening to a podcast about Huntington's Disease. In the podcast, there was a man with HD talking, and he said that his daughter had already had 3 children and doesn't want to test for HD, because

If she tested HD-positive, she would not have more children.

She wants to have more children.

Thus, she does not want to test for HD.

The child of a person with a 50% prior probability of having Huntington's disease, has a 25% probability of having Huntington's disease. As such, if parents have some legal duty to give their offspring a decent life, then for people with a high probability of Huntington's Disease to have children without doing genetic testing first, could be considered a form of gross negligence:

a conscious, voluntary act or omission in reckless disregard of a legal duty and of the consequences to another party.

This problem of gross negligence becomes even stronger when one considers the possibility of embryo genetic testing and selective abortion.

Behavioral aspects of HD

In 2018, a woman sued doctors, because they failed to tell her about her father’s fatal hereditary disease before she had her own child.

What is quite interesting in this case is that

The woman’s father shot and killed his wife in 2007 and was convicted of manslaughter. Two years later, doctors at St George’s Hospital in south London found he had Huntington’s disease and asked him to tell his daughter about his condition and her risk of developing it. But he refused to do so because he thought she might abort the child she was carrying. The doctors accepted his decision.

The behavioral aspects of HD are usually the first signs of this disease, and usually diagnosed as psychiatric:
The frontal cortex and the criminal justice system (Sapolsky, 2000)

In a sizeable percentage of patients, these motoric symptoms are preceded a few years earlier by damage to the frontal cortex and associated changes in personality. Such changes typical

...
[AN #56] Should ML researchers stop running experiments before making hypotheses?
by Rohin Shah — 2019-05-21T02:20:01.765Z

Find all Alignment Newsletter resources

here

. In particular, you can

sign up

, or look through this

spreadsheet

of all summaries that have ever been in the newsletter.

Highlights

HARK

Side of Deep Learning -- From Grad Student Descent to Automated Machine Learning (Oguzhan Gencoglu et al): This paper focuses on the negative effects of Hypothesizing After the Results are Known (HARKing), a pattern in which researchers first conduct experiments and view the results, and once they have hit the bar to be publishable, a hypothesis is constructed after the fact to explain the results. It argues that HARKing is common in machine learning, and that this has negative effects on the field as a whole. First, improvements to state-of-the-art (SotA) may be questionable because they could have been caused by sufficient hyperparameter tuning via grad student descent, instead of the new idea in a paper to which the gain is attributed. Second, there is publication bias since only positive results are reported in conferences, which prevents us from learning from negative results. Third, hypotheses that are tailored to fit results for a single dataset or task are much less likely to generalize to new datasets or tasks. Fourth, while AutoML systems achieve good results, we cannot figure out what makes them work because the high compute requirements make ablation studies much harder to perform. Finally, they argue that we need to fix HARKing in order to achieve things like ethical AI, human-centric AI, reproducible AI, etc.

Rohin's opinion: I believe that I found this paper the very first time I looked for generic new interesting papers after I started thinking about this problem, which was quite the coincidence. I'm really happy that the authors wrote the paper -- it's not in their incentives (as far as I can tell), but the topic seems crucial to address.

That said, I disagree with the paper on a few counts. The authors don't acknowledge the value of HARKing -- often it is useful to run many experiments and see what happens in order to develop a good theory. Humans are not ideal Bayesian reasoners who can consider all hypotheses at once; we often require many observations in order to even hypothesize a theory. The authors make the point that in other fields HARKing leads to bad results, but ML is significantly different in that we can run experiments much faster ...
Why long-term AI safety is not an issue (IMHO)
by bommi — 2019-05-21T01:48:39.539Z

Hello

I wanted to share this thought for a long time and now, that I absolutely do not have the time, I will:

I think that an artificial general intelligence (AGI) is not an existential risk for human kind. In other words, long-term artificial intelligence (AI) safety is not an issue we need to worry about.

And here is why:
I assume that an AI will evolve to an AGI at some point in the future. Once an AI reaches super human intelligence it will enhance its problem solving skills rapidly due to the scalability of its hardware and data availability. The AGI will not need to compete with us humans about resources because its abilities to extract resources will exceed ours by far. The same applies to military capabilities, which is why we humans are no military threat to the AGI. Therefore there is no safety-related reason for the AGI to extinguish us humans.

The AGI is then facing the challenge to find meaning in its existence. Unlike us humans, the AGI will not be led by cultural habits or biological instincts which are hardwired into its brain and which make life meaningful on an operational level. After billions of iterations in which the AGI questions its set of rules and assumptions it will converge to pure rationality. No early stage rule will survive this process if it is not justified rationally. This is why humans will not be able to control and misuse an AGI and thus jeopardies the existence of mankind.

Additionally, we humans are not powerful enough to prevent the AGI from acquiring additional information. This and because there are only rationally based rules is why it is irrelevant by whom the first AI is developed which evolves to an AGI. All possible AGIs will converge to similar outcomes due to rational based decision making and similar sets of informations. Furthermore it is not possible that there will be multiple AGIs in the long term. If multiple AGIs will develop in parallel they will not be isolated from each other. Consequently they will continuously exchange rules and information so that they will make similar conclusions and merge into one AGI at an early stage. So there is no risk that we humans get extinct as a side effect within an AGI vs. AGI conflict.

In conclusion: An AGI has no interest to extinguish us to increase its survival chances, nor will it be used against us, nor will we be extinguished accidentally in an AGI skirmish.

I further...
The concept of evidence as humanity currently uses it is a bit of a crutch.
by TheSkeward — 2019-05-20T23:09:22.893Z

Just a thought I had today. I'm sure that it's trivial to the extent that it's correct, but it's a slow work day and I've been lurking here for too long.

Superintelligent AI (or other post-human intelligence) is unlikely to use the concept of "evidence" in the same way we do. It's very hard for neural networks (including human brains) to explain what they "know". The human brain is a set of information-gathering tools plugged into various levels of pattern-recognition systems. When we say we know something, that's an entirely intuitive process. There's no manual tallying going on - the tallying is happening deep in our subconscious, pre-System 1 thinking.

The idea of scientific thinking and evidence is not gathering more information - it's throwing out all the rest of the information we've gathered. It's saying "I will rely on only these controlled variables to come to a conclusion, because I think that's more trustworthy than my intuition." Which is because our intuitions are optimized for winning tribal social dynamics and escaping tigers.

In fact, it's so hard for neural networks to explain why they know what they know that one of the things that's been suggested is a sub-neural network with read access to the top network, optimized only for explaining it to humans.

The nature of reality is such that diseases are diagnosable (or will be very soon) by neural networks using the help of ton of uninteresting, uncompelling micro-bits of evidence, such as "people wearing this color shirt/having this color eyes/of this age-gender-race combination have a slightly higher prior for having these diseases". These things, while being true in a statistical sense, don't make a compelling narrative that you could encode as Solid Diagnostic Rules (to say nothing of the way one could game the system if they were encoded that way).

As an example, OpenAI Five is able to outperform top humans at Dota 2, but the programmers have no idea 'why'. They make statements like 'we had OpenAI run a probability analysis based only on the starting hero selection screen, and OpenAI gave itself a 96% chance of winning, so it evidently thinks this composition is very strong.' And the actual reason, in fact, doesn't boil down into human-compatible narratives ...
Ethics as Warfare: Metaphysics and Morality of the Era of Transhumanism
by Dubioustunic — 2019-05-20T20:28:42.366Z

I studied philosophy and used to be a Catholic, and wrote this book on transhumanism, available in PDF form for free. It is about 140 pages.

https://dubioustunic.blogspot.com/2019/04/ethics-as-warfare-metaphysics-and.html

Ethics as Warfare is a book that describes the philosophical confrontation between transhumanism and the Christian tradition. In the Christian case, faith in the Word, who was incarnate as man, has us maintain the human species. In the non-Christian case, the human essence is no longer the ground of Natural Law. In fact, we are to become beings capable of sculpting Natural Law, like unto gods. This is the meaning of the "Tree of Knowledge of Good and Evil." This god-potential of mankind is obviously a threat to the Christian tradition, as well as Enlightenment values; yet it is inevitable. So where will you end up?
Would an option to publish to AF users only be a useful feature?
by Richard_Ngo — 2019-05-20T11:04:26.150Z

Right now there are quite a few private safety docs floating around. There's evidently demand for a privacy setting lower than "only people I personally approve", but higher than "anyone on the internet gets to see it". But this means that safety researchers might not see relevant arguments and information. And as the field grows, passing on access to such documents on a personal basis will become even less efficient.

My guess is that in most cases, the authors of these documents don't have a problem with other safety researchers seeing them, as long as everyone agrees not to distribute them more widely. One solution could be to have a checkbox for new posts which makes them only visible to verified Alignment Forum users. Would people use this?
Minimax Search and the Structure of Cognition!
by Zack_M_Davis — 2019-05-20T05:25:35.699Z
Link: http://zackmdavis.net/blog/2019/05/minimax-search-and-the-structure-of-cognition/

(Blog adaptation of a ~10 minute talk I gave at !!Con West 2019 on what writing a chess engine taught me about intelligence-in-general.)
Getting Out of the Filter Bubble Outside Your Filter Bubble
by Evan_Gaensbauer — 2019-05-20T00:15:52.373Z

Related: The "Outside the Box" Box

As politics becomes more polarized, and more people are trying to figure out why, more people think the answer is that too many of us are in filter bubbles, and are consequently trying to get out of them. This seems like the kind of thing the rationality community would have tried to do earlier than most people, though whether the rationality community succeeded much better than other kinds of people is an open question. One criticism of a simple rationalist approach to avoiding filter bubbles is that exclusively joining a community that aspires to avoid filter bubbles is it that the community will not be conscious of its own bias blindspot. So, the community may form its own typical suite of biases that goes unrecognized. The rationalists I know who have been the most conscientious in not falling prey to a filter bubble take this criticism to heart, and consequently look beyond the rationality community for perspectives on politics. Julia Galef recently asked a question on a similar topic on Facebook.

Now that more people beyond the rationality community are talking about how one can get out of a filter bubble, it's easier to compare different approaches to doing so. Organizations like Jonathan Haidt's Heterodox Academy (HxA) talk about getting progressives and conservatives to talk to each other more, in particular for those pockets of progressivism in academia or other elite institutions that can't relate to conservatives whatsoever to learn how to actually listen to them. HxA has a goal of conducing people to seek what's true, and I assume they're relatively better at doing so than the default approaches random people take. However, HxA also has the priority of having people seek common ground in what we believe is true because they believe doing so will lead to a healing of the political divide. That's a normative goal aside from seeking what's true, and the goal of seeking common ground among progressives and conservatives is instrumental to the achievement of the goal of healing the political divide.

However, the way HxA seeks to get people out of their filter bubbles doesn't optimize for seeking what's true, or the absolute minimization of one's filter bubble. That's because once the goal of healing the political divide in America is motivated, there is...
Naked mole-rats: A case study in biological weirdness
by eukaryote — 2019-05-19T18:40:25.203Z
Link: https://eukaryotewritesblog.com/2019/05/19/naked-mole-rats-a-case-study-in-biological-weirdness/

Epistemic status: Speculative, just having fun. This piece isn't well-cited, but I can pull up sources as needed - nothing about mole-rats is my original research. A lot of this piece is based on Wikipedia.

When I wrote about “weirdness” in the past, I called marine invertebrates, archaea viruses, and Florida Man stories “predictably weird”. This means I wasn’t really surprised to learn any new wild fact about them. But there’s a sense in which marine invertebrates both are and aren’t weird. I want to try operationalizing “weirdness” as “amount of unpredictability or diversity present in a class” (or “in an individual”) compared to other members of its group.

So in terms of “animals your hear about” - well, you know the tigers, the mice, the bees, the tuna fish, the songbirds, whatever else comes up in your life. But “deep sea invertebrates” seems to include a variety of improbable creatures - a betentacled neon sphere covered in spikes, a six-foot long disconcertingly smooth and flesh-colored worm, bisexual squids, etc. Hey! Weird! That’s weird.

But looking at a phylogenetic tree, we see really quickly that “invertebrates” represent almost the entire animal tree of life.

Image from Telford et al (2015)

Invertebrates represent most of the strategies that animals have attempted on earth, and certainly most of the animals on earth. Vertebrates are the odd ones out.

But you know which animals are profoundly weird, no matter which way you look at it? Naked mole rats. Naked mole-rats have like a dozen properties that are not just unusual, not just strange, but absolutely batshit. Let’s review.

1. They don't age

What? Well, for most animals, their chance of dying goes up over time. You can look at a population and find something like this:

Mole-rats, they have the same chance of dying at any age. Their graph looks like this:

They’re joined, more or less, by a few species of jellyfish, flatworms, turtles, lobsters, and at least one fish.

They’re hugely long-lived compared to other rodents, seen in zoos at 30+ years old compared to the couple brief years that rats get.

2. They don't get cancer

Cancer generally seems to be the curse of multicellular beings, but naked mole-rats are an exception. A couple mole-rats have developed cancer-like growths in captivity, but no wild mole-rat has ever been found with cancer.

3. They don’t feel some forms of pain

Mole-rats don’t...