trentmkelly/LessWrong-43k huggingface dataset:
Interpreting genetic testing
by jefftk — 2018-12-15T15:56:57.339Z

Several years ago I participated in a study where my DNA was sequenced, and while I ended up not getting the sequence data [1] I did get a file of 23andme-style SNP variant calls. I loaded it into Promethease, and excluded mutations with magnitude below 2 ("looks interesting enough to be worth reading"). I saw 139 mutations marked as "bad," 41 as "good," and 26 as "not set."

Initially I interpreted this to mean that I should be more pessimistic about my health than I was before getting the report, since more of the mutations are bad (2x risk of something) than good (0.5x risk of something else). To figure out how your beliefs should change, though, you need to know how many bad vs good mutations people typically have. For example, if someone might normally have 200 bad mutations and 10 good ones then my report is good news, but if instead normal is 100 bad mutations and 70 good ones then my report is bad news.

In general, I would expect most people to have more negative mutations than positive ones, simply because most mutations with an effect are negative. Randomly changing something is much more likely to break things than make them better.

This also applies when determining total risk of something. For example, lets say I have SNPs that individually give me 3x, 1.5x, 2x, and 0.5x risk for heart disease. I could naively multiply them together, ignoring that they don't stack perfectly, [2] and conclude that I had 4.5x the risk of the general population. But most people will probably have some mutations that increase their risk of heart disease. I think the proper way to handle this is for each case where you have the normal value of a variant you count that as slightly improving your risk, and when you consider all of these tiny improvements you get back to the average person having the average risk. Alternatively, and probably more accurately, you could just naively compute each person's risk, and then normalize.

Is this just a Promethease problem? Do other places that give health reports handle this better? Or do places just avoid giving consumer health information because this is both really hard to do well and highly regulated?

(It's also definitely possible I'm misinterpreting Promethease, or not thinking well about how the stats work here.)

[1] This was really frustrating. They confirmed receipt of...
What is abstraction?
by Adam Zerner — 2018-12-15T08:36:01.089Z

I'm not clear on what the term abstraction really means. It seems to mean different things in different contexts.

I'm a programmer, and when I think of abstraction, the first thing I think of is something that feels like composition. Suppose you are a parent and want to teach your child how to brush their teeth. To do that, you break the task "brush your teeth" into subtasks like 1) prepare your toothbrush, 2) actually brush your teeth, 3) rinse your mouth and 4) clean your toothbrush. In programming, you would break the brushTeeth method into helper methods like prepareToothbrush, actuallyBrushTeeth, rinseMouth and cleanToothbrush. And you would call brushTeeth an abstraction, because it is composed of subtasks. My understanding is that this doesn't just apply to programming: in the real world, if you told your child "brush your teeth", that would also be an abstraction for "prepare your toothbrush, actually brush your teeth, rinse your mouth, and then clean your toothbrush".

But there's something else that I think of when I think of the term "abstraction". Suppose I have a program that has people, dogs, and cats. All three have heights, weights and names, so I create an animal class, and I say that people, dogs and cats are more specific versions of an animal. My understanding is that this too would be an abstraction. In programming, and in the real world. My understanding is that the idea of an animal is an abstraction over the idea of a human, dog or cat.

But there seems to be something very different about each of the two situations. In the first, we would say that the "brush your teeth" abstraction is composed of the subtasks, but we wouldn't say that "animal" is composed of humans, dogs and cats in the second. And we would say that the idea of an animal is a more general version of the idea of a human, dog and cat, but we wouldn't say that "brush your teeth" is a more general version of preparing your toothbrush or rinsing your mouth.

The first example seems to be about composition, and the second about generality, so I'm confused as to what abstraction really is.

And if we look at yet more contexts, there seems to be more inconsistency in how the term is used. In semantics and philosophy, "abstract" means something that you can't perceiv...
Introducing the Longevity Research Institute
by sarahconstantin — 2018-12-14T20:20:00.532Z

I’ve just founded a nonprofit, the Longevity Research Institute — you can check it out here.

The basic premise is: we know there are more than 50 compounds that have been reported to extend healthy lifespan in mammals, but most of these have never been tested independently, and in many cases the experimental methodology is poor.

In other words, there seems to be a lot of low-hanging fruit in aging. There are many long-lived mutant strains of mice (and invertebrates), there are many candidate anti-aging drugs, but very few of these drugs have actually been tested rigorously.

Why? It’s an incentives problem. Lifespan studies for mice take 2-4 years, which don’t play well with the fast pace of publication that academics want; and the FDA doesn’t consider aging a disease, so testing lifespan isn’t on biotech companies’ critical path to getting a drug approved. Mammalian lifespan studies are an underfunded area — which is where we come in.

We write grants to academic researchers and commission studies from contract research organizations. Our first planned studies are on epitalon (a peptide derived from the pineal gland, which has been reported to extend life in mice, rats, and humans, but only in Russian studies) and C3 carboxyfullerene (yes, a modified buckyball, which prevents Parkinsonism in primate models and has been reported to extend life in mice). I’m also working on a paper with Vium about some of their long-lived mice, and a quantitative network analysis of aging regulatory pathways that might turn up some drug targets.

We’re currently fundraising, so if this sounds interesting, please consider donating. The more studies that can be launched in parallel, the sooner we can get results.
Player vs. Character: A Two-Level Model of Ethics
by sarahconstantin — 2018-12-14T19:40:00.520Z

Epistemic Status: Confident

This idea is actually due to my husband, Andrew Rettek, but since he doesn’t blog, and I want to be able to refer to it later, I thought I’d write it up here.

In many games, such as Magic: The Gathering, Hearthstone, or Dungeons and Dragons, there’s a two-phase process. First, the player constructs a deck or character from a very large sample space of possibilities. This is a particular combination of strengths and weaknesses and capabilities for action, which the player thinks can be successful against other decks/characters or at winning in the game universe. The choice of deck or character often determines the strategies that deck or character can use in the second phase, which is actual gameplay. In gameplay, the character (or deck) can only use the affordances that it’s been previously set up with. This means that there are two separate places where a player needs to get things right: first, in designing a strong character/deck, and second, in executing the optimal strategies for that character/deck during gameplay.

(This is in contrast to games like chess or go, which are single-level; the capacities of black and white are set by the rules of the game, and the only problem is how to execute the optimal strategy. Obviously, even single-level games can already be complex!)

The idea is that human behavior works very much like a two-level game.

The “player” is the whole mind, choosing subconscious strategies. The “elephant“, not the “rider.” The player is very influenced by evolutionary pressure; it is built to direct behavior in ways that increases inclusive fitness. The player directs what we perceive, do, think, and feel.

The player creates what we experience as “personality”, fairly early in life; it notices what strategies and skills work for us and invests in those at the expense of others. It builds our “character sheet”, so to speak.

Note that even things that seem like “innate” talents, like the savant skills or hyperacute senses sometimes observed in autistic people, can be observed to be tightly linked to feedback loops in early childhood. In other words, savants practice the thing they like and are good at, and gain “superhuman” skill at it. They “practice” along a faster and more hyperspecialized path than what we think of as a neurotypical “practicing hard,” but it’s still a learning process. Savant skills are m...
How to reset my password?
by hirvinen — 2018-12-14T16:18:02.936Z

How do I change my password? On the account settings page there is a "reset password" button that emails me a link. I thought this would be additional security measure to get to a "set new password" page, but it only tells me that the password was reset. (To what???) However, my password seems to be unchanged.

Since I don't use the site regularly, I would appreciate an email about the answer, if it's not automatic. Thanks!
Meditations on Momentum
by Richard Meadows — 2018-12-14T10:53:05.446Z

Cross-posted and lightly-edited from The Deep Dish.

Epistemic status: describing a general phenomenon; may not be correct on every specific point. may have used scientific terms in an annoying metaphorical fashion. elements of pointing out the bleeding obvious, hopefully framed in a novel way.

tl;dr: positive feedback loops are a thing, thinking in systems/exponentially is hard, intersectionality is underrated.

“For to everyone who has, more will be given, and he will have an abundance. But from the one who has not, even what he has will be taken away.”

—MATTHEW 25:29

In 2013, unknown author Robert Galbraith published his debut novel… to crickets.

The first print run of The Cuckoo’s Calling was 1500 copies. It’s not clear how many actually sold. The book occupied 4709th place on Amazon’s bestseller charts.

And there, perhaps it would have stayed, if the cuckoo in the nest had remained undiscovered. The secret unravelled after a few months: ex-military security contractor Galbraith was a pseudonym for J.K. Rowling. As soon as the news broke, The Cuckoo’s Calling soared to the number one spot on Amazon. Sales increased by 150,000 per cent overnight. Copies from that first neglected print run are now worth thousands of dollars.

What’s the difference between Robert Galbraith and J.K. Rowling? Clearly, being a talented writer is necessary, but not sufficient. Rowling has momentum on her side. At this point, she could publish the contents of a bowl of alphabet soup, and it would still sell better than 99 per cent of novels by hopeful first-time authors.

This is a ‘no duh’ example, designed to get you nodding your head along. But momentum is everywhere, and it’s rarely in plain sight. Without being consciously aware of doing so, I’ve written about it in four domains:

1. Popularity

Popular things often get their start through what amounts to good luck. The rapid ascent is driven by something even more powerful than rocket fuel: social contagion. Our opinions and preferences cluster together, but it’s not because we’ve carefully evaluated them on their merits. We just want to feel close to our fellow social apes, and have something to gossip about around the water cooler.

In other words, popularity is a lot like herpes. After catching a lucky initial break, it manages to spread to a few hosts, then rides the exponential growth curve until it has planted its gentle, blistery kiss

...
Can I use Less Wrong branding in youtube videos?
by Bae's Theorem — 2018-12-14T07:10:16.072Z

I will be starting a youtube channel where I record my Street Epistemology encounters. I want to heavily plug our community, for the benefit of growing it, and also because it's just so darn appropriate (I am literally asking people what they think they know, and how they think they know it).

I will not be claiming ownership of the branding (I'll be diverting people to the website, of course), but I would probably go so far as to use it in the name of the channel (for lack of a better pithy name, 'Less Wrong' is excellent).
Are there any legal problems with this?
Three AI Safety Related Ideas
by Wei Dai — 2018-12-13T21:32:25.415Z

(I have a health problem that is acting up and making it hard to type for long periods of time, so I'm condensing three posts into one.)

1. AI design as opportunity and obligation to address human safety problems

Many AI safety problems are likely to have counterparts in humans. AI designers and safety researchers shouldn't start by assuming that humans are safe (and then try to inductively prove that increasingly powerful AI systems are safe when developed/trained by and added to a team of humans) or try to solve AI safety problems without considering whether their designs or safety approaches exacerbate human safety problems relative to other designs / safety approaches. At the same time, the development of AI may be a huge opportunity to address human safety problems, for example by transferring power from probably unsafe humans to de novo AIs that are designed from the ground up to be safe, or by assisting humans' built-in safety mechanisms (such as moral and philosophical reflection).

2. A hybrid approach to the human-AI safety problem

Idealized humans can be safer than actual humans. An example of idealized human is a human whole-brain emulation that is placed in a familiar, safe, and supportive virtual environment (along with other humans for socialization), so that they are not subject to problematic "distributional shifts" nor vulnerable to manipulation from other powerful agents in the physical world. One way to take advantage of this is to design an AI that is ultimately controlled by a group of idealized humans (for example, has a terminal goal that refers to the reflective equilibrium of the idealized humans), but this seems impractical due to computational constraints. An idea to get around this is to give the AI an advice or hint, that it can serve that terminal goal by learning from actual humans as an instrumental goal. This learning can include imitation learning, value learning, or other kinds of learning. Then, even if the actual humans become corrupted, the AI has a chance of becoming powerful enough to discard its dependence on actual humans and recompute its instrumental goals directly from its terminal goal. (Thanks to Vladimir Nesov for giving me a hint that led to this idea.)

3. Several approached to AI alignment will differentially accelerate intellectual progress that are analogous to solving problems that are low in the polynomial hie

...
The Bat and Ball Problem Revisited
by drossbucket — 2018-12-13T07:16:30.017Z

Cross posted from my personal blog.

In this post, I'm going to assume you've come across the Cognitive Reflection Test before and know the answers. If you haven't, it's only three quick questions, go and do it now.

One of the striking early examples in Kahneman's Thinking, Fast and Slow is the following problem:

(1) A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball.

How much does the ball cost? _____ cents

This question first turns up informally in a paper by Kahneman and Frederick, who find that most people get it wrong:

Almost everyone we ask reports an initial tendency to answer “10 cents” because the sum $1.10 separates naturally into $1 and 10 cents, and 10 cents is about the right magnitude. Many people yield to this immediate impulse. The surprisingly high rate of errors in this easy problem illustrates how lightly System 2 monitors the output of System 1: people are not accustomed to thinking hard, and are often content to trust a plausible judgment that quickly comes to mind.

In Thinking Fast and Slow, the bat and ball problem is used as an introduction to the major theme of the book: the distinction between fluent, spontaneous, fast 'System 1' mental processes, and effortful, reflective and slow 'System 2' ones. The explicit moral is that we are too willing to lean on System 1, and this gets us into trouble:

The bat-and-ball problem is our first encounter with an observation that will be a recurrent theme of this book: many people are overconfident, prone to place too much faith in their intuitions. They apparently find cognitive effort at least mildly unpleasant and avoid it as much as possible.

This story is very compelling in the case of the bat and ball problem. I got this problem wrong myself when I first saw it, and still find the intuitive-but-wrong answer very plausible looking. I have to consciously remind myself to apply some extra effort and get the correct answer.

However, this becomes more complicated when you start considering other tests of this fast-vs-slow distinction. Frederick later combined the bat and ball problem with two other questions to create the Cognitive Reflection Test:

(2) If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets? _____ minutes

(3) In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it t

...
Multi-agent predictive minds and AI alignment
by Jan_Kulveit — 2018-12-12T23:48:03.155Z

Abstract: An attempt to map a best-guess model of how human values and motivations work to several more technical research questions. The mind-model is inspired by predictive processing / active inference framework and multi-agent models of the mind.

The text has slightly unusual epistemic structure:

1st part: my current best-guess model of how human minds work.

2nd part: explores various problems which such mind architecture would pose for some approaches to value learning. The argument is: if such a model seems at least plausible, we should probably extend the space of active research directions.

3rd part: a list of specific research agendas, sometimes specific research questions, motivated by the previous.

I put more credence in the usefulness of research questions suggested in the third part than in the specifics of the model described the first part. Also, you should be warned I have no formal training in cognitive neuroscience and similar fields, and it is completely possible I’m making some basic mistakes. Still, my feeling is even if the model described in the first part is wrong, something from the broad class of “motivational systems not naturally described by utility functions” is close to reality, and understanding problems from the 3rd part can be useful.

How minds work

As noted, this is a “best guess model”. I have large uncertainty about how human minds actually work. But if I could place just one bet, I would bet on this.

The model has two prerequisite ideas: predictive processing and the active inference framework. I'll give brief summaries and links for elsewhere.

In the predictive processing / the active inference framework, brains constantly predict sensory inputs, in a hierarchical generative way. As a dual, action is also “generated” by the same machinery (changing environment to match “predicted” desirable inputs and generating action which can lead to them). The “currency” on which the whole system is running is prediction error (or something in style of free energy, in that language).

Another important ingredient is bounded rationality, i.e. a limited amount of resources being available for cognition. Indeed, the specifics of hierarchical modelling, neural architectures, principle of reusing and repurposing everything, all seem to be related to quite brutal optimization pressure, likely related to brain’s enormous energy consumption...
What went wrong in this interaction?
by t3tsubo — 2018-12-12T19:59:57.790Z

I'm curious about an interaction I had a few weeks ago with someone in the rationality community, I was wondering if someone here can look at the conversation and evaluate what 'went wrong' so to speak.

It began with some comments I made on a blog post, where I disagreed with the author that 'metoo' was good, but rather than discuss the entire point I wanted just to address some counterexamples to something the author said about metoo never having gone too far. The post is here: http://benjaminrosshoffman.com/metoo-is-good/

After a bit of back and forth, it seemed like I should try take it to private chat before it turned into a demon thread. This was the ensuing conversation: https://pastebin.com/epQmxZK2

It seems to me like my points were understood, and likewise I didn't quite get what the author was trying to make me understand.

To me, it seems to me like my points were understood, and likewise I didn’t quite get what the author was trying to make me understand.

The author also seemed hostile and unwilling to engage, and how he disengaged from the conversation seemed like a personal attack that was unjustified. But I’m biased, so I was wondering if it was something about my comments or behavior or tone that I was missing that provoked that response, or if I misread the hostility at all.

And any thoughts about why the author had that kind of a reaction? It was not what I expected since I thought most rational community members would welcome a honest discussion like the one I was trying to start.
Should ethicists be inside or outside a profession?
by Eliezer Yudkowsky — 2018-12-12T01:40:13.298Z

Originally written in 2007.

Marvin Minsky in an interview with Danielle Egan for New Scientist:

Minsky: The reason we have politicians is to prevent bad things from happening. It doesn’t make sense to ask a scientist to worry about the bad effects of their discoveries, because they’re no better at that than anyone else. Scientists are not particularly good at social policy.

Egan: But shouldn’t they have an ethical responsibility for their inventions

Minsky: No they shouldn’t have an ethical responsibility for their inventions. They should be able to do what they want. You shouldn’t have to ask them to have the same values as other people. Because then you won’t get them. They’ll make stupid decisions and not work on important things, because they see possible dangers. What you need is a separation of powers. It doesn’t make any sense to have the same person do both.

The Singularity Institute was recently asked to comment on this interview - which by the time it made it through the editors at New Scientist, contained just the unvarnished quote “Scientists shouldn’t have an ethical responsibility for their inventions. They should be able to do what they want. You shouldn’t have to ask them to have the same values as other people.” Nice one, New Scientist. Thanks to Egan for providing the original interview text.

This makes an interesting contrast with what I said in my “Cognitive biases” chapter for Bostrom’s Global Catastrophic Risks:

Someone on the physics-disaster committee should know what the term “existential risk” means; should possess whatever skills the field of existential risk management has accumulated or borrowed. For maximum safety, that person should also be a physicist. The domain-specific expertise and the expertise pertaining to existential risks should combine in one person. I am skeptical that a scholar of heuristics and biases, unable to read physics equations, could check the work of physicists who knew nothing of heuristics and biases.

Should ethicists be inside or outside a profession?

It seems to me that trying to separate ethics and engineering is like trying to separate the crafting of paintings into two independent specialties: a profession that’s in charge of pushing a paintbrush over a canvas, and a profession that’s in charge of artistic beauty but knows nothing about paint or optics.

The view of ethics as a separate profession is part of ...