trentmkelly/LessWrong-43k huggingface dataset:
RSS Troubles
by SquirrelInHell — 2018-01-13T08:07:25.181Z

Some stuff that is currently broken about handling RSS feeds:

rss link builders on the front page create invalid urls
all feeds except the one with all posts are unusable in practice, because of sth like the low end of the range of posts that is available in them sometimes goes back in time, which makes rss readers pick up old posts as new posts
if user accounts are supposed to work like personal blogs, then they need to have their own rss links which are easy to access from there
ideally, there would be a way to subscribe to an aggregate feed of all blogs which one is following, and also to all non-meta posts
relatedly, I can't find a way to view a list of accounts which I'm following
Is death bad?
by Richard_Ngo — 2018-01-13T04:55:25.788Z
Link: https://thinkingcomplete.blogspot.co.nz/2018/01/is-death-bad.html

My linked blog post explores a variety of arguments and counterarguments about death, but is not aimed at LW readers. However, there's one particular argument which I wanted to highlight and discuss here:

"For simplicity, let's consider the case where every couple has exactly 2 children no matter how long they live. Then we face the following conundrum: is it better to have x people living for y years each, or to increase longevity and have x/n people living for y*n years each? The population at each time is the same either way, it's merely the distribution of those lives which changes.

In this scenario, for death to be good on a population level, it's not necessary that our lives end up with negative value as our lifespans increase. Instead, it simply needs to be the case that our longer lives end up worse, on average, than those of the children who would have replaced us. Is that likely? Anders Sandberg argues that with longer lives we would "waste" less time, proportionately, in early childhood and in ill health during old age. However, less of our lives would be spent on new and exciting experiences.

The importance of the latter effect depends on whether what we value is more like life satisfaction or more like hedonic experiences. If the former, then repeating experiences probably won't provide much more satisfaction in the overall picture - winning Olympic gold twice is probably not twice as satisfying as winning once. Perhaps we could even get a rough idea of drop-off in marginal value by asking people questions like whether they prefer another 25 years of healthy life for certain over a 50% chance of another 60 years of healthy life and a 50% chance of imminent death. (However, it's unclear how much of a bias towards the former would be introduced by risk aversion.) On the other hand, if hedonic experiences are what we care about, then it's very salient to note that people actually report being happiest around their 70s, and so it seems plausible that we could be equivalently happy for much longer than the normal lifespan.

It's also plausible that many people would be significantly happier than that, because a great deal of sorrow often accompanies both the deaths of loved ones and the contemplation of one's upcoming death. However, this grief seems to usually be manageable when such deaths com...
Weaponized Fluffy Sparkles
by Bound_up — 2018-01-12T23:08:53.221Z

See "When Truth Isn't Enough".

(Tevye)
But my father and my mother
Said we'd learn to love each other
And now I'm asking, Golde
Do you love me?

(Golde)
I'm your wife

(Tevye)
"I know..."
But do you love me?

(Golde)
Do I love him?
For twenty-five years I've lived with him
Fought with him, starved with him
Twenty-five years my bed is his
If that's not love, what is?

(Tevye)
Then you love me?

(Golde)
I suppose I do

(Tevye)
And I suppose I love you too

(Both)
It doesn't change a thing
But even so
After twenty-five years
It's nice to know.

---

Love is a commitment, not a feeling

It's a marriage, not a civil union.

It's a baby, not a fetus.

Is this a date?

It contradicts the rest! Contradict? No, I'd say it complements it.

I'm an attorney. Nah, man, you're just a lawyer.

Who are you to judge? I'm not judging, I'm just saying what I think.

---

Denotation is what a word officially means. Connotation refers to the underlying meanings. Many times, two words with the same denotation will have different connotations.

I like to think of each word like the center of a web. Running off from the center of the web are connections to other ideas. For example, "attorney" might be at the center of a web that connects to ideas of respectability and honor, while "lawyer" might be at the center of a web that has lines running off towards sliminess and shallowness. Knowledge of the words' connotations is enough to predict how lawyer's (excuse me, attorneys) introduce themselves.

Wouldn't it be amazing if we could take the positive connotations connected to the attorney concept and just attach them to something else, something that we want people to think well of? Bam! Instant respectability. If only...

Alternatively, imagine if we could take the negative connations around a word like "fascist" and tie those connotations around our enemies' necks! They'd be politically crippled and our victory would be assured.

This is one of the key political moves and it works like so.

Concept 1 has some connotations attached to it.

Concept 2 doesn't have those connations.

You'd like people to think about Concept 2 in the same way they think about Concept 1; you want Concept 2 to have the first concept's connotations.

So, you simply take the word that refers to Concept 1, and insi...
Prune
by alkjash — 2018-01-12T22:50:58.357Z
Link: https://radimentary.wordpress.com/2018/01/12/prune/

Previously, I described human thought-generation as an adversarial process between a low-quality pseudorandom Babble generator and a high-quality Prune filter, roughly analogous to the Generative Adversarial Networks model in machine learning. I then elaborated on this model by reconceptualizing Babble as a random walk with random restarts on an implicitly stored Babble graph.

Rationalist training (and schooling in general) slants towards developing Prune over Babble. I'm trying to solve the dual problem: that of improving the quality of your Babble.

Although the previous posts listed a number of exotic isolation exercises for Babble, I'm guessing nobody was inspired to go out and play more Scrabble, write haikus, or stop using the letter 'e'. That's probably for the best - taking these exercises too seriously would produce exotic but sub-optimal Babble anyway. For a serious solution to this serious problem, we need to understand Prune at a higher resolution.

The main problem with Prune is that it has too many layers. There's a filter for subconscious thoughts to become conscious, another for it to become spoken word, another for the spoken word to be written down, and a further one for the written word to be displayed in public. With this many-layer model in mind, there are plenty of knobs to turn to let more and better Babble through.

The River of Babble

Imagine that your river of Babble at its source, the subconscious: a foaming, ugly-colored river littered with half-formed concepts, too wild to navigate, too dirty to drink from. A quarter mile across, the bellow of the rapids is deafening.

Downstream, you build a series of gates to tame the rushing rapids and perhaps extract something beautiful and pure.

The First Gate, conscious thought, is a huge dam a thousand feet high and holds almost all the incoming thoughts at bay. Behind it, an enormous lake forms, threatening to overflow at any moment. A thick layer of trash floats to the top of this lake, intermixed with a fair amount of the good stuff. The First Gate lets through anything that satisfies a bare minimum of syntactical and semantic constraints. Thoughts that make it past the First Gate are the first ones you become conscious of - that's why they call the output the Stream of Consciousness.

A mile down the Stream of Consciousness is the Second Gate, spoken word, the fil...
Boiling the Crab: Slow Changes (beneath Sensory Threshold) add up
by RTiberiu — 2018-01-12T20:44:46.497Z

Phenomenon: A live crab, when slowly boiled, will not climb out of the pot.

This is an analog of a known feature in human cognition: humans are not good at observing small changes. We are good at noticing sharp changes.

In this post, I outline a few anecdotal examples. Then, the phenomenon is applied to outside view, why being yourself in dating can backfire, and slowly deteriorating relationships.

---

Example of Boiling the Crab: Disappearing the Statue of Liberty: A magic trick.

There's a famous David Copperfield magic trick that involves disappearing the statue of liberty before a live audience. The magician closes the curtain, behind which the statue of liberty sits. When the curtain is lifted, the audience is amazed to find the staute of liberty no longer there! The mechanic behind this magic trick is simple: the participants are sitting on a rotating platform, which accelerates below the the audience's sensory threshold. (Source: Poor Charlie's Almanack, by Charlie Munger).

Example of Boiling the Crab: The smell of leaking gas, and a failure of olfactory perception.

A friend of mine named Robert was sitting doing his homework next to the kitchen in the University dormitory. As he was working, someone turned on the gas stove without turning on the flame.

About half an hour later, friend Alex ran into the kitchen, and said: "Holy shit! That gas smell is so strong that I could smell it from the opposite end of the dormitory."

Robert replied: "What gas smell?"

*

Application: Outside View can observe sharp differences.

An organization, community, or company that lacks outside view may make a series of small changes to their culture, intentional or otherwise. An outsider who comes in will say:

"What the hell?"

Then they point out 10 things that are unusual. Those things may not be bad -- they juts fall outside of normal boundaries.

In this model, the insider is subject to a sequence of small changes from Beginning to End (the boiling crab). The outsider notices the sharp difference, and reacts.

Application: Dating, and being accepted for who you are, is non-automatic. Epistemic Status: Highly conjectural.

"I want to find a girl/boy/other who accepts me for who I am." - a common sentiment.

"Whenever I apply more filters on myself, girls/boys/others that I am interested in gravitate towards me. When I apply fewer fi

...
What the Universe Wants: Anthropics from the POV of Self-Replication
by moridinamael — 2018-01-12T19:03:34.044Z

Meta: Once again I'm trying to submit this to my "personal page" and not the front page, but I really have no idea who will actually see it.

Here's a pretty simple idea I hadn't heard yet anywhere in science or science fiction. It's likely I'm not the first to think of this. It's also likely that it's nonsense, but I think it's fun nonsense. Please enjoy this in the spirit of the wild-ass speculations straddling the border between sci-fi and futurism with which Less Wrong used to be filled.

The rules of the universe are in some sense objectively unlikely, but by the anthropic principle, we shouldn't be surprised to observe them. A universe that doesn't support observers doesn't get observed.

We happen to be animals capable of creating new intelligent life by default, through a fundamental drive to replicate our genes, and a physics that supports that process. We breed.

We should condition "anthropically" on this evidence and update in favor of believing that intelligent observers are usually/commonly also "replicators" driven by natural selection propelled by lower levels of self-replication.

If you'll forgive my abuse of language, genes "want" to self-replicate, "causing" humans to "want" to self-replicate.

There is an argument that the universe may be more complicated than it strictly needs to be to support observers. If you could run a sentient observer in a Conway's Game of Life universe, and you include some kind of complexity-weighted distribution over universes, then shouldn't "most" universes and "most" observers exist in minimally complex universes?

Here is the step of the argument with the most uncertainty: There is a concept that each black hole creates a new universe.

Let's assume that black holes do indeed create new universes. If that is the case, "most" universes should be the ones just complex enough to support creation of new, similar universes. Natural selection in action, again.

However, that should put evolutionary pressure on generating universes just complex enough to create such singularities. Observers are not a requisite.

Under the above assumptions, we're in a universe that's complex enough to support both intelligent, self-replicating observers, and a potential mechanis...
Neural program synthesis is a dangerous technology
by syllogism — 2018-01-12T16:19:00.374Z

Crossposted from my Medium blog

Today’s computer viruses can copy themselves between computers, but they can’t mutate. That might change very soon, leading to a serious crisis.

Research on program synthesis is proceeding quickly, using neural reinforcement learning. With a similar approach, researchers in 2017 showed that a computer can teach itself to master chess and go, entirely from scratch.

The programs currently being generated are very small: dividing two numbers, reversing a string of letters, etc. It could be that there’s another chasm to cross before programs for more interesting tasks can be generated. But what if there’s not?

Consider the problem of automatically generating a program that can install itself on another computer on the network. To make this work, the statistical process generating the program needs some way to know which partial programs are most promising. Metaphorically, the “AI” needs some way to “know” when it’s “on the right track”.

Imagine you were the one generating this program, and you knew nothing except that it should be a sequence of characters. Even knowing nothing, if you could get feedback on each character, you’d quickly be able to get the result through sheer trial and error. You would just have to try each character one-by-one, keeping the character that performed best. If you only got feedback on each character pair, your life would be harder, but you’d still get there eventually. If feedback only came after each complete line, you probably wouldn’t get to the answer in a human lifetime.

The simple point here is that the task of generating malware automatically will be difficult if the “reward” is all-or-nothing — if we only know we’re on the right track once the program is fully successful. If partial success can be rewarded, it will be much easier to find the correct solution.

It might be that there are lots of obvious, intermediate rewards that make generating malware very easy. Exit without error, connection to the target IP, long and varied communication all seem like good signs that the program is on the right track.

If it turns out that writing a virus is much easier than mastering chess, we could be in a lot of trouble. Every time the virus infects another computer, the new host can be put to work improving and spreading the malware. The improvements apply to the process of generating the malware, rather than any...
Collective Aligned Systems
by  — 2018-01-12T14:15:14.681Z

I mainly talk about a system alignment in depth being aligned to one person. This is because I find collective alignment a lot more problematic. Problems include things like founder effects, edit wars and monopolies.

Collective alignment is in some ways easier from the “agent alignment” philosophy, you just have to find a goal that represents a fair aggregate of everyone’s many preferences. From the program alignment point of view you have more problems, because there are many ways that alignment changes can be caused and so many ways for things to go wrong or be problematic. One of the most likely forms of alignment that might go wrong in the collective scenario is the “Instruct” phase. The instruct phase is when a human sees the system is doing something wrong and instructs it to change it’s programming, to change things up to and including its morals and/or goals.

Let us construct a scenario where there is a moral disagreement between different members of the human population. For the sake of not too much argument, let us say that, “Wearing white after labour day” is not just a fashion faux pas but a moral one.

So Anti-white-after-labour-day people may seek to modify the AIs behaviour to do the following.

Prevent outfit suggestions that included white being made to themselves after labour day
Prevent outfit suggestions that included white being made to others after labour day (lest people get tempted into sin)
Cease to ship all white clothing/accessories after labour day.
Seek to prevent Pro-white-after-labour-day people being able to instruct the AI to modify the above rules.

If we have a scenario like this, where the Anti-white group was dominant to start with, or just had earlier access to the collective AGI, then you would expect founder effects as the AGI would also have an impact on the culture of the people interacting with it. So the initial group who can influence the system have an out-sized impact on the behaviour of the collective system and also of the culture that is interacting with it.

If there is a cultural injunction against modifications to stop instruction from other people, you might get into the equivalent of edit wars. Where instruction from each group continually overrides the programming given by the other grpup, on whether the wearing of white is acceptable after labour day.

This is not ideal! Rapidly changing moral m...
[Paper]: The Technological Landscape Affecting Artificial General Intelligence
and the Importance of Nanoscale Neural Probes
by Daniel Eth — 2018-01-12T05:22:30.236Z
Link: http://www.informatica.si/index.php/informatica/article/view/1874

I have a new paper out where I describe the technological landscape affecting AGI and implications for AI safety (largely building on Bostrom's work in Superintelligence).

Here's the abstract:

In this paper, we contrast three major pathways to human level AI, also known as artificial general intelligence (AGI), and we investigate how safety considerations compare between the three. The first pathway is de novo AGI (dnAGI), AGI built from the ground up. The second is Neuromorphic AGI (NAGI), AGI based loosely on the principles of the human brain. And third is Whole Brain Emulation (WBE), AGI built by emulating a particular human brain, in silico. Bostrom has previously argued that NAGI is the least safe form of the three. NAGI would be messier than dnAGI and therefore harder to align to arbitrary values. Additionally, NAGI would not intrinsically possess safeguards found in the human brain – such as compassion – while WBE would. In this paper, we argue that getting WBE first would be preferable to getting dnAGI first. While the introduction of WBE would likely be followed by a later transition to the less-constrained and therefore more-powerful dnAGI, the creation of dnAGI would likely be less dangerous if accomplished by WBEs than if done simply by biological humans, for a variety of reasons. One major reason is that the higher intelligence and quicker speed of thinking in the WBEs compared to biological humans could increase the chances of traversing the path through dnAGI safely. We additionally investigate the major technological trends leading to these three types of AGI, and we find these trends to be: traditional AI research, computational hardware, nanotechnology research, nanoscale neural probes, and neuroscience. In particular, we find that WBE is unlikely to be achieved without nanoscale neural probes, since much of the information processing in the brain occurs on the subcellular level (i.e., the nanoscale). For this reason, we argue that nanoscale neural probes could improve safety by favoring WBE over NAGI.
More Babble
by alkjash — 2018-01-12T01:41:06.965Z
Link: https://radimentary.wordpress.com/2018/01/11/more-babble/

In my last babble, I introduced the Babble and Prune model of thought generation: Babble with a weak heuristic to generate many more possibilities than necessary, Prune with a strong heuristic to find a best, or the satisfactory one. I want to zoom in on this model. If the last babble was colored by my biases as a probabilist, this one is motivated by my biases as a graph theorist.

First, I will speculate on the exact mechanism of Babble, and also highlight the fact Babble and Prune are independent systems that can be mocked out for unit testing.

Second, I will lather on some metaphors about the adversarial nature of Babble and Prune. Two people have independently mentioned Generative Adversarial Networks to me, a model of unsupervised learning involving two neural nets, Generator and Discriminator. The Artist and the Critic are archetypes of the same flavor - I have argued in the past the spirit of the Critic is Satan.

Babble is (Sampling From) PageRank

Previously, I suggested that a Babble generator is a pseudorandom word generator, weighted with a weak, local filter. This is roughly true, but spectacular fails one of the technical goals of a pseudorandom generator: independence. In particular, the next word you Babble is frequently a variation (phonetically or semantically) of the previous one.

PageRank, as far as I know, ranks web pages by the heuristic of "what is the probability of ending up at this page after a random walk with random restarts." That's why a better analogy for Babble is sampling from PageRank i.e. taking a weighted random walk in your Babble graph with random restarts. Jackson Pollock is visual Babble.

Imagine you're playing a game of Scrabble, and you have the seven letters JRKAXN. What does your algorithm feel like?

You scan the board and see an open M. You start Babbling letter combinations that might start with M: MAJR, MRAJ, MRAN, MARN, MARX (oops, proper noun), MARK (great!). That's the weighted random walk. You set MARK aside and look for another place to start.

Time for a restart. You find an open A before a Triple Word, that'd be great to get! You start Babbling combinations that end with A: NARA, NAXRA, JARA, JAKA, RAKA. No luck.

Maybe the A should be in the middle of the word! ARAN, AKAN, AKAR, AJAR (great!). You sense mean stares for taking so long, so you turn off the Babble and score AJAR for (1+8+1+... ,,,