trentmkelly/LessWrong-43k huggingface dataset:
What would happen if all the water on Earth were accumulated into spheres & drop on the surface?
by Long try — 2019-11-05T04:55:38.035Z

I was inspired by the APOD pictures and discussion

here

and

here

. The conditions for the 'experiment' are:

For some mysterious reason, all the water on Earth (minus that in living things - the biosphere) suddenly & immediately converges into 17 identical sphere and let to drop down to the land below right away.
The places of those water balls are the centers of 17 biggest tectonic plates as listed here.
The bottoms of the huge balls just touch the highest points on the ground there. Also from my calculation, which may be wrong, their radius is 258 km.
The ISS is currently at its highest altitude of 460 km with ship(s) in dock.

So, what will happen to us, the Earth and things on it?

My prediction is very grim for humanity, thus I turn to the ISS as our last hope. Supposed that at t=0, the astronauts can choose the station's direction of flying. Can they avoid all the 17 falling balls? Is there any chance they can land back on Earth without any help from Houston when things settle down, in case the water does settle down and not all turn into vapor? (That's a possibility I can't calculate and will need your help).

I'm looking for answers in xkcd style, which IMO is a textbook way to respond to absurd "What if?" questions. It's detailed, with nice pictures to help our imagination, displaying formulas when necessary but also very rookie-friendly, tackle the problem from many different angles & POVs, and takes into account some easy-to-overlook stuffs. Too bad he doesn't answer anymore.

Anyway, thank you just for reading! :)
What AI safety problems need solving for safe AI research assistants?
by John_Maxwell — 2019-11-05T02:09:17.686Z

In his AI Safety “Success Stories” post, Wei Dai writes:

[This] comparison table makes Research Assistant seem a particularly attractive scenario to aim for, as a stepping stone to a more definitive success story. Is this conclusion actually justified?

I share Wei Dai's intuition that the Research Assistant path is neglected, and I want to better understand the safety problems involved in this path.

Specifically, I'm envisioning AI research assistants, built without any kind of reinforcement learning, that help AI alignment researchers identify, understand, and solve AI alignment problems. Some concrete examples:

Possible with yesterday's technology: Document clustering that automatically organizes every blog post about AI alignment. Recommendation systems that find AI alignment posts similar to the one you're reading & identify connections between the thinking of various authors.

May be possible with current or near future technology: An AI chatbot, trained on every blog post about AI alignment, which makes the case for AI alignment to skeptics or attempts to shoot down FAI proposals. Text summarization software that compresses a long discussion between two forum users in a way that both feel is accurate and fair. A NLP system that automatically organizes AI safety writings into a problem/solution table as I described in this post.

May be possible with future breakthroughs in unsupervised learning, generative modeling, natural language understanding, etc.: An AI system that generates novel FAI proposals, or writes code for an FAI directly, and tries to break its own designs. An AI system that augments the problem/solution table from this post with new rows and columns generated based on original reasoning.

What safety problems are involved in creating research assistants of this sort? I'm especially interested in safety problems which haven't yet received much attention, and safety problems with advanced assistants based on future breakthroughs.
Total horse takeover
by KatjaGrace — 2019-11-05T00:10:01.319Z

I hear a lot of talk of ‘taking over the world’. What is it to take over the world? Have I done it if I am king of the world? Have I done it if I burn the world? Have humans or the printing press or Google or the idea of ‘currency’ done it?

Let’s start with something more tractable, and be clear on what it is to take over a horse.

A natural theory is that to take over a horse is to be the arbiter of everything about the horse —to be the one deciding the horse’s every motion.

But you probably don’t actually want to control the horse’s every motion, because the horse’s own ability to move itself is a large part of its value-add. Flaccid horse mass isn’t that helpful, not even if we throw in the horse’s physical strength to move itself according to your commands, and some sort of magical ability for you to communicate muscle-level commands to it. If you were in command of the horse’s every muscle, it would fall over. (If you directed its cellular processes too, it would die; if you controlled its atoms, you wouldn’t even have a dead horse.)

Information and computing capacity

The reason this isn’t so good is that balancing and maneuvering a thousand pounds of fast-moving horse flesh balanced on flexible supports is probably hard for you, at least via an interface of individual muscles, at least without more practice being a horse. I think for two reasons:

Lack of information e.g. about exactly where every part of the horse’s body is and where its hoofs are touching the ground how hard

Lack of computing power to dedicate to calculating desired horse muscle motions from the above information and your desired high level horse behavior

(Even if you have these things, you don’t obviously know how to use them to direct the horse well, but you can probably figure this out in finite time, so it doesn’t seem like a really fundamental problem.)

Tentative claim: holding more levers is good for you only insofar as you have the information and computing capacity to calculate which directions you should want those levers pushed.

So, you seem to be getting a lot out of the horse and various horse subcomponents making their own decisions about steering and balance and breathing and snorting and mitosis and where electrons should go. That is, you seem to be getting a lot out of not being in control of the horse. In fact so far it seems like the more you are in control of the hors...
More variations on pseudo-alignment
by evhub — 2019-11-04T23:24:20.335Z

In "Risks from Learned Optimization," we talked about a variety of different forms of pseudo-alignment—that is, ways in which a trained model's objective (its mesa-objective) can be misaligned off-distribution with the loss function it was trained under (the base objective). In particular, we distinguished between proxy alignment, suboptimality alignment, approximate alignment, and deceptive alignment. I still make heavy use of this classification, though I now believe that there are some additional types of pseudo-alignment which I think are fairly important but which I don't feel like this classification fully addresses. In particular, there are two variations on pseudo-alignment not discussed in the paper which I want to talk about here: corrigible pseudo-alignment and suboptimality deceptive alignment.

Corrigible pseudo-alignment. In the paper, we defined corrigible alignment as the situation in which "the base objective is incorporated into the mesa-optimizer's epistemic model and [the mesa-optimizer's] objective is modified to 'point to' that information." We mostly just talked about this as a form of robust alignment—however, as I note in "Towards a mechanistic understanding of corrigibility," this is a very unstable operation, requiring you to get your pointer just right. Thus, I think it's better to talk about corrigible alignment as the class of possible relationships between the base and mesa-objectives defined by the model having some sort of pointer to the base objective, including both corrigible robust alignment (if the pointer is robust) and corrigible pseudo-alignment (if the pointer is to some sort of non-robust proxy). In particular, I think this distinction is fairly important to why deceptive alignment might be more likely than robust alignment, as it points at why robust alignment via corrigibility might be quite difficult (which is a point we made in the paper, but one which I think is made much clearer with this distinction).

Suboptimality deceptive alignment. In the paper, we defined suboptimality alignment as follows:

A mesa-optimizer is suboptimality aligned if some deficiency, error, or limitation in its optimization process causes it to exhibit aligned behavior on the training distribution. This could be due to computational constraints, unsound reasoning, a lack of information, irrational decision procedures, or any other defect

...
Drug Policy
by jefftk — 2019-11-04T21:30:02.058Z

I've been thinking some about what drug policy should look like, from
a few angles:

Drug prohibition hasn't gone very well. It's very expensive,
leads to a lot of crime and jail time, and dangerous drugs are still
widely available. Plus a lot of the ways that we think "drugs are
dangerous" are actually "getting addicted to something illegal and
inconsistent is dangerous.

Which drugs we prohibit is a historical accident. Alcohol is
substantially worse (Nutt 2007) than many
illegal options.

I don't like
recreational drugs. For a social lubricant I strongly prefer food,
music, dancing, singing, or gaming.

Lots of people disagree with me on this.

I'd like to see a policy that approaches drugs from the perspective
that our society is going to have some, and pushes people in the
direction of drugs and approaches that are least harmful. Something
like high taxes on alcohol and cigarettes, while removing restrictions
on vaping, cannabis, LSD, MDMA, and psilocybin.

For extremely addictive drugs such as heroin and other opioids, there
are probably social environments where they could be consumed safely.
Unlike alcohol, they don't lead people to hurt those around them, and
mostly cause harm through the efforts people will take to obtain more.
You could imagine some sort of "heroin retirement" that mitigates
these issues by having you put up funds to pay for heroin and
nursing-home style care for the rest of your life.

When considering the effects of various drugs there's also the way
they tend to make people change. Stimulants typically increase
productivity, cannabis often reduces ambition, LSD can give people a
sense of oneness that can make them more altruistic, etc. I'm not
sure if policy should consider things like this.

Overall our current policy is so far from reasonable that there are a
lot of potential ways to make it better, and I think the recent
efforts around marijuana legalization are a step forward.
Will transparency help catch deception? Perhaps not
by Matthew Barnett — 2019-11-04T20:52:52.681Z

In the last few months, some AI safety researchers have signaled what Wei Dai called a "wave of optimism" for using transparency tools to catch deceptive behavior in our machine learning models. By a transparency tool, I mean any technique that allows us to peek inside a model to get a better idea of what goes on inside of it. This post outlines one reason to be pessimistic about the usefulness of using transparency tools to catch advanced deception.

First, I construct a model for why we need transparency tools in the first place. I imagine the training process as a game played by two adversaries: a deceptive model, and an overseer trying to catch any deceptive reasoning. The issue for the overseer is that the deceptive model is opaque, which means that the overseer must use some tools to interpret its thoughts and behavior.

While by design, the deceptive model and overseer are comparably intelligent, the nature of using transparency tools enables a "weakest link" dynamic, implying that the deceptive model can exploit weaknesses in the transparency procedure to fool the overseer. In particular, deception can be hidden if the deceptive model has a good understanding of the deployment environment that isn't revealed while using transparency tools.

In the end, I consider whether this weakness implies that transparency tools act as unnecessary middlemen for catching deception, and use an analogy to GANs to argue that training an overseer to detect deception end-to-end might work better.

The model

Our goal is to take a neural network and detect whether there are any inputs to the network that will cause it to act unacceptably. By assumption, we are concerned about deceptive alignment, a scenario where a misaligned model understands its training process, and will therefore act aligned during training while plotting something else for later.

Since we want to detect deceptive thoughts, we are interested in opening up the model and seeing what goes on inside. We have access to an overseer that interprets the model and spends time thinking whether the model has inputs which cause it to act catastrophically, which will allow us to penalize the model accordingly. Crucially, this overseer is not able to understand the model weights by themselves: instead it relies on a transparency procedure that converts the model into a relatable abstraction, which it then uses to determine ...
Politics is work and work needs breaks
by KatjaGrace — 2019-11-04T17:10:01.120Z

A post written a few years ago, posting now during a time of irrelevance (as far as I know with my limited attention on politics or social media) so as not to be accidentally taking a side in any specific political debates.

***

Alexandra What has happened is shocking and everyone should oppose it.

Beatrice I’m eating a sandwich. It is better than I expected.

Alexandra I can’t believe you would write about a sandwich at a time like this. Don’t you oppose what happened?

(Claire is excited by the breakfast bread discussion but guesses that contributing a picture of her bagel is pretty inappropriate and looks at SMBC instead.)

People break up their time into work and leisure. You might think of work vs. leisure as roughly ‘doing stuff because other people pay for it’ vs. ‘doing stuff because you want to’. You can also think of it as roughly ‘effortful’ vs. ‘relaxing’. Often these categories align—other people pay you to do effortful things, and the things you want to do are more relaxing. They don’t always align. Helping your aged relative go to the doctor might be effortful but a thing you do because you want to, or your job might be so easy that you come home with a bunch of energy for challenging tasks.

I’m going to call these ‘resting-‘ vs. ‘challenged-‘ and ‘-boss’ vs. ‘-employee’. So entirely free time is mostly resting-boss and paid work is usually challenged-employee. But you can also get resting-employee and challenged-boss activities. This all maybe relies on some models of rest and attention and effort and such that don’t work out, but they seem at least close the models that most people practically rely on. For whatever reason, most people prefer to spend a decent fraction of their time doing non-effortful things, unless something terrible will happen soon if they don’t.

People mostly use social media as leisure, both in the sense that nobody would intentionally pay them for it, and in the sense that it is not effortful. When important political things are happening, social media naturally turns to discussion of them. Which, if you are lucky, might be thoughtful analysis of world affairs, with a bunch of statistics and rethinking your assumptions and learning new facts about the world. Which is all great, but it is not leisure in the ‘not effort’ sense. When I need a break from researching the likely social consequences of artificial intelligence, moving...
Elon Musk is wrong: Robotaxis are stupid. We need standardized rented autonomous tugs to move customized owned unpowered wagons.
by chaosmage — 2019-11-04T14:04:10.851Z

Sorry for the flashy headline, but I genuinely feel this might be the best idea I ever had.

After the invention of the horseless carriage, it supposedly took people years to realize that without reins to hold, the driver could now sit inside the cabin. Change of the core technology allowed a rethink of the entire product (the vehicle) but that rethink was hard.

With autonomous cars, we already have a rethink. Autonomous vehicles can be called to a person who needs it, so they should not be owned (and sit idle while the owner doesn't need to move) but should be taxis. Lots of people, including Elon Musk, agree that's the way to go. And it is indeed an improvement over the current car ownership model. But it doesn't go far enough.

The robotaxi concept retains an outdated assumption: that a single product, a single vehicle needs to do both the moving and the accommodation of the user.

Get rid of that assumption and imagine instead a system made from two very different parts that can (automatically) couple and uncouple:

A driving unit (tug) with the motor, driving and communication systems. This is usually rented not owned.
An accommodation unit (wagon) for the human(s) and their stuff. This is usually owned not rented.

How is this better than robotaxis? Lots of reasons!

Your personal wagon can have a bed in it, or your valuables, or spare underwear. You can use it as a movable locker, as you do every time you use a personal car to get groceries from multiple shops. You can sleep in it.
Your personal wagon gives you full control over the space you occupy while traveling. A robotaxi requires you to take out your stuff every time you leave and check whether the previous user did every time you enter. And what if the previous occupant used a perfume you're allergic to?
The tug doesn't need seats, windows or air conditioning. It doesn't need to haul all that mass every time it is called to a new user. A lot of complexity is removed from production, so these are much cheaper to build than robotaxis.
Renting out these tugs is much less hassle than renting out robotaxis because they need much less maintenance. They can be pretty thoroughly protected from interference by users. Charging infrastructure for tugs needs much less space than charging infrastructure for robotaxis.
Your personal wagon can be in a huge variety of designs and sizes! Human-pilot

...
Book Review: Man's Search for Meaning by Victor Frankel
by Chris_Leong — 2019-11-04T11:21:05.791Z

(Oops, I meant this to be a shortform)

"Everything can be taken from a man but one thing: the last of the human freedoms—to choose one’s attitude in any given set of circumstances, to choose one’s own way"

“He who has a why to live for can bear almost any how.” —Friedrich Nietzsche

The core premise of this book is well-known: that in order to survive the harshest and most terrible of circumstances one must have some kind of purpose to keep them going. This book describes the kind of mindset that one would have to adopt to survive in such circumstances, but the question that kept arising in my mind was, "Would it actually be worth it?". Would it actually be worth it to endure the degrading and inhumane conditions of a concentration camp given that it would almost certainly come to nothing? Perhaps if I knew that I was going to survive then I could find some meaning and growth in the suffering, but to go through all that suffering and for it all to be a waste... Or perhaps one could endure for love, but what would this amount to if you were likely to never be reunited, if they could already be dead? Or if one thought they could achieve the kind of good Victor Frankel did, but that lot falls to very few.

Victor Frankel is religious so he can find purpose in this. If there is a God and he he has a purpose, then one has played a role in this divine plan by bearing their suffering with dignity and could further hope to be rewarded in the afterlife. So he would view his life as having been meaningful even if he had perished, but what about the rest of us? Ultimately, I suspect that I may be asking too much out of him. These techniques could still provide a large amount of value, even if we reject his premise that any life is still worth leading, for a life can still be worth leading even if it involves a great deal of suffering.

His view that life is always worth living is expressed in the following quote: "Life asks you the meaning of life; you don't ask life. You are questioned by life. It's not what you expect from life, but what life expects from you." As noted, I disagree with this in extreme circumstances, but nonetheless, once it has been established that life is worthwhile or at least that you don't want to die, this attitude is on the whole healthy.

Camp life:

I found these des...
Ways that China is surpassing the US
by Wei Dai — 2019-11-04T09:45:53.881Z
Link: https://palladiummag.com/2019/10/11/the-american-dream-is-alive-in-china/

[Edit: Changed the post title from the original article title to something more meaningful.]

I came across this article today, and I have to agree with it strongly based on my own recent trip to China. The update it triggered for me is the realization that China is genuinely doing better than the US on many fronts, most importantly on governance. How/why did that happen? Did anyone or any political theory predict this ahead of time? (In case it's not clear, this is not meant to be a rhetorical question. I'm surprised and confused and am wondering if someone or some theory can offer an explanation.) What does it imply for things like AI governance and global coordination on x-risks?

Below I'll quote two sections that summarize most of the article. Click the link above for the full text.

China is changing in a deep and visceral way, and it is changing fast , in a way that is almost incomprehensible without seeing it in person. In contrast to America’s stagnation, China’s culture, self-concept, and morale are being transformed at a rapid pace—mostly for the better.

For Americans, this transformation towards competence and prosperity should be the more worrying thing about China. We aren’t well-coordinated with China, and it is starting to surpass the U.S. in important ways. If we ignore this point by focusing only on the moral negatives, of which there are many, we risk underrating the competition until it’s too late, or failing to reflect on the importance of solving our own mounting problems of governance.

More optimistically, we should study Chinese development as a positive example of how to do better on the things that America has been fumbling recently: infrastructure, growth, industrial policy, and positive transformation of everyday life. America has been a shining beacon of this kind of development in the past, and can be again, if we’re willing to look closer at what China does right.

[...]

If I’m being honest, China’s success scares me. There is something deeply disconcerting about watching China surpass America in the ways it is. China is transforming fishing villages into major industrial cities, while we fail to build high-speed rail or new housing. How are we going to catch up?

Is it really as bad as it seems? I understand the instinct to avoid the topic, disbelieve it, and play it down. I’ve even had the instinct to stay quiet about China’s progress m

...