trentmkelly/LessWrong-43k huggingface dataset:
In Defense of the Arms Races… that End Arms Races
by Gentzel — 2020-01-15T21:30:00.828Z

All else being equal, arms races are a waste of resources and often an example of the defection equilibrium in the prisoner’s dilemma. However, in some cases, such capacity races may actually be the globally optimal strategy. Below I try to explain this with some examples.

1: If the U.S. kept racing in its military capacity after WW2, the U.S. may have been able to use its negotiating leverage to stop the Soviet Union from becoming a nuclear power: halting proliferation and preventing the build up of world threatening numbers of high yield weapons. Basically, the earlier you win an arms race, the less nasty it may be later. If the U.S. had won the cold war earlier, global development may have taken a very different course, with decades of cooperative growth instead of immense amounts of Soviet GDP being spent on defense, and ultimately causing its collapse. The principle: it may make sense to start an arms race if you think you are going to win if you start now, provided that a nastier arms race is inevitable later.

2: If neither the U.S. nor Russia developed nuclear weapons at a quick pace, many more factions could have developed them later at a similar time, and this would be much more destabilizing and potentially violent than cases where there is a monopolar or a bipolar power situation. Principle: it is easier to generate stable coordination with small groups of actors than large groups. The more actors there are, the less likely MAD and treaties are to work, the earlier an arms race starts, the more prohibitively expensive it is for new groups to join the race.

3: If hardware design is a bottleneck on the development of far more powerful artificial intelligence systems, then racing to figure out good algorithms now will let us test a lot more things before we get to the point a relatively bad set of algorithms can create an immense amount of harm due to the hardware it has at its disposal (improbable example: imagine a Hitler emulation with the ability to think 1000x faster). Principle: the earlier you start an arms race, the more constrained you are by technological limits.1

I do not necessarily think these arguments are decisive, but I do think it is worth figuring out what the likely alternatives are before deciding if engaging in a particular capacity race is a bad idea. In general:

It’s nice for there to not be tons of violence and death from

...
Fire Alarm for AGI
by  — 2020-01-15T20:41:39.616Z
Link: https://blog.acolyer.org/2020/01/15/programmatically-interpretable-reinforcement-learning/

" While the interpretable model may not get to quite the same level of performance as the Oracle on the exact task used for training, it turns out that the process of generating the interpretable model results in something which generalises much better to new situations."

This sounds like artificial dreaming:

"...after a certain number of search steps for a fixed set H, and after choosing the best available synthesized program for this set, we sample a set of additional histories by simulating the current programmatic policy, and add these samples to H."

The following also struck me as portentous although not as much as the above paper.

https://ai.facebook.com/blog/using-neural-networks-to-solve-advanced-mathematics-equations/
Go F*** Someone
by Jacob Falkovich — 2020-01-15T18:39:33.080Z

As always, cross-posted from Putanumonit.

From Tokyo to TriBeCa, people are increasingly alone. People go on fewer dates, marry less and later, have smaller families if at all. People are having less sex, especially young people. The common complaint: it’s just too hard. Dating is hard, intimacy is hard, relationships are hard. I’m not ready to play on hard mode yet, I’ll do the relationship thing when I level up.

And simultaneously, a cottage industry sprung up extolling the virtue of loneliness. Self-care, self-development, self-love. Travel solo, live solo, you do you. Wait, doesn’t that last one literally mean “go fuck yourself”?

This essay is to tell you: go fuck someone else. Ask someone on a date. At the very least, invite someone to hang out and ask them what they’re struggling with. This essay is not about how to make friends and lovers (a topic I’ll come back to), but an exhortation to actually go and do that. Now instead of later, directly instead of ass-backwards, seek relationships instead of seeking to be deemed worthy of relationships. If you think this is all too obvious to mention, reread the first two paragraphs again.

My argument doesn’t hinge on specific data relating to the intimacy recession and whether the survey counting sex dolls adjusted for inflation. If you’re reading Putanumonit as a brief escape from all the loving relationships smothering you, congrats! If you’re trying as hard as you can to connect and the world isn’t reciprocating, consider this essay as written for those you seek to connect with instead. Reverse all advice as neccessary.

This essay’s epistemic status is whatever The Last Psychiatrist was drinking.

Wherefore all this aloneness? The pink-hairs blame the red-pills who blame the pink-hairs. But really, they’re both in agreement that men and women are natural enemies and any interactions between the two are zero-sum. If you’re stuck in zero-sum thinking you’re probably on the wrong blog, but take this as a first dose of medicine and then go give someone a hug.

One level up from the gender war is the class war. Leftists blame loneliness on capitalism — single people buy twice as many toasters, sex toys, and Netflix subscriptions. Rightists blame socialism — for the state to be your daddy it must first destroy the family. I won’t spend much time on this. If your ability to connect with people depends more than zero on t

...
[AN #82]: How OpenAI Five distributed their training computation
by Rohin Shah — 2020-01-15T18:20:01.270Z

Find all Alignment Newsletter resources

here

. In particular, you can

sign up

, or look through this

spreadsheet

of all summaries that have ever been in the newsletter. I'm always happy to hear feedback; you can send it to me by replying to this email.

Audio version

here

(may not be up yet).

Highlights

Dota 2 with Large Scale Deep Reinforcement Learning

(OpenAI et al) (summarized by Nicholas): In April,

OpenAI Five

(

AN #54

) defeated the world champion Dota 2 team, OG. This paper describes its training process. OpenAI et al. hand-engineered the reward function as well as some features, actions, and parts of the policy. The rest of the policy was trained using PPO with an LSTM architecture at a massive scale. They trained this in a distributed fashion as follows:

- The Controller receives and distributes the updated parameters.

- The Rollout Worker CPUs simulate the game, send observations to the Forward Pass GPUs and publish samples to the Experience Buffer.

- The Forward Pass GPUs determine the actions to use and send them to the Rollout Workers.

- The Optimizer GPUs sample experience from the Experience Buffer, calculate gradient updates, and then publish updated parameters to the Controller.

The model trained over 296 days. In that time, OpenAI needed to adapt it to changes in the code and game mechanics. This was done via model “surgery”, in which they would try to initialize a new model to maintain the same input-output mapping as the old one. When this was not possible, they gradually increased the proportion of games played with the new version over time.

Nicholas's opinion: I feel similarly to my opinion on

AlphaStar

(

AN #73

) here. The result is definitely impressive and a major step up in complexity from shorter, discrete games like chess or go. However, I don’t see how the approach of just running PPO at a large scale brings us closer to AGI because we can’t run massively parallel simulations of real world tasks. Even for tasks that can be simulated, this seems prohibitively expensive for most use cases (I couldn’t find the exact costs, but I’d estimate this model cost tens of millions of dollars). I’d be quite excited to see an example of deep RL being used for a complex real world task without training in simulation.

Technical AI alignment

Technical agendas and prioritization

Just Imitate Humans?

(Micha...
ACDT: a hack-y acausal decision theory
by Stuart_Armstrong — 2020-01-15T17:22:48.676Z

Inspired by my post on problems with causal decision theory (CDT), here is a hacked version of CDT that seems to be able to imitate timeless decision theory (TDT) and functional decision theory[1] (FDT), as well as updateless decision theory (UDT) under certain circumstances.

Call this ACDT, for (a)causal decision theory. It is, essentially, CDT which can draw extra, acausal arrows on the causal graphs, and which attempts to figure out which graph represents the world it's in. The drawback is its lack of elegance; the advantage, if it works, is that it's simple to specify and focuses attention on the important aspects of deducing the graph.

Defining ACDT

CDT and the Newcomb problem

In the Newcomb problem, there is a predictor Ω who leaves two boxes, and predicts whether you will take one ("one-box") or both ("two-box"). If Ω predicts you will one-box, it had put a large prize in that first box; otherwise that box is empty. There is always a small consolation prize in the second box.

In terms of causal graphs, we can represent it this way:

The dark red node is the decision node, which the agent can affect. The green node is a utility node, whose value the agent cares about.

The CDT agent uses the "do" operator from Pearl's Causality. Essentially all the incoming arrows to the decision node are cut (though the CDT agent keeps track of any information gained that way), then the CDT agent maximises its utility by choosing its action:

In this situation, the CDT agent will always two-box, since it treats Ω's decision as fixed, and in that case two-boxing dominates, since you get whatever's in the first box, plus the consolation prize.

ACDT algorithm

The ACDT algorithm is similar, except that when it cuts the causal links to its decision, it also adds potential links from that decision node to all the other nodes in the graph. Then it attempts to figure out which diagram is correct, and then maximises its utility in the CDT way.

Note that ACDT doesn't take a position on what these extra links are - whether they are pointing back in time or are reflecting some more complicated structure (such as the existence of predictors). It just assumes the links could be there, and then works from that.

In a sense, ACDT can be seen as anterior to CDT. How do we know that causality exists, and the rules it runs on? From our experience in the world. If we lived in a world where the Newc

...
In defense of deviousness
by Juan Andrés Hurtado Baeza — 2020-01-15T11:56:12.387Z
Link: https://medium.com/lai4d/in-defense-of-deviousness-e8e877d8820d

Why hard-to-read code can be a good code. A complex criticism to the “keep it simple” universal coding advice

It is easy to find articles and decalogues where experienced software developers manifest what they think are good rules for coding. Usually those recommendations are expressed with strong belief due to the reinforcement from similar opinions of most other authors, and due to they apparently emerge from plausible goals like readability, maintainability or simplicity. Who could question those goals? Probably no one since they are desirable goals but…are they objectively measurable?
After years of trying to think like a machine, what surely has affected my perception of complexity, I can write this response to some of those recommendations, not because of an opposed belief but because a reflection about how those recommendations supposedly drive to the goals is necessary.

I would like to begin with an anecdote

Some years ago I was working in an R&D project trying to solve a really intricate problem. I had been blocked for more than five months when I finally saw the light. One week later I had the corresponding algorithm implemented and it worked. I felt like a victorious warrior after the hardest battle.
Having passed near two years I noticed that a determined casuistic was not correctly treated by the algorithm so I had to review it, nothing special. I read the algorithm’s documentation, I read the source code and, to my surprise, I was unable to understand it!. How was it possible!, I figured out the solution!, I implemented the algorithm!, I commented the source code!, I wrote the extended documentation!, then why wasn’t I able to understand what my own program was doing?

I read all again several times and it took me two hours of thinking about the problem until I achieved the understanding of the algorithm I implemented to solve it. The experience was so disgusting that I promised myself this will never happen again. I had to be more careful and exhaustive when documenting a process as complex as that and when commenting its implementation. Then I prepared to rewrite the extended documentation and…amazingly, I realized that the documentation I initially wrote correctly explained the problem and its solution, and that the comments in the source code were useful and sufficient.
What had actually happened was very simple. After so much time working on o...
Avoiding Rationalization
by dspeyer — 2020-01-15T10:55:33.153Z

Previously: Red Flags for Rationalization

It is often said that the best way to resist temptation is to avoid it. This includes the temptation to rationalize. You can avoid rationalization by denying it the circumstances in which it would take place.

When you avoid rationalization, you don't need to worry about infinite regress of metacognition, nor about overcompensation. As a handy bonus, you can demonstrate to others that you weren't rationalizing, which otherwise involves a lot of introspection and trust.

Here are three ways to do it:

Double Blinding

Identify the thing that would control your rationalization and arrange not to know it.

The trope namer is experimental science. You might be tempted to give better care to the experimental than the control group, but not if you don't know which is which. In many cases, you can maintain the blinding in statistical analysis as well, comparing "group A" to "group B" and only learning which is which after doing the math.

Similarly, if you are evaluating people (e.g. for a job) and are worried about subconscious sexism (or overcompensation for it), write a script or ask a friend to strip all gender indicators from the application.

Unfortunately, this technique requires you to anticipate the rationalization risk. Once you notice you might be rationalizing, it's usually too late to double-blind.

End-to-End Testing

A logical argument is a series of mental steps that build on each other. If the argument is direct, every step must be correct. As such, it is somewhat similar to a computer program. You wouldn't write a nontrivial computer program, look it over, say "that looks right" and push it to production. You would test it. With as close as possible to a full, end-to-end test before trusting it.

What does testing look like for an argument? Take a claim near the end of the argument which can be observed. Go and observe it. Since you have a concrete prediction, it should be a lot easier to make the observation.

Once you've got that, you don't need the long chain of argument that got you there. So it doesn't matter if you were rationalizing along the way. The bit at the end of the argument which you haven't chopped off still needs scrutinizing, but it's shorter, so you can give each bit more attention.

Suppose you're organizing so...
Artificial Intelligence and Life Sciences (Why Big Data is not enough to capture biological systems?)
by HansNauj — 2020-01-15T01:59:25.947Z

Introduction

Artificial intelligence algorithms, and in particular deep learning, are becoming increasingly relevant to biology research. With these algorithms it is possible to analyze large amounts of data that are impossible to analyze with other techniques, helping to detect characteristics that might otherwise be impossible to recognize. This automatic feature detection allows you to classify cellular images, make genomic connections, advance drug discovery, and even find links between different types of data, from genomics and images to electronic medical records.[1]

However, as S. Webb pointed in a technology feature published in Nature, like any computational-biology technique, the results that arise from artificial intelligence are only as good as the data that go in. Overfitting a model to its training data is also a concern. In addition, for deep learning, the criteria for data quantity and quality are often more rigorous than some experimental biologists might expect [2]. Regarding this aspect, if organisms behave more like observers who make decisions depending a constant changing environment, then this strategy is not useful at all because individual decisions introduce constant irregularities in the collected data that cannot be grasped in a simple model or principle[3].

This perspective is not only a challenge to research in biology but seeks for an alternative approach to our understanding of intelligence, since the goal is not only to understand and simply classify individual complex structures and organisms, but is also about how these individual intelligences (heterogeneously) distribute into spaces and (eco) systems, and generate coherence[4] from apparent incoherent world. This perspective also has implications about how any research and derived technologies about intelligence cannot be seen as an isolated phenomenon from the rest of the nature.

Big data as golden pathway in biology?

We currently live in a golden age of the biology in part due to the idea that biological systems consist of a relationship between the parts (e.g. molecules) and the whole (e.g. organisms or ecosystems). Thanks to advances in technology (for instance in microscopy), mathematics and informatics it is possible today to measure and model complex systems by interconnecting different components at different scales in the space and depending on time[5]. To thi...
Clarifying The Malignity of the Universal Prior: The Lexical Update
by interstice — 2020-01-15T00:00:36.682Z

[UPDATE: looks like the lexical update is real after all; see Paul's comment and my reply]

In Paul's classic post What does the universal prior actually look like? he lays out an argument that the universal prior, if it were to be used for important decisions, would likely be malign, giving predictions that would effectively be under the control of alien consequentialists. He argues for this based on an 'anthropic update' the aliens could make that would be difficult to represent in a short program. We can split this update into two parts: an 'importance update' restricting attention to bits fed into priors used to make important decisions, and what I'm calling a 'lexical update' which depends on the particular variant of the universal prior being used. I still believe that the 'importance update' would be very powerful, but I'm not sure anymore about the 'lexical update'. So in this post I'm going to summarize both in my own words then explain my skepticism towards the 'lexical update'.

As background, note that 'straightforwardly' specifying data such as our experiences in the universal prior will take far more bits than just describing the laws of physics, as you'll also need to describe our location in spacetime, an input method, and a set of Everett branches(!), all of which together will probably take more than 10000 bits(compared to the laws alone which likely only take a few hundred) Thus, any really short program(a few hundred bits, say) that could somehow predict our experiences well would likely have a greater probability according to the universal prior than the 'straightforward' explanation.

Paul's post argues that there likely do exist such programs. I'm going to fix a reference prefix machine U which generates a universal prior. The argument goes:

A) there are many long-running programs with short descriptions according to U, such as our universe.

B) If other programs are like our universe's program, aliens could evolve there and end up taking over their programs.

C) Since their program has high measure in U, the aliens will plausibly have been selected to be motivated to control short programs in U.

D) To control U, the aliens could try to manipulate beings using the universal prior who have control over short programs in U (like us, hypothetically)

E) If the aliens are reasonably motivated to manipulate U, we can sample them doing that with few bits.

...
Tips on how to promote effective altruism effectively? Less talk, more action.
by culturechange — 2020-01-14T23:17:34.697Z

Hi!

We are two 20 years old Finnish students from the University of Helsinki, who are very concerned about global issues. We have a lot of energy and have met a couple of times for organizing our thoughts. We found out about effective altruism and it seems to be exactly the ideology that we want to strive for.

But, on our meetings, we ran into a problem: what is the most effective way to concentrate this energy, so we can advance from talking to taking action? Our agenda and goals seem to be too ambiguous at the moment, so we hope that you could shed some light on what on earth (no pun intended) should we do.

As far as we know, effective altruism hasn't really had a breakthrough in making a cultural change on how people think. The movement in Finland is even more marginal: there are only 30 people in the EA Finland Telegram group and around 10+ people are attending the meetups. As we don't have that much experience in popularizing a social movement, we would need help on how to achieve our goals regarding this project? At least ridding us from the anxious ambiguity of not knowing exactly what concrete things we should do.

Best regards,

culturechange

P.S. Has there been any research on effective promoting of effective altruism?