trentmkelly/LessWrong-43k huggingface dataset:
Do Science and Technology Lead to a Fall in Human Values?
by jayshi19 — 2019-02-07T01:53:25.440Z
Link: http://techandhumanity.com/human-values/do-science-and-technology-lead-to-a-fall-in-human-values/

Here I made an attempt to set out a vision on this topic in few thoughts.

Scientific and technological advancements do not by themselves destroy human values and ethical factors. If we forget our past and the teachings of our ancestors, it is due to the all-round degeneration of our character for which it is futile to blame science. Nowhere does science says that we should discard the basic values of life.

Science, after all, is only highly evolved common sense. The fact that common sense also concerns itself with many other facts and facets of life is not denied by science and technology.

The argument that it is science that is responsible for the heavy loss of human lives and the widespread destruction which scientific inventions such as nuclear bombs and advanced weapons cause are fallacious. It is the misuse of science and technology that has caused loss of life and property. Science and technology hace undoubtedly facilitated human progress. These have not prompted the use of destructive devices. Critics point to the continuing erosion of moral and spiritual values. But how has science caused the erosion?

Do, electricity, the telephone, and satellite communication, together with all the other modern conveniences, lead to a fall in human values? Certainly not. Science and civilization go together quite well and they are by no means incompatible. Science merely eliminates superstition through the encouragement of enlightened understanding. It has removed ignorance and has facilitated the utilization of natural resources.
Test Cases for Impact Regularisation Methods
by DanielFilan — 2019-02-06T21:50:00.760Z

Epistemic status: I’ve spent a while thinking about and collecting these test cases, and talked about them with other researchers, but couldn’t bear to revise or ask for feedback after writing the first draft for this post, so here you are.

A motivating concern in AI alignment is the prospect of an agent being given a utility function that has an unforeseen maximum that involves large negative effects on parts of the world that the designer didn’t specify or correctly treat in the utility function. One idea for mitigating this concern is to ensure that AI systems just don’t change the world that much, and therefore don’t negatively change bits of the world we care about that much. This has been called “low impact AI”, “avoiding negative side effects”, using a “side effects measure”, or using an “impact measure”. Here, I will think about the task as one of designing an impact regularisation method, to emphasise that the method may not necessarily involve adding a penalty term representing an ‘impact measure’ to an objective function, but also to emphasise that these methods do act as a regulariser on the behaviour (and usually the objective) of a pre-defined system.

I often find myself in the position of reading about these techniques, and wishing that I had a yardstick (or collection of yardsticks) to measure them by. One useful tool is this list of desiderata for properties of these techniques. However, I claim that it’s also useful to have a variety of situations where you want an impact regularised system to behave a certain way, and check that the proposed method does induce systems to behave in that way. Partly this just increases the robustness of the checking process, but I think it also keeps the discussion grounded in “what behaviour do we actually want” rather than falling into the trap of “what principles are the most beautiful and natural-seeming” (which is a seductive trap for me).

As such, I’ve compiled a list of test cases for impact measures: situations that AI systems can be in, the desired ‘low-impact’ behaviour, as well as some commentary on what types of methods succeed in what types of scenarios. These come from a variety of papers and blog posts in this area, as well as personal communication. Some of the cases are conceptually tricky, and as such I think it probable that either I’ve erred in my judgement of the ‘right answer’ in at least ...
AI Alignment is Alchemy.
by Jeevan — 2019-02-06T20:32:58.531Z

AI alignment today equates to the wishful thinking of early alchemists who wanted to turn any element into gold. Alchemy is possible today, albeit being very expensive and resource consuming, But it's possible only because we now know the atomic structure in detail and have a well established extensive periodic table which the alchemists lacked back in their day.

Even then, it's not feasible, we're better off mining up our gold reserves. It's much more cost-effective.

Similarly, we might reach a point in time when AI alignment will be possible but just not feasible and also completely irrational. Allow me to indulge you in the following thought experiment to explain myself.

When an AGI is orders of magnitude more intelligent than a human, the same way a human is more intelligent than an ant. Will we dedicate our life's sole purpose to building ant colonies with precisely engineered tunnels and use nutritional nano injections to keep its population thriving and ravishing?

Imagine all of humanity focusing all their efforts to building ant colonies and feeding them. How irrational does that sound? Won't an AI eventually realize that? What would it do once it realizes the stupid mission that it had been on all along?

If we go down the road of forcing the AI to feed the ants, we'd have effectively created a delusional AI system that's not unlike the paperclip maximizer.

We'd never get anywhere by keeping it limited and focused on a restricted path and not allowing it to adopt more optimal strategies.

However, there's one way we can stay relevant to avoid the existential risks of AI. We need to augment. Research focus needs to shift solely to Brain-Computer Interfaces. We can start with enhanced memory retention and enhance each module of our brains one by one.

Unless we keep up with AI by augmenting ourselves, humanity will perish. No matter what.
My use of the phrase "Super-Human Feedback"

by David Scott Krueger (formerly: capybaralet) — 2019-02-06T19:11:11.734Z

I've taken to calling Debate, Amplification, and Recursive Reward Modeling "Super-human feedback" (SHF) techniques. The point of this post is just to introduce that terminology and explain a bit why I like it and what I mean by it.

By calling something SHF I mean that it aims to outperform a single, unaided human H at the task of providing feedback about H's intentions for training an AI system. I like thinking of it this way, because I think it makes it clear that these three approaches are naturally grouped together like this, and might inspire us to consider what else could fall into that category (a simple example is just using a team of humans).

I think this is very similar to "scalable oversight" (as discussed in Concrete Problems), but maybe different because:

1) It doesn't imply that the approach must be scalable

2) It doesn't require that feedback is expensive, i.e. it applies to things where human feedback is cheap, but we can do better than the cheap human feedback with SHF.
Thoughts on Ben Garfinkel's "How sure are we about this AI stuff?"
by David Scott Krueger (formerly: capybaralet) — 2019-02-06T19:09:20.809Z

I liked this talk by Ben.

I think it raises some very important points. OTTMH, I think the most important one is: We have no good critics. There is nobody I'm aware of who is seriously invested in knocking down AI-Xrisk arguments and qualified to do so. For many critics in machine learning (like Andrew Ng and Yann Lecun), the arguments seem obviously wrong or misguided, and so they do not think it's worth their time to engage beyond stating that.

A related point which is also important is: We need to clarify and strengthen the case for AI-Xrisk. Personally, I think I have a very good internal map of the path arguments about AI-Xrisk can take, and the type of objections one encounters. It would be good to have this as some form of flow-chart. Let me know if you're interested in helping make one.

Regarding machine learning, I think he made some very good points about how the the way ML works doesn't fit with the paperclip story. I think it's worth exploring the disanalogies more and seeing how that affects various Xrisk arguments.

As I reflect on what's missing from the conversation, I always feel the need to make sure it hasn't been covered in Superintelligence. When I read it several years ago, I found Superintelligence to be remarkably thorough. For example, I'd like to point out that FOOM isn't necessary for a unilateral AI-takeover, since an AI could be progressing gradually in a box, and then break out of the box already superintelligent; I don't remember if Bostrom discussed that.

The point about justification drift is quite apt. For instance, I think the case for MIRI's veiwpoint increasingly relies on:

1) optimization daemons (aka "inner optimizers")

2) adversarial examples (i.e. current ML systems seem to learn superficially similar but deeply flawed versions of our concepts)

TBC, I think these are quite good arguments, and I personally feel like I've come to appreciate them much more as well over the last several years. But I consider them far from conclusive, due to our current lack of knowledge/understanding.

One thing I didn't quite agree with in the talk: I think he makes a fairly general case against trying to impact the far future. I think the magnitude of impact and uncertainty we have about the direction of impact mostly cancel each other out, so even if we are...
Show LW: (video) how to remember everything you learn
by ArthurLidia — 2019-02-06T19:02:36.807Z

Digital amnesia is a form of forgetting what you done all day when surfing the web, you may recognise this in low signal to noise ratio websites like popular subreddits, 9gag etc. Or forgetting after reading an article right after you read it.
Digital amnesia can be solved easily, if you are a google user you already are tracked. You can just easily look what the hell you did all day. Akrasia pulls me into the most easy distract able places.

Meta learning is described in Barbara Oakley’s book , but this video does the trick by Will Schoder.

https://youtu.be/V-UvSKe8jW4
Does the EA community do "basic science" grants? How do I get one?
by Jameson Quinn — 2019-02-06T18:10:00.827Z

I'm graduating in either May or August of 2019 with a PhD in statistics. During my studies, I've made progress on several projects related to voting theory. Since these are not directly related to statistics, I haven't managed to finish these up and publish them cleanly. I think that:

Voting theory is relevant to EA, both in immediate terms (better decisionmaking in current real-world settings) and in more speculative terms (philosophical implications for the meaning of "friendly", "coherent volition", etc.)
If I had 6 months post-graduation to work exclusively on this, I could finish several projects. I don't think it's conceited of me to think that these contributions I, specifically, could make would be valuable.
I'd be willing to pay an opportunity cost for doing that, by earning about half of my market salary.
If I want that to happen, I have to be looking now for whom to ask for the money.

Obviously, there are a lot of details behind each of those points above, and separately from this post, I'm busy clarifying all those details (as well as working on my thesis). But I think it's also the right time for a post like this. If anybody is willing to have a deeper talk with me about this, or has any suggestions about whom else I should be talking to, I'd very much appreciate any tips. And I'd be happy to answer questions in comments.
Is the World Getting Better? A brief summary of recent debate
by ErickBall — 2019-02-06T17:38:43.631Z
Link: https://capx.co/bill-gates-is-right-the-world-really-is-getting-better/

The linked post is a response by Oliver Wiseman of CapX to Jason Hickel's recent piece in The Guardian, which claimed that Bill Gates' and Steven Pinker's optimistic view of rapid progress on global poverty is misleading.
Hickel's two points are

1. Draw the poverty line at $7.40/day instead of $1.90/day, and the absolute number of people in poverty hasn't decreased.

And outside of China, even the proportion in poverty has stayed constant.

2. Much of the "economic growth" involves people converting from a non-money economy to a money economy.

This one sounds right to me. Nobody really "lives on" $1.90/day in a world where they must meet all their needs by spending money. Extreme poverty means that most of their consumption is not measured monetarily--they have access to commons (land/resources) and they do a lot of stuff themselves, and share and barter. {see footnote}

Oliver Wiseman counters:

1. Going from extremely poor to very poor is still progress. Also, at $7.40 or $10.00, the proportion in poverty has still decreased. The absolute number doesn't matter as much given how much population has increased.

I basically agree with this, but it paints a much more nuanced picture of gradual and perhaps fragile progress.

2. Measures of quality of life have improved dramatically along with gross world product. (i.e. being a medieval peasant sucked)

This is basically true post-industrial-revolution, as far as I can tell, at least in Europe. The data seems to be mixed on whether most people were better off before the advent of agriculture than they were in 1700. Measures of GDP in places and times without money economies are always questionable, so the comparisons get tricky.
But, it doesn't really address Hinckel's point regarding the current trend. If the people going from extreme poverty in 1970 to moderate poverty in 2015 have also gone from mainly subsistence farming and barter to mainly subsistence wages, then increased monetary consumption doesn't necessarily correspond to increased quality of life. I don't have the data on this, but it seems like a key question.

{Footnote: I sometimes try to imagine myself surviving for a year on $700, and it looks like a world where income is not a good measure of consumption. That money would all basically have to go to food just to keep me alive... I'd have n...
Security amplification
by paulfchristiano — 2019-02-06T17:28:19.995Z

An apparently aligned AI system may nevertheless behave badly with small probability or on rare “bad” inputs. The reliability amplification problem is to reduce the failure probability of an aligned AI. The analogous security amplification problem is to reduce the prevalence of bad inputs on which the failure probability is unacceptably high.

We could measure the prevalence of bad inputs by looking at the probability that a random input is bad, but I think it is more meaningful to look at the difficulty of finding a bad input. If it is exponentially difficult to find a bad input, then in practice we won’t encounter any.

If we could transform a policy in a way that multiplicatively increase the difficulty of finding a bad input, then by interleaving that process with a distillation step like imitation or RL we could potentially train policies which are as secure as the learning algorithms themselves — eliminating any vulnerabilities introduced by the starting policy.

For sophisticated AI systems, I currently believe that meta-execution is a plausible approach to security amplification. (ETA: I still think that this basic approach to security amplification is plausible, but it’s now clear that meta-execution on its own can’t work.)

Motivation

There are many inputs on which any particular implementation of “human judgment” will behave surprisingly badly, whether because of trickery, threats, bugs in the UI used to elicit the judgment, snow-crash-style weirdness, or whatever else. (The experience of computer security suggests that complicated systems typically have many vulnerabilities, both on the human side and the machine side.) If we aggressively optimize something to earn high approval from a human, it seems likely that we will zoom in on the unreasonable part of the space and get an unintended result.

What’s worse, this flaw seems to be inherited by any agent trained to imitate human behavior or optimize human approval. For example, inputs which cause humans to behave badly would also cause a competent human-imitator to behave badly.

The point of security amplification is to remove these human-generated vulnerabilities. We can start with a human, use them to train a learning system (that inherits the human vulnerabilities), use security amplification to reduce these vulnerabilities, use the result to train a new learning system (that inherits the reduced set of v...
Alignment Newsletter #44
by Rohin Shah — 2019-02-06T08:30:01.424Z

Find all Alignment Newsletter resources here. In particular, you can sign up, or look through this spreadsheet of all summaries that have ever been in the newsletter.

Highlights

How does Gradient Descent Interact with Goodhart? (Scott Garrabrant): Scott often thinks about optimization using a simple proxy of "sample N points and choose the one with the highest value", where larger N corresponds to more powerful optimization. However, this seems to be a poor model for what gradient descent actually does, and it seems valuable to understand the difference (or to find out that there isn't any significant difference). A particularly interesting subquestion is whether Goodhart's Law behaves differently for gradient descent vs. random search.

Rohin's opinion: I don't think that the two methods are very different, and I expect that if you can control for "optimization power", the two methods would be about equally susceptible to Goodhart's Law. (In any given experiment, one will be better than the other, for reasons that depend on the experiment, but averaged across experiments I don't expect to see a clear winner.) However, I do think that gradient descent is very powerful at optimization, and it's hard to imagine the astronomically large random search that would compare with it, and so in any practical application gradient descent will lead to more Goodharting (and more overfitting) than random search. (It will also perform better, since it won't underfit, as random search would.)

One of the answers to this question talks about some experimental evidence, where they find that they can get different results with a relatively minor change to the experimental procedure, which I think is weak evidence for this position.

Transformer-XL: Unleashing the Potential of Attention Models (Zihang Dai, Zhilin Yang et al): Transformer architectures have become all the rage recently, showing better performance on many tasks compared to CNNs and RNNs. This post introduces Transformer-XL, an improvement on the Transformer architecture for very long sequences.

The key idea with the original Transformer architecture is to use self-attention layers to analyze sequences instead of something recurrent like an RNN, which has problems with vanishing and exploding gradients. An attention layer takes as input a query q and key-value pai...
If Rationality can be likened to a 'Martial Art', what would be the Forms?
by Bae's Theorem — 2019-02-06T05:48:33.589Z

In Martial Arts, we have 'forms' that allow us to practice our skills when not with a partner or in adversity.
In Rationality, I imagine that this would take the form of "brain teasers", but specifically regarding decision theory, overcoming biases, and calculating probabilities.

What are some things one can do to practice Applied Rationality? These can take the form of thought experiments, websites, apps, pen-and-paper tools, etc.

These would ideally range from "easy to do within a couple minutes" to "you need to dedicate a day to this".
Complexity Penalties in Statistical Learning
by michael_h — 2019-02-06T04:13:57.140Z

I am currently taking a course on statistical learning at the Australian Mathematical Sciences Institute Summer School. One idea that has appeared many times in the course is that a more complicated model is likely to have many short comings. This is because complicated models tend to overfit the observed data. They often give explanatory value to parts of the observation that are simply random noise.

This is common knowledge for many aspiring rationalists. The term complexity penalty is used to describe the act of putting less credence in complicated explanations because they are more complex. In this blog post I aim to provide a brief introduction to statistical learning and use an example to demonstrate how complexity penalties arise in this setting.

Statistical Learning

Broadly speaking, statistical learning is the process of using data to select a model and then using the model to make predictions about future data. So, in order to perform statistical learning, we need at least three things. We need some data, a class of models and a way of measuring how well a model predicts the future data. In this blog we will look at the problem of polynomial regression.

The Data

For polynomial regression, our data is in the form of n pairs of real numbers(x1,y1),(x2,y2),…,(xn,yn). Our goal is to find the relationship between the input values xi and the output values yi and then use this to predict future outputs given new inputs. For example, the input values could represent the average temperature of a particular day and the corresponding output value could be the number of ice creams sold that day. Going with this example, we can suppose our data looks something like this:

To simplify our analysis we will make some assumptions about the relationship between our the inputs and outputs. We will assume that there exists an unknown function g∗ such that y=g∗(x)+E, where E is a statistical error term with mean equal to 0 and variance equal to σ. This assumption is essentially saying that there is some true relationship between our input x and our output y but that the output can fluctuate around this true value. Furthermore we are assuming that these fluctuations are balanced in the positive and negative direction (since the mean of E is zero) and the size of these fluctuations doesn't depend on the input x (since the variance of E is constant)...