trentmkelly/LessWrong-43k huggingface dataset:
UML VI: Stochastic Gradient Descent
by Rafael Harth — 2020-01-12T21:59:25.606Z

(This is part six in a sequence on Machine Learning based on this book. Click here for part 1.)

Stochastic Gradient Descent is the big Machine Learning technique. It performs well in practice, it's simple, and it sounds super cool. But what is it?

Roughly, Gradient Descent is an approach for function minimization, but it generally isn't possible to apply it to Machine Learning tasks. Stochastic Gradient Descent, which is Gradient Descent plus noise, is a variant which can be used instead.

Let's begin with regular Gradient Descent.

Gradient Descent

Given a function f:Rd→R and a point x∈Rd, the gradient of f at x is the vector of partial derivatives of f at x, i.e.

∇f(x)=(∂f∂x1(x),...,∂f∂xd(x))

So the gradient at a point is an element of Rd. In contrast, the gradient itself (not yet applied to a point) is the function ∇f:Rd→Rd defined by the above rule. Going forward, gradient always means "gradient at a point".

If d=1, then the gradient will be f′(x), a number in R. It can look like this:

→

Or like this:

←

Or also like this:

⟶

Point being, if f′(x)>0 it'll point rightward and if f′(x)<0 it'll point leftward, also it can have different lengths, but that's it. The idea of gradient descent is that the gradient points into the direction of fastest positive change, thus the opposite of the gradient is the direction of fastest negative change. It follows that, to minimize a function, one can start anywhere, compute the gradient, and then move into the opposite direction.

In the above example, the function goes up, therefore the derivative is positive, therefore the gradient points rightward. Gradient Descent tells us to go into the opposite direction, i.e. leftwards. Indeed, leftward is where the function decreases. Clearly, this is a display of utter brilliance.

Importantly, note that the picture is misleading insofar as it suggests there are more directions than two. But actually, the gradient lives in the domain space, in this case, R, not in the Cartesian product of domain space and target space, in this case, R2. Therefore, it cannot point upward or downward.

As silly as the one-dimensional case is, it quickly becomes less trivial as we increase the dimension. Consider this function (picture taken from Wikipedia):

Aga...
Update on Ought's experiments on factored evaluation of arguments
by Owain_Evans — 2020-01-12T21:20:42.317Z
Link: https://ought.org/updates/2020-01-11-arguments

Ought has written a detailed update and analysis of recent experiments on factored cognition. These are experiments with human participants and don’t involve any machine learning. The goal is to learn about the viability of IDA, Debate, and related approaches to AI alignment. For background, here are some prior LW posts on Ought: Ought: Why it Matters and How to Help, Factored Cognition presentation.

Here is the opening of the research update:

Evaluating Arguments One Step at a Time

We’re studying factored cognition: under what conditions can a group of people accomplish complex cognitive tasks if each person only has minimal context?

In a recent experiment, we focused on dividing up the task of evaluating arguments. We created short, structured arguments for claims about movie reviews. We then tried to distinguish valid from invalid arguments by showing each participant only one step of the argument, not the review or the other steps.

In this experiment, we found that:

1. Factored evaluation of arguments can distinguish some valid from invalid arguments by identifying implausible steps in arguments for false claims.

2. However, experiment participants disagreed a lot about whether steps were valid or invalid. This method is therefore brittle in its current form, even for arguments which only have 1–5 steps.

3. More diverse argument and evidence types (besides direct quotes from the text), larger trees, and different participant guidelines should improve results.

In this technical progress update, we describe these findings in depth.

The rest of the post is here.
How would we check if "Mathematicians are generally more Law Abiding?"
by Raemon — 2020-01-12T20:23:05.479Z

In Local Validity, Eliezer notes:

But I would venture a guess and hypothesis that you are better off buying a used car from a random mathematician than a random non-mathematician, even after controlling for IQ. The reasoning being that mathematicians are people whose sense of Law was strong enough to be appropriated for proofs, and that this will correlate, if imperfectly, with mathematicians abiding by what they see as The Law in other places as well. I could be wrong, and would be interested in seeing the results of any study like this if it were ever done. (But no studies on self-reports of criminal behavior, please. Unless there's some reason to believe that the self-report metric isn't measuring "honesty times criminality" rather than "criminality".)

I'm guessing such a study hasn't been done, but it seems like the sort of thing you should be able to actually go and check.

I'm interested in both:

What is the expensive, impractical study you could hypothetically run that would give strong evidence about this question?
What's the cheapest, practical study we could run that would provide at least some meaningful data about it?
What long term good futures are possible. (Other than FAI)?
by Donald Hobson — 2020-01-12T18:04:52.803Z

Does anyone know any potential long term futures that are good, and do not involve the creation of a friendly super-intelligence.

To be clear, long term future means billion years+. In most of my world models, we settle into a state from which it is much easier to predict the future within the next few hundred years. (Ie a state where it seems unlikely that anything much will change)

By good, I mean any future that you would prefer to be in if you cared only about yourself, or would be replaced with a robot that would do just as much good here. A weaker condition would be any future that you would want not to be erased from existence.

A superintelligent agent running around doing whatever is friendly or moral or whatever would meet these criteria, I am excluding it because I already know about that possibility. Your futures may contain Superintelligences that aren't fully friendly. A superintelligence that acts as a ZFC oracle is fine.

Your potential future doesn't have to be particularly likely, just remotely plausible. You may assume that a random 1% of humanity reads your reply and goes out of their way to make that future happen. Ie people optimizing for this goal can use strategies of the form "someone does X" but not "everyone does X". You can get "a majority of humans does X" if X is easy to do and explain and most people have no strong reason not to X.

You should make clear what stops somebody making a UFAI which goes on to destroy the world. (Eg paperclip maximizer)

What stops Moloch, what stops us trashing away everything of value in to win competitions? (Hansons Hardscrabble frontier replicators.)
Malign generalization without internal search
by Matthew Barnett — 2020-01-12T18:03:43.042Z

In my last post, I challenged the idea that inner alignment failures should be explained by appealing to agents which perform explicit internal search. By doing so, I argued that we should instead appeal to the more general concept of malign generalization, and treat mesa-misalignment as a special case.

Unfortunately, the post was light on examples of what we should be worrying about instead of mesa-misalignment. Evan Hubinger wrote,

Personally, I think there is a meaningful sense in which all the models I'm most worried about do some sort of search internally (at least to the same extent that humans do search internally), but I'm definitely uncertain about that.

Wei Dai expressed confusion why I would want to retreat to malign generalization without some sort of concrete failure mode in mind,

Can you give some realistic examples/scenarios of “malign generalization” that does not involve mesa optimization? I’m not sure what kind of thing you’re actually worried about here.

In this post, I will outline a general category of agents which may exhibit malign generalization without internal search, and then will provide a concrete example of an agent in the category. Then I will argue that, rather than being a very narrow counterexample, this class of agents could be competitive with search-based agents.

The switch case agent

Consider an agent governed by the following general behavior,

LOOP:State = GetStateOfWorld(Observation)IF State == 1:PerformActionSequence1()IF State == 2:PerformActionSequence2()...END_LOOP

It's clear that this agent does not perform any internal search for strategies: it doesn't operate by choosing actions which rank highly according to some sort of internal objective function. While you could potentially rationalize its behavior according to some observed-utility function, this would generally lead to more confusion than clarity.

However, this agent could still be malign in the following way. Suppose the agent is 'mistaken' about the state of the world. Say that it believes that the state of the world is 1, whereas the actual state of the world is 2. Then it could take the wrong action, almost like a person who is confident in a falsehood and makes catastrophic mistakes because of their error.

To see how this could manifest as bad behavior in our artificial agents, I will use a motivating...
How to Identify an Immoral Maze
by Zvi — 2020-01-12T12:10:00.766Z

Previously in sequence: Moloch Hasn’t Won, Perfect Competition, Imperfect Competition, Does Big Business Hate Your Family?, What is Life in an Immoral Maze?, Stripping Away the Protections, What is Success in an Immoral Maze?

Immoral mazes (hereafter mazes), as laid out in the book Moral Mazes, are toxic organizations. Working for them puts tremendous pressure on you to prioritize getting ahead in the organization over everything else. Middle managers are particularly affected – they are pushed to sacrifice not only all of their time, but also things such as their morality, family and ability to think clearly. Only those who go all-in doing this get ahead, and even most of them fail.

Even successfully getting ahead is little consolation.

Mazes exert similar pressures on those who do business with them or work in non-managerial roles, to a lesser but substantial degree.

The best defense is to identify mazes before you agree to work for or do business with them, and choose to work or do business elsewhere. At a minimum, one’s eyes should be open, and the costs involved must be fully factored in before making such decisions.

This makes it important to figure out what parts of what organizations are mazes, and to what extent. This is hard to get exactly right.

What is easier is using simple heuristics to get a good approximation, then keeping an eye out for and updating on new evidence.

I offer seven heuristics, the first two of which will do the bulk of the work on their own. You benefit from the ‘right’ answer to all of them even absent concerns about mazes, so they are good questions to get into the habit of asking.

1. How many levels of hierarchy exist?

Full mazes require at least three levels of hierarchy, without which one cannot have middle management.

Each level beyond that makes things worse. The fourth and fifth levels both make things much worse.

With only one level, there’s nothing to worry about.

With only two levels, a boss and those who report to the boss, the boss has skin in the game, no boss causing problems for them, and not enough reason to reward bad outcomes.

With three levels, there are middle managers in the second layer, so one should be wary. But things are unlikely to be too bad. No middle manager has a boss or underling who is also a middle manager. This means that in any interaction between non-equals

...
Key Decision Analysis - a fundamental rationality technique
by Eli Tyre — 2020-01-12T05:59:57.704Z

The technique

This post is signal-boosting and recommending a strategy for improving your decision making that I picked up from the entrepreneur Ivan Mazour. He describes the process here, and publishes his own results every year on his blog.

In his words...

I believe that life is far too fast-paced for us to be able to make rational, carefully thought-out, decisions all the time. This completely contradicts my mathematical upbringing and training, but is something I have come to realise throughout my twenties. We need a way of keeping up with the constant barrage of decisions, even when the inevitable ‘decision fatigue’ sets in. The only way to do this, I find, is to act on instinct, but this only works if your instincts are correct. They cannot be correct all the time, of course, but if we can maximise the chance of making the right decision by instinct, then we have a strategy for coping with a complicated and highly productive life.

To sharpen my instincts, I keep a monthly journal of all key decisions which I make – decisions that could be truly life changing – and my instinctive reasons for why I made them. I go back only after exactly a year has passed, and I note down whether the decision was correct, and more importantly whether my instincts were right. At the end of the year, I go over all twelve months worth of notes, and search for any patterns amongst all of the right and wrong choices.

This is not a short-term strategy, as you can tell. In fact it takes exactly two years from the day you start following it, to the time that you can get some useful insights to sharpen your instincts. Keeping a diary of decisions has other uses, and there are many ways of getting an overview of your life prior to this, but it is only after the two years have passed that a genuine clear pattern presents itself.

Some theory

This accords with some abstract theory about human rationality. A perfect-Bayesian expected utility maximizer doesn't start out with an optimal policy. Rather, its strength is being able to learn from its experience (optimally), so that it converges towards the optimal policy.

Of course, humans have a number of limitations standing between us and perfect-decision-making-in-the-limit. Due to computational constraints, perfect Bayesian updating is out of reach. But among a number of limitations, the first a...
Using Vickrey auctions as a price discovery mechanism
by epiphi — 2020-01-12T01:00:10.181Z
Link: https://kevinlynagh.com/notes/pricing-niche-products/

I recommend "Pricing niche products: Why sell a mechanical keyboard kit for $1,668?" for providing a practical case study in price dynamics that helped with my economic intuitions.

The author's friend had created a new custom keyboard kit. Their friend's previous kit had sold out in minutes, so clearly something was amiss with their "estimate costs and premiums and then set a price" approach:

I’m not a fan of this inside-out approach, for several reasons:

factors like “brand premium” are inherently subjective — the temptation to compare to others limits potential upside and differentiation

picking a (new, higher) price may have reputational downsides (because o_f course_ your customers spend all day in mechanical keyboard chat rooms and may gripe about you “selling out the community”)

you will second guess yourself regardless of the outcome; either you sell out again (goto 0) or you sell too few and then must live with the shame of having $20k worth of unsold keyboard in your garage

The most compelling argument against simply picking a price, though, is that it limits how much you can learn about your market.

Instead, they run a Vickrey auction (or "second-price sealed-bid auction") and find that the demand curve supports 3x the list price they would have chosen:

I can’t overstate the benefits of knowing the demand curve.

In my friend’s case, the auction let them sell far above their initial price and revealed that the market was deep enough to justify a larger production run.

(I discovered this post via The Prepared, a newsletter that I'd strongly recommend.)
Is it worthwhile to save the cord blood and tissue?
by Alexei — 2020-01-11T21:52:24.256Z

My wife and I are expecting a baby this May. I only recently learned about this, but apparently there is an option to save the cord blood and tissue during the delivery. This seems potentially very useful, so I'm wondering if anyone has done research into this or has found trustworthy, informative resources on this topic.

My current understanding is that this tissue has a lot of stem cells. And the stem cells are useful for all sorts of medical procedures, and are even more likely to be used in the future. But currently we don't have an easy (or cheap?) way to get them.

Broad questions I have:

How valuable are the stem cells right now and how valuable are they expected to be in the future?
How hard is it to get stem cells for yourself / your child right now vs in the future?
Will the collected stem cells be only useful for the baby or the mother too?
Can we reasonably expect the cryo companies to last long enough and not go under?
Have you had experience donating it?
Have you had experience storing it?
Please Critique Things for the Review!
by Raemon — 2020-01-11T20:59:49.312Z

I’ve spent a

lot of time defending LW authors’ right to have the conversation they want to have

, whether that be early stage brainstorming, developing a high context idea, or just randomly wanting to focus on some particular thing.

LessWrong is not only a place for finished, flawless works. Good intellectual output requires both

Babble

and

Prune

, and in my experience the best thinkers often require idiosyncratic environments in order to produce and refine important insights. LessWrong is a full-stack intellectual pipeline.

But the 2018 Review is supposed to be late stage in that pipeline. We’re pruning, not babbling here, and criticism is quite welcome. We’re deliberately offering just as much potential prize money ($2000) to reviewers as to the top-rated authors.

Nominated authors had the opportunity to opt out of the review process, and none of them did. Getting nominated is meant to feel something like “getting invited to the grown-ups table”, where your ideas are subjected to serious evaluation, and that scrutiny is seen as a sign of respect.

In my current expectations, the Review is one of the primary ways that LessWrong ensures high epistemic standards. But how well that plan works is proportional to how much effort critics put into it.

The Review and Voting Phases will continue for another until January 19th. During that time, review-comments will appear on the voting page, so anyone considering how to vote on a given post will have the opportunity to see critiques. The reviews will appear abridged initially, so I’d aim for the first couple sentences to communicate your overall takeaway.

The Review norms aren’t “literally anything goes” – ridicule, name-calling etc still aren’t appropriate. I’d describe the intended norms for reviews as “professional”. But, posts nominated for the Review should treated as something like “the usual frontpage norms, but with a heavier emphasis on serious evaluation.”

I’m still not sure precisely what the rules/guidelines should be about what is acceptable for the final Best of 2018 Book. In some cases, a post might make some important points, but also make some unjustified claims. (I personally think Local Validity as Key to Sanity and Civilization falls in this category). My current best guess is that it’d be fine if such posts end up in the book, but I’d want to make sure to also include reviews that h

...
Moral uncertainty vs related concepts
by MichaelA — 2020-01-11T10:03:17.592Z

Overview

How important is the well‐being of non‐human animals compared with the well‐being of humans?

How much should we spend on helping strangers in need?

How much should we care about future generations?

How should we weigh reasons of autonomy and respect against reasons of benevolence?

Few could honestly say that they are fully certain about the answers to these pressing moral questions. Part of the reason we feel less than fully certain about the answers has to do with uncertainty about empirical facts. We are uncertain about whether fish can feel pain, whether we can really help strangers far away, or what we could do for people in the far future. However, sometimes, the uncertainty is fundamentally moral. [...] Even if were to come to know all the relevant non‐normative facts, we could still waver about whether it is right to kill an animal for a very small benefit for a human, whether we have strong duties to help strangers in need, and whether future people matter as much as current ones. Fundamental moral uncertainty can also be more general as when we are uncertain about whether a certain moral theory is correct. (Bykvist; emphasis added)[1]

I consider the above quote a great starting point for understanding what moral uncertainty is; it gives clear examples of moral uncertainties, and contrasts these with related empirical uncertainties. From what I’ve seen, a lot of academic work on moral uncertainty essentially opens with something like the above, then notes that the rational approach to decision-making under empirical uncertainty is typically considered to be expected utility theory, then discusses various approaches for decision-making under moral uncertainty.

That’s fair enough, as no one article can cover everything, but it also leaves open some major questions about what moral uncertainty actually is.[2] These include:

How, more precisely, can we draw lines between moral and empirical uncertainty?

What are the overlaps and distinctions between moral uncertainty and other related concepts, such as normative, metanormative, decision-theoretic, and metaethical uncertainty, as well as value pluralism?

My prior post answers similar questions about how morality overlaps with and differs from related concepts, and may be worth reading before this one.

Is what we “ought to do” under moral uncertainty an objective or subjective mat

...