trentmkelly/LessWrong-43k huggingface dataset:
A rant against robots
by Lê Nguyên Hoang — 2020-01-14T22:03:19.668Z

What comes to your mind when you hear the word "artificial intelligence" (or "artificial general intelligence")? And if you want to prepare the future, what should come to your mind?

It seems that when most people hear AI, they think of robots. Weirdly, this observation includes both laymen and some top academics. Stuart Russell's book (which I greatly enjoyed) is such an example. It often presents robots as an example of an AI.

But this seems problematic to me. I believe that we should dissociate a lot more AIs from robots. In fact, given that most people will nevertheless think of robots when we discuss AIs, we might even want to use the terminology algorithms rather AIs. And perhaps algorithms with superhuman-level world model and planning capabilities instead of AGIs...

To defend this claim, I shall argue that the most critical aspects of today's and tomorrow's world-scale ethical problems (including x-risks) have and will have to do with algorithms; not robots. Moreover, and most importantly, the example of robots raises both concerns and solutions that seem in fact irrelevant to algorithms. Finally, I'll conclude by arguing that the example of today's large-scale algorithms is actually useful, because it motivates AI alignment.

It's about algorithms, not robots!

AIs that matter are algorithms

Today's AI research is mostly driven by non-robotic applications, from natural language processing to image analysis, from protein folding to query answering, from autocompletion to video recommendation. This is where the money is. Google is investing (hundreds of?) millions of dollars in improving its search engine and YouTube's recommendation system. Not in building robots.

Today's ranking, content moderation and automated labeling algorithms are arguably a lot more influential than robots. YouTube's algorithms have arguably become the biggest opinion-maker worldwide. They present risks and opportunities on a massive scale.

And it seems that there is an important probability that tomorrow's most influential algorithms will be somewhat similar, even if they achieve artificial general intelligence. Such algorithms will likely be dematerialized, on the cloud, with numerous copies of themselves stored in several data centres and terminals throughout the world.

And they will be extremely powerful. Not...
Is backwards causation necessarily absurd?
by Chris_Leong — 2020-01-14T19:25:44.419Z

In Newcomb's problem an agent picks either one-box or two-box and finds that no matter which option they picked, a predictor predicted them in advance. I've gone to a lot of effort to explain how this can be without requiring backwards causation (The Prediction Problem, Deconfusing Logical Counterfactuals), yet now I find myself wondering if backwards causation is such a bad explanation after all.

Unfortunately I'm not a physicist, so take what I say with a grain of salt, but I'll sketch out some reasons why backwards causation might not be as ridiculous as it first seems and hopefully someone else develops this in more detail.

One prominent theory of time is Eternalism in which there is no objective flow of time and terms like "past", "present" and "future" can only be used in a relative sense. An argument in favour of this is that it is often very convenient in physics to model space-time as a 4-dimensional space. If time is just another dimension, why should the future be treated differently than the past? Nothing in this model differentiates the two. If we have two blocks X and Y next to each other, we can view either X as the left one or Y as the left one depending on the direction we look at it from. Similarly, if A causes B in the traditional forwards sense, why can't we symmetrically view B as backwards causing A, where again if we viewed it another way A to B would be backwards causation and B to A would be forwards causation.

Another relativistic argument against time flowing is that simultaneity is only defined relative to a reference frame. Therefore, there is no unified present which is supposed to be what is flowing. This doesn't mean that the universe couldn't be described by a unidirectional graph. However, it does greatly undermine any trust in our naive intutions related to time.

Thirdly, entropy has often been the arrow of time with other physical laws claimed to be reversible. We are in a low-entropy world so entropy increases. However, if we were in a high-entropy world, it would decrease, so time and causation would seem to be going backwards (from our perspective). This would seem to suggest that backwards causation is just as valid a phenomenon as backward causation.

I want finish by reminding readers again that I am not a physicist. This post is more intended to spark discussion that anything else.

(Another possibility I haven't discussed ...
Predictors exist: CDT going bonkers... forever
by Stuart_Armstrong — 2020-01-14T16:19:13.256Z

I've been wanting to get a better example of CDT (causal decision theory) misbehaving, where the behaviour is more clearly suboptimal than it is in the Newcomb problem (which many people don't seem to accept as CDT being suboptimal), and simpler to grasp than Death in Damascus.

The "predictors exist" problem

So consider this simple example: the player is playing against Omega, who will predict their actions[1]. The player can take three actions: "zero", "one", or "leave".

If ever they do "leave", then the experiment is over and they leave. If they choose "zero" or "one", then Omega will predict their action, and compare this to their actual action. If the two match, then the player loses 1 utility and the game repeats; if the action and the prediction differs, then the player gains 3 utility and the experiment ends.

Assume that actually Omega is a perfect or quasi-perfect predictor, with a good model of the player. An FDT or EDT agent would soon realise that they couldn't trick Omega, after a few tries, and would quickly end the game.

But the CDT player would be incapable of reaching this reasoning. Whatever distribution they compute over Omega's prediction, they will always estimate that they (the CDT player) have at least a 50% chance of choosing the other option[2], for an expected utility gain of at least 0.5(3)+0.5(−1)=1.

Basically, the CDT agent can never learn that Omega is a good predictor of themselves[3]. And so they will continue playing, and continue losing... for ever.

Omega will make this prediction not necessarily before the player takes their action, not even necessarily without seeing this action, but still makes the prediction independently of this knowledge. And that's enough for CDT. ↩︎

For example, suppose the CDT agent estimates the prediction will be "zero" with probability p, and "one" with probability 1-p. Then if p≥1/2, they can say "one", and have a probability p≥1/2 of winning, in their own view. If p<1/2, they can say "zero", and have a subjective probability 1−p>1/2 of winning. ↩︎

The CDT agent has no problem believing that Omega is a perfect predictor of other agents, however. ↩︎
Red Flags for Rationalization
by dspeyer — 2020-01-14T07:34:42.821Z

Previously: Why a New Rationalization Sequence?

What are Red Flags?

A red flag is a warning sign that you may have rationalized. Something that is practical to observe and more likely in the rationalization case than the non-rationalization.

Some are things which are likely to cause rationalization. Others are likely to be caused by it. One on this list is even based on common cause. (I don't have any based on selection on a common effect, but in theory there could be.)

How to Use Red Flags

Seeing a red flag doesn't necessarily mean that you have rationalized, but it's evidence. Likewise, just because you've rationalized doesn't mean your conclusion is wrong, only that it's not as supported as you thought.

So when one of these flags raises, don't give up on ever discovering truth; don't stop-halt-catch-fire; definitely don't invert your conclusion.

Just slow down. Take the hypothesis that you're rationalizing seriously and look for ways to test it. The rest of this sequence will offer tools for the purpose, but just paying attention is half the battle.

A lot of these things can be present to a greater or lesser degree, so you'll want to set thresholds. I'd guess an optimal setting has about 1/3 of triggers be true. High enough that you keep doing your checks seriously, but low because the payoff matrix is quite asymmetrical.

Basically use these as trigger-action planning. Trigger: anything on this list. Action: spend five seconds doing your agenty best to worry about rationalization.

Conflict of Interest

This is a classic reason to distrust someone else's reasoning. If they have something to gain from you believing a conclusion separate from that conclusion being true, you have reason to be suspicious. But what does it mean for you to gain from you believing something apart from it being true?

Not Such Great Liars

Probably the simplest reason is that you need to deceive someone else. If you're not a practiced liar, the easiest way to do this is to deceive yourself.

Simple example: you're running late and need to give an estimate of when you'll arrive. If you say "ten minutes late" and arrive twenty minutes late, it looks like you hit another ten minutes' worth of bad luck, whereas saying "twenty minutes" looks like your fault. You're not good at s...
Repossessing Degrees
by jefftk — 2020-01-13T21:40:01.884Z

Several months ago I

argued that
we should allow student loans to be discharged through bankruptcy.
Yesterday I

realized
another way to modify student loans to be more compatible with
bankruptcy: allow lenders to effectively repossess degrees. This could
make having student loans not survive bankruptcy more politically
practical.

Sometimes people bring
up
the idea of repossessing degrees as a way of illustrating the
absurdity of student loans, but it's actually pretty reasonable. A
large majority of the benefit of getting a degree comes from having the
credential as opposed to having learned skills: employers use "do
they have a college degree" as a filter. If you took out a student
loan to get a degree, and then later declared bankruptcy, the court
could require you to no longer represent yourself as having a the
degree as a condition of having your debt discharged.

I think the goal is that someone with a repossessed degree should look
the same as someone who completed part of the degree but dropped out
before finishing. Colleges would need to be part of this as well; one
way for this to work is if as a condition of continued participation
in the student loan system colleges would need to agree to revoke
degrees if asked to do so by the bankruptcy court.

One way this could work poorly would be if people would just ignore
the law and tell employers in person that they have a revoked degree?
I think they mostly wouldn't: degrees are often used at an early stage
of hiring screening, where if you don't have a degree you don't even
get to the stage of being able to talk to an interviewer and explain
the situation. Additionally, someone who decides to claim they went
to Harvard after having their degree repossessed looks just like
someone who's lying about having gone to college, or who dropped out.

This would probably need to be combined with some sort of waiting
period, something like five years, to avoid the case where someone
declares bankruptcy before finishing their degree when there's nothing
to repossess, and then completes their degree later.

Implementing this in a way compatible with the first amendment seems
tricky but doable. In a standard non-disclosure
agreement you trade your right to share some information for
something else you value more. That's what we're talking about here,
so it seems like we're ok? But I'm just ...
What is the relationship between Preference Learning and Value Learning?
by Riccardo Volpato — 2020-01-13T21:08:40.334Z

It appears that in the last few years the AI Alignment community has dedicated great attention to the Value Learning Problem [1]. In particular, the work of Stuart Armstrong stands out to me.

Concurrently, during the last decade, researcher such as Eyke Hüllermeier Johannes Fürnkranz produced a significant amount of work on the topics of preference learning [2] and preference-based reinforcement learning [3].

While I am not highly familiar with the Value Learning literature, I consider the two fields closely related if not overlapping, but I have not often seen references the Preference Learning work, and vice-versa.

Is this because the two fields are less related than what I think? And more specifically, how do the two fields relate with each other?

References

[1] - Soares, Nate. "The value learning problem." Machine Intelligence Research Institute, Berkley (2015).

[2] - Fürnkranz, Johannes, and Eyke Hüllermeier. Preference learning. Springer US, 2010.

[3] - Fürnkranz, Johannes, et al. "Preference-based reinforcement learning: a formal framework and a policy iteration algorithm." Machine learning 89.1-2 (2012): 123-156.
Moral uncertainty: What kind of 'should' is involved?
by MichaelA — 2020-01-13T12:13:11.565Z

This post follows on from my prior post; consider reading that post first.

We are often forced to make decisions under conditions of uncertainty. This may be empirical uncertainty (e.g., what is the likelihood that nuclear war would cause human extinction?), or it may be moral uncertainty (e.g., does the wellbeing of future generations matter morally?).

In my prior post, I discussed overlaps with and distinctions between moral uncertainty and related concepts. In this post, I continue my attempt to clarify what moral uncertainty actually is (rather than how to make decisions when morally uncertain, which is covered later in the sequence). Specifically, here I’ll discuss:

Is what we “ought to do” (or “should do”) under moral uncertainty an objective or subjective (i.e., belief-relative) matter?

Is what we “ought to do” (or “should do”) under moral uncertainty a matter of rationality or morality?

An important aim will be simply clarifying the questions and terms themselves. That said, to foreshadow, the tentative “answers” I’ll arrive at are:

It seems both more intuitive and more action-guiding to say that the “ought” is subjective.

Whether the “ought” is a rational or a moral one may be a “merely verbal” dispute with no practical significance. But I’m very confident that interpreting the “ought” as a matter of rationality works in any case (i.e., whether or not interpreting it as a matter of morality does, and whether or not the distinction really matters).

This post doesn’t explicitly address what types of moral uncertainty would be meaningful for moral antirealists and/or subjectivists; I discuss that topic in a separate post.[1]

Epistemic status: The concepts covered here are broad, fuzzy, and overlap in various ways, making definitions and distinctions between them almost inevitably debatable. Additionally, I’m not an expert in these topics (though I have now spent a couple weeks mostly reading about them). I’ve tried to mostly collect, summarise, and synthesise existing ideas (from academic philosophy and the LessWrong and EA communities). I’d appreciate feedback or comments in relation to any mistakes, unclear phrasings, etc. (and just in general!).

Objective or subjective?

(Note: What I discuss here is not the same as the objectivism vs subjectivism debate in metaethics.)

As I noted in a prior post:

Subjective normativity relates to what one should do

...
Why a New Rationalization Sequence?
by dspeyer — 2020-01-13T06:46:33.813Z

This is the first in a five-post mini-sequence about rationalization, which I intend to post one-per-day. And you may ask, why should we have such a sequence?

What is Rationalization and Why is it Bad?

For those of you just tuning in, rationalization is when you take a conclusion you want to reach and try to come up with a argument that concludes it. The argument looks very similar to one in which you started from data, evaluated as well as you could, and reached this conclusion naturally. Almost always similar enough to fool the casual observer, and often similar enough to fool yourself.

If you're deliberately rationalizing for an outside audience, that's out-of-scope for this sequence. All the usual ethics and game theory apply.

But if you're involuntarily rationalizing and fooling yourself, then you've failed at epistemics. And your arts have turned against you. Know a lot about scientific failures? Now you can find them in all the studies you didn't like!

Didn't Eliezer Already Do This?

Eliezer wrote the against rationalization sequence back in 2007/8. If you haven't read it, you probably should. It does a good job of describing what rationalization is, how it can happen, and how bad it can be. It does not provide a lot of tools for you to use in protecting yourself from rationalization. That's what I'll be focusing on here.

And, besides, if we don't revisit a topic this important every decade or so with new developments, then what is this community for?

Is There Hope?

Periodically, I hear someone give up on logical argument completely. "You can find an argument for anything," they say, "Forget logic. Trust [ your gut / tradition / me ] instead." Which brushes over the question of whether the proposed alternative is any better. There is no royal road to knowledge.

Still, the question needs answering. If rationalization looks just like logic, can we ever escape Cartesian Doubt?

The Psychiatrist Paradox

A common delusion among grandiose schizophrenics in institutions is that they are themselves psychiatrists. Consider a particularly underfunded mental hospital, in which the majority of people who "know" themselves to be psychiatrists are wrong. No examination of the evidence will convince them otherwise. No matter how overwhelming, some reason to disbelieve will be found...
Drawing Toward Power
by hiblick — 2020-01-13T04:24:23.960Z

At the Bay Area pre-solstice unconference, I gave a talk which started as a discussion of how rationalists could build tools to enable better organization and conversation online. I'm a largely improvisational speaker though, and it quickly turned into a discussion of the future of the movement.

I'm pretty old now, and I've seen a number of groups of people move from relative obscurity, to positions of power or, at least, to being subjects of general curiosity. The most relevant, I think, for rationalists are the early (pre-2002) Googlers.

If a social group gets projected onto the broader canvas of mass attention, or their interests get magnified through access to the levers of power, you get criticism and praise in varied amounts (Paul Graham has just written a piece about this). But just as visibly, the tiniest omissions and imperfections of the analysis and attitudes of the founders are also magnified.

I gave a talk in, I think, 2009, about this at a Foo Camp. I warned in an environment that was ready to hear it, but not capable of changing matters much, that geeks were about to become dangerous: that we had a set of Tragic Flaws, that we were already seeing magnified in the wider world.

I don't remember all of the characteristics of geekdom I gave then that would lead to its downfall, but a couple of them stuck with me: that we were bending the workplace into our own vision of a pleasant place to be (unbounded by the 9-5, contract-driven, full of interest and unboring, self-driven, university-like), and that meant that we were work-focused, and pushing others to be (even when their lives could not be bent that way). We experienced burn-out as a part of our lives, and now we were driving others into burn-out. We viewed efficiency as a life goal for ourselves, individually, and that was how we were encouraging others to live. We were strongly influenced by our alienation from others as young people, and that meant (paradoxically) that when we did assume power, we would not recognise it, and instead continued to use the habits and attitudes of outsiders with no power. (I called this "Stalinism": Stalin's paranoia and cruelty may have come from being the weak, minority group experiencing cruelty from the more powerful. I'm not sure if that is truly the case, but perhaps we could give as a better example Bill Gates, who fo...
How do you do hyperparameter searches in ML?
by lsusr — 2020-01-13T03:45:46.837Z

I know how to do hyperparameter searches. ☺

This is a survey. I want to know is how you do hyperparameter searches. It doesn't matter whether your system is good or bad. I won't judge you. I just want to know what systems other people are using in the real world right now.

Any information you're willing to share would help me out here, but there are two questions I'm especially interested in.

What algorithm do you use? (Do you use random search, grid search or Bayes search? Do you do some iterative process? Do you do something else entirely?)

Do you cache anything? If so, what's your process?

I'm also curious what industry you're in, but if you're not comfortable sharing that some information is better than none.
How has the cost of clothing insulation changed since 1970 in the USA?
by DanielFilan — 2020-01-12T23:31:56.430Z

I sometimes hear the claim that innovation in the physical world has stagnated since around 1970. More specifically, chapter 1 of The Rise and Fall of American Growth by Robert J Gordon claims that there has been basically no innovation in clothing other than changes in fashion. This is somewhat contrary to my intuition (although I definitely believe that innovation in the 50 years before 1970 was greater that in the 50 years after), and price of insulation seems like a relatively objective metric for this.

My favourite type of response would be time series data of clo per inflation-adjusted dollar, but I'd also appreciate people's subjective experience of this.