trentmkelly/LessWrong-43k huggingface dataset:
The master skill of matching map and territory
by Rafael Harth — 2018-03-27T12:06:53.377Z

Robin Hanson tells us to have fewer opinions on things, to specialize and be agnostic about everything outside your field of expertise. This may be good advice, but most of us won't take it. We're too obsessed with being right about stuff, including things we didn't study. Whether or not there is an external reason why being correct about something is useful, we usually care either way.

Most of our beliefs are based on what other people say, so the key skill seems obvious: identifying which people's views to consider strong evidence. Mastering that means being on par with the greatest experts in every field – not in understanding of the field itself, but in accuracy of views on controversial topics.

Why is there so little talk about this? Maybe because it's controversial, or it could be a status thing. I know I'd be uncomfortable with one entry if I had to share a short list of people who have a sizeable effect on my world-view. But even if that's enough of a reason to avoid talking about conclusions, about which particular people are or aren't trustworthy, it shouldn't stop us from at least going meta.

So the second half of the post will be my first suggestion, a list of cues which I, on reflection, appear to follow, to determine whether or not to take another person's views seriously. I'm almost certainly missing important ones, most likely even some that I use myself but am not conscious of. Items are phrased as actions: the person to be evaluated does X. Cues 1-8 are positive and improve trustworthiness, cues 9-12 are negative – though in reality most of them represent a spectrum and could also be phrased in the opposite way.

1. Is internally consistent

2. Is aligned with things I'm already confident in

3. Brings up points that aren't obvious but make sense

4. Has high IQ

5. Uses implicitly or explicitly consequentialist arguments

6. Uses sentences that in isolation sound as sophisticated as necessary to make the point and not more

7. Performs an unusually low amount of visible signaling

8. Cites people I consider to be highly trustworthy

9. Assumes a high baseline of trustworthiness of other people based on their social status or academic degrees

10. Ascribes well-thought-out views to groups of people that haven't been selected through a plausible filter

11. Bases her judgment of other people larg...
[Preprint for commenting] Digital Immortality:
Theory and Protocol for Indirect Mind Uploading
by avturchin — 2018-03-27T11:49:31.141Z

I would like to get useful input for the following text :

"Digital Immortality: Theory and Protocol for Indirect Mind Uploading"

Abstract. Future superintelligent AI will be able to reconstruct a model of the personality of a person who lived in the past based on informational traces. This could be regarded as some form of immortality if this AI also solves the problem of personal identity in a copy-friendly way. A person who is currently alive could invest now in passive self-recording and active self-description to facilitate such reconstruction. In this article, we analyze informational-theoretical relationships between the human mind, its traces, and its future model; based on this analysis, we suggest the instruments to most cost-effectively collect quality data about a person for future resurrection. These guidelines form a “digital immortality protocol”. Digital immortality is plan C for achieving immortality, after plan A, life extension, and plan B, cryonics.

Keywords: Digital immortality – superintelligence – effective altruism – life extension – mind uploading.

Highlights:

· Future superintelligent AI will be able to simulate past people.

· To help AI improve its simulations, we can collect data about a living person now.

· Passive data collection is constant recording.

· Active data collection is running tests and recording self-description.

· The best way to collect information is to create art, as it is unique, valuable, and predictive.

Full text is open for commenting: https://goo.gl/QkDfdU
Problems with Amplification/Distillation
by Stuart_Armstrong — 2018-03-27T11:12:36.749Z

This post presents my criticisms of Paul Christiano’s Amplification and Distillation framework. I’ll be basing my understanding of Paul’s method mainly on this post. EDIT: Recently added this post looking into corrigibility issues relevant to the framework.

To simplify, the framework starts with a human H and a simple agent A[0]. Then there is an amplification step, creating the larger system Amplify(H, A[0]), which consists of H and many copies of A[0] to aid them. Then there is the distillation step, which defines the artificial agent Distil(Amplify(H, A[0]); this is basically an attempt to create a faster, automated version of Amplify(H, A[0]).

Then A[n+1] is defined recursively as Distil(Amplify(H, A[n])); note that there is always a "human in the loop", as H is used every time we replace A[n] with A[n+1].

I won’t be talking much about the Distil step, to which I have no real objections (whether it works or not is more an empirical fact than a theoretical one). I’ll just note that there is the possibility for (small) noise and error to be introduced by it.

The method relies on three key assumptions. The first one is about Distil; the other two are:

2) The Amplify procedure robustly preserves alignment.
3) At least some human experts are able to iteratively apply amplification to achieve arbitrarily high capabilities at the relevant task.

I’ll also note that, in many informal descriptions of the method, it is assumed that A[n+1] will be taking on more important or more general tasks than A[n]. The idea seems to be that, as long as A[n] does its subtasks safely, Amplify(H, A[n]) can call upon copies of A[n] in confidence, while focusing on larger tasks.

Summary of my critique

I have four main criticisms of the approach:

1. "Preserve alignment" is not a valid concept, and "alignment" is badly used in the description of the method.
2. The method requires many attendant problems to be solved, just like any other method of alignment.
3. There are risks of generating powerful agents within the systems that will try to manipulate it.
4. If those attendant problems are solved, it isn’t clear there’s much remaining of the method.

The first two points will form the core of my critique, with the third as a strong extra worry. I am considerably less convinced about the fourth.

Preserving alignment

How is alignment defined?

An AI aligned with human...
GreaterWrong—several new features & enhancements
by Said Achmiz — 2018-03-27T02:36:59.741Z

(I hope it’s acceptable to post this in the Meta section; if not, the mods can move it to my personal blog.)

GreaterWrong.com (which is an alternative way to browse the new LessWrong) has been adding some new features and enhancements—here’s some of the stuff we’ve got now:

Appearance customization (click the “sliders” button on the top-left)

Images in posts are now properly excluded from changes to brightness / contrast / inversion / etc.

Adjustable text size (look on the top right, just under the content width selector buttons)

Note: This doesn’t work in Firefox, unfortunately.

Most functions / buttons / etc. now have accesskeys, letting you do a lot of navigation (and other things) via the keyboard

All available accesskeys are listed on the About page

Pagination links at the top of all listings pages (next page, previous page, etc.)

Improved mobile layout (especially for the post/comment editing UI)

An updated About page with detailed info on many of GreaterWrong’s features

As always, please let us know if you’re having any problems with the site, bugs, etc. (If you do report bugs or problems, please include your browser, version, and operating system!)
Non-Adversarial Goodhart and AI Risks
by Davidmanheim — 2018-03-27T01:39:30.539Z

In a recent paper by Scott Garrabrant and myself, we formalized and extended the categories Scott proposed for Goodhart-like phenomena. (If you haven't read either his post or the new paper, it's important background for most of this post.)

Here, I lay out my further intuitions about how and where the non-adversarial categories matter for AI safety. Specifically, I view these categories as particularly critical in preventing accidental superhuman AI, or near-term paperclipping. This makes them particularly crucial in the short term.

I do not think that most of the issues highlighted are new, but I think the framing is useful, and hopefully clearly presents why causal mistakes by Agentic AI are harder problems that I think is normally appreciated.

Epistemic Status: Provisional and open to revision based on new arguments, but arrived at after significant consideration. I believe conclusions 1-4 are restatements of well understood claims in AI safety. I believe conclusions 5 and 6 are less well appreciated.

Side Note: I am deferring discussion of adversarial Goodhart to the other paper and a later post; it is arguably more important, but in very different ways. The deferred topics includes most issues with multiple agentic AIs that interact, and issues with pre-specifying a control scheme for a superhuman AI.

Goodhart Effects Review - Read the paper for details!

Regressional Goodhart - When selecting for a proxy measure, you select not only for the true goal, but also for the difference between the proxy and the goal.

Extremal Goodhart - Worlds in which the proxy takes an extreme value may be very different from the ordinary worlds in which the relationship between the proxy and the goal was observed. This occurs in the form of Model Insufficiency, or Change in Regime.

Causal Goodhart - When the causal path between the proxy and the goal is indirect, intervening can change the relationship between the measure and proxy, and optimizing can then cause perverse effects.

Adversarial Goodhart will not be discussed in this post. It occurs in two ways. Misalignment - The agent applies selection pressure knowing the regulator will apply different selection pressure on the basis of the metric. This allows the agent to hijack the regulator's optimization. Cobra Effect - The regulator modifies the agent goal, usually via an incentive, to correlate it with the reg...
A Difficulty With Density-Zero Exploration
by Diffractor — 2018-03-27T01:03:03.000Z

Summary: If exploration rates decay to zero, the obvious way of ensuring that exploration occurs infinitely often (have a trader that sells the sentence saying that exploration will happen) may fail when there are long delays before you get feedback on whether exploration happened, because the trader can go indefinitely into (possible) debt due to slow feedback. And, if the trader budgets itself, then it won't do enough trading to acquire unbounded money from the slowly decaying exploration rate.

So, Density-Zero Exploration was motivated by the following concern: A trader could mess up conditional utilities, and the way in which it did it left the trader capable of taking the same action next turn, as detailed here Of course, ϵ-exploration takes care of this issue, and further, you don't really need the ϵ to remain constant over time, you can have it drop as 1n on the ¯¯¯P-generable weighting which corresponds to the trades of the enforcer trader.

The obvious hope is that exploration would happen infinitely often along the subsequence, so any enforcer trader would lose eventually. Intuitively, that's how it works. But it's a bit harder to get than I naively thought at first.

My first attempt at proving it (modulo some finicky details about ¯¯¯P-generable weightings and how they aren't always 1) was something along the lines of "in the limit, the probability of exploration on the n'th element of the subsequence will be very close to 1n, and because of this, if there's only finitely much exploration, it's possible for a trader to get infinite money by selling stocks of the exploration sentence."

However, there's a problem when you don't get immediate feedback on whether the exploration step occurred. If you have to wait a very long time to hear whether the exploration step happened, then the strategy of "sell stocks in the exploration sentence" may leave the trader unboundedly in debt.

For most of the theorems in the logical induction paper, it was acceptable to take a very long time to exploit an arbitrage opportunity, because by assumption, arbitrage opportunities occurred infinitely often. However, because the frequency of exploration on a subsequence drops as 1n, you can't get a contradiction with the logical inductor criterion if the trader only exploits a sparse subsequence of those days.

Therefore, you can't guarantee infinite exploration steps occur with 1n

...
My Thoughts on Takeoff Speeds
by tristanm — 2018-03-27T00:05:33.482Z

Epistemic Status: Spent a while thinking about one subset of the arguments in this debate. My thoughts here might be based on misunderstanding the details of the arguments, if so, I apologize.

There is a debate going on within the AI risk community about whether or not we will see “gradual”, “slow”, “fast”, or “discontinuous” progress in AGI development, with these terms in quotes because the definitions of these terms can mean entirely different things based on who uses them.

Because progress in AI is very difficult to quantify, these terms largely are forced to be qualitative. For example, “discontinuous” appears to mean that we might observe a huge leap in performance of an AI system, possibly due to a single major insight that allows the system to gain a strategic advantage within one domain and dominate. The term is important because it implies that we might not be able to foresee the consequences of increases in the capability of an AI system before the system is changed. If the rate of progress is “slow” enough, this might give us enough time to prepare and reduce the risk posed by the next increase in capability.

For that reason, we might wonder what type of progress we should expect, and if there is any evidence that points to “slow”, “fast”, or “discontinuous” progress at any given point in AI development.

The evidence that is usually used in favor of the discontinuous narrative is that evolution seemed to produce humans from previous ape-like ancestors who were no more capable than our Chimpanzee relatives, but in a relatively short time-scale (geologically speaking) humans came to develop language, generate art, culture, and science, and eventually dominate the world.

An interesting development in this conversation occurred recently with a contribution by Paul Christiano, who argued in his post Takeoff Speeds, that he did not find our observations about evolution to be very convincing arguments in favor of the discontinuous narrative.

To summarize his argument, we first note that evolution does not in general optimize for “intelligence.” Evolution optimizes for fitness, which may happen to result in increases in intelligence if that improvement happens to be beneficial in a given environment. Evolution does not pick out improvements that will be beneficial in the future, only ones that are beneficial in the current environment. These improvement...
Basic model of Sending a Message (Communication 101)
by Elo — 2018-03-25T23:55:52.888Z

My hope in writing this out is that you go "well duh! Of course" and then pretend like you knew this all already. Whether or not you did is up to you.

With communication there is going to be a Sender and a Receiver. These can and will regularly swap around in a healthy relationship. There will also be a Message. The message goes from the sender to the receiver. Often the sender feels most heard when we get confirmation or affirmation that the message was received and it was the same message that we sent. This can happen by repetition (See also Handshaking (computer science)).

It's not all that complicated.

A short time later. A confirmation of the first message or a new message being sent.

There are many ways that a message can go wrong.

Here is a few of them:

Over emphasis - you get the message across but it’s super harsh. "Don't walk on my left side" is heard as "never ever do that ever again"
Under emphasis - you get the message across but it’s a mild form and not taken seriously. "can you make sure you message me when you are running late" becomes "if you remember to text me, that would be great"
Opposite message - you successfully send the opposite message. "I appreciate your attention" becomes "leave me alone"
Wrong message - you successfully sent a different message. "I want you to tell me that you like what I am wearing" becomes "I want you to lie to me to make me feel better"
Under specific - you sent a message but it’s not clear what the specific problem is. Or why you are sending this message. "you need to be a cleaner person" when you wanted to say, "clean your bathroom because there is mould on the walls and it's making you sick"
Over specific - you get the message across but it seems like it only applies to the past and not other similar situations. See also rules-lawyering your relationships - “you said you didn’t want me to go to dinner, you didn’t say anything about lunch”. “you said you didn’t like me holding hands, you didn’t say anything about walking arm in arm... why are you so upset! Come back and talk to me!”
garbled - it’s clear you are sending a message but it’s not clear what. "Hey when you do that thing I wish you would do something different instead". "lets meet up some time to talk".
Inco

...
On Cognitive Distortions
by Quaerendo — 2018-03-25T22:44:10.482Z

Follow-up to: On Defense Mechanisms

In my previous post, I suggested that rationalists examine the concept of defense mechanisms: self-deceiving ways of coping with the anxiety caused by internal conflicts. Psychologists argue that defense mechanisms are often inferior to constructive coping techniques (e.g. meditation, systematic problem-solving, positive reinterpretation) in terms of personal adjustment.

Defense mechanisms can make us do weird things. We bury distressing thoughts in our subconscious minds and we deny that there is anything wrong. Even when we acknowledge a problem, we often rationalize our own behavior with ad-hoc excuses, or blame other people. Sometimes we unleash pent-up frustration onto innocent bystanders and act in immature ways. Other times, we hide our true feelings by acting contrary to them or by focusing on the abstract aspects of the situation. We try to "cancel out" guilt with atonement, and compensate for our perceived shortcomings in exaggerated ways. We identify with accomplished people and fantasize about our wishes without taking action.

But wait, there's more! A related concept is that of cognitive distortions.

As you might guess, a distorted cognition is a thought that is based on insufficient evidence and is therefore likely to be an exaggeration or misperception of reality. More specifically, cognitive distortions are automatic thought patterns that are usually slanted in a negative direction. The theory propounded by psychologists like Albert Ellis, Aaron Beck and David Burns is that our cognitions (thoughts) influence our emotions (feelings), which influence how we respond to a situation. This can create a feedback loop whereby inaccurate perceptions of reality cause people to experience negative psychological states like stress, anxiety, depression or low self-esteem, which then reinforce the irrational thoughts.

Whereas defense mechanisms have their genesis in Freudian psychoanalytic theory (where uncomfortable emotions are caused by tensions between the id, ego and superego), the theory of cognitive distortions is based in the more modern cognitive paradigm of psychology. Two promoters of the cognitive approach in therapy, Albert Ellis and Aaron Beck, pinpointed negative self-talk and catastrophic thinking as common causes of maladaptive responses to stress (see chapter four in Weiten, Dunn & Hammer...
Welcome to Baltimore Lesswrong Meetup [Edit With Your Details]
by adrusi — 2018-03-25T21:49:42.255Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
Broken images in the sequences
by sfree — 2018-03-25T14:41:33.951Z

This post is to call to attention a large number of broken images in the sequences that completely hinder the ability to comprehend the posts.

For example, the Quantum Mechanics sequence is almost completely unreadable:

https://www.lesserwrong.com/s/Kqs6GR7F5xziuSyGZ/p/5vZD32EynD9n94dhr

https://www.lesserwrong.com/s/Kqs6GR7F5xziuSyGZ/p/ybusFwDqiZgQa6NCq

The URLs of the broken images are such as the following:

https://www.lesserwrong.com/static/imported/2008/04/10/fig4_2.gif
The Art of the Artificial: Insights from 'Artificial Intelligence: A Modern Approach'
by TurnTrout — 2018-03-25T06:55:46.204Z

‘‘Most people won't agree to kill themselves for 50 million dollars."Stuart Russell and Peter Norvig

Foreword

One of the fruits of growing older is revisiting your old favorites, whether they be foods, books, or songs. As you take your warm, rose-tinted mental bath, you appreciate subtleties which had escaped you, laugh heartily at jokes which hadn't even registered, and grasp concepts whose importance you had never fully realized. Similarly, some songs never lose their charm, no matter how abused the 'replay' button becomes. AI: AMA is a paean to the triumphs of Computer Science and a rallying cry towards those hills we have yet to climb (ahem).

Exposition

My process for the first 60% of this 1,052-page behemoth was fairly normal: an hour or two of studying per day over the course of a few weeks. The last bit went a little differently.

Whatever It Takes

Two days ago, my winter trimester ended; I had upwards of 400 pages to go, half of this post to write, and dozens of exercises to complete. I found myself with three full days to spare before my research resumed the proceeding week. I did what any young man in his early twenties would do - I concocted a plan for studying upwards of 30 hours in that time span.

I knew that this plan was preposterous; in all likelihood, I'd finish 3 chapters and burn out. That's why I wheeled out Murphyjitsu.

Preparing for Failure Modes

Burnout was prepared for by:

The imposition of some dark-arts beliefs:1

My ego does not deplete.
I am 100% certain that I will succeed.
Given enough time, I can understand anything.

The institution of hourly brief exercise breaks. Cutting through the crisp, cold air at high speed got my heart beating and rekindled my passion for crushing the task at hand.
The construction of a narrative, with me as the protagonist, slowly increasing in capability, working diligently against the racing clock.

Yes, I basically turned myself into a trope.
No, I don't mind: I'm willing to take advantage of whatever psychological factors are at my disposal to help me succeed.
Yes, I listened to the Rocky soundtrack a few times. It was great.

Mental exhaustion (distinct from emotional / physical burnout in that your mind can't process concepts as efficiently as it could seven hours ago when you started) was prepared for by:

Regularly taking breaks to chill out, listen to music, and dance a bit.

... ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,