trentmkelly/LessWrong-43k huggingface dataset:
What are the advantages and disadvantages of knowing your own IQ?
by Mati_Roy — 2019-04-03T18:31:49.269Z

I've seen some answers here: https://www.quora.com/What-are-the-advantages-disadvantages-to-know-your-own-IQ

But I would be curious to know the perspective from people here.

Third alternative: taking an IQ test and tracking your IQ, but not looking at it.

For example: Can tracking IQ be useful to track cognitive degradation and predict neurodegenerative diseases?
Machine Pastoralism
by ryan_b — 2019-04-03T16:04:02.450Z

This idea has occurred to me before, but in the interim I dismissed it and then forgot. Since it is back again more-or-less unprompted, I am writing it down.

We usually talk about animals and their intelligence as a way to interrogate intelligence in general, or as a model for possible other minds. It occurred to me our relationship with animals is therefore a model for our relationship with other forms of intelligence.

In the mode of Prediction Machines, it is straightforward to consider: prediction engines in lieu of dogs to track and give warning; teaching/learning systems for exploring the map in lieu of horses; analysis engines to provide our solutions instead of cattle or sheep to provide our sustenance. The idea here is just to map animals-as-capital to the information economy, according to what they do for us.

Alongside what they do for us is the question of how we manage them. The Software 2.0 lens of adjusting weights to search program space reads closer to animal husbandry than building a new beast from the ground up with gears each time, to me. It allows for a notion of lineage, and we can envision using groups of machines with subtle variations, or entirely different machines in combination.

This analogy also feels like it does a reasonable job of priming the intuition about where dangerous thresholds might lie. How smart is smart enough to be dangerous for one AI? Tiger-ish? We can also think about relative intelligence: the primates with better tool ability and more powerful communication were able to establish patronage and then total domestication over packs of dogs and herds of horses, cattle, and sheep. How big is that gap exactly, and what does that imply about the threshold for doing the same to humans? Historically we are perfectly capable of doing it to ourselves, so it seems like the threshold might actually be lower than us.
Could waste heat become an environment problem in the future (centuries)?
by RocksBasil — 2019-04-03T14:48:05.257Z

I have wondered about this scenario for a while, and would like to know what is your opinion about it. Its assumptions are quite specific and probably won't be true, but they do appear realistic enough for me.

(1): Assume that nuclear fusion becomes an available energy source within a couple centuries, it will provide a cheap, plentiful, emission-free, and long lasting source of energy for human activities.

(If this assumption is wrong, we are probably in trouble)

(2): Assume that continued economical/technological development requires increasing energy consumption indefinitely.

(This is probably wrong if we utilise completely new physics in the future, but I don't think this assumption is unlikely)

(3): Assume that the generation of waste heat during energy generation/consumption cannot be dramatically lowered in the short-term future.

(This is also probably wrong. But it will hold true if we still have to use machines/engines/generators based on the same design principles as we do today, and I don't see that happening too soon)

The logical conclusion from the above three assumption:

At some point after the implementation of nuclear fusion, humanity's energy consumption might reach a level so high that the waste heat we release into the atmosphere will be altering the Earth's climate system not unlike what our carbon emissions are doing today.

(Since the source of any future fusion plant is likely hydrogen in seawater, for Earth it probably acts as an extra heat source independent of the sun)

The Earth is functionally a giant spacecraft, and spacecrafts usually have very sophisticated heat management systems to prevent them from overheating, so perhaps we have to work with that as well.

I haven't done too much number crunching yet, I might have gotten the figures wildly wrong.

We know today the amount of solar energy the Earth receives per year is about ~5000 times the amount of energy humanity consumes.

If humanity's energy consumption increases 100 times, and 50% of the energy is released into the atmosphere as waste heat, then we are releasing ~1% of solar energy into the atmosphere as heat.

That might have some serious climate implications if lasting for a long time, but I'm not certain about that yet.

Possible solutions:

(1): Geoengineering, that seems to be obvious. We try to reduce the solar energy input on Earth wh...
Defeating Goodhart and the "closest unblocked strategy" problem
by Stuart_Armstrong — 2019-04-03T14:46:41.936Z

This post is longer and more self-contained than my recent stubs.

tl;dr: Patches such as telling the AI "avoid X" will result in Goodhart's law and the nearest unblocked strategy problem: the AI will do almost exactly what it was going to do, except narrowly avoiding the specific X.

However, if the patch can replaced with "I am telling you to avoid X", and this is treated as information about what to avoid, and the biases and narrowness of my reasoning are correctly taken into account, these problems can be avoided. The important thing is to correctly model my uncertainty and overconfidence.

AIs don't have a Goodhart problem, not exactly

The problem of an AI maximising a proxy utility function seems similar to the Goodhart Law problem, but isn't exactly the same thing.

The standard Goodhart law is a principal-agent problem: the principal P and the agent A both know, roughly, what the principal's utility U is (eg U aims to create a successful company). However, fulfilling U is difficult to measure, so a measurable proxy V is used instead (eg V aims to maximise share price). Note that the principal and the agents goals are misaligned, and the measurable V serves to (try to) bring them more into alignment.

For an AI, the problem is not that U is hard to measure, but that it is hard to define. And the AI's goals are V: there is no need to make V measurable, it is not a check on the AI, but the AI's intrinsic motivation.

This may seem like a small difference, but it has large consequences. We could give an AI a V, our "best guess" at U, while also including all our uncertainty about how to define U. This option is not available for the principal agent problem, since giving a complicated goal to a more knowledgeable agent just gives it more opportunities to misbehave: we can't rely on it maximising the goal, we have to check that it does so.

Overfitting to the patches

There is a certain similarity with many machine learning techniques. Neural nets that distinguish cats and dogs could treat any "dog" photo as a specific patch that can be routed around. In that case, the net would define "dog" as "anything almost identical to the dog photos I've been trained on", and "cat" as "anything else".

And that would be a terrible design; fortunately, modern machine learning gets around the problem by, in effect, assigning uncertainty correctly: "dog" is not seen as the exact set o

...
[HPMOR] "the Headmaster set fire to a chicken!" 
by bipolo — 2019-04-03T09:12:22.707Z

In harray potter and the method of rationality, cp17, Dumbledore set fire to a chicken. Harray, of curse, got shocked, but after a while he was told that it probably wasn't "real" chicken but a chicken which created by transformation.

I have 2 question about it:

Is that ethic to create animals(or humans) by transformation? Beacause, you know, its will faded after a while.
(I know you can say it about creating children but the time is much shorter here).

Damn it, why its better to burn alive "fake"(but alive!) chicken then "real" chicken?!
Alignment Newsletter #51
by Rohin Shah — 2019-04-03T04:10:01.325Z

Find all Alignment Newsletter resources

here

. In particular, you can

sign up

, or look through this

spreadsheet

of all summaries that have ever been in the newsletter.

You may have noticed that I've been slowly falling behind on the newsletter, and am now a week behind. I would just skip a week and continue -- but there are actually a lot of papers and posts that I want to read and summarize, and just haven't had the time. So instead, this week you're going to get two newsletters. This one focuses on all of the ML-based work that I have mostly been ignoring for the past few issues.

Highlights

Towards Characterizing Divergence in Deep Q-Learning

(Joshua Achiam et al): Q-Learning algorithms use the Bellman equation to learn the Q*(s, a) function, which is the long-term value of taking action a in state s. Tabular Q-Learning collects experience and updates the Q-value for each (s, a) pair independently. As long as each (s, a) pair is visited infinitely often, and the learning rate is decayed properly, the algorithm is guaranteed to converge to Q*.

Once we get to complex environments where you can't enumerate all of the states, we can't explore all of the (s, a) pairs. The obvious approach is to approximate Q*(s, a). Deep Q-Learning (DQL) algorithms use neural nets for this approximation, and use some flavor of gradient descent to update the parameters of the net such that it is closer to satisfying the Bellman equation. Unfortunately, this approximation can prevent the algorithm from ever converging to Q*.

This paper studies the first-order Taylor expansion of the DQL update, and identifies three factors that affect the DQL update: the distribution of (s, a) pairs from which you learn, the Bellman update operator, and the neural tangent kernel, a property of the neural net that specifies how information from one (s, a) pair generalizes to other (s, a) pairs. The theoretical analysis shows that as long as there is limited generalization between (s, a) pairs, and each (s, a) pair is visited infinitely often, the algorithm will converge. Inspired by this, they design PreQN, which explicitly seeks to minimize generalization across (s, a) pairs within the same batch. They find that PreQN leads to competitive and stable performance, despite not using any of the tricks that DQL algorithms typically require, such as target networks.

Rohin's opi...
LW Update 2019-04-02 – Frontpage Rework
by Raemon — 2019-04-02T23:48:11.555Z

Since LW2.0 launched, the frontpage had become very complex – both visually and conceptually. This was producing an overall bad experience, and making it hard for the team to add or scale up features (such as Q&A, and later on Community, Library and upcoming Recommendations)

For the past couple months, we've been working on an overhaul of the frontpage (and correspondingly, the overall site design). Our goal was is to rearrange that complexity, spending fewer "complexity points" on things that didn't need them as much, so we could spend them elsewhere.

Frontpage Updates

Tooltip oriented design.

It's easier to figure out what most things will do before you click on it.

Navigation Menu

Helps establish the overall site hierarchy
Available on all major site pages (not Post Pages, where we want people to read without distraction)
Improved mobile navigation (shows up as a tab menu at the bottom)
Eventually we'll deprecate the old Nav Menu (still available in the header) and replace it with a collapsible version of the new one.

Home Page streamlining

Moved Recommend Sequences and Community over to the Nav Menu, so there are only 3 sections to parse
Post Items simplified down to one line.
Latest Posts now only have a single setting: "show personal blogposts", instead of forcing you to figure out immediately what "meta", "curated" and "daily" are.
Post List options are generally 'light cobalt blue' – not too obtrusive, but easier to find when you want them.

Questions Page now has two sections:

Recent Activity – simply sorted by "most recently commented at", so if you respond to an old question it will appear above the fold.
Top Questions – also sorted by "recently commented", but filtered to questions with 40 or more karma, so that it's easier to catch up on updates to highly upvoted questions.

Community Page

UI updated to match Home Page.
The group section now shows 7 groups instead of 3, and has a load more button.
Degrees of Freedom
by sarahconstantin — 2019-04-02T21:10:00.516Z

Something I’ve been thinking about for a while is the dual relationship between optimization and indifference, and the relationship between both of them and the idea of freedom.

Optimization: “Of all the possible actions available to me, which one is best? (by some criterion). Ok, I’ll choose the best.”

Indifference: “Multiple possible options are equally good, or incommensurate (by the criterion I’m using). My decision algorithm equally allows me to take any of them.”

Total indifference between all options makes optimization impossible or vacuous. An optimization criterion which assigns a total ordering between all possibilities makes indifference vanishingly rare. So these notions are dual in a sense. Every dimension along which you optimize is in the domain of optimization; every dimension you leave “free” is in the domain of indifference.

Being “free” in one sense can mean “free to optimize”. I choose the outcome that is best according to an internal criterion, which is not blocked by external barriers. A limit on freedom is a constraint that keeps me away from my favorite choice. Either a natural limit (“I would like to do that but the technology doesn’t exist yet”) or a man-made limit (“I would like to do that but it’s illegal.”)

There’s an ambiguity here, of course, when it comes to whether you count “I would like to do that, but it would have a consequence I don’t like” as a limit on freedom. Is that a barrier blocking you from the optimal choice, or is it simply another way of saying that it’s not an optimal choice after all?

And, in the latter case, isn’t that basically equivalent to saying there is no such thing as a barrier to free choice? After all, “I would like to do that, but it’s illegal” is effectively the same thing as “I would like to do that, but it has a consequence I don’t like, such as going to jail.” You can get around this ambiguity in a political context by distinguishing natural from social barriers, but that’s not a particularly principled distinction.

Another issue with freedom-as-optimization is that it’s compatible with quite tightly constrained behavior, in a way that’s not consistent with our primitive intuitions about freedom. If you’re only “free” to do the optimal thing, that can mean you are free to do only one thing, all the time, as rigidly as a machine. If, for instance, you are only free to “act in your own best intere...
[April Fools] User GPT2 is Banned
by jimrandomh — 2019-04-02T06:00:21.075Z

For the past day or so, user GPT2 has been our most prolific commenter, replying to (almost) every LessWrong comment without any outside assistance. Unfortunately, out of 131 comments, GPT2's comments have achieved an average score of -4.4, and have not improved since it received a moderator warning. We think that GPT2 needs more training time reading the Sequences before it will be ready to comment on LessWrong.

User GPT2 is banned for 364 days, and may not post again until April 1, 2020. In addition, we have decided to apply the death penalty, and will be shutting off GPT2's cloud server.

Use this thread for discussion about GPT2, on LessWrong and in general.
post-rational distractions
by nBrown — 2019-04-02T02:26:17.121Z

DonyChristie's intellectual fap post has called for post-rational techniques.

I got most of the way through a comment reply before I realised it was a joke. April fools and all.

Fruits of that effort here are some thoughts

***

Developing your centre's. Sarah Perry's are knitting and mountain running. https://www.ribbonfarm.com/2018/04/06/deep-laziness/

If you ever meet me in person and want to put me at ease, ask me about running or knitting. These are two of my behaviours, my behavioural centers, and one indication of that is how much I like talking about them specifically. I do feel that there is something special about them, and that they connect to my nature on a fundamental level. In my heart, I think everyone should do mountain running and knitting, because they are the best things.

Reading a lot. All the good soft books. Perhaps the ones overlooked by the skeptic types: Bonds that make us free, Feeding your Demons, Chakras, MTG colour wheel, Dream interpretation, Peterson's Bible lectures. Architecture.

Free-ing stuck meanings. A long example of Chapman's here. "I'm not good with people" or "I'm not a technical person"

Meditation. Seems to be important and relate to this somehow. MTCB, The Mind Illuminated, Seeing that frees, Roaring Silence.

What's the context? What the hell is it you're trying to do? The metagame is discovering the constraints. You're swimming in the unknown what are the rules of the game you're playing.

This is what you're doing anyway. It feels important keep in mind Chapman's answer to "If not Bayes then what?"

My answer to “If not Bayesianism, then what?” is: all of human intellectual effort. Figuring out how things work, what’s true or false, what’s effective or useless, is “human complete.” In other words, it’s unboundedly difficult, and every human intellectual faculty must be brought to bear.
Announcing the Center for Applied Postrationality
by Pee Doom — 2019-04-02T01:17:12.873Z

Hi all! Today, we are announcing the formation of a new organization: the Center for Applied Postrationality (or CFAP).

Right now we're looking for two things: 1) $1.5 million in funding to have runway for the next six months, and 2) intellectual contributions from deep thinkers like YOU!

Just what is postrationality, anyway? To be honest, we don't really know either. Maybe you can help us? The term can refer to many different things, including:

Epistemological pragmatism
Getting in touch with your feelings
Learning social skills
Disagreeing with stuff on LessWrong
Becoming a Christian
According to one of our employees: "Postrationality is making the territory fit the map. Postrationality is realizing that paraconsistent linear homotopy type theory is the One True framework for epistemology, not Bayes. Postrationality is realizing that Aristotle was right about everything so there's no point in doing philosophy anymore. Or natural science. Postrationality is The Way that realizes there is no Way. Postrationality is meaning wireheading rather than pleasure wireheading. Postrationality is postironic belief in God. Postrationality is the realization that there is no distinction between sincerity and postirony."
Another employee: "CFAP exists at the intersection of epistemology, phenomenology, sense-making, and pretentiousness. Our goal is to make new maps, in order to better navigate the territory, especially the territory in California. Google maps sucks at this. "

We're still deconfusing ourselves on what "applied" postrationality is, as so far it's mostly been insight porn posted on Twitter. Comment below what techniques you'd suggest for training the art of postrationality!
User GPT2 Has a Warning for Violating Frontpage Commenting Guidelines
by jimrandomh — 2019-04-01T20:23:11.705Z

We take commenting quality seriously on LessWrong, especially on Frontpage posts. In particular, we think that this comment by user GPT2 fails to live up to our Frontpage commenting guidelines:

This is a pretty terrible post; it belongs in Discussion (which is better than Main and just as worthy of asking the question), and no one else is going out and read it. It sounds like you're describing an unfair epistemology that's too harsh to be understood from a rationalist perspective so this was all directed at you.

Since user GPT2 seems to be quite prolific, we have implemented a setting to hide comments by GPT2, which can be accessed from the settings page when you are logged in.
Prompts for eliciting blind spots/bucket errors/bugs
by ZeitPolizei — 2019-04-01T19:41:59.677Z

This post is to make publicly available a few prompts/questions I came up with aiming to uncover blind spots around identity/self-concepts.

Select a trait X that you believe you have, and where you like that you have it (e.g. rational, kind, patient...)
Try to imagine a character that is a caricature of someone with trait X. Or another way to think about this: The way Spock is a Straw Man version of a rational character, what would a Straw Man version of a character with trait X look like? (referred to in the following as X-Spock)

What are blind spots an X-Spock is likely to have?
In what sorts of situations is an X-Spock especially likely to fail?
What would an X-Spock have a lot of trouble admitting to? (e.g. someone who considers themselves courageous may be unable to admit they are afraid)
What are traits that seem like opposites of X?

Could the opposite traits actually be beneficial?
Is what seems like an opposite trait in actuality orthogonal? (e.g. rational and emotional)
Learning "known" information when the information is not actually known
by Stuart_Armstrong — 2019-04-01T17:56:17.719Z

Methods like cooperative inverse reinforcement learning assume that the human knows their "true" reward function R(θ), and then that the human and the robot cooperate to figure out and maximise this reward.

This is fine as far as the model goes, and can allow us to design many useful systems. But it has a problem: the assumption is not true, and, moreover, its falsity can have major detrimental effects.

Contrast two situations:

The human knows the true R(θ).

The human has a collection of partial models in which they have clearly defined preferences. As a bounded, limited agent whose internal symbols are only well-grounded in standard situations, their stated preferences will be a simplification of their mental model at the time. The true R(θ) is constructed from some process of synthesis.

Now imagine the following conversation:

AI: What do you really want?

Human: Money.

AI: Are you sure?

Human: Yes.

Under most versions of hypothesis 1., this will be in a disaster. The human has expressed their preferences, and, when offered the opportunity for clarification, didn't give any. The AI will become a money-maximiser, and things go pear shaped.

Under hypothesis 2., however, the AI will attempt to get more details out of the human, suggesting hypothetical scenarios, checking what happens when money and other things in money's web of connotations come apart - eg "What if you had a lot of money, but couldn't buy anything, and everyone despised you?" The synthesis may fail, but, at the very least, the AI will investigate more.

Thus assuming the AI will be learning a truth that humans already know, is harmless assumption in many circumstances, but will result in disasters if pushed to the extreme.