trentmkelly/LessWrong-43k huggingface dataset:
Strategy is the Deconfusion of Action
by ryan_b — 2019-01-02T20:56:28.124Z

Reading the New Research Directions update from MIRI, I was struck by the description of deconfusion:

By deconfusion, I mean something like “making it so that you can think about a given topic without continuously accidentally spouting nonsense.”

I find this concept deeply impressive. I have also lately been considering the problem of strategy or lack thereof, so the idea popped up almost immediately: strategy is deconfusion in the action domain.

I'm going to draw on three sources for this post: the first is the aforementioned New Research Directions post from MIRI; the second is a paper from Parameters 46 Winter issue by Jeffrey W. Meiser, "Are Our Strategic Models Flawed? Ends+Ways+Means = (Bad) Strategy"; the third is an article from November 2012 in The Atlantic by Thomas E. Ricks, "General Failure." I recommend them all individually, but I won't assume you have read them.

Returning to the concept of deconfusion, it is easy to change the quoted section only a little to capture what I mean:

By strategy, I mean something like "making it so that you can act towards a given end-state without continuously accidentally wasting effort."

I think this is an important connection to draw. There is a lot of information available on strategy: each military philosopher of note supports an entire corpus of commentary, and likewise for every conqueror; every war has lots of official and academic analysis done on its results; there is an absurd profusion of filtering the military and historical information through the lens of self-help or business-speak. It has all very consistently failed to guide the development and execution of good strategies, and I think deconfusion does a good job of pointing to why.

Failure to Notice Confusion and the Lykke Model

Meiser's paper is about the current norms in the US military and the Army in particular. The focus of the paper is the Lykke model and the ways in which it encourages bad strategy. It consists of the following:

“Strategy equals ends (objectives toward which one strives) plus ways (courses of action) plus means (instruments by which some end can be achieved).”

The problem in practice, Meiser argues, is that strategy development is dominated by means-based planning. This is because the theory is from 1989, developed in reaction to the failures of the Vietnam War; the thinking went that i...
Electrons don’t think (or suffer)
by Shmi — 2019-01-02T16:27:13.159Z
Link: http://backreaction.blogspot.com/2019/01/electrons-dont-think.html

There is an EA fringe that talks about suffering in elementary particles. Physicist Sabine Hossenfelder reminds her readers why this panpsychist idea is nonsense.

TL;DR: "if you want a particle to be conscious, your minimum expectation should be that the particle can change. It’s hard to have an inner life with only one thought. But if electrons could have thoughts, we’d long have seen this in particle collisions because it would change the number of particles produced in collisions."

The whole post and the comments are an entertaining read.
Events in Daily?
by Raemon — 2019-01-02T02:30:06.788Z

I'm in the process of refactoring how we handle the default posts view. The core goal was to implement some anti-spam features into it. But it highlighted some oddness in how we're currently filtering things.

I wanted to check how this might effect people on GreaterWrong, and people who use the Daily page.

tldr: would you feel better or worse if Events showed up in the daily feed? (for reference, there seems to be roughly one event listing per two days)

(And, for people implementing LW readers, would it be a problem if events and meta posts appeared in the default post view?)

Technical details:

Currently our default view filters out meta posts, event posts, and posts associated with a particular community group. But, almost all the actual views we use on lesswrong.com don't really need that default filtering:

The curated feed only includes thing we manually put there (so it's not especially useful to filter out meta/events/group-descriptions, since we just wouldn't put them there)
Same for frontpage
Same for sequences
The events view only shows events anyhow
User profiles currently aren't showing events / meta / group posts, and this is actually a problem – it means it's harder for a user to find all their posts
Recent discussion already deliberately includes all these things (but has to go out of it's way to

The only two views where we actually would want to exclude things are in "All Posts" and "Daily", which serve similar functions.

"Event" posts are location-specific and so aren't as often relevant to people, but also, if your primary way of engaging with site is on /daily, you might not otherwise get to see events.

I'm curious if people who primarily read daily would feel better or worse about Events getting included there. (Again, the frequency seems to be about 1 listing per two days)
Learning-Intentions vs Doing-Intentions

by Ruby — 2019-01-01T22:22:39.364Z

Epistemic Status: In truth, only a slight repackaging of familiar ideas with a new handle I’ve found myself wanting. See

The Lean Startup

and

Riskiest Assumption Testing

for other resources.

Suppose you are Bob Steele, structural engineer extraordinaire, and you’ve recently completed your doctorate thesis in advanced bridge aerodynamics. You see how a new generation of bridge technology could significantly improve human welfare. Bridges are not as direct as bed nets or cash transfers, but improved transport infrastructure in developing regions boosts economic productivity flowing through to healthcare, education, and other life-improving services. There’s no time to waste. You found Bridgr.io, put the hard in hardware startup, and get to work bringing your revolutionary technologies to the world.

Common advice is that startups should have a few core metrics which capture their goals, help them track their progress, and ensure they stay focused. For Bridgr.io that reasonably might be revenue, clients, and number of bridges built. There is a danger in this, however.

Although Bridgr.io’s ultimate goal is to have built bridges in the right place, the most pressing tasks are not construction tasks. They’re research tasks. Refining the designs and construction process. Until Bridgr.io hits on a design which works and can be scaled, there is no point sourcing steel and construction workers for a thousand bridges. The first step should be building a sufficient number of test and prototype-bridges (or simulations) not with goal that these bridges will transport anyone, just with the goal of learning.

Phase 1: Figure what to do and how to do it.

Phase 2: Do it.

It’s true that if Bridgr.io tries to build as many bridges as possible as quickly as possible that they will learn along the way what works and what doesn’t, that R&D will automatically happen. But I claim that this kind of learning that happens as a product of trying to do the thing (prematurely) is often inefficient, ineffective, and possibly lethal to your venture.

Superficially, building bridges to have bridges and building bridges to figure out which bridges to build both involve building bridges. Yet in the details they diverge. If you’re trying to do the thing, you often spend your time trying to mobilize enough resources for the all-out effort. You throw everything you’ve got at it, because that’s what it ...
What observations do you as a customer would expect to matter, if your experience was 1000 times "stronger"?
by Mary Chernyshenko — 2019-01-01T22:03:39.480Z

No, this is not for marketing purposes because for marketing purposes it is useless.

I work in a small bookshop. Please stop here and feel that calm feeling of being introduced to a topic you already have some idea about, even if you have only ever been on the buying end of the business. This is your chance to ask yourself what expectations that intangible knowledge provides you - are there any particular things you think would "have weight", if your bookshop experience was a thousand times more, eh, experienced? (It's a nice round number, let's roll with it.) By "things having weight" I mean smth like "you still notice it after all this time" (1) and/or "you built a heuristic around it" (2) and/or "you observe it and know it to be the common way of things" (3) and/or "you know something for a convention that can, if need arises, be cast aside" (4) and/or "you seek it out as your clue for a specific action" (5).

And what do you think about what other people's answers would look like?

...And what do you think a bookseller's answers would look like?

(The broader question is (PLEASE read only after you have thought about the specific one) hfvat jung pbaprcgf jbhyq lbh qrfpevor lbhe nccebnpu gb ercrngrq bofreingvba naq npgvba va fbzr snveyl arhgeny, pbzzba frggvat, juvpu qbrfa'g checbfrshyyl ahqtr lbh gb ersvar lbhe fgengrtvrf (cebonoyl), hayvxr gur tnzrf jurer lbh zvtug abg xabj jung'f tbvat ba ohg lbh qb xabj lbh'er rvgure YBFVAT be JVAAVAT. Ohg V qvqa'g guvax vg'f gvzr gb nfx gung dhrfgvba lrg.)
What exercises go best with 3 blue 1 brown's Linear Algebra videos?
by Raemon — 2019-01-01T21:29:37.599Z

3 blue 1 brown is a youtube channel that teaches math concepts. I've found it a much better introduction than other resources I've looked at. The Essence of Calculus series was particularly good.

But one issue is that, while the videos come with a few exercises sprinkled within (typically one per concept), they don't come with enough to really check whether I understand a thing.

Last year I tried binging the channel but kept running into issues where I'd want to do additional practice, so I'd try Khan Academy or Brilliant.org. But it felt like the exercises were introducing concepts in a fairly different order, or without using the same metaphors/explanations/keywords that 3-blue-1-brown was. So it was hard to tell what exercises corresponded with what. (Brilliant.org was better overall than Khan Academy but still felt like a completely different lesson plan, with somewhat less-good-explanations)

Eventually, bouncing between brilliant.org and 3 blue 1 brown resulted in me running out of steam and giving up.

I've emailed the creator about it but even if they were excited about it I imagine it'd be quite a while before anything useful came of it.

I'd be generally interested in a project of crowdsourcing exercises that correspond to individual 3-blue-1-brown videos. Since I'm currently trying to re-ignite my interest in the Linear Algebra series I thought I'd ask specifically about that, although if people happen to have cached thoughts about other series that'd be cool.
Advantages of Card Rebalancing
by Zvi — 2019-01-01T13:10:02.224Z

Previously: Artifact Embraces Card Balance Changes, Card Collection and Ownership, Card Balance and Artifact, Card Rebalancing, Card Oversupply and Economic Considerations in Digital Card Games

VIII. Card Rebalancing and Gameplay Benefits

Let us presume we’ve gotten economics out of the way. We’ve chosen a model compatible with limited ownership of cards and/or paid the price for not doing so. Most players will have access to whatever level of cards they usually have access to, and at least don’t mind the changes to their cards.

What are the effects on game play? What are the good and bad scenarios?

I’ll lay out the good scenarios and features, and offer eight good reasons to rebalance your cards.

Next time, I’ll lay out the bad scenarios and features, and why it all might be a terrible mistake.

Changing cards keeps the game fresh and new.

After the most recent set has been released for a while, it is easy for card games to become stale and repetitive. Players know the best decks and strategies, and the same patterns happen repeatedly. Changing a few key cards even in subtle ways can shake that up, as there are often lots of subtle factors determining how decks look and play.

Consider Axe moving from seven power to six power, and from eleven to ten health. This dramatically improved the value of heroes with seven health, which among means Zeus, Ogre Magi, Drow Ranger (although she was nerfed at the same time) and Bounty Hunter. Blue became much stronger, and black decks have reason to pick Bounty Hunter, which is also now a lock to kill Axe by default on the second turn, even if you lose both coin flips. There are also now a lot more scenarios where heroes are left at one health, which also helps Ogre Magi and Zeus, making blue that much more appealing. This isn’t only about Axe doing six on turn one, it is also about him being much worse with Duel and his companion card Berserker’s Rage. It also makes him want a weapon that much more to get back to where he needs to be, making Short Sword, Jasper Daggers and Demagicking Maul more attractive.

Drow Ranger’s change is also transformative. Decks no longer have to worry that green can knock out an entire lane if given initiative, so long as they can put two heroes in non-adjacent locations. This in turn makes it horribly difficult for green to go wide or commit to deep board presence in a lane for fear of Annihilat...
Optimization Regularization through Time Penalty
by Linda Linsefors — 2019-01-01T13:05:33.131Z

For an overview of the problem of Optimization Regularization, or Mild Optimization, I refer to MIRI's paper Alignment for Advanced Machine Learning Systems, section 2.7

My solution

Start with a bounded utility function, U(T), that is evaluated based on state of the world at a single time T (ignoring for now simultaneity is ill-defined in Relativity). Examples:

Umaximize(T)=tanh(# of paper-clips at time T)
Usatisfier(T)=P(# of paper-clips > 1000, at time T)
Uhuman values(T)= If a human at time 0 (at the start of the optimization process) are shown the world state at time T, how much would they like it (mapped to to the interval [0,1]).

Then maximize U(T)−λT, where λ>0 is a regularization parameter chosen by the AI engineer, and T is a free variable chosen by the AI.

Time is measured from the start of the optimization process. Because the utility is evaluated based on the world at time T, this value is the amount of time the AI spends on the task. It is up to the AI to decide how much time it wants. Choosing T should be seen as part of choosing the policy, or be included in the action space.

Because the utility function is bounded, the optimization process will eventually hit diminishing returns, and will then choose to terminate, because of the time penalty.

Why time penalty?

Unbounded optimization pressure is dangerous. Without any form of regularization, we need to get the alignment exactly right. However, with regularization we merely need to get it almost exactly right, which I believe is much easier.

However, impact regularization have turned out to be very hard. We don't want the impact measure to depend on the AI's understanding of human values, because that will not provide extra safety. But a value neutral impact measure is almost impossible, because the world has too many degrees of freedom. However, time is both value neutral and has only a single degree of freedom.

Why not use a fixed finite time horizon?

The reason T is a variable chosen by the AI and not a constant chosen by us, is because we don't know when the optimization process will start hitting diminishing returns. Leaving T up to the AI solves this dynamically. In addition, we will still get to choose a maximum time horizon by the choice of U and λ. The AI will never keep going beyond Tmax=Umax−Uminλ

What happens after t=T ?

That depen... ,
Alignment Newsletter #39
by Rohin Shah — 2019-01-01T08:10:01.379Z

Happy New Year!

Find all Alignment Newsletter resources here. In particular, you can sign up, or look through this spreadsheet of all summaries that have ever been in the newsletter.

Highlights

Constructing Unrestricted Adversarial Examples with Generative Models (Yang Song et al): This paper predates the unrestricted adversarial examples challenge (AN #24) and shows how to generate such unrestricted adversarial examples using generative models. As a reminder, most adversarial examples research is focused on finding imperceptible perturbations to existing images that cause the model to make a mistake. In contrast, unrestricted adversarial examples allow you to find any image that humans will reliably classify a particular way, where the model produces some other classification.

The key idea is simple -- train a GAN to generate images in the domain of interest, and then create adversarial examples by optimizing an image to simultaneously be "realistic" (as evaluated by the generator), while still being misclassified by the model under attack. The authors also introduce another term into the loss function that minimizes deviation from a randomly chosen noise vector -- this allows them to get diverse adversarial examples, rather than always converging to the same one.

They also consider a "noise-augmented" attack, where in effect they are running the normal attack they have, and then running a standard attack like FGSM or PGD afterwards. (They do these two things simultaneously, but I believe it's nearly equivalent.)

For evaluation, they generate adversarial examples with their method and check that humans on Mechanical Turk reliably classify the examples as a particular class. Unsurprisingly, their adversarial examples "break" all existing defenses, including the certified defenses, though to be clear existing defenses assume a different threat model where an adversarial example must be an imperceptible perturbation to one of a known set of images. You could imagine doing something similar by taking the imperceptible-perturbation attacks and raise the value of ϵ until it is perceptible -- but in this case the generated images are much less realistic.

Rohin's opinion: This is the clear first thing to try with unrestricted adversarial examples, and it seems to work reasonably well. I'd love to see whether adversarial trainin...
Why do Contemplative Practitioners Make so Many Metaphysical Claims?
by romeostevensit — 2018-12-31T19:44:30.358Z

To paraphrase Culadasa: awakening is a set of special insights that lead to drastically reduced suffering. This seems straightforward enough, and might lead one to question, if this is the case, why the vast landscape of teachers and practitioners making what seem to be some fairly wild claims about reality? Even if it is the case that these claims are some combination of mistaken, pedagogical in intention, reframes of more mundane points using unfortunate language etc, it would still raise the concern that these practices are, de facto, making their practitioners less connected with reality and decent epistemic standards in their mental models and communication with others. What gives?

I believe I have an explanation that covers some of the territory here. I don't claim it covers all of the phenomenon in question. Hopefully it will be of some benefit in clearing up certain confusions.

In order to have the necessary insights, practitioners engage in cultivation of prerequisite skills. One long lived and fairly straightforward model of such skills is the 7 Factors of Enlightenment:

Physical Relaxation
Equanimity
Joy
Energy
Determination to Investigate
Concentration
Mindfulness

These skills are not binary. Each one deepens along a spectrum as you practice. As the skills deepen, you begin to have more direct perception, on a moment-by-moment basis, of how beliefs and values (is and ought) are formed and interact with one another. This direct perception very often leads to changes, as unhelpful linkages are noticed and either drop away if no longer needed, or are upgraded into versions more closely aligned with how the world is or skillfully realizing values. For those familiar with Cognitive Behavioral Therapy, something very similar is at play here. In CBT, your attention is drawn to the way that a situation can trigger a feeling, which triggers an associated thought pattern, which drives a compensatory action etc. Perception of the linkages provides more intervention points.

Depending on where a person starts (existing linkages between beliefs and values) they may be led to come up with a variety of ideas about the 'true nature of reality' along the way as these linkages change. Even if this map-territory error isn't made, a significant and unexpected shift in how you relate to your own life, ie the story you use to make sense of your current beli...
What do you do when you find out you have inconsistent probabilities?
by NunoSempere — 2018-12-31T18:13:51.455Z

I've recently been reading about a rationalist blogger who converted to Catholicism. She may have assigned subjective probabilities like:

P(Objective Morality)=0.9P(God)=0.05

Then she may have introspected and come up with:

P(Objective Morality | God)=0.99P(Objective Morality | No God)=0.02

We can calculate:

P(God|Objective Morality)=P(Objective Morality|God)×P(God)P(Objective Morality)=0.99×0.050.9=0.55P(God|No Objective Morality)=P(No Objective Morality|God)×P(God)P(No Objective Morality)=0.01×0.050.1=0.005

Abbreviating Objective Morality by "OM", and "God" by "G", this state of affairs is inconsistent, because we intuitively see that:

P(G)≠P(G|OM)×P(OM)+P(G|¬OM)×P(¬OM)

To resolve it, she could either increase her subjective probability of there being a God

P(God)=P(G | OM )×P(OM)+P(G | ¬OM )×P(¬OM )=0.55×0.9+0.005×0.1=0.4955≈0.5

or she could reduce her probability of there being some kind of objective morality

P(OM)=P(OM | G )×P(G)+P(OM | ¬G )×P(¬G )=0.99×0.1+0.02×0.95=0.118

She could also reconsider P(God|Objective Morality) or P(Objective Morality|God).

Anyways, I find myself very confused by this state of affairs. Is this a solved question? Is there a purely principled way of resolving this which only takes into account the 4 numbers P(OM), P(G), P(OM|G) and P(G|OM)? Is there a standard way of using some kind of metaprobabilities?