trentmkelly/LessWrong-43k huggingface dataset:
noticing internal experiences 
by brown — 2018-10-16T01:36:04.539Z

Try:

Open up a blank text file or take a blank sheet of lined paper, Fill the page with whatever comes to mind. write down what is running through your internal monologue. Just get it all out onto the page. Try to capture as much as possible even if it disappears too quickly. This is called free writing or morning pages.[1]

Here's a worked example:

"The feel of the new keyboard as I look down and see my fingers moving I notice the rushing of the wind through the open window and I just used the word notice twice the keys clack and I'm not looking at the screen at all need to learn touch typing new keyboard though so that's okay wonder if I've done enough yet don't want to get lost in the meta the typing is slowing down I feel warm" [2]

Our inner worlds are incredible interesting subtle and strange. If you’re going to learn what kind of thing you are you need to look internally & notice the subtle flinches that drive you. Amplify the texture and substance of your thoughts through gentle observation. Use your awareness to explore your mind.

Noticing is done through your peripherals, your awareness not what you're currently focused on. A mental habit trained through intention. The feeling of something leaping into consciousness you notice theres is an enjoyable song playing, also flinches can be bodily.

To change the way a mind normally reactions you need to observe what it does by default. Our actions can be built of long chains of if then statements. Given specific context -> response. Hungry at home & no bread & low energy level -> peanut butter wrap.

Keywords: Subconscious awareness

[1] Brienne calls this Reflective Recording. Much better explanations there o'course.

[2] It can be really nice to "validate" your internal experience by writing it down.
Alignment Newsletter #28
by Rohin Shah — 2018-10-15T21:20:11.587Z

Highlights

Motivating the Rules of the Game for Adversarial Example Research (Justin Gilmer, George E. Dahl et al) (summarized by Dan H): In this position paper, the authors argue that many of the threat models which motivate adversarial examples are unrealistic. They enumerate various previously proposed threat models, and then they show their limitations or detachment from reality. For example, it is common to assume that an adversary must create an imperceptible perturbation to an example, but often attackers can input whatever they please. In fact, in some settings an attacker can provide an input from the clean test set that is misclassified. Also, they argue that adversarial robustness defenses which degrade clean test set error are likely to make systems less secure since benign or nonadversarial inputs are vastly more common. They recommend that future papers motivated by adversarial examples take care to define the threat model realistically. In addition, they encourage researchers to establish “content-preserving” adversarial attacks (as opposed to “imperceptible” l_p attacks) and improve robustness to unseen input transformations.

Dan H's opinion: This is my favorite paper of the year as it handily counteracts much of the media coverage and research lab PR purporting ``doom'' from adversarial examples. While there are some scenarios in which imperceptible perturbations may be a motivation---consider user-generated privacy-creating perturbations to Facebook photos which stupefy face detection algorithms---much of the current adversarial robustness research optimizing small l_p ball robustness can be thought of as tackling a simplified subproblem before moving to a more realistic setting. Because of this paper, new tasks such as Unrestricted Adversarial Examples (AN #24) take an appropriate step toward increasing realism without appearing to make the problem too hard.

Technical AI alignment

Agent foundations

A Rationality Condition for CDT Is That It Equal EDT (Part 2) (Abram Demski)

Learning human intent

Learning under Misspecified Objective Spaces (Andreea Bobu et al): What can you do if the true objective that you are trying to infer is outside of your hypothesis space? The key insight of this paper is that in this scenario, the human feedback that you get will likely not make sense for any reward function in your hypothesis spa...
The Kelly Criterion
by Zvi — 2018-10-15T21:20:03.430Z

Epistemic Status: Reference Post / Introduction

The Kelly Criterion is a formula to determine how big one should wager on a given proposition when given the opportunity.

It is elegant, important and highly useful. When considering sizing wagers or investments, if you don’t understand Kelly, you don’t know how to think about the problem.

In almost every situation, reasonable attempts to use it will be somewhat wrong, but superior to ignoring the criterion.

What Is The Kelly Criterion?

The Kelly Criterion is defined as (from Wikipedia):

For simple bets with two outcomes, one involving losing the entire amount bet, and the other involving winning the bet amount multiplied by the payoff odds, the Kelly bet is:

where:

f * is the fraction of the current bankroll to wager, i.e. how much to bet;
b is the net odds received on the wager (“b to 1″); that is, you could win $b (on top of getting back your $1 wagered) for a $1 bet
p is the probability of winning;
q is the probability of losing, which is 1 − p.

As an example, if a gamble has a 60% chance of winning (p = 0.60, q = 0.40), and the gambler receives 1-to-1 odds on a winning bet (b = 1), then the gambler should bet 20% of the bankroll at each opportunity (f* = 0.20), in order to maximize the long-run growth rate of the bankroll.

(A bankroll is the amount of money available for a gambling operation or series of wagers, and represents what you are trying to grow and preserve in such examples.)

For quick calculation, you can use this rule: bet such that you are trying to win a percentage of your bankroll equal to your percent edge. In the above case, you win 60% of the time and lose 40% on a 1:1 bet, so you on average make 20%, so try to win 20% of your bankroll by betting 20% of your bankroll.

Also worth remembering is if you bet twice the Kelly amount, on average the geometric size of your bankroll will not grow at all, and anything larger than that will on average cause it to shrink.

If you are trying to grow a bankroll that cannot be replenished, Kelly wagers are an upper bound on what you can ever reasonably wager, and 25%-50% of that amount is the sane range. You should be highly suspicious if you are considering wagering anything above half that amount.

(Almost) never go full Kelly.

Kelly betting, or betting full

...
Where is my Flying Car?
by PeterMcCluskey — 2018-10-15T18:39:38.010Z
Link: http://www.bayesianinvestor.com/blog/index.php/2018/10/14/where-is-my-flying-car/

Book review: Where Is My Flying Car? A Memoir of Future Past, by J.
Storrs Hall (aka Josh).

If you only read the first 3 chapters, you might imagine that this is
the history of just one industry (or the mysterious lack of an
industry).

But this book attributes the absence of that industry to a broad set of
problems that are keeping us poor. He looks at the post-1970 slowdown in
innovation that Cowen describes in The Great
Stagnation[1].
The two books agree on many symptoms, but describe the causes
differently: where Cowen says we ate the low hanging fruit, Josh says
it's due to someone "spraying
paraquat on the
low-hanging fruit".

The book is full of mostly good insights. It significantly changed my
opinion of the Great Stagnation.

The book jumps back and forth between polemics about the Great
Strangulation (with a bit too much outrage
porn), and nerdy
descriptions of engineering and piloting problems. I found those large
shifts in tone to be somewhat disorienting - it's like the author can't
decide whether he's an autistic youth who is eagerly describing his
latest obsession, or an angry old man complaining about how the world is
going to hell (I've met the author at
Foresight conferences, and got similar but
milder impressions there).

Josh's main explanation for the Great Strangulation is the rise of Green
fundamentalism[2], but he also describes other cultural /
political factors that seem related. But before looking at those, I'll
look in some depth at three industries that exemplify the Great
Strangulation.

The good old days of Science Fiction

The leading SF writers of the mid 20th century made predictions for
today that looked somewhat close to what we got in many areas, with a
big set of exceptions in the areas around transportation and space
exploration.

The absence of flying cars is used as an argument against futurists'
ability to predict technology. This can't be dismissed as just a minor
error of some obscure forecasters. It was a widespread vision of leading
technologists.

Josh provides a decent argument that we should treat that absence as a
clue to why U.S. economic growth slowed in the 1970s, and why growth is
still disappointing.

Were those SF writers clueless optimists, making mostly random
forecasting errors? No! Josh shows that for the least energy intensive
technologies, their optimism was about right, and the more energy
intensive the technolog

...
Exponential Altruism: A Strategy For A New World
by larriant — 2018-10-15T11:57:44.549Z
Link: https://medium.com/exponential-altruism/exponential-altruism-a-strategy-for-a-new-world-e3ad56794434

Our world faces multiple global crises that are growing in both severity and urgency. These are not limited to a climate and ecological catastrophe¹, a worldwide rise in nationalism², and an unsustainable growth-based economy that has led to extreme wealth inequality ³.

Simultaneously, we’re observing multiple revolutions in human behaviour and values. One example from the last century is that we’ve extended our empathy to a wider and wider spectrum of conscious beings through the civil, animal, LGBTQ and other rights movements.⁴

On top of that, rapid technological change is giving us more power than ever before, bringing us some hugely positive revolutions, but at the same time driving newfound crises. For example, advances in robotics and AI promise to free us from menial labour but also threaten to unleash widespread unemployment and further deepen inequality.

One important point is that technology isn’t just a neutral tool we use to achieve our aims.⁵ Instead it shapes our very values themselves. For example, our growing addiction to digital devices is slowly drowning us in a sea of distraction, making us less connected to one another and less aware of humanity’s situation.

These revolutions and crises are all interacting, cross-pollinating and influencing one another, culminating in the giant clusterfuck of what it is to be a human in the twenty first century. And it’s only going to get crazier.

Humanity walking the tightrope

Humanity’s current predicament is like that of a tightrope walker, and an inexperienced one at that. We’re teetering on a rope above a deep canyon, walking a narrow path between ruin and global flourishing.

If we fall into the abyss it will mean at best the deaths of billions of sentient beings and the suffering of billions more, and at worst the end of life on earth. Don’t slip, humanity. One false move and it’s all over — maybe forever.

Some astronomers have even hypothesised that the reason we haven’t yet encountered alien life (the Fermi paradox) is because all intelligent species inevitably face these challenges, but most of them fail and don’t live to tell the tale.⁶ The abyss is littered with the corpses of tightrope-walking civilisations who slipped and fell.

On the far side of the abyss, is the dramatically better world we should be aiming for. This world has the potential, at least, to provide fulfilling and meaningful lives for a...
Coordination Problems in Evolution: The Rise of Eukaryotes
by Martin Sustrik — 2018-10-15T06:18:47.576Z

Introduction

This is a series of posts about coordination problems, as they appear in the course of biological evolution. It is based on the book "The Major Transitions in Evolution" by John Maynard Smith and Eörs Szathmáry. Previous part, discussing Eigen's paradox as well as the origin of chromosomes, can be found here.

In this part we are going to look at the origin of eukaryotic cell, specifically at its acquisition of endosymbiotic organelles, and at the origin of multicellularity.

Prokaryota vs. eukryota

While all single-celled organisms may look like similar wiggly little creatures to us, there is a huge difference between prokaryota (like bacteria) and eukaryota (like protozoa or, for that matter, our own cells). The cell wall is different. The interior of the cell is different. One has rigid cell wall, in the other it's the cytoskeleton that holds the cell together. One has a single-origin DNA strand attached to the cell wall, the other has nucleus containing chromosomes. One has mitochondria and chloroplasts, the other does not. Even the mechanism of cell division is different. If we haven't known that we share part of the genome, it would be easy to make a mistake and believe that the life on Earth had originated at two separate occasions.

The transition from prokayotes to eukaryotes is likely the most complex transition in the entire course of evolution. It took two billion years to happen. More than the emergence of life itself.

All that being said, we are going to look only at a single part of the evolution of eukaryotes, namely at their acquisition of mitochondria. Mitochondria were free-living cells once. But then they've became an inseparable part of eukaryotic cell. Hence, back to the coordination problems!

How did it come to be that some cells started living within other cells? Well, assuming that flexible cell wall and phagocytosis evolved before the domestication of mitochondria, getting them inside wouldn't be a big problem. It happens each time one cell eats another.

What's more interesting is how did the guest cell survive and how did the cooperative behavior between the host and the guest evolve.

Symbiosis

Let's make a digression and think about symbiosis for a second. If we assume that there are only two strategies for the host (cultivate the symbiont or try to kill it) and two strategies for the symbiont (cooperate with the host or para

...
Logical Counterfactuals are low-res
by Shmi — 2018-10-15T03:36:32.380Z
Link: https://donerkebabphilosophy.wordpress.com/2018/10/14/logical-counterfactuals-are-low-res/

Cross-posted from my blog. Related to my several previous posts.

(Epistemic status: I have no idea why such an obvious observation is never even mentioned by the decision theorists. Or maybe it is, I have not seen it.)

A logical counterfactual, as described by Nate Soares:

In a setting with deterministic algorithmic agents, how (in general) should we evaluate the expected value of the hypothetical possible worlds in which the agent’s decision algorithm takes on different outputs, given that all but one of those worlds is logically impossible?

So we know that something happened in the past, but want to consider something else having happened instead under the same circumstances. Like a ubiquitous and familiar to everyone lament of a decision made: “I knew better than doing X and should have done Y instead”. It feels like there was a choice, and one could have made a different choice even while being the same person under the same circumstances.

Of course, a “deterministic algorithmic agent” would make the same decisions in the same circumstances, so what one really asks is “what kind of a [small] difference in the agent’s algorithm, and/or what kind of [minimally] different inputs into the algorithm’s decision-making process would result in a different output?” When phrased in this way, we are describing different “hypothetical possible worlds”. Some of these worlds correspond to different algorithms, and some to different inputs to the same algorithm, and in this way they are just as “logically possible” as the world where the agent we observed took an action we observed.

Why does it feel like there was both a logical possibility and impossibility, “the agent’s decision algorithm takes on different outputs”? Because the world we see is in low resolution! Like when you see a road from high up, it looks like a single lane:

But when you zoom in, you see something looking like this, with a maze of lanes, ramps and overpasses:

So how does one answer Nate’s question? (Which was not really a question, but maybe should have been) Zoom In! See what the agent’s algorithm is like, what hardware it runs on, what unseen-from-high-up inputs can cause it to switch lanes and take a different turn. So that the worlds which look counterfactual from high up (e.g. a car could have turned left, even though it has turned right) becomes physically unlikely when zoomed in (e.g. the car was...
Effective Altruism Book Review: Radical Abundance (Nanotechnology)
by Isnasene — 2018-10-14T23:57:36.099Z

Book Review: Radical Abundance

I. Introduction

As a materials engineering major with, roughly speaking, a year of full-time experience with molecular dynamics simulations, I have a special place in my heart for high impact materials both literal and figurative. As a lurker in effective altruism, I was delighted to see something potentially relevant to my experience show up. I made a post in the "Nanotechnolgy in EA" Facebook group expressing/re-iterating the following thoughts:

Nanotechnology will be chaotic and hard-to-predict, requiring massive advancements in computation before effective machines can be designed.
Our technology in terms of actually producing APM seems plausible but very uncertain at this point

From my post, I got some interesting comments suggesting an alternative to 1. along with a recommendation to read K. Eric Drexler's new book on atomically precise manufacturing (APM), Radical Abundance: How a Revolution in Nanotechnology Will Change Civilization. This, in part with some googling, gave me the impression that I was also wrong about 2.

The goals of this book report from least to most significance are: providing a decent summary of Radical Abundance, expressing my own thoughts on what I find to be the most salient aspects of nanotechnology, and clarifying possible common misconceptions that contribute to the frequent confusion about nanotechnology's place in effective altruism.

Anyway, Radical Abundance lightly discusses three inter-woven concepts: how APM would work, how APM could impact the world, and how APM can be achieved. Let's do this.

II. How APM Works

A. Advanced APM in a Nutshell

The description and understanding of APM can be best characterized by Drexler's own words:

APM-based materials processing technology will employ nanoscale mechanical devices that operate at high frequencies and produce patterns of atoms...think of an APM system as a kind of printer that builds objects out of atoms just as a printer builds images out of patterns of ink, constrained by a limited gamut, not of colors, but of output materials.

This clarifies two distinct points that I, as someone working in molecular dynamics and , would miss. First, despite my priors from experience with soft matter molecular dynamics, APM systems will be based on mechanically-inspired rigid mechanisms rather than biologically-inspired...
Outline of Metarationality, or much less than you wanted to know about postrationality
by Gordon Seidoh Worley — 2018-10-14T22:08:16.763Z

There was a recent discussion on Facebook that led to an ask for a description of postrationality that isn't framed in terms of how it's different from rationality (or rather perhaps more a challenge that such a thing could not be provided). I'm extra busy right now until at least the end of the year so I don't have a lot of time for philosophy and AI safety work, but I'd like to respond with at least an outline of a constructive description of post/meta-rationality. I'm not sure everyone who identifies as part of the metarationality movement would agree with my construction, but this is what I see as the core of our stance.

Fundamentally I think the core belief of metarationality is that epistemic circularity (a.k.a. the problem of the criterion, the problem of perception, the problem of finding the universal prior) necessitates metaphysical speculation, viz. we can't reliably say anything about the world and must instead make one or more guesses to overcome at least establishing the criterion for assessing truth. Further, since the criterion for knowing what is true is unreliably known, we must be choosing that criterion on some other basis than truth, and so instead view that prior criterion as coming from usefulness to some purpose we have.

None of this is radical; it's in fact all fairly standard philosophy. What makes metarationality what it is comes from the deep integration of this insight into our worldview. Rather than truth or some other criteria, telos (usefulness, purpose) is the highest value we can serve, not by choice, but by the trap of living inside the world and trying to understand it from experience that is necessarily tainted by it. The rest of our worldview falls out of updating our maps to reflect this core belief.

To say a little on this, when you realize the primacy of telos in how you make judgments about the world, you see that you have no reason to privilege any particular assessment criterion except in so far as it is useful to serve a purpose. Thus, for example, rationality is important to the purpose of predicting and understanding the world often because we, through experience, come to know it to be correlated with making predictions that later happen, but other criteria, like compellingness-of-story and willingness-to-life, may be better drivers in terms of creating the world we would lik...