trentmkelly/LessWrong-43k huggingface dataset:
Harry Potter and methods of rationality alternative ending.
by Klen Salubri — 2020-07-01T18:51:09.629Z

*Spoiler Alert*

As it is clear from the title, following text deals with the plot of the book. I strongly recommend to anyone no familiar with the subject to read the book first and form a personal opinion. Also, I will be glad to see in the discussion anyone who were directly involved in creation of this artwork.

First of all, I want to thank the community and its creator and express deep respect and gratitude for the work done. I got incomparable pleasure from studying unexpected plot intricacies, consistent and well-structured thoughts of the protagonist, as well as references to outstanding works, many of which I, to my shame, did not know.

However, the finale of the book left more questions than answers.

What will Potter's friends do when they realize that he made them orphans and did not even give the opportunity to say goodbye to parents? Why didn't he even try to persuade his fellows to confront their parents and do their best to force them to change their minds and traditions?

What will Hermione do when she will realize that Harry could stop her, yet instead left her defenceless and then turned her into golem and personal bodyguard with help of their enemy?

Will she being literally walking dead be able to give birth? Will she give birth to some kind of unicorn trolls or nearly invincible magicians on steroids? Will her descendants even try to build diplomatic relations with humans or will turn into immortal fascist?

What will other mages do, when they get to know that Harry was so afraid to loose his inner dark counselor which is part of Voldie’s soul that he spared You-Know-Who and hid him under his own and Hermione’s protection? Somehow reminds me story of Isildur from LOTR.

Maybe i'm too optimistic, but i just hope that current ending is just possible future that Hermione had seen when she had lost her consciousness during fight with slytherins. Maybe we could do better. I dunno, maybe she could ask Hufflepuff (which are depicted as local counter-intelligence service) to help her. There’s is always a better solution.

Best regards, Klen.
What's the most easy, fast, efficient way to create and maintain a personal Blog?
by Ferdinand Cachoeira — 2020-07-01T18:51:05.904Z

I have seen a lot of members here have personal blogs and encourage creating anti-Ra content. Although I have some experience on creating python programs/android apps, I don't know much about web development. I started watching some html/css/javascript tutorials, but there are so many tutorials and so much babble on the way, I wouldn't like to waste too much time to start.

If your past-self was looking to create a very simple text blog, what material would you send him to learn from? If there's anything you can recommend me, feel free to do it
Second-Order Existential Risk
by Ideopunk — 2020-07-01T18:46:52.140Z

Cross-posted.

[Epistemic status: Low confidence]

[I haven’t seen this discussed elsewhere, though there might be overlap with Bostrom’s “crunches” and “shrieks”]

How important is creating the conditions to fix existential risks versus actually fixing existential risks?

We can somewhat disentangle these. Let’s say there are two levels to “solving existential risk.” The first level includes the elements deliberately ‘aimed’ at solving existential risk. This includes researchers, their assistants, their funding. On the second level are the social factors that come together to produce humans and institutions with the knowledge and skills to even be able to contribute to existential risk. This second level includes things like “a society that encourages curiosity” or “continuity of knowledge” or “a shared philosophy that lends itself to thinking in terms of things like existential risk (humanism?).” All of these have numerous other benefits to society, and they could maybe be summarized as “create enough surplus to enable long-term thinking.”

Another attribute of this second level is that these are all conditions that allow us to tackle existential risk. Here are a few more of these conditions:

Humans continue to reproduce.
Humans tend to see a stable career as their preferred life-path.
Research institutions exist.
Status is allocated to researchers and their institutions.

If any of these were reversed, it seems conceivable that our capacity to deal with existential risk would be heavily impacted. Is there a non-negligible risk of these conditions reversing? If so, then perhaps research should be put into dealing with this “second-order” existential risk (population collapse, civilization collapse) the same way it’s put into dealing with “first-order” existential risk (nuclear war, AI alignment).

Reasons why second-order x-risk might be a real concern:

The above conditions are not universal and thus can’t be taken for granted.
Some of these conditions are historical innovations and thus can’t be taken for granted.
The continued survival of our institutions could be based more on inertia than any real strength.
Civilizations do collapse. Due to our increased

...
How to Find Sources in an Unreliable World
by Elizabeth — 2020-07-01T18:30:02.468Z

I spent a long time stalling on this post because I was framing the problem as “how to choose a book (or paper. Whatever)?”. The point of my project is to be able to get to correct models even from bad starting places, and part of the reason for that goal is that assessing a work often requires the same skills/knowledge you were hoping to get from said work. You can’t identify a good book in a field until you’ve read several. But improving your starting place does save time, so I should talk about how to choose a starting place.

One difficulty is that this process is heavily adversarial. A lot of people want you to believe a particular thing, and a larger set don’t care what you believe as long as you find your truth via their amazon affiliate link (full disclosure: I use amazon affiliate links on this blog). The latter group fills me with anger and sadness; at least the people trying to convert you believe in something (maybe even the thing they’re trying to convince you of). The link farmers are just polluting the commons.

With those difficulties in mind, here are some heuristics for finding good starting places.

Search “best book TOPIC” on google

Most of what you find will be useless listicles. If you want to save time, ignore everything on a dedicated recommendation site that isn’t five books.

If you want to evaluate a list, look for a list author with deep models on both the problem they are trying to address, and why each book in particular helps educate on that problem. Examples:

Fivebooks’ Best Books on Learning from the Great Depression

Fivebooks’ Best Books on Evolution

A bad list will typically have a topic rather than a question they are trying to answer, and will talk about why books they recommend are generically good, rather than how they address a particular issue. Quoting consumer reviews is an extremely bad sign and I’ve never seen it done without being content farming.

Jerry Jenkin’s The 12 Best Books on Writing I’ve Ever Read

Culture Trip’s 7 Must-Read Books Based on India’s History

Search for your topic on Google Scholar

Look at highly cited papers. Even if they’re wrong, they’re probably important for understanding what else you read.

Look at what they cite or are cited by

Especially keep an eye out for review articles

Search for web forums on your topic (easy mode: just check reddit). Sometimes these will have intro guides

...
Evan Hubinger on Inner Alignment, Outer Alignment, and Proposals for Building Safe Advanced AI
by Palus Astra — 2020-07-01T17:30:48.351Z

It’s well-established in the AI alignment literature what happens when an AI system learns or is given an objective that doesn’t fully capture what we want. Human preferences and values are inevitably left out and the AI, likely being a powerful optimizer, will take advantage of the dimensions of freedom afforded by the misspecified objective and set them to extreme values. This may allow for better optimization on the goals in the objective function, but can have catastrophic consequences for human preferences and values the system fails to consider. Is it possible for misalignment to also occur between the model being trained and the objective function used for training? The answer looks like yes. Evan Hubinger from the Machine Intelligence Research Institute joins us on this episode of the AI Alignment Podcast to discuss how to ensure alignment between a model being trained and the objective function used to train it, as well as to evaluate three proposals for building safe advanced AI.

Topics discussed in this episode include:

Inner and outer alignment
How and why inner alignment can fail
Training competitiveness and performance competitiveness
Evaluating imitative amplification, AI safety via debate, and microscope AI

You can find the page for this podcast here

Transcript:

Lucas Perry: Welcome to the AI Alignment Podcast. I’m Lucas Perry. Today we have a conversation with Evan Hubinger about ideas in two works of his: An overview of 11 proposals for building safe advanced AI and Risks from Learned Optimization in Advanced Machine Learning Systems. Some of the ideas covered in this podcast include inner alignment, outer alignment, training competitiveness, performance competitiveness, and how we can evaluate some highlighted proposals for safe advanced AI with these criteria. We especially focus in on the problem of inner alignment and go into quite a bit of detail on that. This podcast is a bit jargony, but if you don’t have a background in computer science, don’t worry. I don’t have a background in it either and Evan did an excellent job making this episode accessible. Whether you’re an AI alignment researcher or not, I think you’ll find this episode quite informative and digestible. I learned a lot about a whole other dimension of alignment that I previously wasn’t aware of, and feel ...
[AN #106]: Evaluating generalization ability of learned reward models
by Rohin Shah — 2020-07-01T17:20:02.883Z

Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.

Audio version here (may not be up yet).

HIGHLIGHTS

Quantifying Differences in Reward Functions (Adam Gleave et al) (summarized by Rohin): Current work on reward learning typically evaluates the learned reward models by training a policy to optimize the learned reward, and seeing how well that policy performs according to the true reward. However, this only tests how well the reward works in the particular environment you test in, and doesn’t tell you how well the reward will generalize. For example, suppose the user loves apricots, likes plums, but hates durians. A reward that has apricots > durians > plums works perfectly -- until the store runs out of apricots, in which case it buys the hated durian.

So, it seems like we should evaluate reward functions directly, rather than looking at their optimal policies. This paper proposes Equivalent-Policy Invariant Comparison (EPIC), which can compare two reward functions while ignoring any potential shaping that doesn’t affect the optimal policy.

EPIC is parameterized by a distribution of states and actions DS and DA, as well as a distribution DT over transitions (s, a, s’). The first step is to find canonical versions of the two rewards to be compared, such that they have expected zero reward over DS and DA, and any potential shaping is removed. Then, we look at the reward each of these would assign to transitions in DT, and compute the Pearson correlation. This is transformed to be in the range [0, 1], giving the EPIC distance.

The authors prove that EPIC is a pseudometric, that is, it behaves like a distance function, except that it is possible for EPIC(R1, R2) to be zero even if R1 and R2 are different. This is desirable, since if R1 and R2 differ by a potential shaping function, then their optimal policies are guaranteed to be the same regardless of transition dynamics, and so we should report the “distance” between them to be zero.

The authors show how to approximately compute the EPIC distance in high dimensional environments, and run experiments to showcase EPIC’s properties. Their first experiment de...
PSA: Cars don't have 'blindspots'
by romeostevensit — 2020-07-01T17:04:06.690Z

Not once in my life have I gone to drive someone's car and not seen incorrect mirror setup. Infographic here.

With correct adjustment, as cars leave your rear view mirror they enter your side view, and as they exit your side view they enter your peripheral vision. You still need to move your head to check for motorcycles and for maintaining situational awareness, but properly adjusted mirrors drastically reduces being surprised.
Forecasting Newsletter. June 2020.
by NunoSempere — 2020-07-01T09:46:04.555Z

Highlights

Facebook launches Forecast, a community for crowdsourced predictions.

Foretell, a forecasting tournament by the Center for Security and Emerging Technology, is now open.

A Preliminary Look at Metaculus and Expert Forecasts: Metaculus forecasters do better.

Sign up here, view this newsletter on the EA Forum here, or browse past newsletters here

Index

Highlights.

In the News.

Prediction Markets & Forecasting Platforms.

Negative Examples.

Hard to Categorize.

Long Content.

In the News.

Facebook releases a forecasting app (link to the app, press release, TechCrunch take, hot-takes). The release comes before Augur v2 launches, and it is easy to speculate that it might end up being combined with Facebook's stablecoin, Libra.

The Economist has a new electoral model out (article, model) which gives Trump an 11% chance of winning reelection. Given that Andrew Gelman was involved, I'm hesitant to criticize it, but it seems a tad overconfident. See here for Gelman addressing objections similar to my own.

COVID-19 vaccine before US election. Analysts see White House pushing through vaccine approval to bolster Trump's chances of reelection before voters head to polls. "All the datapoints we've collected make me think we're going to get a vaccine prior to the election," Jared Holz, a health-care strategist with Jefferies, said in a phone interview. The current administration is "incredibly incentivized to approve at least one of these vaccines before Nov. 3."

"Israeli Central Bank Forecasting Gets Real During Pandemic". Israeli Central Bank is using data to which it has real-time access, like credit-card spending, instead of lagging indicators.

Google produces wind schedules for wind farms. "The result has been a 20 percent increase in revenue for wind farms". See here for essentially the same thing on solar forecasting.

Survey of macroeconomic researchers predicts economic recovery will take years, reports 538.

Prediction Markets & Forecasting platforms.

Ordered in subjective order of importance:

Foretell, a forecasting tournament by the Center for Security and Emerging Technology, is now open. I find the thought heartening that this might end up influencing bona-fide politicians.

Metaculus

posted A Preliminary Look at Metaculus and Expert Forecasts: Metaculus forecasters do better, and the piece is a nice reference point.

was fea

...
Inviting Curated Authors to Give 5-Min Online Talks
by Ben Pace — 2020-07-01T01:05:39.794Z

If you've written a post that's been curated by the LessWrong team, then you're eligible to give a talk at one of our upcoming weekend events. If you'd like to give one, here's the form to fill out your availability.

The events looks something like this:

20-40 LessWrong users show up.
3-5 curated authors give 5 minute talks over zoom
After each talk, we have 5-10 mins of Q&A from the attendees.
After the talks (which last ~1 hour), we split into breakout rooms for a 30-60 min hangout and discuss the ideas.
After the event, Ben and Jacob make transcripts of the talks and publish them.

Speakers so far have been Vaniver, TurnTrout, Abram Demski, John Wentworth, Eukaryote, Othonormal, Charlie Steiner, Alkjash, Daniel Kokotajlo.

Examples of transcripts: Alkjash, Abram.

Your talk can discuss a post, a comment, or a new idea you've not been able to write down yet.

Here's the form.
Situating LessWrong in contemporary philosophy: An interview with Jon Livengood
by Suspended Reason — 2020-07-01T00:37:00.695Z

Jonathan Livengood is a current associate professor of philosophy at Urbana-Champaign, who hung around the first iteration of LessWrong in the late 2000s and early 2010s as a graduate student at the University of Pittsburgh, where he was writing a dissertation on causal inference under John Norton, Peter Spirtes, and Edouard Machery. He also blogs at the excellent Unshielded Colliders.

One of the central criticisms of mainstream philosophy at LessWrong has always been aimed at its tendency (sometimes called "conceptual analysis") to reify cognitive concepts ir linguistic terms—to perceive them, in other words, as having a simple, one-to-one correspondence with regularities or features of the world (see "Taboo Your Words," "Concepts Don't Work That Way," "LessWrong Rationality and Mainstream Philosophy"). Livengood and I discuss the state of conceptual analysis in philosophy departments, and its recent replacement by "conceptual engineering." We also discuss some of the problems of academic philosophy, continuities between LessWrong and analytic thought, and the status of insights like Bayesianism, verificationism, the pragmatist motto "making beliefs pay rent," and Korzybski's "map and territory."

Some context for this interview can be found in an earlier post, "Conceptual engineering: The revolution in philosophy you've never heard of," as well as in short pieces on my personal blog about LessWrong vs. contemporary philosophy (1, 2). But I'll add some definitions up front, to give context to our conversation for those who haven't read the backlog:

conceptual analysis: a method of philosophy in which a concept is assumed to have necessary and sufficient criteria which can be described simply and robustly; for instance, there might be a set of criteria which elegantly compress and describe all native-speaker utterances of a concept like "truth." Typically, a philosophical opponent will rebut a proposed set of criteria by offering counterexamples: cases in which a use-case of a concept does not meet the proposed criteria (or in which a non-member of the conceptual does meet them). Michael Bishop's "The Possibility of Conceptual Clarity in Philosophy" is an excellent, if skeptical, introduction.

conceptual engineering: a recently ...
AvE: Assistance via Empowerment
by FactorialCode — 2020-06-30T22:07:50.220Z
Link: https://arxiv.org/abs/2006.14796

This might be relevant to the AI safety crowd. Key quote:

"Our key insight is that agents can assist humans without inferring their goals or limiting their autonomy by instead increasing the human’s controllability of their environment – in other words, their ability to affect the environment through actions. We capture this via empowerment, an information-theoretic quantity that is a measure of the controllability of a state through calculating the logarithm of the number of possible distinguishable future states that are reachable from the initial state [41]. In our method, Assistance via Empowerment (AvE), we formalize the learning of assistive agents as an augmentation of reinforcement learning with a measure of human empowerment. The intuition behind our method is that by prioritizing agent actions that increase the human’s empowerment, we are enabling the human to more easily reach whichever goal they want. Thus, we are assisting the human without information about their goal[...]Without any information or prior assumptions about the human’s goals or intentions, our agents can still learn to assist humans."[Emphasis and omissions are mine]

From the abstract: One difficulty in using artificial agents for human-assistive applications lies in the challenge of accurately assisting with a person's goal(s). Existing methods tend to rely on inferring the human's goal, which is challenging when there are many potential goals or when the set of candidate goals is difficult to identify. We propose a new paradigm for assistance by instead increasing the human's ability to control their environment, and formalize this approach by augmenting reinforcement learning with human empowerment. This task-agnostic objective preserves the person's autonomy and ability to achieve any eventual state. We test our approach against assistance based on goal inference, highlighting scenarios where our method overcomes failure modes stemming from goal ambiguity or misspecification. As existing methods for estimating empowerment in continuous domains are computationally hard, precluding its use in real time learned assistance, we also propose an efficient empowerment-inspired proxy metric. Using this, we are able to successfully demonstrate our method in a shared autonomy user study for a challenging simulated teleoperation task with human-in-the-loop training.

How does this fit in wit

...
I am Bad at Flirting; Realizing that by Noticing Confusion
by snog toddgrass — 2020-06-30T20:05:29.151Z

This post is about applying rationality to my dating life. It is gooey and rich in self-disclosure. But it was a great triumph over motivated reasoning.

I notice my confusion

When I was 22 I that my romances began mainly when I was busy. They began most often, paradoxically, when I was unable to pay attention to my interest in early courtship. The observation was interesting but did not then replace my preferred explanation for romantic successes and failures.

At the time I subscribed to, what I call, the “mental health” hypothesis[1]. When I was healthy and confident women could sense it and chose to date me. Sometimes I became “unhealthy” and women became disinterested. Partners sensed “unhealth” by uncontrollable micro-cues. Thinking about dating will only make you more self-conscious and worsen your chances. This explanation agreed with the results of my favorite epistemology: asking my female friends what they think happened (critique of this method). At the time I had not learned Bayes theorem, Occamian reasoning or any social influence literature, so the “mental health” hypothesis seemed plausible. Besides, I liked my earlier beliefs. I just need to be “better” then women will like me. It was a simple, appealing narrative. If Iost that explanation, the alternatives might be “you must be a jerk” or “you are unattractive”, which scared me. Besides, Me_2016 did not know the benefits of saying “oops”.

By 2019, the “mental health” explanation was under increasing strain. The conventional advice was to make your life full by exercising, working on your mental health, improving your career, and the partners will come”. The problem was that it did not work, despite soundind wise. In the Summer of 2019 I was healthy,4; I had a good job, strong friendship networks, passionate hobbies, and plenty of exercise. It was the ideal time to seek a partner and I put great energy into the search. Despite my apparent health, I had the worst results in years. When I pointed this out, my poor friends could only shake their heads and think “there he goes, trying to solve the unsolvable.”

The nail in the coffin of the “mental health” hypothesis came that winter. I lost my job, was briefly jailed in a foreign country, and e...