trentmkelly/LessWrong-43k huggingface dataset:
Reflections on Larks’ 2020 AI alignment literature review
by Alex Flint — 2021-01-01T22:53:36.120Z

This work was supported by OAK, a monastic community in the Berkeley hills. It could not have been written without the daily love of living in this beautiful community.

Larks has once again evaluated a large fraction of this year’s research output in AI alignment. I am, as always, deeply impressed not just by the volume of his work but by Larks’ willingness to distill from these research summaries a variety of coherent theses on how AI alignment research is evolving and where individual donors might give money. I cannot emphasize enough how much more difficult this is than merely summarizing the entire year’s research output, and summarizing the entire year’s research output is certainly a heroic undertaking on its own!

I’d like to reflect briefly on a few points that came up as I read the post.

Depth

The work that I would most like to see funded is technical work that really moves our understanding of how to build beneficial AI systems forward. I will call this “depth”. It is unfortunately very difficult to quickly assess the depth of a given piece of research. Larks touches on this point when he discusses low-quality research:

[...] a considerable amount of low-quality work has been produced. For example, there are a lot of papers which can be accurately summarized as asserting “just use ML to learn ethics”. Furthermore, the conventional peer review system seems to be extremely bad at dealing with this issue.

Yet even among the papers that did get included in this year’s literature review, I suspect that there is a huge variation in depth, and I have no idea how to quickly assess which papers have it. Consider: which of the research outputs from, say, 2012 really moved our understanding of AI safety forward? How about from 2018? My sense is that these are fearsomely difficult questions to answer, even with several years’ hindisight.

Larks wisely does not fall into the trap of merely counting research outputs, or computing any other such simplistic metric. I imagine that he reads the papers and comes to an informed sense of their relative quality without relying on any single explicit metric. My own sense is that this is exactly the right way to do it. Yet the whole conclusion of the literature review does rest critically on this one key question: what is it that constitutes valuable research in the field of AI alignment? My sense is that depth is the most...
Is Free Will A Myth?
by Precious Oluwatobi Emmanuel — 2021-01-01T20:56:50.221Z

Free Will is basically the ability to choose between different possible courses of action unimpeded, according to Wikipedia (seriously, I shudder to imagine how the world will be without Wikipedia. It’s so useful) It is when you can choose to do what you want when you want without any form of opposition or duress.

For a long time, at least since the Enlightenment when philosophers began popping up like flies, one of the most often-debated questions has been whether humans truly have free will. On one hand, humans have very strong feelings towards freedom. We love it so much. We hold it up as an ideal. We have killed each other over it. This our strong sense of freedom always leads us to believe that we have free will. We even believe that civilization as we know it depends on this free will and losing it will have calamitous consequences. We have built many aspects of society on the assumption of that we have free will – from our code of ethics to democracy to aspects of our religion to criminal law. If we take away free will, society stands on shaky ground. As my friend P says “I believe free will exists. The alternatives if it doesn’t, are too scary to consider.”

But is this belief in free will just in our heads?

Science these days has been bolder in claiming that there is really no free will, that all human behavior, from love to anger to chivalry to everything else, can be explained through precise clockwork laws of cause and effect and chemical reactions. This claim can be traced to 150 years ago, when Charles Darwin published ‘On the Origin of Species’, where he laid out the theory of evolution. His cousin Francis Galton began to infer some radical implications. He said that if we have evolved, then our mental faculties (such as intelligence) were hereditary. And since we use these faculties (which, from evolution, some people are bound to have in a greater degree than others), our ability to choose is not free, it depends on our heredity*. Hence, according to his theory, Einstein's descendants have a greater free will than, say, Trump's.

This was obviously controversial then, but these days, not so much. Brain scanners have allowed us to peer into our brains when we make decisions and see intricate networks of neurons that fire when we make most of our decisions. Many point to the research and experiments of the American physiologist Benjamin Libet in the...
Thoughts on being mortal
by Joe Carlsmith — 2021-01-01T19:17:17.697Z

(Cross-posted from Hands and Cities)

(Content warning: discussion of death and intense pain)

This post is an amalgam of thoughts about death, prompted centrally by Atul Gawande’s book Being Mortal.

I.

Gawande’s book describes a lot of different people who are dying, at different speeds, with different kinds of suffering and support. I often found it piercingly sad, in a particular way I associate with death, and with people saying final goodbyes to life and to each other.

But for all the pain and degeneration in the book, the tone still feels somehow “smooth.” Gawande imbues the suffering of his subjects with a lot of dignity; he focuses, often, on what they can still do, and what they love. Most of the doctors and nurses he portrays are competent and well-intentioned.

Beneath this, though, one senses a roiling ocean of pain, confusion, fear, despair, and loss. Here I am reminded of Scott Alexander’s post on death in hospitals, which emphasizes, much more than Gawande, patients who are screaming, attacking doctors, trying to pull out their tubes, being restrained, going actively insane. See also Samuel Shem’s House of God, for a portrayal modern medicine much more macabre and ridiculous and horrifying than Gawande’s (I’ve only read the beginning, but it felt like the first few chapters were enough to get the gist). Anne Boyer’s The Undying — an account of her experience with breast cancer and its treatment — is also more relentless than Gawande in her focus on the pure pain — examining it from a huge variety of angles, searching for some sort of linguistic adequacy. She opens with words from the Iliad: “Not even if I had ten tongues and ten mouths.”

Sometimes, on the comparatively rare occasions when I experience even-somewhat-intense sickness or pain, I think back to descriptions like this, and am brought more directly into the huge number of subjective worlds filled with relentless, inescapable pain. These glimpses often feel like a sudden shaking off of a certain kind of fuzziness; a clarifying of something central to what’s really going on in the world; and it also comes with fear of just how helpless we can become.

(Boyer and Shem are also much more interested than Gawande in the corruptions of the medical system, but these won't be my focus here.)

I don’t fault Gawande the book’s smoothness: the topic is hard enough as it is, and his tasteful writing...
List of things I think are worth reading (last updated 3/1/2021)
by just_browsing — 2021-01-01T17:05:56.933Z

For personal reference, I'm going to keep track of things I read which I think are worth reading. By "worth reading" I specifically mean it scores high on this metric:

(insight gained)(time needed to absorb)⋅log(current popularity)

Each entry will have a title, link, date it was added, summary with salient examples / quotes, and outcome (what I personally got out of reading it). The entry is probably only worth your time if you think you would be better off achieving that outcome.

This list was last updated on 3/1/2021.

AI

The New Business of AI (and How It’s Different From Traditional Software) (added 1/1/2021) Summary: AI businesses can be modeled as a hybrid of software and service businesses. State-of-the-art AI is expensive (>= $100k) to train: you have to pay for the compute and (cloud) storage. Compute needed for current models is increasing faster than hardware (e.g. NVIDIA GPUs) is advancing. Edge cases make AI accuracy have an S-curve shape ("as much as 40-50% of intended functionality for AI products we’ve looked at can reside in the long tail of user intent"). Deploying to each customer can have high marginal cost if you are fine-tuning the model. ("Even customers that appear similar – two auto manufacturers doing defect detection, for example – may require substantially different training data, due to something as simple as the placement of video cameras on their assembly lines.") AI model architectures are usually public and data usually belongs to the customer, which makes profitability trickier. Data has "diseconomics of scale" because each subsequent edge case is more difficult to address. To make AI more practical: reduce model complexity (so less tuning is necessary), focus on low-complexity (but high-scale) tasks, enter the services market rather than pure tech ("running a taxi service rather than selling self-driving cars"). Outcome: I know a bit more about concerns when doing large-scale, practice-oriented AI.

The Bitter Lesson (added 3/1/2021) Summary: Progress in AI is driven by general-purpose techniques which scale with increased computation—the two main examples being search and learning. Outcome: Abstract understanding of what the frontier of modern ML research includes and doesn't include.

What Failure Looks Like: Distilling the Discussion (summary of original post) (added 3/1/2021) Summary: Two ways AI ...
Empiricism in NLP : Test Operate Text Exit (TOTE)
by ChristianKl — 2021-01-01T16:09:10.546Z

Epistemic status: Presentation of an established technique and history. I learned most of my NLP knowledge from Chris Mulzer who’s one of Bandler’s top students. The Origins of Neuro-Linguistic Programming by John Grinder and Frank Pucelik is my main source for the history.

What's NLP? In 1971 Frank Pucelik and Richard Bandler started teaching Fritz Perls’s Gestalt therapy in a group at the University of California, Santa Cruz where the two were in a Bachelor program of psychology. They were joined by John Grinder who was an assistant professor in Linguistics who had just finished writing his PHD thesis on the topic of deletions. As a linguist he had projects like modeling the language of the Tanzanian Wagogo tribe to be able to communicate with them. He had the idea that if he would create a model of how Fritz Perls was using language to get the results he got in his Gestalt therapy work, he should be able to achieve the same results.

Just like modeling the customs of the Wagogo tribe, the goal was to copy the linguistic patterns that were present in Perls’s work to be able to achieve the same results. As a side job Bandler was transcribing lectures of the late Fritz Perls, so they had plenty of video material to study. In addition to the videos Grinder could also study Bandler and Pucelik as they were doing their Gestalt work.

Modeling in NLP

Later they described the modeling process that they followed as a five-step process description of NLP Modeling:

1. Identification of and obtaining access to a model in the context where he or she is performing as a genius.

2. Unconscious uptake of model’s patterns without any attempt to understand them consciously.

3. Practice in a parallel context to replicate the pattern. The intention is to achieve a performance of the model’s patterns which is equal to the model him/herself.

4. Once the modeler can consistently reproduce the pattern in an applied fashion with equal results, the modeler begins the coding process.

5. Testing to determine if the pattern as coded can be transferred successfully to others who will in turn be able to get equally effective results from the coded results … and then, ultimately to teach those processes to others

Every Monday Bandler and Pucelik had their Gestalt group and the Thursday afterwards Grinder tried to do the same thing as their group to another group of students. They also got...
Forecasting Newsletter: December 2020
by NunoSempere — 2021-01-01T16:07:39.015Z

Highlights

Nigh unbeatable forecaster gives 85% chance that the newly identified COVID-19 strain is >30% more transmissible.
Prediction markets and betting platforms mostly resolved the election in favor of Biden already. However, new markets have been created about whether Trump will still be president in February. Those who believe he won't can still earn a circa 10% return.
Metaculus announces their AI Progress tournament, with $50,000 in rewards.

Index

Prediction Markets & Forecasting Platforms
US Presidential Election Betting
In the News
Negative Examples
Hard to Categorize
Long Content
Yearly Housekeeping

Sign-up or view past newsletters here.

Prediction Markets & Forecasting Platforms

Metaculus organized the AI Progress tournament, covered by a Forbes contributor here; rewards are hefty ($50,000 in total). Questions for the first round, which focuses on the state of AI exactly six months into the future, can be found here. In the discussion page for the first round, some commenters point out that the questions so far aren't that informative or intellectually stimulating. Metaculus has also partnered with The Economist for a series of events in 2021, and were mentioned in this article (sadly paywalled). They are also hiring.

@lxrjl, a moderator in the platform, has gathered a list of forecasts about Effective Altruism organisations which are currently on Metaculus. This includes a series introducing ACE (Animal Charity Evaluators) to using forecasting as a tool to inform their strategy, which Misha Yagudin and I came up with after some back-and-forth with ACE.

Augur partnered with the crypto currency/protocol $COVER/CoverProtocol to provide protection from losses in case Augur got hacked. In effect, traders could also bet for or against the proposition that a given Augur market will be hacked, and they could do this in a separate protocol. Traders could then use these bets to provide or acquire insurance (source, secondary source).

In an ironic twist of fate, soon after commencing the partnership CoverProtocol was hacked (details, secondary source). Even though the money was later returned, it seems that $COVER is being delisted from major cryptocurrency trading exchanges.

I added catnip.exchange to this list of prediction markets. More suggestions are welcome, and can be made by leaving a comment in the linked document.

During Christma...
Luna Lovegood and the Chamber of Secrets - Part 13
by lsusr — 2021-01-01T08:01:08.646Z

"Wait," said Luna, "This is the Lost Diadem of Ravenclaw. It makes the wearer smarter. You might want it."

Professor Quirrel took the diadem in his hands. He feinted as if to place it over his head.

"I am an Occlumens," said Professor Quirrel, "Ravenclaw's device rips the incoherence out of doublethink. If I were to place this device over my head I would be lucky if it did not shred my mind. Nice try."

Professor Quirrel tossed the diadem back to Luna. Luna kowtowed.

"I heard stories of the First Wizarding War. You never cared much for individual human beings but you were always very careful not to destroy wizardkind," said Luna, "I get the feeling you put some effort into protecting the universe."

"So?" said Professor Quirrel.

"You are bored. This plane is too small for you," said Luna.

You-Know-Who did not murder her.

"You should not be a villain," said Luna.

"If you tell me to be a hero then you will die painfully," said Professor Quirrel.

"You should be a god," said Luna.

Luna willingly bestowed the astrolabe to Professor Quirrel.

"Is that all?" said Professor Quirrel.

"Yes," said Luna.

"Avada Kedavra," said Professor Quirrel.

Luna collapsed. Professor Quirrel sheathed his wand. His slender skeleton fingers untangled the clockwork. Professor Quirrel unfolded the astrolabe around him. He ascended to a higher plane of existence.

Luna stepped out of the Forgotten Library. She held the Sword of Gryffindor in her left hand and Wanda in her right. She buried Wanda in Hagrid's pumpkin patch.

The final duel of Lockhart's tournament was that afternoon. Professor Flitwick refereed. Luna lost.

Clang. Luna dropped the Sword of Gryffindor on Professor Lockhart's empty chair. She sat down for dinner in her seat at the end of the Ravenclaw table. A student stood behind her.

"You fought well in Lockhart's dueling tournament," said Ginevra Weasley, "Why don't you try sitting with us Gryffindors for a change?"

The astrolabe displayed "7" on one dial and "0" on all the rest. A tall, slender snakelike figure stepped into Heaven's throne room where a god rested. The trespasser threw a tactical reality anchor like a javelin. It stuck into the wall behind the throne. The trespasser stabbed his second tactical reality anchor behind himself into the floor of the entrance.

"LET'S DUEL."

Credits

You may do whatever you want with this story. You may expand it. You may abridge it. You may retcon ...
Mistakes to want
by KatjaGrace — 2021-01-01T04:10:14.279Z

The end of the year is a classic time for reflecting on the year. And a classic part of reflecting is noticing mistakes you have made. I admit that I don’t relish this: having made mistakes, admitting to them, and looking at them further all pain me, and I find it hard to call things mistakes. It’s because to make a mistake would seem to be to make the world worse than it could have been, and thus to indelibly reduce the total goodness of the universe at the end of time, which feels like a big deal and the worst (only?) evil.

Possibly others don’t have the same mental hangups around such things as I do, or have thought clearly about this earlier, but just in case not, I’ll spell out how it actually isn’t bad at all, even in these terms (or at least offer some somewhat scattered thoughts on the matter).

Let us distinguish between two kinds of mistakes: ‘innocent mistakes’ and ‘evil mistakes’.

Suppose you made a mistake. Was it a mistake given what you knew and understood at the time? Then that’s an ‘evil mistake’. For instance, if you think kicking dogs is worse than not kicking them, and you kick one anyway to experience what badness feels like. (It sounds kind of strange to call it a mistake, since it was what you intended, but whatever.)

If your mistake was not an error given your understanding the time, then it’s an ‘innocent mistake’. For instance, if you made an unsuccessful choice about what projects to do, or if you hurt your brother’s feelings because it hasn’t occurred to you that he would be sensitive about his appearance. It is tempting to say that an innocent mistake wasn’t a mistake in the relevant sense. You couldn’t have really done better in your actual circumstances of limited knowledge and limited thought, at least not predictably. Your conscience should be clear, at least. You did the best you could, so you do not seem to deserve blame or regret.

Yet if blame and regret are for teaching people, then it seems you should have them, at least if they are the ways for you to notice or feel your mistake. For instance, perhaps your sister should say ‘you were so mean to Bob!’ or you should find your thoughts sadly dwelling on your choice of projects. On the other hand, there is not much point blaming the evil mistake-doer, unless your blame hurts them in a way that might put them off selfishly in the future, supposing that they haven’t come around. Ju...
AI Alignment, Philosophical Pluralism, and the Relevance of Non-Western Philosophy
by xuan — 2021-01-01T00:08:07.034Z

This is an extended transcript of the talk I gave at EAGxAsiaPacific 2020. In the talk, I present a somewhat critical take on how AI alignment has grown as a field, and how, from my perspective, it deserves considerably more philosophical and disciplinary diversity than it has enjoyed so far. I'm sharing it here in the hopes of generating discussion about the disciplinary and philosophical paradigms that (I understand) the AI alignment community to be rooted in, and whether or how we should move beyond them. Some sections cover introductory material that most people here are likely to be familiar with, so feel free to skip them.

The Talk

Hey everyone, my name is Xuan (IPA: ɕɥɛn), and I’m doctoral student at MIT doing cognitive AI research. Specifically I work on how we can infer the hidden structure of human motivations by modeling humans using probabilistic programs. Today though I’ll be talking about something that’s more in the background that informs my work, and that’s about AI alignment, philosophical pluralism, and the relevance of non-Western philosophy.

This talk will cover a lot of ground, so I want to give an overview to keep everyone oriented:

First, I’ll give a brief introduction to what AI alignment is, and why it likely matters as an effective cause area.
I’ll then highlight some of the philosophical tendencies of current AI alignment research, and argue that they reflect a relatively narrow set of philosophical views.
Given that these philosophical views may miss crucial considerations, this situation motivates the need for greater philosophical and disciplinary pluralism.
And then as a kind of proof by example, I’ll aim to demonstrate how non-Western philosophy might provide insight into several open problems in AI alignment research.

A brief introduction to AI alignment

So what is AI alignment? One way to cache it out is the project of building intelligent systems that robustly act in our collective interests — in other words, building AI that is aligned with our values. As many people in the EA community have argued, this is a highly impactful cause area if you believe the following:

AI will determine the future of our civilization, perhaps by replacing humanity as the most intelligent agents on this planet, or by having some other kind of transformative impact, like enabling authoritarian dystopias.
AI will be likely be misaligned with our collectiv

...