trentmkelly/LessWrong-43k huggingface dataset:
Quadratic Voting and Collusion
by leogao — 2021-11-17T00:19:15.737Z

Quadratic voting is a proposal for a voting system that ensures participants cast a number of votes proportional to the degree they care about the issue by making the marginal cost of each additional vote linearly increasing - see this post by Vitalik for an excellent introduction.

One major issue with QV is collusion - since the marginal cost of buying one vote is different for different people, if you could spread a number of votes out across multiple people, you could buy more votes for the same amount of money. For instance, suppose you and a friend have $100 each, and you care only about Cause A and they care only about Cause B, and neither of you care about any of the other causes up for vote. You could spend all of your $100 on A and they could spend all of theirs on B, or you could both agree to each spend $50 on A and $50 on B, which would net √2 times the votes for both A and B as opposed to the default.

The solution generally proposed in response to this issue is to ensure that the vote is truly secret, to the extent that you cannot even prove to anyone else who you voted for. The thinking is that this creates a prisoner's dilemma where by defecting, you manage to obtain both the $50 from your friend and also the full $100 from yourself for your own cause, and that because there is no way to confirm how you voted, there are no possible externalities to create incentives for not defecting.

Unfortunately, I have two objections to this solution, one theoretical and one practical. The theoretical objection is that if the two agents are able to accurately predict each others' actions and reason using FDT, then it is possible for the two agents to cooperate à la FairBot - this circumvents the inability to prove what you voted for after the fact by proving ahead of time what you will vote. The practical objection is that people tend to cooperate in prisoner's dilemmas a significant amount of the time anyways, and in general a lot of people tend to uphold promises they make even if the other party has no way to verify. I think there's even an argument to be made that the latter is at least partially a real world instance of the former.

I'm still optimistic that this might not be a showstopping problem in practice, since there is a limited pool of people who you would trust in practice, which puts a ceiling on how many times your vote can count...
Taking a simplified model
by dominicq — 2021-11-16T22:21:22.890Z

(cross-posted from my blog, Sunday Stopwatch)

This may be trivial but I didn't see it written down explicitly anywhere, so I'm writing it. There's a specific trick I've noticed that some people do that makes for better conversations, and I don't know if there's a name for it, but I think there should be, so I nominate taking a simplified model.

Say that you want to figure out some big societal issue, like how taxes should work, or how much people should be paid, or how much of a company should belong to the founder, and how much to the workers.

The usual way to go about this is to launch into specifics or into generalities. You either talk about the tax situation in, I don't know, post-war Netherlands and how the price of oil affected the shipping industry which then led to something which led to something else - I'm just fabricating stuff, but you know, a line of reasoning where you extract some lesson from highly specific events.

Or you talk about very general principles - like, we should all be treated equally, or we should reward those with ingenuity and drive, stuff like that.

But the cool trick is taking a simplified model: e.g. imagining a society of only ten people. And then running through the combinations, maybe extracting lessons. For example:

Imagine that in this society, John accidentally discovers oil (or an imaginary resource, doesn't matter). John works hard and makes life better for the rest of the society. How should he be rewarded (if at all)?

Imagine that John doesn't work very hard, but is just the first person out of many who might have plausibly discovered this new resource. Does this change anything?

Imagine that John works on this project single-handedly but wants to leave his heritage not to the village but to his two sons. How do we feel about that?

And so on. This works for a broad set of questions, and I've found that taking a simplified model always brings clarity. The trick is that it doesn't work in highly soldier-like debates, where such hypotheticals might be perceived as gotchas. But when trying to figure things out, reducing the number of variables works really good. You answer a simpler question, and only then make the question more like real life.
The Greedy Doctor Problem
by Jan — 2021-11-16T22:06:15.724Z
Link: https://universalprior.substack.com/p/the-greedy-doctor-problem

TL;DR: How to reason about people who are smarter than you. A few proposals, interspersed with reinforcement learning and humorous fiction. Ending on a surprising connection to logical inductors.

What is the Greedy Doctor Problem?

I came up with a neat little thought experiment[1] :

You are very rich and you want to make sure that you stay healthy. But you don't have any medical expertise and, therefore, you want to hire a medical professional to help you monitor your health and diagnose diseases. The medical professional is greedy, i.e. they want to charge you as much money as possible, and they do not (per se) care about your health. They only care about your health as far as they can get money from you. How can you design a payment scheme for the medical professional so that you actually get the ideal treatment?

Over the last few weeks, I've been walking around and bugging people with this question to see what they come up with. Here I want to share some of the things I learned in the process with you, as well as some potential answers. I don't think the question (as presented) is completely well-formed, so the first step to answering it is clarifying the setup and deconfusing the terms. Also, as is typical with thought experiments, I do not have a definitive "solution" and invite you (right now!) to try and come up with something yourself[2].

Some background on the problem

The subtext for the thought experiment is: How should you act when interacting with someone smarter than yourself? What can you say or do, when your interlocutor has thought of everything you might say and more? Should you trust someone's advice, when you can't pinpoint their motivation? As a Ph.D. student, I run into this problem around three to five times a week, when interacting with colleagues or my advisor[3].

After bugging a few people I learned that (of course) I'm not the first person to think about this question. In economics and political science, the situation is known as the principal-agent problem and is defined as "a conflict in priorities between a person or group and the representative authorized to act on their behalf. An agent may act in a way that is contrary to the best interests of the principal." This problem arises f.e. in the context of conflicts between corporate management and shareholders, clients and their lawyers, or elected officials and their voters. Wel...
Equity premium puzzles
by Ege Erdil — 2021-11-16T20:50:16.959Z
Link: https://www.metaculus.com/notebooks/8532/equity-premium-puzzles/

Crossposted from the Metaculus Journal with minor modifications. The original is available here, and you can forecast on the questions cited in the essay yourself after signing up.

From 1870 to 2015, stocks in the United States have returned around 8.4% per year in real (inflation adjusted) terms, while short-term government bonds (called Treasury bills in the US) have only returned around 2.1% per year. Though the United States is somewhat of an outlier in how well its equity markets have performed in this period, the picture in other countries isn't much different: stocks have much higher returns than bonds. This fact is called the equity premium puzzle, and I'll get into why exactly it's so puzzling in this essay. I note here that while the discussion about the equity premium puzzle often focuses on broad stock indices, the puzzle is actually present in all asset markets: junk bonds, real estate, foreign exchange, et cetera.

If the difference between 8.4% and 2.1% per year looks small to you, remember that the logic of compounding works such that at these rates of return an initial investment into stocks (with cash from dividends, buybacks, etc. reinvested into the same portfolio) would double in value roughly every 8 years, while the same investment into bonds would double every 33 years. If the time horizon is on the order of a few decades, the difference between the value of the two portfolios becomes enormous. This, and many other interesting findings, are available in The Rate of Return on Everything.

The most common explanation offered for this difference in returns is that stocks are riskier and therefore it's natural that they command a higher rate of return: people who buy stocks are compensated for the risk that they are taking on. To see why this explanation is by itself not sufficient to explain anything, it's enough to notice that a short position on the stock market is at least just as risky (and, in fact, more risky) than a long position, and yet it has an abysmal and deeply negative mean return. Aside from being short the S&P 500, you can probably think of plenty of other risks you could take for which you would not be rewarded: if you jump from the top floor of a tall building and hope to make it safely to the ground, you take an enormous risk and nobody will reward you for doing so.

A better explanation is not that people are rewarded fo...
Why I am no longer driven
by dominicq — 2021-11-16T20:43:00.927Z

(cross-posted from my blog, Sunday Stopwatch)

I used to be very driven. I'm talking "wake up at 4 AM, go run with my dog, take a cold shower, study for two hours, go work, meditate, cook, go to MMA practice, read a book before bed" driven. I was very ready to, you know, get after it. As you might notice from the title, this is no longer true. Why?

The first and obvious explanation is that I'm getting older. I have no way of confirming or denying this - it really could be the case that it's some mashup of biological processes that works independently of whatever my subjective experience of life is. But I'm 29, which doesn't seem old at all, so what are the other reasons?

The first reason I can think of is that normal, everyday life is actually difficult. This may be an entirely personal phenomenon, but I find that I semi-consciously categorize my life into two areas: "takes energy" and "doesn't take energy". Things that take energy are the big important things, like studying for some important professional certification or completing a big project or whatever. And things that don't take energy include all the backend admin work: the cooking, the cleaning, the handling of people, the coordinating, the who's gonna take the dog for a walk, the did you remember that we have a family lunch this weekend, and more.

You don't have to be particularly attentive to realize that the "doesn't take energy" category is completely bogus. It's a scam, one that I've willingly enrolled myself into. My general theory for why I've done this to myself is the implicit attitude that if you do things "right", things from this other category shouldn't take time or energy. Like, these are the basics, right, and the basics shouldn't be hard. I don't know, it doesn't make a lot of sense when I look into it, which is why it's a total scam. Things that don't take energy, take energy.

There's something more that's sorta related. Sometimes, you do a thing that "takes energy" and that's it, you've done it, done. But projects usually require maintenance after you do the project itself, so now you have to, again, do the backend admin work. If you study something, you have to apply it or at least go through some flash cards because otherwise, you're losing it. If you train some skill, same thing. If you get promoted, you have a harder baseline job. If you buy a house, congratulations on your second j...
Super intelligent AIs that don't require alignment
by Yair Halberstadt — 2021-11-16T19:55:01.258Z

An idealized computer program acts on a defined set of inputs, and produces a defined set of outputs.

When we talk about AI misalignment there's two fears:

The AI will hack the system and produce outputs that it's not theoretically meant to be able to produce at all.
The AI will produce incorrect or undesired outputs that lead to bad things happening.

For example consider a sandboxed AI that's only able to answer yes or no questions.

It might realize that by making sure specific instructions run on specific CPUs at specific times it can exploit a hardware bug to change the memory of another process running on the same hardware, allowing it to take a far wider range of actions. This would be misalignment type 1.

Or it might be asked whether it's safe to be given more power, lie and answer yes, and then take over the world. This would be misalignment type 2.

The 1st type of misalignment is fundamentally an engineering problem. It's the same problem as making sure your code and hardware has no security flaws that will allow a hacker to take over the system. It's an extremely hard problem, but one which we have a lot of experience solving, and we have a rich set of tools which are getting better all the time. For example code is now being in written in much safer languages like rust instead of C++, and automated theorem provers are increasingly being used to verify properties of your code.

I'm not saying this is easy. As any security expert will tell you, against a clever enough hacker it's almost impossible. But it's the kind of impossible we know how to shut up and do. We know how to write toy programs which are provably bug free. Writing real bug free programs is the same thing, just much much harder. When crunch time comes, we've got a fighting chance of getting this right.

The second type of misalignment is much more insidious. We don't really have any clue how to get an AI to only give the responses we want. Even if we knew that humanities future hung in the balance, we still wouldn't have a clue where to even start solving this one.

So instead of solving it, let's dissolve it.

The second type of misalignment only occurs because the AI has slack in how it responds. Since it can give multiple answers, it can pick one based on it's inner values. And we have no way to ensure those inner values correspond with our own.

What if we only asked the AI questions where:

There's

...
Why Save The Drowning Child: Ethics Vs Theory
by Raymond Douglas — 2021-11-16T19:07:00.612Z

(cross-posted on the EA Forum)

A consequentialist, a deontologist, and a virtue ethicist are walking through a forest discussing the fundamental nature of morality. So heated is their disagreement that they startle a nearby small child, who falls into a nearby small pond. At once, they all cease their arguing and leap in to save the child. But why?

Recap: The Big Three Schools

The Consequentialist thinks ethics is about consequences. She thinks that a drowned child is a bad consequence, and a saved child is a good one. She thinks you should save a drowning child because the end result will be a child that hasn't drowned.

The Deontologist thinks ethics is about rules. He thinks that one of those rules is "whenever you get the chance to save someone's life for very little cost, take it". He thinks you should save a drowning child because otherwise you're breaking a moral rule.

The Virtue Ethicist thinks ethics is about virtues. They think that saving a drowning child is courageous and kind. They think that you should save a drowning child because you should be the kind of person who saves a drowning child.

This isn't doing credit to the range and nuances of any of them, but it will do for now. What matters isn't the specifics of the theory but rather that

all three have fundamentally different ideas about what ethics is about (and these aren't even all the ideas)
they still all save the child

Although they disagree about some very fundamental questions, they seem to broadly agree on a lot of actions. Is there anything they can learn from each other?

(For the sake of brevity I'm going to focus specifically on the interplay between Consequence and Virtue)

The Consequence Of Virtue

Why might a Consequentialist care about Virtue? Well, first of all, our Consequentialist would like to give a full account of ethics, and it seems to her that "what kind of person should I be?" is a question of ethics. Even if the answer is basically 'the kind of person who produces good consequences', it would be good to be able to say what kind of person that is.

Second, whether or not she believes that virtue is what ethics is about, she can't help but notice that helping people become wiser, kinder, more courageous, and more prudent, tends to have good consequences.

Let's make a distinction: Virtue Ethics is about virtue as a foundation for ethics, and Virtue Theory is about providing a theory f...
Two Stupid AI Alignment Ideas
by aphyer — 2021-11-16T16:13:20.134Z

Epistemic status: this is not my field. I am unfamiliar with any research in it beyond what I've seen on LW. These are the best ideas I could come up with when stopping to think about the problem, and aren't things I've seen on LW, but I don't imagine that they're original and I would hope that e.g. MIRI has already thought about them. If there are layman-accessible arguments explaining why these ideas are stupid, I'd appreciate hearing them; if e.g. MIRI has already looked into these ideas and written papers about them, I'd appreciate links.

Safe Testing Spaces via Extremely High Discount Rates.

Hyperbolic discounting is unstable under self-modification. If you are an agent that values 1 unit of utility at time T as follows:

Current time: 1

Current time + 1: 0.1

Current time + 2: 0.09

and you are faced with a choice between 1 unit of utility at time T+1 or 5 units of utility at time T+2, you would currently prefer the 5 units at time T+2, but your T+1-self will predictably prefer the 1 unit at time T+1. You will therefore be incentivized to modify your future self to remove this hyperbolic discounting. If you are an AI, and capable of self-modification, and we give you a hyperbolic discount curve, you will self-modify to remove that.

Exponential discounting is stable under self-modification. So long as you have a consistent exponential discount rate, a valuation of:

Current time: 1

Current time + 1: (1-e)

Current time + 2: (1-e)^2

is stable under reflection. If you are an AI, and capable of self-modification, and we give you an exponential discount rate, you will retain it.

Suppose that we have a paperclip-making AI in a testing environment. We offer it the opportunity to steal resources and make 10 extra paperclips, in a way that we think our safety precautions should prevent. It does not take that opportunity. Does that mean our safety precautions worked?

No, because the AI may be able to predict the indirect effects of its actions. If it steals those resources, it will make 10 extra paperclips, and then we will turn it off and it will make no more. If it does not steal those resources, we will trust it, and then next week it can take over the world and convert the mass of Earth into 6*10^27 paperclips. Since 6*10^27 > 10, it will take this option.

But suppose that we have given the ...
[linkpost] Project Blueprint: ‘Measuring and then maximally reversing the quantified biological age of my organs’
by matteodimaio — 2021-11-16T02:48:27.913Z

Here’s a blog post in which Kernel (which aims to make mass-market Brain imaging headsets) CEO Bryan Johnson details his new endeavor “Project Blueprint”, in which he aims to spend upwards of $1m a year measuring the biological age of 70+ of his body organs in an attempt to “maximally reverse” body aging. The process relies on DNA methylation testing. The second link is to the dashboard for the project. Bryan’s “gamified” the process, gaining an “age reduction point” for each year a validated marker is below his chronological age.

He writes:

“Many people believe that anti-aging, the fountain-of-youth radical type, is decades if not centuries away (if ever) and will arrive in the form of a magic pill. Blueprint is a stock ticker of sorts that will reveal, through the tracking of my biological versus chronological age, the status of today's anti-aging science (even if an N=1 for now).”

Right now, Johnson’s project is expensive and extensive—but ought to provide a good test case to watch for the state of the art of anti-aging practice, when budget and effort are not worries.

Most of the markers are currently sitting at “TBM”, but more are coming online each month.

Link one

Link two
A positive case for how we might succeed at prosaic AI alignment
by evhub — 2021-11-16T01:49:48.464Z

This post is my attempt at something like a response to Eliezer Yudkowsky’s recent discussion on AGI interventions.

I tend to be relatively pessimistic overall about humanity’s chances at avoiding AI existential risk. Contrary to some others that share my pessimism, however—Eliezer Yudkowsky, in particular—I believe that there is a clear path forward for how we might succeed within the current prosaic paradigm (that is, the current machine learning paradigm) that looks plausible and has no fundamental obstacles.

In the comments on Eliezer’s discussion, this point about whether there exists any coherent story for prosaic AI alignment success came up multiple times. From Rob Bensinger:

I think it's pretty important here to focus on the object-level. Even if you think the goodness of these particular research directions isn't cruxy (because there's a huge list of other things you find promising, and your view is mainly about the list as a whole rather than about any particular items on it), I still think it's super important for us to focus on object-level examples, since this will probably help draw out what the generators for the disagreement are.

In that spirit, I’d like to provide my own object-level story for how prosaic AI alignment might work out well.

Of course, any specific story for how we might succeed is going to be wrong simply because it specifies a bunch of details, and this is such a specific story. The point, however, is that the fact that we can write plausible stories for how we might succeed that don’t run into any fundamental obstacles implies that the problem isn’t “we don’t even know how we could possibly succeed” but rather “we know some ways in which we might succeed, but they all require a bunch of hard stuff that we have to actually execute on,” which I think is a pretty different place to be.

One thing I do agree with Eliezer on, however, is that, when you’re playing from behind—as I think we are—you play for variance. That means embracing strategies that might not work in expectation, but that have long tails in the positive direction, and I definitely see my picture here as falling into that category.

Furthermore, as one should probably expect with any full roadmap for solving such a complicated problem, there’s still a lot left out of my picture—especially my intuitions for why I think each of these steps is actually plausible. I am ...