trentmkelly/LessWrong-43k huggingface dataset:
Complex Behavior from Simple (Sub)Agents
by moridinamael — 2019-05-10T21:44:04.895Z

Epistemic Status: Simultaneously this is work that took me a long time and a lot of thought, and also a playful and highly speculative investigation. Consider taking this seriously but not literally.

Introduction

Take a simple agent (GitHub; Python), with no capacity for learning, that exists on a 2D plane. It shares the plane with other agents and objects, to be described shortly.

The agent intrinsically doesn't want anything. But it can be assigned goal-like objects, which one might view as subagents. Each individual goal-like subagent can possess a simple preference, such as a desire to reach a certain region of space, or a desire to avoid a certain point.

The goal-like subagents can also vary in the degree to which they remain satisfied. Some might be permanently satisfied after achieving their goal once; some might quickly become unsatisfied again after a few timesteps.

Every timestep, the agent considers ten random movements of unit-distance, and executes the movement corresponding to the highest expected valence being reported by its goal-like subagents, in a winner-take-all fashion.

Even with such an intentionally simplistic model, a surprising and illuminating level of behavioral complexity can arise.

Sections 1-8 concern interesting or amusing behaviors exhibited by the model.

Sections 8-12 outline future directions for the model and ruminations on human behavior.

1. Baseline

In this image, the path of the agent is painted with points, the color of the points changing slowly with the passage of time. This agent possesses three subagents with preferences for reaching the three green circles, and a fourth mild preference for avoiding the red circle.

Once it comes within a set distance of one of the green circles, the corresponding subagent is satisfied, and thus the movement with the highest expected valence switches to the next-highest valence goal. The satisfaction gradually wears off, and the agent begins to be drawn to the goal again. Thus, the agent moves inexorably around the triangle of green circles, sometimes in a circuit, sometimes backtracking.

2. "Ugh field"

If the aversion to the red circle is amplified above a certain threshold, this behavior results. The subagent with preferences for reaching the top green circle still exists, but it will never be satisfied, because expected negative valence of passing near the red circle is too hig...
Disincentives for participating on LW/AF
by Wei Dai — 2019-05-10T19:46:36.010Z

I was in a research retreat recently with many AI alignment researchers, and found that the vast majority of them do not participate (post or comment) on LW/AF or participate to a much lesser extent than I would prefer. It seems important to bring this up and talk about whether we can do something about it. Unfortunately I didn't get a chance to ask them about why that is the case as there were other things to talk about, so I'm going to have to speculate here based on my personal experiences and folk psychology. (Perhaps the LW team could conduct a survey and get a better picture than this.)

Criticism that feels overly harsh or is otherwise psychologically unpleasant

Downvotes

Not getting enough upvotes as one feels deserved

Not getting enough engagement

More adversarial (zero-sum) nature of public discussion / preferring private discussions for more collaborative nature

Feeling like "losing" a debate when the other person gets more upvotes than you

More effort needed to write comments than to talk to people IRL

Not real-time / time lag between replies

Feeling ignored when someone stops responding

Others?

ETA: Potentially leaving a record of being wrong in public

One meta problem is that different people have different sensitivities to these disincentives, and having enough disincentives to filter out low-quality content from people with low sensitivities just necessarily means some potential high-quality content from people with high sensitivities are also kept out. But it seems like there are still some things worth doing or experimenting with. For example:

Support for real-time collaborative discussion which subsequently get posted and voted upon as one unit (with votes going equally to both participants)

Disabling downvotes on AF

Having more indications/reminders of how much posting to LW/AF benefits the individual posters and the wider community, in terms of making intellectual progress and spreading good ideas. I'm not sure what form this could take, but maybe things like an indication of how many times a post is read.

My previous feature suggestion to help with the "feeling ignored" problem

Being less critical of new users and engaging more positively with them

There's a separate issue that some people don't read LW/AF as much as I would prefer but I have much less idea what is going on there.

On a tangentially related topic, is LW making any prep

...
Financial engineering for funding drug research
by ryan_b — 2019-05-10T18:46:03.029Z

Concept

A group of people from MIT's Sloan School of Management have put together a proposal for using financial engineering to get across the valley of death in drug development. The pitch is, approximately, to securitize a batch of drugs, and then re-securitize them after each stage of FDA trials. By using these securities, it becomes possible to build a fund which will finance a slew of drugs via both debt and equity, with rates of return comparable to things regular financial markets invest in. This makes it possible to put the financial might of mutual funds, retirement accounts, and people interested in hedging like medical insurers, behind drug development.

Megafunds

This takes a lot of money. A lot of money. In the original paper they modeled drug development as an investment of $200M, which after 10 years had a 5% success rate, with an average return of $12.3B on success (which number is the expected present value of $2B a year for 10 years, which itself is the expected length of monopoly guarantee). This is a heavy-duty risk, and most investors are unwilling or unable to take it. While it is clearly still worth it for drug companies, there is only so much money they can invest, and so the total investment is constrained. In the basic model they use 150 independent drugs, which means a megafund of 150 * $200M = $30B. The less independent the drugs, the more capital is required. So they need a way to distribute the risks such that they have access to this huge amount of capital.

Securities

The tool that they have developed for this is the Research-Backed Obligation, or RBO. Quoting from the original paper linked above:

A common form of securitization involves “cashflow” transactions in which a portfolio of assets—typically mortgages, auto loans, student loans, or credit-card receivables—is acquired using money raised by issuing equity and bonds of different seniorities. These assets and the cashflow they generate are pledged as collateral for the debt. In our proposed application, the assets include the initial capital raised from investors, all the subsequent biomedical R&D and licenses acquired, and all the profits generated by these activities or through sales of these assets in later periods. The application of securitization to early-stage clinical and preclinical biomedical has not been described previously, and we shall refer to debt that i

...
Where are the Benefits from Conversation?
by Davis_Kingsley — 2019-05-10T17:49:27.563Z

(epistemic status: fairly speculative)

Recently, I've been thinking about where the main benefits of conversation lie, and I've come to some interesting early conclusions.

Most of the benefits of private conversations come from effects they have on the immediate participants. However, this doesn't necessarily hold true for other forms of conversation. For instance, in a public debate, there is often little benefit to the debaters themselves, preparation aside [1], but often a benefit to the audience. On the other hand, in a public double-crux, the participants as well as the audience can benefit.

One area that I think this bears special interest for is that of online discussion. Online discussions are in some cases private, but because of the permanence of online conversations I feel they often have much more of a "bystander benefit" than conversations in person do - the arguments being offered are available not just to those who happened to be in the room at the time, but to many others who can view the archives.

For instance, I think most of the benefit of comments on LW doesn't come from the immediate interaction between the commenter and the original poster, but rather from their discussion being read by others. One piece of evidence that helps support this is how frequently many more people vote on a particular comment than are active in the conversation itself - this indicates that the people active in the discussion are often a small fraction of those reading, and when you consider people who don't vote, who "lurk", or who don't have accounts the number of readers probably increases further.

Real-time text conversation, like that offered by Slack, Discord, or IRC, is perhaps "in between" in-person and asynchronous online communication. However, I tend to think that similar assumptions hold, at least on public servers - there are many more "lurkers" who read but don't necessarily comment than there are active discussion participants. On Slack or Discord, which allow emoji reacts, this is especially apparent - I frequently see reactions added to conversations well after they have concluded, indicating that people have read the conversation and reacted asynchronously. [2]

One potential counterargument to this theory is that, while many more people may interact with arguments or statemen...
Correspondence visualizations for different interpretations of "probability"
by So8res — 2019-05-10T17:10:23.317Z

(Written for Arbital in 2016.)

Recall that there are three common interpretations of what it means to say that a coin has a 50% probability of landing heads:

The propensity interpretation: Some probabilities are just out there in the world. It's a brute fact about coins that they come up heads half the time; we'll call this the coin's physical "propensity towards heads." When we say the coin has a 50% probability of being heads, we're talking directly about this propensity.
The frequentist interpretation: When we say the coin has a 50% probability of being heads after this flip, we mean that there's a class of events similar to this coin flip, and across that class, coins come up heads about half the time. That is, the frequency of the coin coming up heads is 50% inside the event class (which might be "all other times this particular coin has been tossed" or "all times that a similar coin has been tossed" etc).
The subjective interpretation: Uncertainty is in the mind, not the environment. If I flip a coin and slap it against my wrist, it's already landed either heads or tails. The fact that I don't know whether it landed heads or tails is a fact about me, not a fact about the coin. The claim "I think this coin is heads with probability 50%" is an expression of my own ignorance, which means that I'd bet at 1 : 1 odds (or better) that the coin came up heads.

One way to visualize the difference between these approaches is by visualizing what they say about when a model of the world should count as a good model. If a person's model of the world is definite, then it's easy enough to tell whether or not their model is good or bad: We just check what it says against the facts. For example, if a person's model of the world says "the tree is 3m tall", then this model is correct if (and only if) the tree is 3 meters tall.

Definite claims in the model are called "true" when they correspond to reality, and "false" when they don't. If you want to navigate using a map, you had better ensure that the lines drawn on the map correspond to the territory.

But how do you draw a correspondence between a map and a territory when the map is probabilistic? If your model says that a biased coin has a 70% chance of coming up heads, what's the correspondenc...
Fear as fossil fuel
by aaq — 2019-05-10T15:29:43.709Z

Previous post: Extraordinary ethics require extraordinary arguments

My blog entries are about a personal battle against depression and anxiety, from the point of view of someone who has been immersed in rationalist/LW ideas and culture for a few years now.

In a chat with my little brother two nights ago, I told him about my "pain-strain theorem" - physical strain can be used to alleviate mental pain, and mental strain can be used to alleviate physical pain. (Proof as exercise for the reader.)

I introduced the idea to tie up several narrative-of-our-lives style observations:

I worked very hard on my schoolwork in elementary school. A large part of my motivation was that I had a severe chronic illness across my whole body. Keeping myself mentally occupied offered some sweet relief from the constant pain.

Our father also has a chronic illness that causes him a lot of pain. Before he developed this illness, he was a gregarious, vivacious twentysomething - a hard worker to be sure, but also a great jokester and eager team sport player. By the time we were being reared, however, he had pulled away from all of that and started sinking all of his time into his decently mentally challenging work.

Both my brother and I are diagnosed with mental illnesses. Both my brother and I find a solid, high intensity workout to be one of the best ways to escape the vicious thinking cycles we get locked into. For me, sometimes, the unthinking that comes with being 40 minutes into an elliptical workout is akin to sleep.

My chronic illness thankfully began to relent as I became a teenager; my grades began to slip. My depression deepened; I isolated myself in academia; my grades returned to their high point. I went to community college and let the maw swallow me. I had very close to a 4.0 before transferring out of my community college to an Ivy League, when everything else in my life was terrible - no money, no career prospects, few friends and newly feralized social skills. Now that I'm here, with much better treated depression, almost complete remission of my first chronic illness, a paid research for the summer in my field, and a loving circle of friends and family, I'm at risk of failing one class and I'm almost certainly taking an incomplete in another.

Of course I'm telling myself a story to fit this data in. To an outside human my life is an n=1 narrative, but from the inside,

...
Training human models is an unsolved problem
by Charlie Steiner — 2019-05-10T07:17:26.916Z

I

We can’t write down our precise values any more than we can write down the algorithm we use for judging whether an image contains a cat. If we want an AI to abide by human values, it's going to have to acquire them without us writing them down. We usually think of this in terms of a process of value learning.

The easiest kind of value learning involves starting with some pre-written model of humans - this part for the beliefs, and this part for the values, and so on - and tuning its internal parameters until it does a good job on a corpus of training data. The problem is that we want this human model to have lots of nice properties, each of which makes it harder to find a model that will satisfy us.

There's a tension between models that have clear values, and models that are psychologically realistic. The ideal value-haver is homo economicus, the sterile decision-theoretic agent. Consider what such a model must make of a training dataset that includes humans buying lottery tickets, and not wearing seatbelts, and being sold products by modern ad campaigns. The model has no leeway. It must assume that humans are behaving optimally, and therefore that there is some intrinsic value in lottery-tickets and seatbelt-free driving that should be preserved into the far future. As for the survival of humanity as a whole - well, if humans aren't taking the optimal action to ensure it, it must not matter all that much.

The homo economicus model is too psychologically unrealistic to learn what we mean by human values. But if you allow that humans might be lazy, or biased, or incapable of getting a handle on the consequences of their actions, then you're adding more and more degrees of freedom to your model. The more you allow for human action to not reflect their modeled values, the more underdetermined the modeled values are.

One of the various guarantees that people try to extract from value learning schemes is that if humans really did work according to your model, your value learning scheme would eventually make your model of the human converge to the human. With even fairly tame models of human bias, you quickly lose this sort of guarantee as the model becomes rich enough to learn unintended answers.

II

Let's change gears and talk about neural networks. It's not too big of a topic switch, though, because neural networks are a family of models...
Episode 4 of Tsuyoku Naritai! (the 'becoming stronger' podcast): TAPs
by Bae's Theorem — 2019-05-10T02:16:47.857Z

Latest episode is up! In this episode, we experiment with forming habits, to hopefully be more effective people. Transcript in description/show notes.

https://www.youtube.com/watch?v=L1_erZGknEA&feature=youtu.be

https://anchor.fm/tsuyokunaritai/episodes/Episode-4---TAPs-e402l1
Tales From the American Medical System
by Zvi — 2019-05-10T00:40:00.768Z

Epistemic Status: Overheard in New York

I am walking and talking with my friend, a Type I Diabetic, when he receives a phone call from his doctor’s office.

As a Type I Diabetic, my friend needs insulin. The effects of not having insulin are very bad, and include death.

He has run out of refills on his prescription, and will run out of insulin on Saturday. He called about a week ago to attempt to remedy this situation and get refills.

That’s for background. This isn’t about the order of magnitude higher my friend’s copay is in America, compared to the entire retail price in Canada.

This is about my friend’s attempt to get legal permission to continue buying life-saving medication for a lifelong condition with no known cure.

Because for some reason, in America you need legal permission to buy it, and you need to renew that periodically, despite there being chance of the need for said life-saving medicine going away.

I overheard the phone call from my friend’s end. The rest is filled in based on a combination of talking to him after, and extrapolation. He has reviewed this for accuracy.

The nurse tells my friend he needs to go see his doctor, because it has been seven months, and the doctor feels he should see his doctor every three.

My friend replies that he agrees he should see his doctor, and he has made an appointment in a few weeks when he has the time to do that.

The nurse says that he can’t get his prescription refilled until he sees the doctor.

My friend explains that he does not have the time to drop what he is doing and see the doctor the next day. That he is happy to see the doctor in a few weeks. But that until then, he requires insulin to live.

The nurse says that he can’t get his prescription refilled until he sees the doctor. That if he wants it earlier he can find another doctor.

My friend explains again that he does not have the time to see any doctor the next day, nor can one find a doctor on one day’s notice in reasonable fashion. And that he has already made an appointment, and needs insulin to live. And would like to speak with the doctor.

The nurse refuses to get the prescription filled. The nurse does not offer to let him speak to the doctor, and says that he can either wait, make an appointment for the next day, or find a new doctor.

My friend points out that without insulin, he will die. He asks if the nurse wants him to die. Or what the nurse suggest...
Interpretations of "probability"
by So8res — 2019-05-09T19:16:21.650Z

(Written for Arbital in 2016.)

What does it mean to say that a flipped coin has a 50% probability of landing heads?

Historically, there are two popular types of answers to this question, the "frequentist" and "subjective" (aka "Bayesian") answers, which give rise to radically different approaches to experimental statistics. There is also a third "propensity" viewpoint which is largely discredited (assuming the coin is deterministic). Roughly, the three approaches answer the above question as follows:

The propensity interpretation: Some probabilities are just out there in the world. It's a brute fact about coins that they come up heads half the time. When we flip a coin, it has a fundamental propensity of 0.5 for the coin to show heads. When we say the coin has a 50% probability of being heads, we're talking directly about this propensity.
The frequentist interpretation: When we say the coin has a 50% probability of being heads after this flip, we mean that there's a class of events similar to this coin flip, and across that class, coins come up heads about half the time. That is, the frequency of the coin coming up heads is 50% inside the event class, which might be "all other times this particular coin has been tossed" or "all times that a similar coin has been tossed", and so on.
The subjective interpretation: Uncertainty is in the mind, not the environment. If I flip a coin and slap it against my wrist, it's already landed either heads or tails. The fact that I don't know whether it landed heads or tails is a fact about me, not a fact about the coin. The claim "I think this coin is heads with probability 50%" is an expression of my own ignorance, and 50% probability means that I'd bet at 1 : 1 odds (or better) that the coin came up heads.

For a visualization of the differences between these three viewpoints, see Correspondence visualizations for different interpretations of "probability". For examples of the difference, see Probability interpretations: Examples. See also the Stanford Encyclopedia of Philosophy article on interpretations of probability.

The propensity view is perhaps the most intuitive view, as for many people, it just feels like the coin is intrinsically random. However, this view is difficult to reconcile with the idea that once we...